Artificial Analysis releases the image-to-video benchmark AA-Video-I2V v2.1
Artificial Analysis launches the image-to-video evaluation benchmarks AA-Video-I2V v2.1 and AA-Video-I2V-Silent v2.1, ranking models at up to 1080p resolution across 10 use-case categories, 10 capabilities, and multiple visual styles. The two versions are based on more than 58,000 human preference votes across over 1,000 prompts and more than 44,000 votes across 500 prompts, respectively, with only 5% of prompts made public in the public arena and rotated monthly. In the first version of the leaderboard, Dreamina Seedance 2.5 ranks first overall, statistically tied with the Seedance 2.0 and MiniMax H3 series, at a price of $34.12/min; the open-weight MiniMax H3 (768p) ranks third, at a price of $4.80/min.
Why read this
AI score 59
The new benchmark breaks image-to-video into 10 use-case categories and 10 capability scores, and provides a price and ranking comparison for each model to make it easier to judge cost-effectiveness.
Topics
Body
We are launching AA-Video-I2V v2.1 and AA-Video-I2V-Silent v2.1, our new benchmark for evaluating image to video models. It judges every model at up to 1080p on a new set of image and text prompts, and ranks models across 10 use cases, 10 capabilities and a diverse range of visual styles.
Image to video makes AI video production more controllable. Creators start from a product shot, a storyboard frame or a character design that needs to move. A strong model keeps the people, objects and setting of the starting image consistent while delivering the motion, camera work and sound the prompt asks for.
In our benchmark, every prompt pairs a 1080p starting image with a prompt written for that image, and is tagged by use case (such as Marketing & Advertising and Live-Action Film), by the capability it tests (such as Human Anatomy and Text Rendering), and by visual style. The overall ranking samples evenly across use cases and capabilities, so it reflects a model's versatility, not just its best category.
We are launching AA-Video-I2V v2.1 with more than 58,000 human preference votes from private evaluators based in the US/UK over 1,000 prompts, and AA-Video-I2V-Silent v2.1 with more than 44,000 votes over 500 prompts. Only 5% of our prompts are available on the public arena, keeping the evaluation resistant to public set contamination, with the prompts rotated every month.
Initial insights from the top of the AA-Video-I2V v2.1 Leaderboard:
➤ Dreamina Seedance 2.5 ranks #1 overall, statistically tied with Seedance 2.0 and the MiniMax H3 family, and leads 11 category boards, more than any other model, including Human Anatomy, Camera Control and Retail & Ecommerce. At $34.12/min it is the most expensive model in the top 10.
➤ Dreamina Seedance 2.0 ranks #2 and is #1 on Audio Synchronization and Lighting & Materials.
➤ MiniMax H3 (768p), an open weights model, ranks #3 at $4.80/min, about 1/7 of the price of Seedance 2.5. It is #1 on Text Rendering.
➤ MiniMax H3 Max, post-trained by fal, ranks #4 with audio and #1 on AA-Video-I2V-Silent v2.1, and leads Spatio-Temporal Consistency and UI/UX & Motion Design.
➤ Bach 1.0 Pro from Video Rebirth ranks #3 on AA-Video-I2V-Silent v2.1, statistically tied for #1, at $7.20/min.
See below for how image to video differs from text to video, and the use case, capability and style breakdowns 🧵