Understand this story
Tsinghua University, NVIDIA, CMU, UC Berkeley, Princeton, NUS and other institutions have jointly launched StreamEval, an evaluation benchmark for streaming interactive video generation. The benchmark covers five task categories—digital human dialogue, virtual human performance, world generation, instruction editing, and reference editing—with a total of 300 test cases and 38 models evaluated. The evaluation incorporates instruction execution, state consistency, visual quality, and audio-visual coordination into a single 0–100 scoring system, with the code, dataset, and scoring rules all open-sourced, and the leaderboard planned to be updated monthly. The results show that visual quality and instruction following are relatively independent capabilities; for example, Vidu S2-Avatar scores 81.71 overall on the virtual human performance task, but only 36.90 on instruction following.
AI-generated from public reports
Developments
1 confirmed developments
Reports and discussion timeline
Browse reports and discussion by time. Their evidence levels remain distinct.
Oct 27
36Kr · AI
Tsinghua University, NVIDIA and other institutions jointly release StreamEval, a real-time interactive video evaluation benchmark, with 38 models participating
Tsinghua University, NVIDIA, CMU, UC Berkeley, Princeton, NUS and other institutions jointly released StreamEval, an evaluation benchmark for streaming interactive video generation, covering five task categories: digital human dialogue, virtual human performance, world generation, instruction editing and reference editing, with a total of 300 test cases and 38 models. The evaluation incorporates instruction execution, state consistency, visual quality and audio-visual coordination into a single 0–100 scoring system; the code, dataset and scoring rules are all open-sourced, and the leaderboard is planned to be updated monthly. Results show that visual quality and instruction following are relatively independent capabilities; for example, Vidu S2-Avatar scored 81.71 overall on the virtual human performance task, but only 36.90 on instruction following.