NVIDIA and MIT Team Open-Source a 4K Video Enhancement Solution Based on MiniMax H3

Recently, the Efficient-Large-Model team, composed of researchers from NVIDIA and MIT, released the model weights for the video enhancement project Open-H3-Regenerate-2K on Hugging Face, with the accompanying inference code now open in the NVlabs/Sana repository. The solution is based on MiniMax H3 and regenerates low-resolution video drafts into high-resolution video, supporting 2560×1440 2K output and achieving native 4K regeneration at up to 3840×2160.

The complete MiniMax H3 system includes multimodal context processing, base audio-video generation, and high-resolution regeneration. Among these, H3-Base, whose weights have been released, handles 768P audio-video generation, while the official H3-Regenerate-2K module is offered as an API service. The project released this time reuses the generation capabilities of the H3 base model to provide a self-deployable open implementation for the high-resolution regeneration stage.

During the training phase of the refinement model, the research team kept the original parameters of the H3 Transformer backbone and the video VAE unchanged, training only the newly added refinement LoRA. At inference, the refinement LoRA is merged with community adapters such as LightX2V FL2VA Turbo and LMS Sharp, and through three-step denoising, regenerates image details at the target resolution. The scene, layout, and motion information in the video draft provide the basis for this process.

To improve inference efficiency, the project adopts a latent-space upscaler developed by the community to directly upscale the video's latent representation, reducing the overhead of repeated decoding and encoding at high resolution. At the same time, the solution combines SoL-Engine's sparse attention, low-precision computation, and a fast VAE decoder to accelerate the refinement and decoding stages.

Tests published by the project team show that, on a single NVIDIA B200 GPU under warmed-up test conditions, the combined time for latent-space upscaling, three-step refinement, and 2K decoding of an approximately 5-second video dropped from 87.26 seconds in the unoptimized implementation to 22.45 seconds, roughly 3.9 times faster than the original. Among these, the fast decoder reduced video decoding time from about 15 seconds to about 1.6 seconds.

In terms of image quality, the project team conducted comparative evaluations on RefinerBench-150. Under the same input and same 2K output conditions, Open-H3-Regenerate-2K scored an average of 60.0 on the perceptual quality benchmark UniPercept, while the official H3-Regenerate-2K API scored 53.4. In the VBench evaluation, the solution scored higher on aesthetic quality, imaging quality, and subject consistency, while the official API remained slightly better on motion smoothness and dynamic degree.

MiniMax H3 opened its weights in August this year, and fine-tuning, distillation, and tool adaptation around H3 rapidly increased, attracting participation from research teams and community developers at home and abroad. Within one month, downloads exceeded 24 million, and more than 300 public derivative models were produced, making H3 one of the most downloaded models globally in 2026.

Among these, the joint development by NVIDIA and MIT around H3 covered different directions. In addition to this video enhancement solution, the team also released the streaming video rendering model StreamRender-H3 built on H3 Ref2VA, and the fast decoder Sol-H3-VAE distilled from the H3 video VAE, aimed at interactive video rendering and decoding efficiency optimization, respectively.

Previously, LightX2V's MiniMax-H3 Turbo offered distilled versions with four steps, eight steps, and others, while the community fine-tuned model Singularity focused on optimizing motion performance, facial details, and visual style. Overseas teams FastVideo and Pika also released the FastH3 four-step preview version and the six-step sampling Pika H3 Flex, respectively; ComfyUI and Unsloth, through adapted weights and quantized versions, brought H3 into more local creative workflows.