stdstu123/LynnReal-Onmi-beta-0.1

LynnReal-Omni · Standard four-step 🎬

One model for text-to-video, image-to-video, body- and hand-pose control, omni-reference generation, video editing, frame-by-frame repair and streaming long video — generated with four denoiser forwards per clip.

Code · Paper · YouTube demo · Bilibili demo

Demo

Compressed preview (≈10 MB, 960×540, audio included). Full-quality demo: YouTube · Bilibili · paper.

ComfyUI 🧩 — start here

The ComfyUI port runs these checkpoints out of the box, with no launcher flags. Everything it needs is in this repository: the workflows, the small custom node pack and the six model files, all in ComfyUI format under comfyui/models/.

1. Copy comfyui/custom_nodes/ComfyUI-LynnReal into ComfyUI/custom_nodes/.
2. Copy the workflows below into ComfyUI/user/default/workflows/.
3. Copy the model files under comfyui/models/ into the matching ComfyUI/models/ folders (see the table further down for what each file is).
4. Open a workflow and press Run. A warm 4-step 1344×768 run takes ≈50 s on an H100 for a five-second video.

Task	Workflow	Node pack	Demo input
Text → video + audio	t2v_lynnreal_4step.json	—	—
First frame → video	i2v_lynnreal_4step.json	—	transparent_rgb_gaming_mouse.png
Reference images → video	r2v_lynnreal_4step.json	—	red_superboy_on_city_roof.png, mecha_dragon_lightning.png
Pose control clip → video	pose2v_lynnreal_4step.json	required	pose_boxing_first.png, pose_boxing_control.mp4
Video continuation	v2v_lynnreal_4step.json	—	snowboard.mp4

The demo inputs sit next to the workflows under comfyui/input/. comfyui/README.md documents every file each task loads, the optional INT8 switch and the VRAM behaviour; the workflows are also in the GitHub repository.

The ComfyUI port is experimental and under active construction 🚧 — it runs the same checkpoints and schedules, but the pipeline around them is ComfyUI's, so take performance numbers from the original scripts in script/sample/; the port is measurably slower today. Same-seed output is not comparable between the two engines: compare quality, not pixel identity. 💨 The Flash three-step checkpoint is coming as a separate release with its Light VAE — on an H100 it generates a 1344×768 five-second t2v in about 10 s warm.

What the model does

LynnReal-Omni connects controllable visual workflows through one shared multimodal diffusion transformer (32B, following the MiniMax H3 architecture). A native packing layout distinguishes appearance references, frame-aligned controls and causal history, so a single checkpoint covers:

Generate	Control	Edit & repair
Text-to-video and first-frame conditioning	Single- and multiple-subject references	Instruction-guided image editing
Video continuation	Body- and hand-pose control	Video appearance editing
Speech and video generation	Game- and mesh-video rendering	General frame-by-frame video repair

Long video is generated in chunks that condition on the preceding context and emit 17 new frames per step, with a fixed head-overlap interface that bounds continuation memory. This release also contains the Standard DiT in ComfyUI format (comfyui/models/), including an optional W8A8 INT8 checkpoint built from the same weights.

On a single H100, warm generation and decoding of a 22-frame 540p clip takes 843 ms with the Standard model and 377 ms with Flash, using the original scripts; Flash, a 27B three-step variant with a lightweight VAE decoder, is the path toward real-time streaming.

Files in this repository

File	Size	Where it goes (ComfyUI)
comfyui/models/diffusion_models/lynnreal_omni_standard_bf16.safetensors	61.7 GiB	models/diffusion_models/
comfyui/models/diffusion_models/lynnreal_omni_standard_int8.safetensors	41.4 GiB	models/diffusion_models/ (optional, INT8 switch)
comfyui/models/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors	14.6 GiB	models/text_encoders/
comfyui/models/vae/minimax_h3_video_vae_fp16.safetensors	4.9 GiB	models/vae/
comfyui/models/vae/minimax_h3_audio_vae_fp32.safetensors	577 MiB	models/vae/
comfyui/models/embeddings/minimaxh3_art_is_explosion.safetensors	500 KiB	models/embeddings/
assets/lynnreal-demo-preview.mp4	9.4 MiB	— (demo preview)

The diffusers-format bundle (transformer shards, text encoder, VAE and schedulers) stays at the repository root and is what the script/sample/ launchers in the GitHub repository load.

More

• Setup, tasks and launchers (reference implementation): GitHub README
• ComfyUI bundle details: comfyui/README.md · end-to-end verification record
• Flash (three-step) weights: stdstu123/LynnReal-Onmi-flash-beta-0.1 (coming soon)
• Lightweight VAE: stdstu123/LynnReal-Onmi-light-vae (coming soon)

This is an early beta: bugs, compatibility issues and inconsistent generation quality may remain. We plan to release the training code, part of the training data and a more efficient DiT. Issues and pull requests are welcome. 🙌

Acknowledgments

We thank the MiniMax H3 team for the model that forms the foundation of LynnReal-Omni, and Qwen3-VL for the multimodal encoder, tokenizer and processor implementation used for conditioning. Model licenses and upstream attribution are retained in the bundle; see LICENSE.