README.md

metadata

title: Cosmos3 Action Viewer
emoji: 🤖
colorFrom: blue
colorTo: green
sdk: docker
app_port: 7860
pinned: false
license: other

Cosmos3 Action Viewer

An interactive, browser-based viewer for robot action datasets that runs Cosmos3 world-model generation locally. Three modes are supported:

• Forward dynamics — predict future video frames from a sequence of actions.
• Inverse dynamics — predict the actions that connect observed frames.
• Policy — predict the next actions to take.

The app serves the Viser frontend and runs Cosmos3 generation in a single process. Packaged demo data ships under assets/examples/, so you can try every mode without downloading any datasets — you only need the model checkpoint.

Run Locally

You need an NVIDIA GPU with >= 48GiB memory and the NVIDIA Container Toolkit installed.

One time, download the Cosmos3-Nano checkpoint:

hf download nvidia/Cosmos3-Nano --local-dir app_data/models/Cosmos3-Nano

To run:

# Build the image.
docker build -t cosmos3-action-viewer .

# Launch docker, mounting the directory that holds the model.
docker run --rm --gpus all -p 8020:7860 \
  -v "$(pwd)/app_data:/app_data" \
  cosmos3-action-viewer

Then open http://localhost:8020.

The container looks for the checkpoint under /app_data/models/ (the mount above), so any of these on-disk layouts work:

• app_data/models/Cosmos3-Nano
• app_data/models/nvidia--Cosmos3-Nano

On first launch the model is warmed up in the background. A generation requested before warmup finishes simply waits for it to complete, then reuses the loaded pipeline — so the first generation is the slowest and later ones are faster.

Customizing a Launch

All behavior is controlled through environment variables, which you can pass to docker run with -e NAME=value. The most useful ones:

Checkpoint / data locations:

• ACTION_VIZ_CHECKPOINT_LOCAL_PATH — explicit checkpoint path, or a colon-separated list of candidate paths to try in order.
• ACTION_VIZ_MODEL_MOUNT_PATH — explicit path to a mounted Hugging Face model.
• ACTION_VIZ_APP_DATA_ROOT — root for app data inside the container (default /app_data); models are resolved under <root>/models/.
• ACTION_VIZ_OUTPUT_ROOT — where generated artifacts are written (default /tmp/action_viz_sessions). Artifacts are cleaned up after each generation/session.

Generation settings:

• ACTION_VIZ_GENERATE_IMAGE_SIZE — output resolution (default 480).
• ACTION_VIZ_GENERATE_NUM_FRAMES — frames per generation (default 24).
• ACTION_VIZ_GENERATE_NUM_STEPS — sampling steps (default 30).
• ACTION_VIZ_DEFAULT_DATASET — dataset selected on load (default bridge).
• ACTION_VIZ_USE_TORCH_COMPILE — torch.compile the pipeline (default 1). The first generation pays a one-time compile cost in exchange for faster subsequent runs.
• ACTION_VIZ_BACKGROUND_WARMUP — warm the model on startup (default 1); set to 0 for a lightweight UI-only start that skips loading the model.

Example — lower resolution, fewer steps, and a different default dataset:

docker run --rm --gpus all -p 8020:7860 \
  -v "$(pwd)/app_data:/app_data" \
  -e ACTION_VIZ_GENERATE_IMAGE_SIZE=320 \
  -e ACTION_VIZ_GENERATE_NUM_STEPS=15 \
  -e ACTION_VIZ_DEFAULT_DATASET=droid \
  cosmos3-action-viewer

Diagnostics

Check how the checkpoint is being resolved (read-only, no model load):

docker run --rm --gpus all \
  -v "$(pwd)/app_data:/app_data" \
  cosmos3-action-viewer python /app/scripts/checkpoint_diagnostic.py

Run a one-step generation smoke test without the browser UI (temporary artifacts are cleaned up by default):

docker run --rm --gpus all \
  -v "$(pwd)/app_data:/app_data" \
  cosmos3-action-viewer \
  python /app/scripts/local_generation_smoke.py --model-mode forward_dynamics --num-steps 1

docker run --rm --gpus all \
  -v "$(pwd)/app_data:/app_data" \
  cosmos3-action-viewer \
  python /app/scripts/local_generation_smoke.py --model-mode policy --num-steps 1