sc-5445: ship lean pre-quantized Q4 snapshot (Q4 DiT 32.8->8.9GB + quantization manifest; prune redundant raw pickles; refresh card)

README.md

@@ -21,7 +21,7 @@ base_model: zai-org/SCAIL-2
 
 ## What changed vs. upstream
 Every component is repackaged to the safetensors layout the SceneWorks Rust/MLX loaders consume — no PyTorch at runtime:
-- **DiT** (`model/1/fsdp2_rank_0000_checkpoint.pt`, an FSDP2/SAT checkpoint) was key-remapped to the `SCAIL2Model` parameter naming using the upstream `convert.py` contract (fused `query_key_value`→`q`/`k`/`v`, `key_value`→`k`/`v`, `clip_feature_key_value_list`→`k_img`/`v_img`) and cast **fp32 → bf16** → `dit.safetensors`. Bit-faithful (987 source keys → 1307 model keys; exact key+shape match against `SCAIL2Model.from_config(config-14b.json)`).
+- **DiT** (`model/1/fsdp2_rank_0000_checkpoint.pt`, an FSDP2/SAT checkpoint) was key-remapped to the `SCAIL2Model` parameter naming using the upstream `convert.py` contract (fused `query_key_value`→`q`/`k`/`v`, `key_value`→`k`/`v`, `clip_feature_key_value_list`→`k_img`/`v_img`), cast **fp32 → bf16**, then **pre-quantized to group-wise-affine Q4** on disk → `dit.safetensors`. The attention (`q`/`k`/`v`/`o` + I2V `k_img`/`v_img`) and FFN (`ffn.0`/`ffn.2`) Linears are packed (`weight` u32 codes + `scales` + `biases` via MLX `quantize`, byte-equal to `nn.quantize`, group size 64); the patch/text/time/image embeddings, norms, and output head stay dense bf16. A `config.json` `quantization` block marks the snapshot so the loader builds the quantized Linears directly from the packs (no dense bf16 materialized at load). Bit-faithful key remap (987 source keys → 1307 model keys; exact key+shape match against `SCAIL2Model.from_config(config-14b.json)`).
 - **VAE** (`Wan2.1_VAE.pth`, the stock Wan2.1 z16 VAE) → `vae.safetensors` (**f32**, channels-last conv transpose, keys unchanged — the `sanitize_wan_vae_weights` contract shared with Bernini/wan). Loaded by `mlx_gen_wan::WanVae`.
 - **Text encoder** (`umt5-xxl/models_t5_umt5-xxl-enc-bf16.pth`, stock UMT5-XXL) → `t5_encoder.safetensors` (**bf16**, sole rename `.ffn.gate.0.`→`.ffn.gate_proj.`). Loaded by `mlx_gen_wan::Umt5Encoder` with `tokenizer.json`.
 - **Image encoder** (`models_clip_...onlyvisual.pth`, open-CLIP XLM-RoBERTa ViT-H/14) → `clip.safetensors` (**f32**, de-prefixed `visual.*` keys). Loaded by `mlx_gen_scail2::ScailClip` (32-layer visual tower, `use_31_block` penultimate features).
@@ -31,21 +31,22 @@ The converted VAE/UMT5 are byte-size-identical (modulo safetensors header) to Be
 ## Contents (turnkey MLX snapshot)
 | file | source | loader | notes |
 |---|---|---|---|
-| `dit.safetensors` | converted | `Scail2Dit` | SCAIL-2 14B DiT, **bf16**, 1307 tensors (~31 GB) |
+| `dit.safetensors` | converted | `Scail2Dit` | SCAIL-2 14B DiT, **Q4 packed** (attn + FFN) + dense bf16 (embeds/norms/head), ~8.9 GB |
 | `vae.safetensors` | converted | `WanVae` | Wan2.1 z16 VAE, **f32**, stride (4,8,8) (~0.5 GB) |
 | `t5_encoder.safetensors` | converted | `Umt5Encoder` | UMT5-XXL encoder, **bf16** (~11 GB) |
 | `clip.safetensors` | converted | `ScailClip` | open-CLIP ViT-H/14 visual tower, **f32**, 1280-dim (~2.5 GB) |
 | `tokenizer.json` | upstream, stock | `load_tokenizer` | UMT5-XXL HF tokenizer (root copy) |
-| `config.json` | upstream `configs/config-14b.json` | `Scail2Config` | `model_type: i2v`, `dim 5120`, `ffn 13824`, `40` layers/heads, `in_dim 20`, `mask_dim 28`, `out_dim 16` |
-| `bias-aware-dpo-lora.pt` | upstream, stock | (sc-5451) | optional Bias-Aware DPO refinement LoRA |
+| `config.json` | upstream `configs/config-14b.json` + `quantization` block | `Scail2Config` | `model_type: i2v`, `dim 5120`, `ffn 13824`, `40` layers/heads, `in_dim 20`, `mask_dim 28`, `out_dim 16`; `quantization: {bits 4, group_size 64}` |
+| `bias-aware-dpo-lora.pt` | upstream, stock | `mlx_gen_scail2` (sc-5451) | optional Bias-Aware DPO refinement LoRA |
 
-Quantization (Q4/Q8) is applied at **load time** by the SceneWorks worker (`mlx_gen` `.quantize()`); this snapshot ships the dense bf16/f32 weights.
-
-The raw upstream pickles (`Wan2.1_VAE.pth`, `umt5-xxl/models_t5_umt5-xxl-enc-bf16.pth`, `models_clip_...onlyvisual.pth`) remain in this repo for provenance; the Rust loaders use only the converted safetensors above, so a lean SceneWorks pull can skip them.
+The DiT ships **pre-quantized to Q4 on disk** (the SceneWorks worker default), so the loader reads the packs directly — there is no dense-bf16 load transient. The VAE / UMT5 / CLIP ship dense (f32 / bf16). This repo ships **only** the loadable safetensors + tokenizer + the optional DPO LoRA; the redundant raw upstream pickles (`Wan2.1_VAE.pth`, `umt5-xxl/models_t5_...pth`, `models_clip_...onlyvisual.pth`) have been **pruned** — they are reproducible from the upstream release and the Rust loaders never used them.
 
 ## Architecture (summary)
 Wan2.1-14B **I2V** dense DiT. Conditioning is a **token-axis packed** stream — reference + video + pose patch-embedded (three Conv3d stems) with additive 28-channel color-coded mask embeddings, concatenated into one self-attention sequence — plus a **per-source RoPE** with integer T/H/W shifts (the `replace_flag` flips the reference H-shift, toggling animation vs. replacement). The reference image is encoded by the CLIP visual tower and injected via Wan-I2V image cross-attention. Sampling is plain CFG (guide 5.0), flow-matching UniPC/DPM++.
 
+## Runtime (Apple Silicon)
+The production default — **832×480 / 5 s** (one 81-frame driving segment) — runs the DiT in **f32 compute** (bf16 overflows to NaN at that packed-sequence length), with shared FFN/attention activation chunking and a temporal-tiled VAE decode, at a measured process footprint of **~70–76 GB**. SceneWorks gates SCAIL-2 to **96 GB**-class Macs. The Q4 DiT keeps the resident weights and the snapshot download lean (≈ 24 GB total).
+
 ## License & attribution
 This repackaging redistributes upstream weights under the license declared on the upstream model card (**MIT**); the upstream code repository is Apache-2.0. Please consult and cite the original:
 - Model: https://huggingface.co/zai-org/SCAIL-2

Wan2.1_VAE.pth

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:38071ab59bd94681c686fa51d75a1968f64e470262043be31f7a094e442fd981
-size 507609880

config.json

@@ -2,14 +2,18 @@
   "_class_name": "WanSCAILModel",
   "_diffusers_version": "0.30.0",
   "dim": 5120,
-  "eps": 1e-06,
+  "eps": 1e-6,
   "ffn_dim": 13824,
   "freq_dim": 256,
   "in_dim": 20,
-  "mask_dim": 28, 
+  "mask_dim": 28,
   "model_type": "i2v",
   "num_heads": 40,
   "num_layers": 40,
   "out_dim": 16,
+  "quantization": {
+    "bits": 4,
+    "group_size": 64
+  },
   "text_len": 512
-}
+}

dit.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:edb1d362c34ae98be657cd7aa4fbd6eaa9737eacf675823be0c04b64537088fd
-size 32791225720
+oid sha256:8aa6a80345bbeab2a42bdf09977d64ad63920a5846a3850a3fcc5b61cc5eddc1
+size 9578472097

models_clip_open-clip-xlm-roberta-large-vit-huge-14-onlyvisual.pth

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:020daacf4c0ce94284df584d243cefb6ddfadefff8772226a8e85431df5de2da
-size 2528485611

umt5-xxl/models_t5_umt5-xxl-enc-bf16.pth

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:7cace0da2b446bbbbc57d031ab6cf163a3d59b366da94e5afe36745b746fd81d
-size 11361920418