Image-to-Video
Diffusers
text-to-video
video-to-video
image-text-to-video
audio-to-video
text-to-audio
video-to-audio
audio-to-audio
text-to-audio-video
image-to-audio-video
image-text-to-audio-video
ltx-2
ltx-2-3
ltx-video
ltxv
lightricks
nvfp4
quantized
fp4me
Instructions to use MrReclusive/LTX-2.3-FP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use MrReclusive/LTX-2.3-FP4 with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MrReclusive/LTX-2.3-FP4", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -32,18 +32,166 @@ tags:
|
|
| 32 |
- ltx-video
|
| 33 |
- ltxv
|
| 34 |
- lightricks
|
|
|
|
|
|
|
|
|
|
| 35 |
pinned: true
|
| 36 |
demo: https://console.ltx.video/playground/
|
| 37 |
---
|
| 38 |
|
| 39 |
-
# LTX-2.3 NVFP4
|
| 40 |
|
| 41 |
-
**
|
| 42 |
|
|
|
|
| 43 |
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
|
|
|
| 48 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
|
|
|
|
|
|
| 32 |
- ltx-video
|
| 33 |
- ltxv
|
| 34 |
- lightricks
|
| 35 |
+
- nvfp4
|
| 36 |
+
- quantized
|
| 37 |
+
- fp4me
|
| 38 |
pinned: true
|
| 39 |
demo: https://console.ltx.video/playground/
|
| 40 |
---
|
| 41 |
|
| 42 |
+
# π¬ LTX-2.3 NVFP4 Quantized Models
|
| 43 |
|
| 44 |
+
> **Compress LTX-2.3 from 46GB β 13GB while maintaining *most* of the quality**
|
| 45 |
|
| 46 |
+
These are NVFP4 quantized variants of LTX-2.3, optimized for users with limited VRAM who still want high-quality video generation. Along with the models, I'm sharing **Comfy Bathroom** β a custom node suite designed specifically for using LoRAs with these heavily quantized models.
|
| 47 |
|
| 48 |
+
---
|
| 49 |
+
|
| 50 |
+
## π¦ Available Models
|
| 51 |
+
|
| 52 |
+
| Model | Size | Description |
|
| 53 |
+
|-------|------|-------------|
|
| 54 |
+
| `LTX-2.3-FP4` | ~21GB | Standard NVFP4 quantization |
|
| 55 |
+
| `LTX-2.3-FP4ME` | ~13GB | Mixed Extreme - Blocks 0, 47 in BF16; Block 1, 46 in FP8 |
|
| 56 |
+
|
| 57 |
+
## π¦ Coming Soon Models
|
| 58 |
+
| Model | Size | Description |
|
| 59 |
+
|-------|------|-------------|
|
| 60 |
+
| `LTX-2.3-FP4MEL` | ~15GB | Mixed Extreme LoRA-friendly |
|
| 61 |
+
|
| 62 |
+
> β οΈ **Note:** FP4ME variants have known LoRA compatibility issues. Use the included Comfy Bathroom nodes for best results.
|
| 63 |
+
|
| 64 |
+
---
|
| 65 |
+
|
| 66 |
+
## πΏ Comfy Bathroom β Custom Node Suite
|
| 67 |
+
|
| 68 |
+
**Comfy Bathroom** is a LoRA loading system designed to work with NVFP4 overly quantized models. It provides per-block weight control to avoid artifacts when using LoRAs on aggressively quantized models.
|
| 69 |
+
|
| 70 |
+
### Why "Bathroom"?
|
| 71 |
+
|
| 72 |
+
Because I made the FP4 models with **Comfy Kitchen**, and when something goes wrong in the kitchen... you end up in the bathroom. π½
|
| 73 |
+
|
| 74 |
+
### πͺ₯ Nodes Included
|
| 75 |
+
|
| 76 |
+
| Node | Icon | Description |
|
| 77 |
+
|------|------|-------------|
|
| 78 |
+
| **Toothbrush** | πͺ₯ | LoRA loader with built-in FP4 presets |
|
| 79 |
+
| **Mirror (Simple)** | πͺ | Per-block on/off toggles |
|
| 80 |
+
| **Mirror (Fancy)** | πͺ | Per-block 0.0-1.0 strength sliders |
|
| 81 |
+
| **Shower** | πΏ | Quick preset applier |
|
| 82 |
+
| **Towel** | π§Ύ | LoRA packet info display |
|
| 83 |
+
| **Bathroom Sink** | π° | Stack multiple LoRAs with global strength |
|
| 84 |
+
|
| 85 |
+
---
|
| 86 |
+
|
| 87 |
+
## π§ How It Works
|
| 88 |
+
|
| 89 |
+
### The Problem
|
| 90 |
+
|
| 91 |
+
When using LoRAs on NVFP4 quantized models, certain transformer blocks cause artifacts:
|
| 92 |
+
|
| 93 |
+
- **Low blocks (0-10)** β Smoky/cloudy output (noise generation interference)
|
| 94 |
+
- **High blocks (40-47)** β Double vision/ghosting (detail alignment issues)
|
| 95 |
+
- **Middle blocks (11-39)** β Usually stable for LoRA injection
|
| 96 |
+
|
| 97 |
+
### The Solution
|
| 98 |
+
|
| 99 |
+
Comfy Bathroom applies intelligent weight curves to transformer blocks, ramping up LoRA strength gradually in early blocks and tapering down in late blocks.
|
| 100 |
+
|
| 101 |
+
### Built-in Presets
|
| 102 |
+
|
| 103 |
+
| Preset | Description | Best For |
|
| 104 |
+
|--------|-------------|----------|
|
| 105 |
+
| **FP4ME Light** | Gentle ramp up/down, block 1 OFF | Style LoRAs, general use |
|
| 106 |
+
| **FP4ME Heavy** | Aggressive taper to 0 at block 46 | Heavy style modifications |
|
| 107 |
+
| **FP4MEL Light** | Offset for BF16 blocks 1, 46 | FP4MEL model + style LoRAs |
|
| 108 |
+
| **FP4MEL Heavy** | Aggressive taper with BF16 safe zones | FP4MEL + content LoRAs |
|
| 109 |
+
|
| 110 |
+
### Weight Curve Visualization
|
| 111 |
+
|
| 112 |
+
```
|
| 113 |
+
FP4ME Light:
|
| 114 |
+
Block: 0 1 2 3 4 5 6 7 8 9 10 11 ... 39 40 41 42 43 44 45 46 47
|
| 115 |
+
Weight: 1.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 1.0 1.0 ... 1.0 0.95 0.9 0.85 0.8 0.75 0.7 0.65 0.5 1.0
|
| 116 |
+
BF16 FP8 βββ RAMP UP βββ FULL βββββββ TAPER DOWN βββ FP8 BF16
|
| 117 |
+
```
|
| 118 |
+
|
| 119 |
+
---
|
| 120 |
+
|
| 121 |
+
## πΈ Example Workflow
|
| 122 |
|
| 123 |
+

|
| 124 |
|
| 125 |
+
### Basic Usage
|
| 126 |
+
|
| 127 |
+
```
|
| 128 |
+
Toothbrush (LoRA + Preset) βββΊ Bathroom Sink βββΊ Model
|
| 129 |
+
β
|
| 130 |
+
global_strength
|
| 131 |
+
```
|
| 132 |
+
|
| 133 |
+
### Advanced Usage
|
| 134 |
+
|
| 135 |
+
```
|
| 136 |
+
Toothbrush βββΊ Shower (swap preset) βββΊ Mirror (fine tune) βββΊ Bathroom Sink βββΊ Model
|
| 137 |
+
β β
|
| 138 |
+
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 139 |
+
Multiple LoRAs can be stacked
|
| 140 |
+
```
|
| 141 |
+
|
| 142 |
+
---
|
| 143 |
+
|
| 144 |
+
## π‘ Usage Tips
|
| 145 |
+
|
| 146 |
+
### For Style LoRAs (90% of use cases)
|
| 147 |
+
- Use **FP4ME Light** preset
|
| 148 |
+
- The base model already knows anatomy/structure
|
| 149 |
+
- LoRA only needs to affect the "style layer" (middle blocks)
|
| 150 |
+
|
| 151 |
+
### For Content LoRAs (specific characters, unusual anatomy)
|
| 152 |
+
- Try **FP4ME Heavy** preset
|
| 153 |
+
- May need to experiment with Mirror nodes for fine-tuning
|
| 154 |
+
- Content LoRAs teach new concepts β need more block coverage
|
| 155 |
+
|
| 156 |
+
### For Multiple LoRAs
|
| 157 |
+
- Connect multiple Toothbrush nodes to Bathroom Sink
|
| 158 |
+
- Use global_strength to scale everything together
|
| 159 |
+
- Individual LoRA strengths multiply with global
|
| 160 |
+
|
| 161 |
+
---
|
| 162 |
+
|
| 163 |
+
## π Requirements
|
| 164 |
+
|
| 165 |
+
- ComfyUI (latest version recommended)
|
| 166 |
+
- `comfy_kitchen` (for NVFP4 model loading)
|
| 167 |
+
- BF16/FP16/FP8/FP4 capable GPU (known to work on 3xxx and above)
|
| 168 |
+
- 16GB+ VRAM recommended, seen people use it on 12GB
|
| 169 |
+
|
| 170 |
+
---
|
| 171 |
+
|
| 172 |
+
## π Installation
|
| 173 |
+
|
| 174 |
+
1. Download model files to `ComfyUI/models/checkpoints/` *unless you have a working unet loader for ltx2.3
|
| 175 |
+
2. Install Comfy Bathroom:
|
| 176 |
+
```
|
| 177 |
+
ComfyUI/custom_nodes/comfy_bathroom.py
|
| 178 |
+
```
|
| 179 |
+
3. Restart ComfyUI
|
| 180 |
+
|
| 181 |
+
---
|
| 182 |
+
|
| 183 |
+
## π Credits
|
| 184 |
+
|
| 185 |
+
- **LTX-2.3** by [Lightricks](https://github.com/Lightricks/LTX-2)
|
| 186 |
+
- **NVFP4 Quantization** via Modified Comfy Kitchen
|
| 187 |
+
- **Comfy Bathroom Nodes** written by GLM5 (Z.ai)
|
| 188 |
+
|
| 189 |
+
---
|
| 190 |
+
|
| 191 |
+
## π License
|
| 192 |
+
|
| 193 |
+
This model is subject to the [LTX-2 Community License Agreement](https://github.com/Lightricks/LTX-2/blob/main/LICENSE).
|
| 194 |
+
|
| 195 |
+
---
|
| 196 |
|
| 197 |
+
*When your LoRA is full of artifacts, head to the Bathroom. πΏ*
|