tracyjxm/SYNTHIA

🪄 Affordance-based Novel Concept Generator (Kandinsky-3 Fine-Tuned)

This is a fine-tuned version of the Kandinsky-3 text-to-image pipeline, designed to generate novel object and furniture concepts by combining affordance-driven functionalities (e.g., "sofa + bed + cargo + bicycle").

---

🚀 How to Use

import os
import sys
import torch
from kandinsky3 import get_T2I_pipeline, get_T2I_Flash_pipeline

# Add kandinsky3 to Python path
sys.path.append('..')

# Set device and dtype maps
device_map = torch.device('cuda:0')
dtype_map = {
    'unet': torch.float32,
    'text_encoder': torch.float32,
    'movq': torch.float32,
}

# Load the Flash text-to-image pipeline
t2i_pipe = get_T2I_Flash_pipeline(
    device_map=device_map,
    dtype_map=dtype_map,
    cache_dir="./cache/"
)

# Load fine-tuned UNet weights
t2i_pipe.unet.load_state_dict(torch.load(
    "unet_model_checkpoint.pt",
    map_location=device_map
))

# Generate image from prompt
res = t2i_pipe(
    text="a new furniture design that has functions from sofa, bed, cargo, bicycle",
    steps=50
)[0]

# Save the result
res.save("generated_image.jpg")