DistilViT2 for Transformers.js

This model is compatible with transformers.js image-to-text pipeline.

Usage

import { pipeline } from '@huggingface/transformers';

const captioner = await pipeline('image-to-text', 'tarekziade/distilvit2');
const result = await captioner('path/to/image.jpg');
console.log(result);

Architecture

• Vision: SigLIP-base-patch16-224 (frozen during training)
• Projector: Trained linear/MLP projection (768 → 576)
• Text: SmolLM-135M with merged LoRA adapters

Training

• Dataset: Flickr30k, COCO
• Task: Image captioning
• Trainable parameters: 2.2M (1% of total)