Hugging Face's logo

google
/
gemma-3-12b-it

Image-Text-to-Text
Transformers
Safetensors
gemma3
conversational
text-generation-inference
Model card Files
xet
 Community
34

batched tokenization support

#34
by peiyan88 - opened
Discussion
peiyan88

Does Gemma 3 tokenizer support batched tokenization?
pannaga10
 Google org

Hi @peiyan88
Yes, batched tokenization is supported. Gemma 3 tokenizers follow the standard Hugging Face Transformers tokenizer interface, which accepts both single inputs and batches.

Thanks

Sign up or log in to comment