Add tokenizer.json

by Xenova HF Staff - opened Jan 7, 2024

←

•

This PR adds the tokenizer.json file so that it can be used with transformers.js.

Linked PR: #1

Xenova changed pull request title from Upload tokenizer.json with huggingface_hub to Add tokenizer.json Jan 7, 2024

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Ready to merge

This branch is ready to get merged automatically.

· Sign up or log in to comment