How to use buddhist-nlp/sanstib with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="buddhist-nlp/sanstib")
# Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("buddhist-nlp/sanstib") model = AutoModel.from_pretrained("buddhist-nlp/sanstib", device_map="auto")
https://huggingface.co/buddhist-nlp/sanstib/resolve/main/tokenizer_config.json
hf download hf://buddhist-nlp/sanstib/tokenizer_config.json
curl -L -o tokenizer_config.json https://huggingface.co/buddhist-nlp/sanstib/resolve/main/tokenizer_config.json
