Serverless Inference API Token Limits/Settings

Beginners
1

Hi,

As written in Topic Title :
My request :
Where can i found information about the Token limits/Settings using Free Serverless Inference API ?

Context :
Trying automation for testing purposes from Make.com with HF_Models the responses generated by the models are Incomplete/truncated
Maybe there are some limits or Maybe there are some settings to do in
Makemodule : HTTP : make an API Key AUTH Request
NB : My knowledge is low

NB: Before opening the request I consulted the links below without finding an answer.
https://huggingface.co/docs/api-inference/rate-limits
/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fsearch%3Fq%3DAPI%2520limit%2520token%3C%2Fa%3E%3C%2Fp%3E

Any Help Welcome.
Thank You.

1 Like
2

For the record:
Useful documentation :
https://huggingface.co/docs/api-inference/tasks/text-generation?code=js#api-specification
https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1?inference_api=true

It would appear that there is a default limit of 500 tokens.
I still have to figure out how to send the ‘max_new_tokens’ parameter at the same time as my request from a Make module.
I am continuing my research on
https://community.make.com/c/how-to/66

regards

1 Like
3

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.