BLOOM outputs only few tokens

Inference Endpoints on the Hub
1

Hi, I am trying to inference on BLOOM using Inference API. It only produces a few tokens (max 1-3 sentences), even if I set ‘min_length’ very high, for instance “min_length”: 1024. How can I generate more tokens with BLOOM? Is there a limitation around this, or am I entering the parameters incorrectly?

2

This is what my entry looks like:

API_URL = “/static-proxy?url=https%3A%2F%2Fapi-inference.huggingface.co%2Fmodels%2Fbigscience%2Fbloom%3C%2Fa%3E%E2%80%9D%3C%2Fp%3E

HF_TOKEN = ‘api_org_XXXXXXX’

headers = {“Authorization”: f"Bearer {HF_TOKEN}"}

prompt_text = “”“Text:
Some prompt text is here because”“”

json_ = {“inputs”: prompt_text,
“parameters”:
{
“top_p”: 1.0,
“temperature”: .72,
“min_length”: 1024,
“return_full_text”: False
}, “options”:
{
“use_cache”: False,
“wait_for_model”:True
},}
response = requests.post(API_URL, headers=headers, json=json_)
output = response.json()