Can't run Janus with HuggingFaceEndpoint

Intermediate
1

I’m trying to integrate huggingface endpoint with LangChain project using HuggingFaceEndPoint proviede by LangChain.

llm = HuggingFaceEndpoint(
    repo_id="deepseek-ai/Janus-Pro-7B",
    task="image-text-to-text",
    max_new_tokens=512,
    do_sample=False,
    repetition_penalty=1.03
)

But it returns with 500 Server Error, and unknown variant ‘any-to-any’. I’m using task like ‘image-text-to-text’ and ‘image-to-text’ and they have same return value.

image
image1943×190 15 KB

Not only Janus-Pro-7B can’t run, other VLM like Qwen2.5-VL-7B-Instruct and glm-4v-9b can’t run. I suspect this HuggingFaceEndpoint could not run VLM.

1 Like
2

Oh…

This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The HF Inference API does not support any-to-any models for transformers library.

3

Thanks! What VLM can I use with HF inference API?

1 Like
4

This?

5

This? Qwen/Qwen2-VL-7B-Instruct

6

Sorry!This model return thishuggingface_hub.errors.HfHubHTTPError: 503 Server Error: Service Temporarily Unavailable for url: /static-proxy?url=https%3A%2F%2Frouter.huggingface.co%2Fhf-inference%2Fmodels%2FQwen%2FQwen2.5-VL-7B-Instruct%2Fv1%2Fchat%2Fcompletions%3C%2Fcode%3E%3Cbr%3E And I can exclude network reason because I can run microsoft/Phi-3-mini-4k-instruct

1 Like