Image-Text-to-Text
Transformers
Safetensors
multilingual
minicpmv
feature-extraction
minicpm-v
vision
ocr
multi-image
video
custom_code
conversational
Eval Results
Instructions to use openbmb/MiniCPM-V-4_5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use openbmb/MiniCPM-V-4_5 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="openbmb/MiniCPM-V-4_5", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("openbmb/MiniCPM-V-4_5", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use openbmb/MiniCPM-V-4_5 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "openbmb/MiniCPM-V-4_5" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/MiniCPM-V-4_5", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/openbmb/MiniCPM-V-4_5
- SGLang
How to use openbmb/MiniCPM-V-4_5 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "openbmb/MiniCPM-V-4_5" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/MiniCPM-V-4_5", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "openbmb/MiniCPM-V-4_5" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/MiniCPM-V-4_5", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use openbmb/MiniCPM-V-4_5 with Docker Model Runner:
docker model run hf.co/openbmb/MiniCPM-V-4_5
Commit History
-
Keep the pipeline tag as Auto Detected.
Verified signatureHey team! The generated code snippet for the model creates an issue as suggested [here](https://github.com/huggingface/huggingface.js/issues/1803). The issue is that the pipeline is tagged as `image-text-to-text`, which uses `AutoModelForImageTextToText`. The model does not support the API and only supports `AutoModel` (with remote code). The easiest option to rectify the issue is to keep `pipeline_tag` as Auto-detected, which would in turn produce the following code snippet that works ⬇ ```py pipe = pipeline("feature-extraction", model="openbmb/MiniCPM-V-4_5", trust_remote_code=True) ``` The other option (a bit convoluted) is to write a custom pipeline. The docs of adding a new pipeline is here: https://huggingface.co/docs/transformers/en/add_new_pipeline#adding-a-new-pipeline Let me know what you think is best! 🤗53054da
-
Update: README.md
Verified signature0e2e644
-
Update: README.md
Verified signatured328a17 -
Update README.md
Verified signature9fa194f
-
Update README.md
Verified signaturea3bb84d -
Update: README.md
Verified signaturea626b89 -
Update README.md
Verified signature76bc8b9 -
Update: license
Verified signatureebb1f6c
-
Update: README
Verified signatured7bbb09
-
Update README.md
Verified signaturea18bbd9
-
Update README.md
Verified signaturef5b97f1
-
Update README.md
Verified signature0fe9c69 -
Update README.md
Verified signature0f80664
-
update readme
tc-mbSigned-off-by: tc-mb <caitianchi@modelbest.cn>1fd8118 -
update readme
tc-mbSigned-off-by: tc-mb <caitianchi@modelbest.cn>c87166b -
add cookbook link
tc-mbSigned-off-by: tc-mb <caitianchi@modelbest.cn>54249c4 -
update readme
tc-mbSigned-off-by: tc-mb <caitianchi@modelbest.cn>7fee240
-
Update README.md
Verified signature17353d1 -
Update README.md
Verified signaturec255f57
-
Update README.md
Verified signaturea8dd5db
-
Update README.md
Verified signature37c5181 -
Update README.md
Verified signaturea426d2a -
add multi-image infer usage
wangchongyid272cfd
-
update readme
tc-mbSigned-off-by: tc-mb <caitianchi@modelbest.cn>73cb2e7 -
add stream demo
wangchongyi7584097
-
Update README.md
Verified signature11bfb79 -
Update README.md
Verified signature2bed3a2
-
Update README.md
Verified signature52e6b29 -
Update README.md
Verified signature6d2099c -
Update README.md
Verified signature37edc54 -
Update README.md
Verified signature100c576
-
Initial commit: MiniCPM-V-4_5 model
tc-mb2e3dbac -
Initial commit: MiniCPM-V-4_5 model
tc-mbc13aabe -
Update README.md
Verified signature3acc1cf -
Update README.md
Verified signature61961f9 -
Initial commit: MiniCPM-V-4_5 model
tc-mbea73194 -
Initial commit: MiniCPM-V-4_5 model
tc-mb3bbd798 -
add readme
tc-mbbf6f912 -
Initial commit: MiniCPM-V-4_5 model
tc-mb1dfa9dc
-
initial commit
Verified signaturee8a8761