Instructions to use daryl149/llama-2-7b-chat-hf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use daryl149/llama-2-7b-chat-hf with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="daryl149/llama-2-7b-chat-hf")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("daryl149/llama-2-7b-chat-hf")
model = AutoModelForCausalLM.from_pretrained("daryl149/llama-2-7b-chat-hf")

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use daryl149/llama-2-7b-chat-hf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "daryl149/llama-2-7b-chat-hf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "daryl149/llama-2-7b-chat-hf",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/daryl149/llama-2-7b-chat-hf

SGLang

How to use daryl149/llama-2-7b-chat-hf with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "daryl149/llama-2-7b-chat-hf" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "daryl149/llama-2-7b-chat-hf",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "daryl149/llama-2-7b-chat-hf" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "daryl149/llama-2-7b-chat-hf",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use daryl149/llama-2-7b-chat-hf with Docker Model Runner:
```
docker model run hf.co/daryl149/llama-2-7b-chat-hf
```

What's the difference between this and the official llama version?

by chaochaoli - opened Sep 12, 2023

Discussion

chaochaoli

Sep 12, 2023

thks

spectral9

Nov 9, 2023

•

edited Nov 9, 2023

The difference between llama-2-7b-chat and llama-2-7b-chat-hf is that the latter is in hugging-face-format. The "chat" version of the llama2 is optimized for dialogue use cases.

XueyingJia

Apr 6, 2024

May I ask what is hugging-face-format?

clessvna

Apr 17, 2024

May I ask what is hugging-face-format?

I have the same question

wenkid

Feb 19, 2025

Is it the same model as Meta's?

daryl149

Owner Feb 21, 2025

It is the same model as meta's llama-2-7b-chat-hf
I hosted it myself at the time, because 2 years ago when meta published Llama2, you had to fill in a form and they had to approve your downloads of the weights.
My mirror requires no filling in any forms with meta, you can just immediately download the weights.
That being said, just use llama3, this is really outdated for most purposes.

Back in 2023, meta did not always use huggingface format for their weights:

llama-2-7b-chat (notice the absence of -hf at the end) you could NOT use with the huggingface transformers library. You had to convert it using a custom script.
llama-2-7b-chat-hf you can use with the huggingface transformers library without any extra model weight conversions.

daryl149 changed discussion status to closed Feb 21, 2025

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment