Instructions to use meta-llama/Meta-Llama-3-8B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use meta-llama/Meta-Llama-3-8B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="meta-llama/Meta-Llama-3-8B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")
model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use meta-llama/Meta-Llama-3-8B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "meta-llama/Meta-Llama-3-8B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meta-llama/Meta-Llama-3-8B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/meta-llama/Meta-Llama-3-8B-Instruct

SGLang

How to use meta-llama/Meta-Llama-3-8B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "meta-llama/Meta-Llama-3-8B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meta-llama/Meta-Llama-3-8B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "meta-llama/Meta-Llama-3-8B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meta-llama/Meta-Llama-3-8B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use meta-llama/Meta-Llama-3-8B-Instruct with Docker Model Runner:
```
docker model run hf.co/meta-llama/Meta-Llama-3-8B-Instruct
```

Request to Reconsider Rejected Access — Academic Research

#276

by shenyun - opened Jul 23

Discussion

shenyun

Jul 23

Hello Meta Llama team,

My access request for meta-llama/Meta-Llama-3-8B-Instruct
was rejected. I would like to respectfully request a manual
reconsideration or have my request moved back to pending.

Hugging Face username: shenyun

Use case:
I am reproducing the open-source OASIS large-scale social
simulation framework for non-commercial academic research.
The model is required to reproduce the original paper's
LLM-agent experiments and evaluate large-scale agent simulation.

I will comply with the Meta Llama 3 Community License.
The model weights will only be used locally for research and
will not be redistributed, publicly hosted, or used for resale.

Could you please reconsider my request or reset it so that I
can submit corrected information?

Thank you.

zhengyupei

15 days ago

•

edited 15 days ago

Title: Request for Reconsideration of Rejected Access — Academic Research

Hello Meta Llama Team,

My access request for meta-llama/Meta-Llama-3-8B-Instruct was rejected, and I would like to respectfully request a manual reconsideration or have my request moved back to pending so that I can provide more complete information.

I am a master's student conducting non-commercial academic research on dialogue strategy prediction in psychological support conversations. I need this model to reproduce the LLaMA3-8B baseline reported in the EmoDynamiX paper and evaluate it on the public ESConv and AnnoMI datasets.

The model will be used locally for LoRA-based fine-tuning and evaluation only. I will not redistribute or share the model weights, expose the model through a public API, or use it for commercial deployment.

I will comply with the Meta Llama 3 Community License and Acceptable Use Policy.

Hugging Face username:zhengyupei
Affiliation: Jianghan University
Role: Master's student

If my previous request was rejected because the information was incomplete or unclear, I would appreciate the opportunity to correct it and resubmit.

Thank you for your time and consideration.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment