Instructions to use lemon-mint/gemma-ko-7b-instruct-v0.62 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lemon-mint/gemma-ko-7b-instruct-v0.62 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lemon-mint/gemma-ko-7b-instruct-v0.62")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lemon-mint/gemma-ko-7b-instruct-v0.62")
model = AutoModelForCausalLM.from_pretrained("lemon-mint/gemma-ko-7b-instruct-v0.62")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use lemon-mint/gemma-ko-7b-instruct-v0.62 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lemon-mint/gemma-ko-7b-instruct-v0.62"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lemon-mint/gemma-ko-7b-instruct-v0.62",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lemon-mint/gemma-ko-7b-instruct-v0.62

SGLang

How to use lemon-mint/gemma-ko-7b-instruct-v0.62 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lemon-mint/gemma-ko-7b-instruct-v0.62" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lemon-mint/gemma-ko-7b-instruct-v0.62",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lemon-mint/gemma-ko-7b-instruct-v0.62" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lemon-mint/gemma-ko-7b-instruct-v0.62",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lemon-mint/gemma-ko-7b-instruct-v0.62 with Docker Model Runner:
```
docker model run hf.co/lemon-mint/gemma-ko-7b-instruct-v0.62
```

Gemma Ko 7B Instruct v0.62

Eval Loss: 1.2946
Train Loss: 1.1717
lr: 2e-5
optimizer: adamw
lr_scheduler_type: cosine

Model Details

Model Description

The Gemma Ko 7B Instruct v0.62 model is designed for generating human-like text in the Korean language. It can be used for a variety of natural language processing tasks, such as language translation, text summarization, question answering, and conversation generation. This model is particularly well-suited for applications that require high-quality, coherent, and contextually relevant Korean text generation.

Developed by: lemon-mint
Model type: Gemma
Language(s) (NLP): Korean, English
License: gemma-terms-of-use
Finetuned from model: openchat/openchat-3.5-0106-gemma

Limitations and Ethical Considerations

As Gemma Ko 7B has been trained on extensive web data, biases present in the training data may be reflected in the model. Additionally, there is a possibility that it may generate sentences containing errors or incorrect information. Therefore, rather than blindly trusting the model's output, it is necessary to refer to it with caution.

Downloads last month: 27

Safetensors

Model size

9B params

Tensor type

BF16

Model tree for lemon-mint/gemma-ko-7b-instruct-v0.62

Base model

openchat/openchat-3.5-0106-gemma

Finetuned

(2)

this model

lemon-mint
/

gemma-ko-7b-instruct-v0.62

Gemma Ko 7B Instruct v0.62

Model Details

Model Description

Limitations and Ethical Considerations

Model tree for lemon-mint/gemma-ko-7b-instruct-v0.62

Space using lemon-mint/gemma-ko-7b-instruct-v0.62 1