Turkish-LLM-14B-Instruct-GGUF

GGUF quantized versions of Turkish-LLM-14B-Instruct, a 14.7B parameter Turkish language model fine-tuned from Qwen2.5-14B-Instruct.

Available Quantizations

File	Quant	Size	RAM Needed	Best For
Turkish-LLM-14B-Instruct-F16.gguf	F16	28 GB	32-35 GB	Full precision, A100/H100, 2x 24GB GPU
Turkish-LLM-14B-Instruct-Q8_0.gguf	Q8_0	15 GB	18-20 GB	32GB RAM, RTX 3090/4090
Turkish-LLM-14B-Instruct-Q5_K_M.gguf	Q5_K_M	9.8 GB	13-14 GB	16GB+ RAM, M2/M3 Mac
Turkish-LLM-14B-Instruct-Q4_K_M.gguf	Q4_K_M	8.4 GB	11-12 GB	16GB RAM laptop, M1/M2 Mac

Recommended Quantization

• Q4_K_M for most consumer hardware (best size/quality ratio)
• Q5_K_M if you have extra RAM and want slightly better quality
• Q8_0 for minimal quality loss with enough RAM
• F16 for research/evaluation with high-end hardware

Usage

llama.cpp

# Download a quantization
huggingface-cli download ogulcanaydogan/Turkish-LLM-14B-Instruct-GGUF Turkish-LLM-14B-Instruct-Q4_K_M.gguf

# Run inference
./llama-cli -m Turkish-LLM-14B-Instruct-Q4_K_M.gguf \
  -p "<|im_start|>system\nSen yardimci bir Turkce yapay zeka asistanisin.<|im_end|>\n<|im_start|>user\nTurkiye'nin baskenti neresidir?<|im_end|>\n<|im_start|>assistant\n" \
  -n 256 --temp 0.7

Ollama

ollama run hf.co/ogulcanaydogan/Turkish-LLM-14B-Instruct-GGUF:Q4_K_M

LM Studio

Download any GGUF file and load it directly in LM Studio.

Chat Template

This model uses the ChatML format:

<|im_start|>system
Sen yardimci bir Turkce yapay zeka asistanisin.<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant
{assistant_response}<|im_end|>

Model Details

• Base Model: Turkish-LLM-14B-Instruct (Qwen2.5-14B-Instruct + SFT)
• Parameters: 14.7B
• Architecture: Qwen2, 48 layers, 5120 hidden size
• Context Length: 32,768 tokens
• Vocabulary: 152,064 tokens
• Quantized with: llama.cpp

Related

• Turkish-LLM-14B-Instruct - Original model (safetensors)
• Turkish-LLM-7B-Instruct - Lighter 7B variant
• Turkish-LLM-14B-Chat - Live demo

Author

Ogulcan Aydogan | GitHub | LinkedIn