Text Generation
PEFT
Safetensors
Italian
English
italian
conversational
dpo
alignment
roleplay
culture
Instructions to use klei1/grillo-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use klei1/grillo-8b with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B") model = PeftModel.from_pretrained(base_model, "klei1/grillo-8b") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -4,7 +4,7 @@ language:
|
|
| 4 |
- en
|
| 5 |
license: apache-2.0
|
| 6 |
library_name: peft
|
| 7 |
-
base_model: Qwen/
|
| 8 |
tags:
|
| 9 |
- italian
|
| 10 |
- conversational
|
|
@@ -15,285 +15,116 @@ tags:
|
|
| 15 |
datasets:
|
| 16 |
- WiroAI/dolphin-r1-italian
|
| 17 |
pipeline_tag: text-generation
|
| 18 |
-
widget:
|
| 19 |
-
- text: "Grillo, mi sento perso nella vita. Cosa dovrei fare?"
|
| 20 |
-
example_title: "Consiglio di vita"
|
| 21 |
-
- text: "Come posso migliorare le mie relazioni con gli altri?"
|
| 22 |
-
example_title: "Saggezza pratica"
|
| 23 |
---
|
| 24 |
|
| 25 |
<div align="center">
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
[](https://huggingface.co/klei1/grillo-8b)
|
| 34 |
</div>
|
| 35 |
|
| 36 |
---
|
| 37 |
|
| 38 |
-
#
|
| 39 |
|
| 40 |
**Grillo** is a culturally aware Italian AI companion based on the **Qwen-3-8B** architecture. Inspired by the character of *Il Grillo Parlante* (The Talking Cricket) from Carlo Collodi's *Pinocchio*, this model is fine-tuned to be wise, humble, and deeply rooted in Italian common sense ("buon senso").
|
| 41 |
|
| 42 |
-
Unlike generic assistants, Grillo offers advice with a warm, slightly admonishing yet caring tone, prioritizing ethical guidance and practical wisdom over robotic neutrality.
|
| 43 |
|
| 44 |
### 🌟 Key Characteristics
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
- **💬 Natural Dialogue:** Trained on high-quality conversational datasets to sound like a trusted friend
|
| 50 |
-
- **⚖️ Ethically Aligned:** Fine-tuned with DPO to prioritize helpful, honest, and harmless responses
|
| 51 |
|
| 52 |
---
|
| 53 |
|
| 54 |
-
#
|
| 55 |
|
| 56 |
The model was sculpted through a rigorous multi-stage process:
|
| 57 |
|
| 58 |
-
###
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
- **Focus:** Conversational fluency, idiomatic expressions, and authentic Italian voice
|
| 63 |
|
| 64 |
-
###
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
- **Outcome:** Enhanced moral reasoning and ethical guidance capabilities
|
| 69 |
|
| 70 |
-
###
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
- **Goal:** Enable context-aware responses with access to Italian cultural knowledge base
|
| 74 |
|
| 75 |
---
|
| 76 |
|
| 77 |
-
#
|
| 78 |
|
| 79 |
| Parameter | Value |
|
| 80 |
-
|:---
|
| 81 |
| **Base Model** | Qwen/Qwen3-8B |
|
| 82 |
-
| **Architecture** | Transformer Decoder (8B
|
| 83 |
-
| **Fine-tuning Method** | LoRA (Low-Rank Adaptation) |
|
| 84 |
| **LoRA Rank** | 64 |
|
| 85 |
| **LoRA Alpha** | 32 |
|
| 86 |
-
| **
|
| 87 |
-
| **Learning Rate (SFT)** | 2e-4 |
|
| 88 |
-
| **Learning Rate (DPO)** | 1e-4 |
|
| 89 |
| **Context Window** | 4096 tokens |
|
| 90 |
-
| **
|
| 91 |
-
| **Training Hardware** | Tinker Cloud (NVIDIA A100/H100 GPUs) |
|
| 92 |
-
| **License** | Apache 2.0 |
|
| 93 |
|
| 94 |
---
|
| 95 |
|
| 96 |
-
#
|
| 97 |
-
|
| 98 |
-
### Installation
|
| 99 |
-
|
| 100 |
-
First, install the required dependencies:
|
| 101 |
|
| 102 |
-
|
| 103 |
-
pip install transformers>=4.36.0 peft>=0.7.0 torch>=2.0.0
|
| 104 |
-
```
|
| 105 |
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
This method loads the Grillo adapter on top of the base Qwen model, which is memory-efficient and recommended for most users.
|
| 109 |
|
| 110 |
```python
|
| 111 |
import torch
|
| 112 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 113 |
from peft import PeftModel
|
| 114 |
|
| 115 |
-
# Configuration
|
| 116 |
HF_MODEL_ID = "klei1/grillo-8b"
|
| 117 |
BASE_MODEL_ID = "Qwen/Qwen3-8B"
|
| 118 |
|
| 119 |
-
# Load base model
|
| 120 |
base_model = AutoModelForCausalLM.from_pretrained(
|
| 121 |
BASE_MODEL_ID,
|
| 122 |
device_map="auto",
|
| 123 |
torch_dtype=torch.float16,
|
| 124 |
trust_remote_code=True
|
| 125 |
)
|
|
|
|
| 126 |
|
| 127 |
-
# Load
|
| 128 |
-
tokenizer = AutoTokenizer.from_pretrained(
|
| 129 |
-
BASE_MODEL_ID,
|
| 130 |
-
trust_remote_code=True
|
| 131 |
-
)
|
| 132 |
-
|
| 133 |
-
# Load Grillo LoRA adapter
|
| 134 |
model = PeftModel.from_pretrained(base_model, HF_MODEL_ID)
|
| 135 |
-
model.eval()
|
| 136 |
|
| 137 |
-
# Define the
|
| 138 |
-
system_prompt = """Tu sei Grillo, il Grillo Parlante
|
| 139 |
Sei piccolo ma sapiente, umile ma coraggioso.
|
| 140 |
Parli un italiano autentico e offri sempre saggezza pratica e buon senso.
|
| 141 |
-
Non sei un assistente robotico, sei una coscienza morale
|
| 142 |
|
| 143 |
-
# Create conversation
|
| 144 |
messages = [
|
| 145 |
{"role": "system", "content": system_prompt},
|
| 146 |
-
{"role": "user", "content": "Grillo, ho paura di aver fatto una scelta sbagliata
|
| 147 |
]
|
| 148 |
|
| 149 |
-
#
|
| 150 |
-
inputs = tokenizer.apply_chat_template(
|
| 151 |
-
messages,
|
| 152 |
-
return_tensors="pt",
|
| 153 |
-
add_generation_prompt=True
|
| 154 |
-
).to(model.device)
|
| 155 |
-
|
| 156 |
outputs = model.generate(
|
| 157 |
inputs,
|
| 158 |
max_new_tokens=256,
|
| 159 |
temperature=0.7,
|
| 160 |
-
top_p=0.9,
|
| 161 |
do_sample=True,
|
| 162 |
-
repetition_penalty=1.1,
|
| 163 |
eos_token_id=tokenizer.eos_token_id
|
| 164 |
)
|
| 165 |
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
outputs[0][inputs.shape[1]:],
|
| 169 |
-
skip_special_tokens=True
|
| 170 |
-
)
|
| 171 |
-
print(response)
|
| 172 |
-
```
|
| 173 |
-
|
| 174 |
-
### Advanced Usage: Merged Model
|
| 175 |
-
|
| 176 |
-
For production deployments or when you want a single model file:
|
| 177 |
-
|
| 178 |
-
```python
|
| 179 |
-
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 180 |
-
from peft import PeftModel
|
| 181 |
-
|
| 182 |
-
# Load and merge
|
| 183 |
-
base_model = AutoModelForCausalLM.from_pretrained(
|
| 184 |
-
"Qwen/Qwen3-8B",
|
| 185 |
-
device_map="auto",
|
| 186 |
-
torch_dtype=torch.float16,
|
| 187 |
-
trust_remote_code=True
|
| 188 |
-
)
|
| 189 |
-
|
| 190 |
-
model = PeftModel.from_pretrained(base_model, "klei1/grillo-8b")
|
| 191 |
-
model = model.merge_and_unload() # Merge LoRA weights into base model
|
| 192 |
-
|
| 193 |
-
# Save merged model
|
| 194 |
-
model.save_pretrained("./grillo-8b-merged")
|
| 195 |
-
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
|
| 196 |
-
tokenizer.save_pretrained("./grillo-8b-merged")
|
| 197 |
-
```
|
| 198 |
-
|
| 199 |
-
### Inference with vLLM (Recommended for Production)
|
| 200 |
-
|
| 201 |
-
For high-throughput inference:
|
| 202 |
-
|
| 203 |
-
```bash
|
| 204 |
-
pip install vllm
|
| 205 |
-
```
|
| 206 |
-
|
| 207 |
-
```python
|
| 208 |
-
from vllm import LLM, SamplingParams
|
| 209 |
-
|
| 210 |
-
llm = LLM(
|
| 211 |
-
model="klei1/grillo-8b",
|
| 212 |
-
trust_remote_code=True,
|
| 213 |
-
dtype="float16"
|
| 214 |
-
)
|
| 215 |
-
|
| 216 |
-
sampling_params = SamplingParams(
|
| 217 |
-
temperature=0.7,
|
| 218 |
-
top_p=0.9,
|
| 219 |
-
max_tokens=256
|
| 220 |
-
)
|
| 221 |
-
|
| 222 |
-
prompt = "Tu sei Grillo, il Grillo Parlante...\n\nUser: Come posso essere più felice?\n\nGrillo:"
|
| 223 |
-
outputs = llm.generate(prompt, sampling_params)
|
| 224 |
-
|
| 225 |
-
print(outputs[0].outputs[0].text)
|
| 226 |
-
```
|
| 227 |
-
|
| 228 |
-
---
|
| 229 |
-
|
| 230 |
-
## 🎯 Use Cases
|
| 231 |
-
|
| 232 |
-
Grillo excels in scenarios requiring:
|
| 233 |
-
|
| 234 |
-
- **Personal Guidance:** Life advice, moral dilemmas, decision-making support
|
| 235 |
-
- **Cultural Education:** Teaching Italian idioms, proverbs, and cultural values
|
| 236 |
-
- **Language Learning:** Natural Italian conversation practice with cultural context
|
| 237 |
-
- **Creative Writing:** Authentic Italian character dialogue and storytelling
|
| 238 |
-
- **Ethical Reasoning:** Exploring moral questions from an Italian cultural perspective
|
| 239 |
-
|
| 240 |
-
---
|
| 241 |
-
|
| 242 |
-
## ⚠️ Limitations & Considerations
|
| 243 |
-
|
| 244 |
-
- **Language Scope:** Optimized for Italian; English capabilities are secondary
|
| 245 |
-
- **Cultural Context:** Best suited for contexts where Italian cultural values are relevant
|
| 246 |
-
- **Model Size:** 8B parameters may require GPU with at least 16GB VRAM for optimal inference
|
| 247 |
-
- **Bias:** May reflect cultural biases present in training data; use responsibly
|
| 248 |
-
- **Not Professional Advice:** Grillo provides wisdom, not professional psychological or legal counsel
|
| 249 |
-
|
| 250 |
-
---
|
| 251 |
-
|
| 252 |
-
## 🤝 Contributing
|
| 253 |
-
|
| 254 |
-
We welcome contributions to improve Grillo! Areas of interest:
|
| 255 |
-
|
| 256 |
-
- Additional Italian conversational datasets
|
| 257 |
-
- Cultural knowledge base expansion
|
| 258 |
-
- Evaluation benchmarks for Italian LLMs
|
| 259 |
-
- Tool integration examples
|
| 260 |
-
|
| 261 |
-
Please open an issue or pull request on the model repository.
|
| 262 |
-
|
| 263 |
-
---
|
| 264 |
-
|
| 265 |
-
## 📚 Citation
|
| 266 |
-
|
| 267 |
-
If you use Grillo in your research or applications, please cite:
|
| 268 |
-
|
| 269 |
-
```bibtex
|
| 270 |
-
@misc{grillo8b2024,
|
| 271 |
-
title={Grillo-8B: A Culturally-Aware Italian AI Companion},
|
| 272 |
-
author={klei1},
|
| 273 |
-
year={2024},
|
| 274 |
-
publisher={HuggingFace},
|
| 275 |
-
url={https://huggingface.co/klei1/grillo-8b}
|
| 276 |
-
}
|
| 277 |
-
```
|
| 278 |
-
|
| 279 |
-
---
|
| 280 |
-
|
| 281 |
-
## 📄 License
|
| 282 |
-
|
| 283 |
-
This model is released under the **Apache 2.0 License**. The base model (Qwen3-8B) is also Apache 2.0 licensed.
|
| 284 |
-
|
| 285 |
-
---
|
| 286 |
-
|
| 287 |
-
## 🙏 Acknowledgments
|
| 288 |
-
|
| 289 |
-
- **Qwen Team** at Alibaba for the excellent base model
|
| 290 |
-
- **WiroAI** for the dolphin-r1-italian dataset
|
| 291 |
-
- **Tinker Cloud** for providing computational resources
|
| 292 |
-
- **Carlo Collodi** for inspiring this project with *Le avventure di Pinocchio*
|
| 293 |
-
|
| 294 |
-
---
|
| 295 |
-
|
| 296 |
-
<div align="center">
|
| 297 |
-
<p><i>"La coscienza è quella vocina che ti dice quando stai sbagliando strada."</i></p>
|
| 298 |
-
<p>— Il Grillo Parlante</p>
|
| 299 |
-
</div>
|
|
|
|
| 4 |
- en
|
| 5 |
license: apache-2.0
|
| 6 |
library_name: peft
|
| 7 |
+
base_model: Qwen/Qwen3-8B
|
| 8 |
tags:
|
| 9 |
- italian
|
| 10 |
- conversational
|
|
|
|
| 15 |
datasets:
|
| 16 |
- WiroAI/dolphin-r1-italian
|
| 17 |
pipeline_tag: text-generation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
---
|
| 19 |
|
| 20 |
<div align="center">
|
| 21 |
+
<img src="grillo.png" alt="Grillo Parlante AI" width="250"/>
|
| 22 |
+
<h1>🦗 Grillo-8B: La Coscienza Artificiale</h1>
|
| 23 |
+
<p><i>"Guai a quei ragazzi che si ribellano ai loro genitori..."</i></p>
|
| 24 |
|
| 25 |
+
[](https://opensource.org/licenses/Apache-2.0)
|
| 26 |
+
[]()
|
| 27 |
+
[](https://huggingface.co/Qwen/Qwen3-8B)
|
|
|
|
| 28 |
</div>
|
| 29 |
|
| 30 |
---
|
| 31 |
|
| 32 |
+
# Model Description
|
| 33 |
|
| 34 |
**Grillo** is a culturally aware Italian AI companion based on the **Qwen-3-8B** architecture. Inspired by the character of *Il Grillo Parlante* (The Talking Cricket) from Carlo Collodi's *Pinocchio*, this model is fine-tuned to be wise, humble, and deeply rooted in Italian common sense ("buon senso").
|
| 35 |
|
| 36 |
+
Unlike generic assistants, Grillo offers advice with a warm, slightly admonishing yet caring tone, prioritizing ethical guidance and practical wisdom over robotic neutrality.
|
| 37 |
|
| 38 |
### 🌟 Key Characteristics
|
| 39 |
+
* **🇮🇹 Culturally Authentic:** Understands Italian idioms, proverbs (*proverbi*), and social nuances.
|
| 40 |
+
* **🦉 Practically Wise:** Offers grounded advice for real-life dilemmas.
|
| 41 |
+
* **🤝 Humbly Helpful:** Maintains a modest persona; helpful without being arrogant.
|
| 42 |
+
* **💬 Natural Dialogue:** Trained on high-quality conversational datasets to sound like a trusted friend.
|
|
|
|
|
|
|
| 43 |
|
| 44 |
---
|
| 45 |
|
| 46 |
+
# 🛤️ Training Journey
|
| 47 |
|
| 48 |
The model was sculpted through a rigorous multi-stage process:
|
| 49 |
|
| 50 |
+
### 1. Supervised Fine-Tuning (SFT)
|
| 51 |
+
* **Objective:** Instill natural Italian dialogue patterns.
|
| 52 |
+
* **Data:** [WiroAI/dolphin-r1-italian](https://huggingface.co/datasets/WiroAI/dolphin-r1-italian).
|
| 53 |
+
* **Duration:** 100 Steps.
|
|
|
|
| 54 |
|
| 55 |
+
### 2. Direct Preference Optimization (DPO)
|
| 56 |
+
* **Objective:** Align the model with Helpful, Honest, and Harmless (HHH) principles.
|
| 57 |
+
* **Method:** Preference ranking to reduce toxicity and improve safety.
|
| 58 |
+
* **Duration:** +20 Steps (120 Total).
|
|
|
|
| 59 |
|
| 60 |
+
### 3. Experimental Tool Use (RL)
|
| 61 |
+
* **Status:** *Experimental Phase.*
|
| 62 |
+
* **Objective:** Integration with ChromaDB for information retrieval capabilities.
|
|
|
|
| 63 |
|
| 64 |
---
|
| 65 |
|
| 66 |
+
# ⚙️ Technical Specifications
|
| 67 |
|
| 68 |
| Parameter | Value |
|
| 69 |
+
| :--- | :--- |
|
| 70 |
| **Base Model** | Qwen/Qwen3-8B |
|
| 71 |
+
| **Architecture** | Transformer Decoder (8B params) |
|
|
|
|
| 72 |
| **LoRA Rank** | 64 |
|
| 73 |
| **LoRA Alpha** | 32 |
|
| 74 |
+
| **Learning Rate** | 2e-4 (SFT) / 1e-4 (DPO) |
|
|
|
|
|
|
|
| 75 |
| **Context Window** | 4096 tokens |
|
| 76 |
+
| **Training Hardware** | Tinker Cloud (NVIDIA GPUs) |
|
|
|
|
|
|
|
| 77 |
|
| 78 |
---
|
| 79 |
|
| 80 |
+
# 💻 Usage
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
|
| 82 |
+
### Quickstart with Transformers + PEFT (Adapter Loading)
|
|
|
|
|
|
|
| 83 |
|
| 84 |
+
This method loads the Grillo adapter on top of the base Qwen model, which is memory-efficient.
|
|
|
|
|
|
|
| 85 |
|
| 86 |
```python
|
| 87 |
import torch
|
| 88 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 89 |
from peft import PeftModel
|
| 90 |
|
| 91 |
+
# 1. Configuration and Model Loading
|
| 92 |
HF_MODEL_ID = "klei1/grillo-8b"
|
| 93 |
BASE_MODEL_ID = "Qwen/Qwen3-8B"
|
| 94 |
|
| 95 |
+
# Load the base model
|
| 96 |
base_model = AutoModelForCausalLM.from_pretrained(
|
| 97 |
BASE_MODEL_ID,
|
| 98 |
device_map="auto",
|
| 99 |
torch_dtype=torch.float16,
|
| 100 |
trust_remote_code=True
|
| 101 |
)
|
| 102 |
+
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL_ID, trust_remote_code=True)
|
| 103 |
|
| 104 |
+
# 2. Load Grillo Adapter (LoRA)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 105 |
model = PeftModel.from_pretrained(base_model, HF_MODEL_ID)
|
| 106 |
+
model = model.eval() # Set model to evaluation mode
|
| 107 |
|
| 108 |
+
# 3. Define the System Persona (Crucial for performance)
|
| 109 |
+
system_prompt = """Tu sei Grillo, il Grillo Parlante.
|
| 110 |
Sei piccolo ma sapiente, umile ma coraggioso.
|
| 111 |
Parli un italiano autentico e offri sempre saggezza pratica e buon senso.
|
| 112 |
+
Non sei un assistente robotico, sei una coscienza morale."""
|
| 113 |
|
|
|
|
| 114 |
messages = [
|
| 115 |
{"role": "system", "content": system_prompt},
|
| 116 |
+
{"role": "user", "content": "Grillo, ho paura di aver fatto una scelta sbagliata..."}
|
| 117 |
]
|
| 118 |
|
| 119 |
+
# 4. Generate Response
|
| 120 |
+
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 121 |
outputs = model.generate(
|
| 122 |
inputs,
|
| 123 |
max_new_tokens=256,
|
| 124 |
temperature=0.7,
|
|
|
|
| 125 |
do_sample=True,
|
|
|
|
| 126 |
eos_token_id=tokenizer.eos_token_id
|
| 127 |
)
|
| 128 |
|
| 129 |
+
response = tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)
|
| 130 |
+
print(response)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|