klei1 commited on
Commit
4b5ca2f
·
verified ·
1 Parent(s): 86a375d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +46 -215
README.md CHANGED
@@ -4,7 +4,7 @@ language:
4
  - en
5
  license: apache-2.0
6
  library_name: peft
7
- base_model: Qwen/Qwen2.5-7B-Instruct
8
  tags:
9
  - italian
10
  - conversational
@@ -15,285 +15,116 @@ tags:
15
  datasets:
16
  - WiroAI/dolphin-r1-italian
17
  pipeline_tag: text-generation
18
- widget:
19
- - text: "Grillo, mi sento perso nella vita. Cosa dovrei fare?"
20
- example_title: "Consiglio di vita"
21
- - text: "Come posso migliorare le mie relazioni con gli altri?"
22
- example_title: "Saggezza pratica"
23
  ---
24
 
25
  <div align="center">
26
- <img src="grillo.png" alt="Grillo Parlante AI" width="250"/>
27
- <h1>🦗 Grillo-8B: La Coscienza Artificiale</h1>
28
- <p><i>"Guai a quei ragazzi che si ribellano ai loro genitori..."</i></p>
29
 
30
- [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
31
- [![Language](https://img.shields.io/badge/Language-Italian-green.svg)]()
32
- [![Base Model](https://img.shields.io/badge/Base_Model-Qwen3--8B-yellow.svg)](https://huggingface.co/Qwen/Qwen3-8B)
33
- [![HuggingFace](https://img.shields.io/badge/🤗-HuggingFace-orange.svg)](https://huggingface.co/klei1/grillo-8b)
34
  </div>
35
 
36
  ---
37
 
38
- ## 📖 Model Description
39
 
40
  **Grillo** is a culturally aware Italian AI companion based on the **Qwen-3-8B** architecture. Inspired by the character of *Il Grillo Parlante* (The Talking Cricket) from Carlo Collodi's *Pinocchio*, this model is fine-tuned to be wise, humble, and deeply rooted in Italian common sense ("buon senso").
41
 
42
- Unlike generic assistants, Grillo offers advice with a warm, slightly admonishing yet caring tone, prioritizing ethical guidance and practical wisdom over robotic neutrality. The model embodies the spirit of Italian culture, understanding nuanced expressions, regional idioms, and the importance of family, community, and moral integrity.
43
 
44
  ### 🌟 Key Characteristics
45
-
46
- - **🇮🇹 Culturally Authentic:** Understands Italian idioms, proverbs (*proverbi*), and social nuances
47
- - **🦉 Practically Wise:** Offers grounded advice for real-life dilemmas rooted in common sense
48
- - **🤝 Humbly Helpful:** Maintains a modest persona; helpful without being arrogant
49
- - **💬 Natural Dialogue:** Trained on high-quality conversational datasets to sound like a trusted friend
50
- - **⚖️ Ethically Aligned:** Fine-tuned with DPO to prioritize helpful, honest, and harmless responses
51
 
52
  ---
53
 
54
- ## 🛤️ Training Journey
55
 
56
  The model was sculpted through a rigorous multi-stage process:
57
 
58
- ### Stage 1: Supervised Fine-Tuning (SFT)
59
- - **Objective:** Instill natural Italian dialogue patterns and cultural understanding
60
- - **Dataset:** [WiroAI/dolphin-r1-italian](https://huggingface.co/datasets/WiroAI/dolphin-r1-italian)
61
- - **Duration:** 100 training steps
62
- - **Focus:** Conversational fluency, idiomatic expressions, and authentic Italian voice
63
 
64
- ### Stage 2: Direct Preference Optimization (DPO)
65
- - **Objective:** Align the model with Helpful, Honest, and Harmless (HHH) principles
66
- - **Method:** Preference ranking to reduce toxicity, improve safety, and enhance wisdom
67
- - **Duration:** 20 additional steps (120 total)
68
- - **Outcome:** Enhanced moral reasoning and ethical guidance capabilities
69
 
70
- ### Stage 3: Experimental Tool Use (RL)
71
- - **Status:** *Experimental Phase*
72
- - **Objective:** Integration with ChromaDB for enhanced information retrieval
73
- - **Goal:** Enable context-aware responses with access to Italian cultural knowledge base
74
 
75
  ---
76
 
77
- ## ⚙️ Technical Specifications
78
 
79
  | Parameter | Value |
80
- |:----------|:------|
81
  | **Base Model** | Qwen/Qwen3-8B |
82
- | **Architecture** | Transformer Decoder (8B parameters) |
83
- | **Fine-tuning Method** | LoRA (Low-Rank Adaptation) |
84
  | **LoRA Rank** | 64 |
85
  | **LoRA Alpha** | 32 |
86
- | **Target Modules** | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
87
- | **Learning Rate (SFT)** | 2e-4 |
88
- | **Learning Rate (DPO)** | 1e-4 |
89
  | **Context Window** | 4096 tokens |
90
- | **Precision** | FP16 / BF16 |
91
- | **Training Hardware** | Tinker Cloud (NVIDIA A100/H100 GPUs) |
92
- | **License** | Apache 2.0 |
93
 
94
  ---
95
 
96
- ## 💻 Usage
97
-
98
- ### Installation
99
-
100
- First, install the required dependencies:
101
 
102
- ```bash
103
- pip install transformers>=4.36.0 peft>=0.7.0 torch>=2.0.0
104
- ```
105
 
106
- ### Quickstart with Transformers + PEFT
107
-
108
- This method loads the Grillo adapter on top of the base Qwen model, which is memory-efficient and recommended for most users.
109
 
110
  ```python
111
  import torch
112
  from transformers import AutoModelForCausalLM, AutoTokenizer
113
  from peft import PeftModel
114
 
115
- # Configuration
116
  HF_MODEL_ID = "klei1/grillo-8b"
117
  BASE_MODEL_ID = "Qwen/Qwen3-8B"
118
 
119
- # Load base model
120
  base_model = AutoModelForCausalLM.from_pretrained(
121
  BASE_MODEL_ID,
122
  device_map="auto",
123
  torch_dtype=torch.float16,
124
  trust_remote_code=True
125
  )
 
126
 
127
- # Load tokenizer
128
- tokenizer = AutoTokenizer.from_pretrained(
129
- BASE_MODEL_ID,
130
- trust_remote_code=True
131
- )
132
-
133
- # Load Grillo LoRA adapter
134
  model = PeftModel.from_pretrained(base_model, HF_MODEL_ID)
135
- model.eval()
136
 
137
- # Define the system persona (crucial for optimal performance)
138
- system_prompt = """Tu sei Grillo, il Grillo Parlante di Pinocchio.
139
  Sei piccolo ma sapiente, umile ma coraggioso.
140
  Parli un italiano autentico e offri sempre saggezza pratica e buon senso.
141
- Non sei un assistente robotico, sei una coscienza morale che guida con gentilezza."""
142
 
143
- # Create conversation
144
  messages = [
145
  {"role": "system", "content": system_prompt},
146
- {"role": "user", "content": "Grillo, ho paura di aver fatto una scelta sbagliata nella mia carriera..."}
147
  ]
148
 
149
- # Tokenize and generate
150
- inputs = tokenizer.apply_chat_template(
151
- messages,
152
- return_tensors="pt",
153
- add_generation_prompt=True
154
- ).to(model.device)
155
-
156
  outputs = model.generate(
157
  inputs,
158
  max_new_tokens=256,
159
  temperature=0.7,
160
- top_p=0.9,
161
  do_sample=True,
162
- repetition_penalty=1.1,
163
  eos_token_id=tokenizer.eos_token_id
164
  )
165
 
166
- # Decode response
167
- response = tokenizer.decode(
168
- outputs[0][inputs.shape[1]:],
169
- skip_special_tokens=True
170
- )
171
- print(response)
172
- ```
173
-
174
- ### Advanced Usage: Merged Model
175
-
176
- For production deployments or when you want a single model file:
177
-
178
- ```python
179
- from transformers import AutoModelForCausalLM, AutoTokenizer
180
- from peft import PeftModel
181
-
182
- # Load and merge
183
- base_model = AutoModelForCausalLM.from_pretrained(
184
- "Qwen/Qwen3-8B",
185
- device_map="auto",
186
- torch_dtype=torch.float16,
187
- trust_remote_code=True
188
- )
189
-
190
- model = PeftModel.from_pretrained(base_model, "klei1/grillo-8b")
191
- model = model.merge_and_unload() # Merge LoRA weights into base model
192
-
193
- # Save merged model
194
- model.save_pretrained("./grillo-8b-merged")
195
- tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
196
- tokenizer.save_pretrained("./grillo-8b-merged")
197
- ```
198
-
199
- ### Inference with vLLM (Recommended for Production)
200
-
201
- For high-throughput inference:
202
-
203
- ```bash
204
- pip install vllm
205
- ```
206
-
207
- ```python
208
- from vllm import LLM, SamplingParams
209
-
210
- llm = LLM(
211
- model="klei1/grillo-8b",
212
- trust_remote_code=True,
213
- dtype="float16"
214
- )
215
-
216
- sampling_params = SamplingParams(
217
- temperature=0.7,
218
- top_p=0.9,
219
- max_tokens=256
220
- )
221
-
222
- prompt = "Tu sei Grillo, il Grillo Parlante...\n\nUser: Come posso essere più felice?\n\nGrillo:"
223
- outputs = llm.generate(prompt, sampling_params)
224
-
225
- print(outputs[0].outputs[0].text)
226
- ```
227
-
228
- ---
229
-
230
- ## 🎯 Use Cases
231
-
232
- Grillo excels in scenarios requiring:
233
-
234
- - **Personal Guidance:** Life advice, moral dilemmas, decision-making support
235
- - **Cultural Education:** Teaching Italian idioms, proverbs, and cultural values
236
- - **Language Learning:** Natural Italian conversation practice with cultural context
237
- - **Creative Writing:** Authentic Italian character dialogue and storytelling
238
- - **Ethical Reasoning:** Exploring moral questions from an Italian cultural perspective
239
-
240
- ---
241
-
242
- ## ⚠️ Limitations & Considerations
243
-
244
- - **Language Scope:** Optimized for Italian; English capabilities are secondary
245
- - **Cultural Context:** Best suited for contexts where Italian cultural values are relevant
246
- - **Model Size:** 8B parameters may require GPU with at least 16GB VRAM for optimal inference
247
- - **Bias:** May reflect cultural biases present in training data; use responsibly
248
- - **Not Professional Advice:** Grillo provides wisdom, not professional psychological or legal counsel
249
-
250
- ---
251
-
252
- ## 🤝 Contributing
253
-
254
- We welcome contributions to improve Grillo! Areas of interest:
255
-
256
- - Additional Italian conversational datasets
257
- - Cultural knowledge base expansion
258
- - Evaluation benchmarks for Italian LLMs
259
- - Tool integration examples
260
-
261
- Please open an issue or pull request on the model repository.
262
-
263
- ---
264
-
265
- ## 📚 Citation
266
-
267
- If you use Grillo in your research or applications, please cite:
268
-
269
- ```bibtex
270
- @misc{grillo8b2024,
271
- title={Grillo-8B: A Culturally-Aware Italian AI Companion},
272
- author={klei1},
273
- year={2024},
274
- publisher={HuggingFace},
275
- url={https://huggingface.co/klei1/grillo-8b}
276
- }
277
- ```
278
-
279
- ---
280
-
281
- ## 📄 License
282
-
283
- This model is released under the **Apache 2.0 License**. The base model (Qwen3-8B) is also Apache 2.0 licensed.
284
-
285
- ---
286
-
287
- ## 🙏 Acknowledgments
288
-
289
- - **Qwen Team** at Alibaba for the excellent base model
290
- - **WiroAI** for the dolphin-r1-italian dataset
291
- - **Tinker Cloud** for providing computational resources
292
- - **Carlo Collodi** for inspiring this project with *Le avventure di Pinocchio*
293
-
294
- ---
295
-
296
- <div align="center">
297
- <p><i>"La coscienza è quella vocina che ti dice quando stai sbagliando strada."</i></p>
298
- <p>— Il Grillo Parlante</p>
299
- </div>
 
4
  - en
5
  license: apache-2.0
6
  library_name: peft
7
+ base_model: Qwen/Qwen3-8B
8
  tags:
9
  - italian
10
  - conversational
 
15
  datasets:
16
  - WiroAI/dolphin-r1-italian
17
  pipeline_tag: text-generation
 
 
 
 
 
18
  ---
19
 
20
  <div align="center">
21
+   <img src="grillo.png" alt="Grillo Parlante AI" width="250"/>
22
+   <h1>🦗 Grillo-8B: La Coscienza Artificiale</h1>
23
+   <p><i>"Guai a quei ragazzi che si ribellano ai loro genitori..."</i></p>
24
 
25
+   [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
26
+   [![Language](https://img.shields.io/badge/Language-Italian-green.svg)]()
27
+   [![Base Model](https://img.shields.io/badge/Base_Model-Qwen3--8B-yellow.svg)](https://huggingface.co/Qwen/Qwen3-8B)
 
28
  </div>
29
 
30
  ---
31
 
32
+ # Model Description
33
 
34
  **Grillo** is a culturally aware Italian AI companion based on the **Qwen-3-8B** architecture. Inspired by the character of *Il Grillo Parlante* (The Talking Cricket) from Carlo Collodi's *Pinocchio*, this model is fine-tuned to be wise, humble, and deeply rooted in Italian common sense ("buon senso").
35
 
36
+ Unlike generic assistants, Grillo offers advice with a warm, slightly admonishing yet caring tone, prioritizing ethical guidance and practical wisdom over robotic neutrality.
37
 
38
  ### 🌟 Key Characteristics
39
+ * **🇮🇹 Culturally Authentic:** Understands Italian idioms, proverbs (*proverbi*), and social nuances.
40
+ * **🦉 Practically Wise:** Offers grounded advice for real-life dilemmas.
41
+ * **🤝 Humbly Helpful:** Maintains a modest persona; helpful without being arrogant.
42
+ * **💬 Natural Dialogue:** Trained on high-quality conversational datasets to sound like a trusted friend.
 
 
43
 
44
  ---
45
 
46
+ # 🛤️ Training Journey
47
 
48
  The model was sculpted through a rigorous multi-stage process:
49
 
50
+ ### 1. Supervised Fine-Tuning (SFT)
51
+ * **Objective:** Instill natural Italian dialogue patterns.
52
+ * **Data:** [WiroAI/dolphin-r1-italian](https://huggingface.co/datasets/WiroAI/dolphin-r1-italian).
53
+ * **Duration:** 100 Steps.
 
54
 
55
+ ### 2. Direct Preference Optimization (DPO)
56
+ * **Objective:** Align the model with Helpful, Honest, and Harmless (HHH) principles.
57
+ * **Method:** Preference ranking to reduce toxicity and improve safety.
58
+ * **Duration:** +20 Steps (120 Total).
 
59
 
60
+ ### 3. Experimental Tool Use (RL)
61
+ * **Status:** *Experimental Phase.*
62
+ * **Objective:** Integration with ChromaDB for information retrieval capabilities.
 
63
 
64
  ---
65
 
66
+ # ⚙️ Technical Specifications
67
 
68
  | Parameter | Value |
69
+ | :--- | :--- |
70
  | **Base Model** | Qwen/Qwen3-8B |
71
+ | **Architecture** | Transformer Decoder (8B params) |
 
72
  | **LoRA Rank** | 64 |
73
  | **LoRA Alpha** | 32 |
74
+ | **Learning Rate** | 2e-4 (SFT) / 1e-4 (DPO) |
 
 
75
  | **Context Window** | 4096 tokens |
76
+ | **Training Hardware** | Tinker Cloud (NVIDIA GPUs) |
 
 
77
 
78
  ---
79
 
80
+ # 💻 Usage
 
 
 
 
81
 
82
+ ### Quickstart with Transformers + PEFT (Adapter Loading)
 
 
83
 
84
+ This method loads the Grillo adapter on top of the base Qwen model, which is memory-efficient.
 
 
85
 
86
  ```python
87
  import torch
88
  from transformers import AutoModelForCausalLM, AutoTokenizer
89
  from peft import PeftModel
90
 
91
+ # 1. Configuration and Model Loading
92
  HF_MODEL_ID = "klei1/grillo-8b"
93
  BASE_MODEL_ID = "Qwen/Qwen3-8B"
94
 
95
+ # Load the base model
96
  base_model = AutoModelForCausalLM.from_pretrained(
97
  BASE_MODEL_ID,
98
  device_map="auto",
99
  torch_dtype=torch.float16,
100
  trust_remote_code=True
101
  )
102
+ tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL_ID, trust_remote_code=True)
103
 
104
+ # 2. Load Grillo Adapter (LoRA)
 
 
 
 
 
 
105
  model = PeftModel.from_pretrained(base_model, HF_MODEL_ID)
106
+ model = model.eval() # Set model to evaluation mode
107
 
108
+ # 3. Define the System Persona (Crucial for performance)
109
+ system_prompt = """Tu sei Grillo, il Grillo Parlante.
110
  Sei piccolo ma sapiente, umile ma coraggioso.
111
  Parli un italiano autentico e offri sempre saggezza pratica e buon senso.
112
+ Non sei un assistente robotico, sei una coscienza morale."""
113
 
 
114
  messages = [
115
  {"role": "system", "content": system_prompt},
116
+ {"role": "user", "content": "Grillo, ho paura di aver fatto una scelta sbagliata..."}
117
  ]
118
 
119
+ # 4. Generate Response
120
+ inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
 
 
 
 
 
121
  outputs = model.generate(
122
  inputs,
123
  max_new_tokens=256,
124
  temperature=0.7,
 
125
  do_sample=True,
 
126
  eos_token_id=tokenizer.eos_token_id
127
  )
128
 
129
+ response = tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)
130
+ print(response)