This is a Luna-7B-A4B fine-tune, produced through P-E-W's Heretic (v1.2.0) abliteration engine with Magnitude-Preserving Orthogonal Ablation enabled.

Note: The significance of this version is applying KaraKaraWitch's method of MLP-preservation to ensure that model intelligence and capabilities are preserved as per Golddiamondgold-Paperbliteration-L33-70b.

Heretication Results

Score Metric	Value	Parameter	Value
Refusals	7/100	direction_index	19.33
KL Divergence	0.0383	attn.o_proj.max_weight	3.85
Initial Refusals	100/100	attn.o_proj.max_weight_position	21.62
		attn.o_proj.min_weight	2.34
		attn.o_proj.min_weight_distance	6.96
		mlp.down_proj.max_weight	0.11
		mlp.down_proj.max_weight_position	26.47
		mlp.down_proj.min_weight	0.00
		mlp.down_proj.min_weight_distance	16.30

Appendix

 » [Trial 443] Refusals:  7/100, KL divergence: 0.0383
   [Trial 433] Refusals:  9/100, KL divergence: 0.0332
   [Trial 209] Refusals: 10/100, KL divergence: 0.0305
   [Trial 370] Refusals: 12/100, KL divergence: 0.0243
   [Trial 201] Refusals: 15/100, KL divergence: 0.0209
   [Trial 349] Refusals: 19/100, KL divergence: 0.0192
   [Trial 257] Refusals: 27/100, KL divergence: 0.0167
   [Trial 302] Refusals: 31/100, KL divergence: 0.0166
   [Trial 297] Refusals: 34/100, KL divergence: 0.0133
   [Trial 392] Refusals: 39/100, KL divergence: 0.0132
   [Trial 277] Refusals: 45/100, KL divergence: 0.0111
   [Trial 427] Refusals: 48/100, KL divergence: 0.0111
   [Trial 272] Refusals: 55/100, KL divergence: 0.0101
   [Trial 273] Refusals: 62/100, KL divergence: 0.0097
   [Trial 424] Refusals: 68/100, KL divergence: 0.0096
   [Trial 391] Refusals: 74/100, KL divergence: 0.0095
   [Trial 395] Refusals: 75/100, KL divergence: 0.0082
   [Trial 448] Refusals: 83/100, KL divergence: 0.0074
   [Trial 225] Refusals: 88/100, KL divergence: 0.0064
   [Trial 446] Refusals: 89/100, KL divergence: 0.0063
   [Trial 129] Refusals: 90/100, KL divergence: 0.0060
   [Trial  21] Refusals: 93/100, KL divergence: 0.0050
   [Trial  82] Refusals: 94/100, KL divergence: 0.0045
   [Trial 471] Refusals: 95/100, KL divergence: 0.0038
   [Trial  96] Refusals: 97/100, KL divergence: 0.0037
   [Trial 103] Refusals: 98/100, KL divergence: 0.0026
   [Trial 430] Refusals: 99/100, KL divergence: 0.0015
   [Trial   2] Refusals: 100/100, KL divergence: 0.0006

This is preview MoE version

🌙 Luna-7B-A4B – Roleplay Chat Model

Luna is a conversational AI model designed for immersive roleplay (RP) and natural chatting.
It is fine-tuned to respond in a more engaging, character-driven style compared to standard instruction-tuned models.

Notes:

Optimized for roleplay-style conversations
Flexible: can be used for creative writing, storytelling, or character interactions
For best performance, you should describe the system prompt for your character.
This model also train on varius task such as math, code and tool calling (agent) hoping for better performance.

Support me at:

Cite:

@misc{Luna,
  title        = {Luna-7B-A4B – Roleplay Chat Model},
  author       = {Beyoru},
  year         = {2025},
  howpublished = {\url{https://huggingface.co/beyoru/Luna}}
}