Qwen3-Neurotic-Experts-8x0.6b-v1

Qwen3-Neurotic-Experts-8x0.6b-v1 is a Mixture of Experts (MoE) made with the following models using LazyMergekit:

• Qwen/Qwen3-0.6B
• suayptalha/Qwen3-0.6B-Code-Expert
• suayptalha/Qwen3-0.6B-Medical-Expert
• suayptalha/Qwen3-0.6B-Math-Expert
• suayptalha/Qwen3-0.6B-Diagnose
• suayptalha/Qwen3-0.6B-Psychological-Support
• suayptalha/Qwen3-0.6B-IF-Expert
• 90dkn0ws/OpenR1-Distill-0.6B

🧩 Configuration

base_model: Qwen/Qwen3-0.6B
dtype: float16
gate_mode: hidden
experts_per_token: 2

experts:
  - source_model: Qwen/Qwen3-0.6B
    positive_prompts:
      - "chat"
      - "conversation"
      - "dialogue"
      - "discuss"
      - "ask questions"
      - "share thoughts"
      - "explore ideas"
      - "personal assistant"
      - "friendly helper"

  - source_model: suayptalha/Qwen3-0.6B-Code-Expert
    positive_prompts:
      - "code"
      - "programming"
      - "python"
      - "javascript"
      - "c++"
      - "debug"
      - "write a function"
      - "implement"
      - "algorithm"
      - "software"

  - source_model: suayptalha/Qwen3-0.6B-Medical-Expert
    positive_prompts:
      - "medical"
      - "medicine"
      - "health"
      - "clinical"
      - "treatment"
      - "pharmacology"
      - "anatomy"
      - "physiology"

  - source_model: suayptalha/Qwen3-0.6B-Math-Expert
    positive_prompts:
      - "math"
      - "mathematics"
      - "algebra"
      - "calculus"
      - "equation"
      - "solve"
      - "proof"
      - "derivation"

  - source_model: suayptalha/Qwen3-0.6B-Diagnose
    positive_prompts:
      - "diagnose"
      - "diagnosis"
      - "symptoms"
      - "differential"
      - "case study"
      - "clinical reasoning"

  - source_model: suayptalha/Qwen3-0.6B-Psychological-Support
    positive_prompts:
      - "mental health"
      - "emotional support"
      - "stress"
      - "anxiety"
      - "depression"
      - "coping"
      - "therapy"
      - "feelings"

  - source_model: suayptalha/Qwen3-0.6B-IF-Expert
    positive_prompts:
      - "follow instructions"
      - "step by step"
      - "task"
      - "do the following"
      - "instruction"
      - "format"
      - "guidelines"

  - source_model: 90dkn0ws/OpenR1-Distill-0.6B
    positive_prompts:
      - "reason"
      - "think carefully"
      - "step by step"
      - "logic"
      - "explain why"
      - "analyze"
      - "chain of thought"

💻 Usage

!pip install -qU transformers bitsandbytes accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "ItsVictorTube/Qwen3-Neurotic-Experts-8x0.6b-v1"

tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)

messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])