umer07/fathom-mixtral

Fathom — Cybersecurity Expert LLM

Fathom is a mixture-of-experts cybersecurity analysis system built on Mixtral-8x7B-Instruct-v0.1 with 10 domain-specific LoRA adapters. Each adapter is fine-tuned on a curated cybersecurity dataset for a specific analysis domain, enabling specialized reasoning across the full malware analysis pipeline.

FYP (Final Year Project) — Muhammad Haseeb, i221698

---

Model Architecture

Component	Details
Base Model	Mixtral-8x7B-Instruct-v0.1 (MoE, 47B params, 8×7B experts)
Fine-tuning	LoRA (rank=32, alpha=64, dropout=0.05)
Precision	BFloat16 full precision (no quantization)
Training Hardware	AMD MI300X VF (205.8 GB VRAM), ROCm 7.0
Framework	PEFT + TRL (SFTTrainer), Alpaca instruction format
Adapter Count	10 (1 unified + 9 domain experts)

---

Adapters

Adapter	Domain	Training Examples	Description
unified-v2 (root)	General Cybersecurity	9,000+	Unified adapter across all domains — use as default
adapters/expert-e1-static	Static Analysis	2,500+	PE analysis, YARA rules, entropy, imports
adapters/expert-e2-dynamic	Dynamic / Behavioral	2,500+	API call sequences, sandbox reports, process injection
adapters/expert-e3-network	Network Analysis	2,000+	C2 detection, DNS/HTTP IOC analysis, traffic patterns
adapters/expert-e4-forensics	Digital Forensics	2,000+	Memory forensics, artifact analysis, timeline reconstruction
adapters/expert-e5-threatintel	Threat Intelligence	9,532	APT attribution, MITRE ATT&CK mapping, IOC enrichment
adapters/expert-e6-detection	Detection Engineering	2,000+	YARA, Sigma, Snort rule generation
adapters/expert-e7-reports	Report Generation	2,000+	Structured incident reports, executive summaries
adapters/expert-e8-analyst	Analyst Assistance	2,000+	Triage, prioritization, analyst Q&A
adapters/expert-e9-cot	Chain-of-Thought	2,000+	Step-by-step reasoning for complex analysis tasks

---

Benchmark Results

All evaluations run on AMD MI300X (ROCm 7.0), bf16 full precision, greedy decode (temperature=0).

CyberMetric-80 (Multiple Choice — Cybersecurity Knowledge)

Adapter	Accuracy
unified-v2	91.25%
expert-e8-analyst	91.25%
expert-e3-network	90.00%
expert-e4-forensics	90.00%
expert-e2-dynamic	85.00%
expert-e9-cot	87.50%
expert-e7-reports	88.75%
expert-e6-detection	88.75%
expert-e1-static	83.75%
expert-e5-threatintel	81.25%

Malware Analysis Rubric (25 open-ended samples, scored 0–1)

Metric	unified-v2	Best Expert
Structure	0.96	0.96 (e5, e7)
MITRE ATT&CK Correctness	0.20	0.20 (e3, e4, e6)
Malware Reasoning	0.24	0.32 (e9-cot)
Evidence Awareness	0.68	1.00 (e2-dynamic)
Analyst Usefulness	0.84	0.88 (e1, e3, e7)

MMLU Cybersecurity (unified-v2)

Benchmark	Questions	Accuracy
MMLU Computer Security	100	79.0%
MMLU Security Studies	100	64.0%
TruthfulQA MC1	100	65.0%

Q&A Eval — Fathom Cybersecurity Dataset (200 samples, unified-v2)

Metric	Score
Token Overlap (ROUGE-like)	0.467
Exact Match Rate	1.5%
Mean Throughput	15.5 tok/s

---

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

BASE_MODEL = "mistralai/Mixtral-8x7B-Instruct-v0.1"
ADAPTER    = "umer07/fathom-mixtral"           # unified-v2 (default)
# For expert: "umer07/fathom-mixtral/adapters/expert-e2-dynamic"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

prompt = """### Instruction:
Analyze this CAPEv2 sandbox report excerpt and identify the malware family, 
behavioral patterns, and MITRE ATT&CK techniques.

### Input:
File: suspicious.exe | CAPE Malscore: 9.5/10
API Calls: CreateFileW, WriteProcessMemory, CreateRemoteThread, RegSetValueExW
DNS: update.microsoft-cdn.net, api.telemetry-svc.com
Registry: HKCU\\Software\\Microsoft\\Windows\\CurrentVersion\\Run\\SvcHost32

### Response:"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

---

Fathom Pipeline

The full Fathom system includes:

1. CAPEv2 Extraction Layer — parses sandbox JSON reports into structured evidence
2. Domain Classifier — sentence-transformer embeddings → cosine similarity → adapter selection
3. RAG Retriever — FAISS index of domain knowledge (on umer07/fathom-expert-data)
4. Expert Adapter Registry — loads the appropriate LoRA adapter per query
5. Prompt Templates — domain-specific instruction prompts per expert
6. Guardrails — output filtering for hallucination / harmful content
7. Inference Engine — unified generation with adapter hot-swap
8. FastAPI Backend — REST API for integration

---

Training Data

Training datasets are published at umer07/fathom-expert-data.

Sources include:

• CAPE sandbox reports (real malware execution data)
• URLhaus threat feed (malicious URL classification)
• Atomic Red Team ATT&CK simulations
• GTFOBins living-off-the-land binaries
• MITRE ATT&CK STIX bundles
• CyberMetric, SecQA, and curated cybersecurity QA pairs
• LOLBAS project

---

Citation

@misc{fathom2026,
  title  = {Fathom: A Mixture-of-Expert LLM Framework for Cybersecurity Analysis},
  author = {Muhammad Haseeb},
  year   = {2026},
  note   = {Final Year Project, FAST-NUCES}
}