ThoxLLM-327M-v2

From-Scratch THOX Language Model — a THOX language model trained from scratch: a 327M-parameter decoder with a standard SwiGLU feed-forward design, built for clean GGUF export

Safetensors weights

Find your model: THOX Model Factory · Collection: Research & From-Scratch Models

Your AI. Your Data. Your Rules.™

Summary

ThoxLLM-327M-v2 is a THOX language model trained from scratch: a 327M-parameter decoder with a standard SwiGLU feed-forward design, built for clean GGUF export. It is fully THOX-owned. For llama.cpp and Ollama use, the GGUF builds live in Thox-ai/ThoxLLM-327M-v2-GGUF.

Your AI. Your Data. Your Rules.™

Highlights

• ThoxLLM-327M-v2 is a THOX language model trained from scratch: a 327M-parameter decoder with a standard SwiGLU feed-forward design, built for clean GGUF export.
• This checkpoint uses the custom THOX thoxllm decoder; its config.json carries no architectures key, so vanilla AutoModel loading may not work.

Specifications

Base model	See card body
Parameters	327M
Formats	Safetensors weights
License	Apache-2.0. Trained from scratch by THOX.ai LLC; the model and additions in this repo are covered under Apache-2.0.
This repository contains	Full weights and tokenizer in this repo

Files

File	Size	Purpose
model.safetensors	648,339,368 B	Model weights (safetensors)
tokenizer.json	3,557,680 B	Tokenizer files
loss-curve.svg	6,185 B	Repository file
.gitattributes	1,519 B	Repository file
eval-curve.tsv	668 B	Repository file
config.json	394 B	Model configuration
tokenizer_config.json	286 B	Tokenizer files

Architecture

Field	Value
Architecture	Custom THOX decoder (standard SwiGLU)
Layers	20
Hidden size	1024
Attention heads	16
KV heads	8 (GQA)
FFN / intermediate	3413
Vocab	50,304
Max context	2048
Tied embeddings	yes
dtype	bfloat16

Run it

This checkpoint uses the custom THOX thoxllm decoder; its config.json carries no architectures key, so vanilla AutoModel loading may not work. Load it with the THOX thoxllm package, or use the GGUF build:

hf download Thox-ai/ThoxLLM-327M-v2-GGUF --include "*.gguf" --local-dir ./ThoxLLM-327M-v2-GGUF
ollama run hf.co/Thox-ai/ThoxLLM-327M-v2-GGUF:Q4_K_M

Platforms

Platform / device	Run path
Hugging Face	Download the builds listed under Files from this repo (see Run it)

Intended use

• On-device and edge text generation within the THOX stack.
• Not a safety-aligned public assistant unless deployed behind THOX guardrails.
• Out of scope: medical, legal, financial, safety-critical, or other high-stakes decisions without human review; unattended control of physical devices without a confirmation step.

Training data and procedure

Trained by THOX from scratch as part of the THOX small-model research line. The corpus and procedure are as described in this card at release; no external base model was used.

Evaluation

No THOX GATE49 result is published for this model.

Limitations

• A 327M base model: completion-style behaviour, limited reasoning and factual reliability; verify important answers.
• Behaviour and bias reflect the training data, which is mostly English; quality in other languages is lower.
• Custom architecture: standard loaders may not load the safetensors checkpoint directly (see Run it).

Provenance

Published by THOX.ai as Thox-ai/ThoxLLM-327M-v2. THOX from-scratch research line; no upstream base model. This card was conformed to THOX Model Card Standard v1.0 on 2026-10-31 (frontmatter, tags, header, files, and section structure); the technical content is as published at release.

License

Apache-2.0. Trained from scratch by THOX.ai LLC; the model and additions in this repo are covered under Apache-2.0.

THOX.ai — Your AI. Your Data. Your Rules.™

Citation

@misc{thoxai2026thoxllm327mv2,
      title={ThoxLLM-327M-v2},
      author={THOX.ai},
      year={2026},
      url={https://huggingface.co/Thox-ai/ThoxLLM-327M-v2}
}