SecurityGPT 14B

SecurityGPT is a 14-billion parameter code generation model fine-tuned for security-focused development tasks. Built on Qwen2.5-Coder-14B-Instruct, it specializes in generating secure, production-ready code with emphasis on best practices for web applications, API development, and cybersecurity.

Model Description

Developed by: [email protected]
Model type: Causal Language Model (Decoder-only Transformer)
Language(s): English
Base model: Qwen/Qwen2.5-Coder-14B-Instruct
License: Apache 2.0 (same as base model)
Finetuned from: Qwen2.5-Coder-14B-Instruct
Context length: 32,768 tokens
Parameters: 14 billion

Model Architecture

Architecture: Qwen2ForCausalLM
- Hidden size: 5,120
- Num layers: 48
- Attention heads: 40
- KV heads: 8 (GQA)
- Intermediate size: 13,824
- Vocab size: 152,064
- RoPE theta: 1,000,000
- Activation: SiLU

Key Features

✅ Security-First Design

Secure password hashing (argon2, NEVER bcrypt)
SQL injection prevention
XSS protection patterns
Input validation & sanitization
Proper authentication flows

✅ Best Practice Enforcement

RESTful API design (/api/v1/ versioning)
Modern dependency management (Poetry for Python)
Production-ready error handling
Comprehensive audit logging

✅ Technology Stack Coverage

Backend: Python, FastAPI, Flask, SQLAlchemy
Frontend: React, TypeScript, Tailwind CSS
Databases: PostgreSQL, Redis, OpenSearch
DevOps: Docker, FreeBSD, GitLab CI/CD

Intended Use

Primary Use Cases

Secure API Development - Generate FastAPI/Flask endpoints with proper authentication, validation, and error handling
Web Application Development - Create React/TypeScript components following modern patterns
Security Code Review - Identify and fix security vulnerabilities in existing code
Infrastructure as Code - Generate secure deployment configurations
DevOps Automation - Create CI/CD pipelines and automation scripts

Out-of-Scope Use

⚠️ This model is NOT intended for:

Malicious code generation or exploit development
Production security auditing (use professional security tools)
Medical, legal, or financial advice
Real-time critical systems without human review

Training Details

Training Method

QLoRA (Quantized Low-Rank Adaptation) using Unsloth for optimization.

LoRA Configuration:

Rank (r): 128
Alpha: 256
Dropout: 0 (Unsloth optimized)
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Quantization: 4-bit (QLoRA)

Training Hyperparameters:

Batch size: 8 per device
Gradient accumulation: 4 steps (effective batch = 32)
Learning rate: 1e-4
Epochs: 5
Max sequence length: 2,048 tokens
Optimizer: AdamW 8-bit
LR scheduler: Cosine
Weight decay: 0.01
Precision: BF16

Training Data

The model was fine-tuned on 16,000 instruction-output pairs focused on:

Secure coding patterns and practices
Web application development (FastAPI, React)
Database operations and security
Authentication and authorization
API design and implementation
DevOps and infrastructure configuration

Data composition:

Security-focused coding examples
Real-world application patterns
Best practice demonstrations
Common vulnerability mitigations

Training Loss

Final training loss: 0.026

Usage

Quick Start with Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model_name = "pki/securitygpt-14b"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

# Format prompt with Qwen chat template
messages = [
    {"role": "system", "content": "You are a helpful AI coding assistant specialized in secure software development."},
    {"role": "user", "content": "Create a FastAPI endpoint for user signup with email and password validation."}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

# Generate
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
    temperature=0.4,
    top_p=0.9,
    do_sample=True
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Using with Ollama (Recommended for Deployment)

Step 1: Convert to GGUF (if not already converted)

# Convert merged model to GGUF
python llama.cpp/convert_hf_to_gguf.py merged_model/ \
  --outfile securitygpt-14b-f16.gguf --outtype f16

# Quantize for deployment (Q8 recommended)
llama.cpp/llama-quantize \
  securitygpt-14b-f16.gguf \
  securitygpt-14b-q8.gguf Q8_0

Step 2: Create Modelfile

FROM ./securitygpt-14b-q8.gguf

PARAMETER temperature 0.5
PARAMETER top_p 0.9
PARAMETER num_ctx 32768
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"

TEMPLATE """<|im_start|>system
You are a helpful AI coding assistant specialized in secure software development.<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

SYSTEM """You are SecurityGPT, a specialized AI assistant for secure software development. You follow security best practices including: argon2 password hashing, input validation, SQL injection prevention, XSS protection, proper authentication, and comprehensive error handling."""

Step 3: Deploy with Ollama

ollama create securitygpt:14b -f Modelfile
ollama run securitygpt:14b

Example Prompts

1. Secure Authentication Endpoint

Create a FastAPI endpoint for user login with JWT token generation.
Use argon2 for password hashing and include proper error handling.

2. React Component with Security

Create a React login form component with email validation,
password strength checking, and CSRF protection.

3. Database Security

Write a SQLAlchemy model for user authentication with
secure password storage and audit logging.

4. API Security Review

Review this API endpoint for security vulnerabilities:
[paste code]

Performance & Benchmarks

Response Quality

Code correctness: High (generates syntactically correct code)
Security adherence: Excellent (consistently applies security best practices)
Best practice compliance: Excellent (follows modern development patterns)

Limitations & Biases

Known Limitations

Domain Specificity
- Optimized for web development (FastAPI, React)
- May be less effective for other domains (embedded systems, game development)
Training Data Constraints
- Trained on patterns up to knowledge cutoff
- May not reflect latest framework versions
- Limited to English language code and documentation
Context Length
- Maximum 32,768 tokens (though effectively handles ~16-24K for quality)
- Very large codebases may need chunking
Security Limitations
- Code generation should ALWAYS be reviewed by humans
- Not a replacement for professional security audits
- May not catch all edge cases or vulnerabilities

Potential Biases

Technology stack bias: Strong preference for specific tech stack (FastAPI, React, PostgreSQL)
Pattern repetition: May favor certain code patterns from training data
Verbosity: Sometimes generates more comprehensive solutions than requested

Mitigation Strategies

✅ Always review generated code before production use ✅ Run security scanners on generated code ✅ Test thoroughly including edge cases ✅ Use alongside professional security tools ✅ Keep dependencies updated as model may reference older versions

Ethical Considerations

Responsible Use

This model should be used responsibly:

✅ DO: Use for learning, prototyping, and accelerating development
✅ DO: Review and test all generated code
✅ DO: Follow applicable security standards and regulations
⚠️ DON'T: Use for malicious purposes or exploit development
⚠️ DON'T: Deploy generated code without human review
⚠️ DON'T: Rely solely on AI for security-critical systems

Environmental Impact

Inference efficiency: QLoRA and quantization reduce deployment costs
Optimization: Unsloth reduces training time and energy consumption

Citation

If you use SecurityGPT in your research or projects, please cite:

@misc{securitygpt2026,
  title={SecurityGPT: A Security-Focused Code Generation Model},
  author={[email protected]},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/pki/securitygpt-14b}},
  note={Fine-tuned from Qwen2.5-Coder-14B-Instruct}
}

Base model citation:

@article{qwen2.5,
  title={Qwen2.5-Coder Technical Report},
  author={Qwen Team},
  journal={arXiv preprint},
  year={2024}
}

Model Card Contact

For questions, issues, or collaboration:

Issues: Open an issue on the model repository
Discussions: Use Hugging Face discussions tab
Email: Contact through Hugging Face profile

Changelog

v1.0.0 (2025-12)

Initial release
Fine-tuned on 16,000 security-focused examples
Supports 32K context window
Optimized for FastAPI, React, and security best practices

Acknowledgments

Base model: Qwen Team for Qwen2.5-Coder-14B-Instruct
Training framework: Unsloth AI for optimization
Quantization: llama.cpp for GGUF conversion
Deployment: Ollama for inference serving

License

This model is released under the Apache 2.0 License, same as the base Qwen2.5-Coder model.


Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Disclaimer: This model is provided as-is for research and development purposes. Always review and test generated code before production deployment. The authors are not responsible for any damages resulting from the use of this model.

Downloads last month: 20

Model tree for pki/securitygpt-14b

Base model

Qwen/Qwen2.5-14B

Finetuned

Qwen/Qwen2.5-Coder-14B

Finetuned

Qwen/Qwen2.5-Coder-14B-Instruct

Finetuned

(57)

this model