Instructions to use Vortex5/Lunar-Abyss-12B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Vortex5/Lunar-Abyss-12B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Vortex5/Lunar-Abyss-12B")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Vortex5/Lunar-Abyss-12B") model = AutoModelForCausalLM.from_pretrained("Vortex5/Lunar-Abyss-12B") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Vortex5/Lunar-Abyss-12B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Vortex5/Lunar-Abyss-12B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Vortex5/Lunar-Abyss-12B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Vortex5/Lunar-Abyss-12B
- SGLang
How to use Vortex5/Lunar-Abyss-12B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Vortex5/Lunar-Abyss-12B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Vortex5/Lunar-Abyss-12B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Vortex5/Lunar-Abyss-12B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Vortex5/Lunar-Abyss-12B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Vortex5/Lunar-Abyss-12B with Docker Model Runner:
docker model run hf.co/Vortex5/Lunar-Abyss-12B
๐ Lunar-Abyss-12B
Born where moonlight touches the deep โ thought meets desire, and reason dreams.
๐ Overview
Lunar-Abyss-12B was made to combine the coherency and stability of LunaMaid-12B with the evocative prose and edgy flair of Abyssal-Seraph-12B.
โ๏ธ Merge Method โ DELLA
๐งฉ Base: Vortex5/MegaMoon-Karcher-12B
๐ Inputs: Vortex5/LunaMaid-12B + Vortex5/Abyssal-Seraph-12B
Configuration
models:
- model: Vortex5/LunaMaid-12B
parameters:
weight:
- filter: self_attn
value: [0.35, 0.4, 0.6, 0.8, 1.0, 0.9, 0.6, 0.3]
- filter: mlp
value: [0.20, 0.25, 0.35, 0.45, 0.45, 0.40, 0.30, 0.20]
- value: [0.25, 0.3, 0.35, 0.4, 0.4, 0.35, 0.3, 0.25]
density: 0.55
epsilon: 0.3
- model: Vortex5/Abyssal-Seraph-12B
parameters:
weight:
- filter: mlp
value: [0.3, 0.5, 0.8, 1.0, 1.0, 0.9, 0.7, 0.4]
- value: [0.2, 0.3, 0.4, 0.5, 0.5, 0.4, 0.3, 0.2]
density: 0.5
epsilon: 0.4
merge_method: della
base_model: Vortex5/MegaMoon-Karcher-12B
parameters:
lambda: 1.0
normalize: true
dtype: bfloat16
tokenizer:
source: Vortex5/Abyssal-Seraph-12B
๐ Essence of the Merge
Like moonlight reflecting on dark water, Lunar-Abyss carries both clarity and depth.
It thinks with the calm focus of LunaMaid yet speaks with the emotional pulse of Abyssal-Seraph.
Every response flows with a quiet duality โ logic beneath, creativity above โ neither overpowering the other.
For fans of expressive writing and immersive roleplay, it offers a tone thatโs reflective, and mysterious.
๐ญ Roleplay & Creative Focus
Designed for narrative storytelling, introspective dialogue, and emotion-driven writing.
๐ Acknowledgements ๐
- โ๏ธ mradermacher โ static / imatrix quantization
- ๐ DeathGodlike โ EXL3 quants
- ๐ฉถ All original model authors and contributors whose work made this model possible.
๐งพ Models Merged
Models merged in this creation:
- Downloads last month
- 1
