Harmony-Bird-12B is a merged model intended for roleplay and storytelling.
Built with MergeKit.
Vermilion-Sage-12B
merges with
Impish-Nemo-12B
using the harmony_forge method (focus=8.0, blend=0.7).
merge_method: harmony_forge models: - model: Vortex5/Vermilion-Sage-12B - model: SicariusSicariiStuff/Impish_Nemo_12B parameters: focus: 8.0 blend: 0.7 tokenizer: source: union dtype: bfloat16
It is a custom adaptive merge method that blends models through consensus weighting across both **spatial** (parameter alignment) and **frequency** (spectral structure) domains. It is designed for stable, noise-resistant merges that preserve the shared strengths of multiple models while reducing conflicts and outlier effects. It supports standard and task-vector merging (when a `base_model` is provided). **How it works:** The method first centers all model weights (relative to either a `base_model` or their median) and normalizes their scales to ensure balance. It then analyzes correlations in parameter space (spatial features) and in the frequency domain (via FFT) to measure similarity and coherence between models. Each model receives a *goodness score* based on how well it aligns with others, adjusted by stability and outlier suppression terms. These scores are converted into normalized merge weights using a **softmax** function, which smoothly scales scores so that higher values receive more weight while all weights sum to 1. The `focus` parameter controls how sharply these weights are distributed — low focus blends models evenly, while high focus concentrates more weight on the most consistent ones. The `blend` parameter mixes how much spatial versus frequency information influences the final weighting. The merged parameters are then computed as a weighted sum across models. *Key parameters:* * **focus:** Controls decisiveness of weighting (higher = more selective). Default: `1.0` * **blend:** Uses a 0–1 scale to control merge emphasis — 0 represents full reliance on spatial similarity, meaning model weights are compared directly in parameter space. 1 represents full reliance on frequency-domain similarity, where weights are compared by their spectral (FFT) patterns. Default: 0.5 blends both equally for balanced structural and behavioral alignment. By weighting models through adaptive consensus across spatial and frequency domains, Harmony Forge emphasizes aligned, stable patterns—encouraging coherent, balanced merges that often inherit the strongest traits of each source model.