GLiNER2 Relation Verifier

Semantic verifier for filtering false positive relations extracted by GLiNER2.

Overview

GLiNER2's relation extraction can produce false positives when entities co-occur without expressing the claimed relation. This verifier uses a lightweight MLP trained on relation extraction datasets to filter these false positives.

Performance:

Reduces false positive rate from ~50% to ~14% at threshold 0.55
Precision: 75%, Recall: 44% at recommended threshold
Trained on ZeroRel + Re-DocRED datasets

Installation

pip install gliner2

Usage

Standalone

from gliner2.verifiers import RelationVerifier

# Load verifier
verifier = RelationVerifier.from_pretrained("oneryalcin/gliner2-relation-verifier")

# Verify relations
text = "Steve Jobs founded Apple in 1976."
relations = {
    "founded": [
        {"head": {"text": "Steve Jobs", "start": 0, "end": 10},
         "tail": {"text": "Apple", "start": 19, "end": 24}}
    ]
}
verified = verifier.verify(text, relations)
print(verified)
# {'founded': [{'head': {...}, 'tail': {...}, 'verifier_score': 0.85}]}

With GLiNER2

from gliner2 import GLiNER2
from gliner2.verifiers import RelationVerifier

# Load models
model = GLiNER2.from_pretrained("fastino/gliner2-base-v1")
verifier = RelationVerifier.from_pretrained("oneryalcin/gliner2-relation-verifier")

# Extract and verify
text = "John read a book about Apple."
results = model.extract_relations(text, ["works_for"], return_dict=True)
verified = verifier.verify(text, results.get("relation_extraction", {}))
# Empty - correctly filtered the false positive!

Threshold Selection

Threshold	Precision	Recall	FPR	Use Case
0.50	70%	58%	25%	Balanced
0.55	75%	44%	14%	Recommended
0.60	80%	36%	9%	High precision
0.75	87%	21%	3%	Very conservative

# Adjust threshold
verifier.set_threshold(0.60)  # More conservative

Model Architecture

Encoder: DeBERTa-v3-base (768-dim)
Features: Head/tail/between-span embeddings + distance + order (3105-dim)
Classifier: 3-layer MLP with LayerNorm and GELU

Training

Trained on:

ZeroRel: Zero-shot relation extraction dataset
Re-DocRED: Revised DocRED with cleaned annotations

See training details.

Citation

@misc{gliner2-verifier,
  author = {Oner Yalcin},
  title = {GLiNER2 Relation Verifier},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/oneryalcin/gliner2-relation-verifier}
}

License

Apache 2.0

Downloads last month: 19