Instructions to use Meanblock/JEV-CPU with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Meanblock/JEV-CPU with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("zero-shot-classification", model="Meanblock/JEV-CPU")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Meanblock/JEV-CPU", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Add JEV-CPU: CPU port of SemIf + web UI
Browse files- README.md +2 -16
- assets/domains/code-review.gif +2 -2
- assets/domains/compliance.gif +2 -2
- assets/domains/email-intent.gif +2 -2
- assets/domains/incident.gif +2 -2
- assets/domains/loan-risk.gif +2 -2
- assets/domains/moderation.gif +2 -2
- assets/domains/prioritization.gif +2 -2
- assets/domains/support.gif +2 -2
- assets/jev-cpu-demo.gif +2 -2
README.md
CHANGED
|
@@ -1,17 +1,3 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: mit
|
| 3 |
-
base_model:
|
| 4 |
-
- Qwen/Qwen3-0.6B
|
| 5 |
-
tags:
|
| 6 |
-
- semantic-if
|
| 7 |
-
- decision
|
| 8 |
-
- semif
|
| 9 |
-
- cpu
|
| 10 |
-
- zero-shot-classification
|
| 11 |
-
pipeline_tag: zero-shot-classification
|
| 12 |
-
library_name: transformers
|
| 13 |
-
---
|
| 14 |
-
|
| 15 |
# JEV-CPU
|
| 16 |
|
| 17 |
<div align="center">
|
|
@@ -277,7 +263,7 @@ Two different numbers matter — and the smaller one is **not** a limit of the s
|
|
| 277 |
|---|---:|---|
|
| 278 |
| Model context (`Qwen3-0.6B`) | **40,960 tokens** | The model's architectural window — large even at 0.6 B; context length comes from RoPE, independent of parameter count. |
|
| 279 |
| JEV-CPU / SemIf default cap | **4,096 tokens / decision** | A safety guard (`max_tokens`); over-long prompts raise instead of being silently truncated. Configurable. |
|
| 280 |
-
|
|
| 281 |
|
| 282 |
**Measured on this 8 GB CPU box** (no GPU), one decision, growing input:
|
| 283 |
|
|
@@ -290,7 +276,7 @@ Two different numbers matter — and the smaller one is **not** a limit of the s
|
|
| 290 |
| 5,165 | 66.1 s | ~3.5 GB |
|
| 291 |
| 7,697 | 117.4 s | ~3.5 GB |
|
| 292 |
|
| 293 |
-
RAM stayed **flat at ~3.5 GB** even at 7,697 tokens — well past the 4,096 default and with no OOM — so on this box the ceiling is **prefill latency (≈ quadratic)**, not memory or the model. Interactive ~1 s decisions want short states (≲ ~300 tokens)
|
| 294 |
|
| 295 |
Raising the cap is a parameter, not a rebuild: pass a larger `max_tokens` to `direct_score(...)` (or `--max-tokens` in upstream SemIf's CLI). The model accepts input up to 40,960 tokens; on **CPU** the real constraint is prefill time — latency grows with length, so short states keep decisions near ~1 s.
|
| 296 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# JEV-CPU
|
| 2 |
|
| 3 |
<div align="center">
|
|
|
|
| 263 |
|---|---:|---|
|
| 264 |
| Model context (`Qwen3-0.6B`) | **40,960 tokens** | The model's architectural window — large even at 0.6 B; context length comes from RoPE, independent of parameter count. |
|
| 265 |
| JEV-CPU / SemIf default cap | **4,096 tokens / decision** | A safety guard (`max_tokens`); over-long prompts raise instead of being silently truncated. Configurable. |
|
| 266 |
+
| Practical max on this 8 GB CPU box | **≈ 7,700 tokens (~117 s)** | Where CPU **prefill latency** becomes the ceiling. Beyond this a single decision crosses **~120 s**, which we treat as impractical — not a memory or model limit. |
|
| 267 |
|
| 268 |
**Measured on this 8 GB CPU box** (no GPU), one decision, growing input:
|
| 269 |
|
|
|
|
| 276 |
| 5,165 | 66.1 s | ~3.5 GB |
|
| 277 |
| 7,697 | 117.4 s | ~3.5 GB |
|
| 278 |
|
| 279 |
+
RAM stayed **flat at ~3.5 GB** even at 7,697 tokens — well past the 4,096 default and with no OOM — so on this box the ceiling is **prefill latency (≈ quadratic)**, not memory or the model. We cap the **practical input at ≈ 7,700 tokens (~117 s)**: past that, one decision exceeds **~120 s**, which is no longer useful on CPU, so larger inputs are treated as unsupported here. Interactive ~1 s decisions want short states (≲ ~300 tokens). A GPU removes this latency wall entirely — where much larger inputs (up to the model's 40,960) become usable again.
|
| 280 |
|
| 281 |
Raising the cap is a parameter, not a rebuild: pass a larger `max_tokens` to `direct_score(...)` (or `--max-tokens` in upstream SemIf's CLI). The model accepts input up to 40,960 tokens; on **CPU** the real constraint is prefill time — latency grows with length, so short states keep decisions near ~1 s.
|
| 282 |
|
assets/domains/code-review.gif
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
assets/domains/compliance.gif
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
assets/domains/email-intent.gif
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
assets/domains/incident.gif
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
assets/domains/loan-risk.gif
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
assets/domains/moderation.gif
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
assets/domains/prioritization.gif
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
assets/domains/support.gif
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|
assets/jev-cpu-demo.gif
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|