Meanblock commited on
Commit
9b0441f
·
verified ·
1 Parent(s): 96c8f6e

Add JEV-CPU: CPU port of SemIf + web UI

Browse files
README.md CHANGED
@@ -1,17 +1,3 @@
1
- ---
2
- license: mit
3
- base_model:
4
- - Qwen/Qwen3-0.6B
5
- tags:
6
- - semantic-if
7
- - decision
8
- - semif
9
- - cpu
10
- - zero-shot-classification
11
- pipeline_tag: zero-shot-classification
12
- library_name: transformers
13
- ---
14
-
15
  # JEV-CPU
16
 
17
  <div align="center">
@@ -277,7 +263,7 @@ Two different numbers matter — and the smaller one is **not** a limit of the s
277
  |---|---:|---|
278
  | Model context (`Qwen3-0.6B`) | **40,960 tokens** | The model's architectural window — large even at 0.6 B; context length comes from RoPE, independent of parameter count. |
279
  | JEV-CPU / SemIf default cap | **4,096 tokens / decision** | A safety guard (`max_tokens`); over-long prompts raise instead of being silently truncated. Configurable. |
280
- | Stable on this 8 GB CPU box | *(measured below)* | Where CPU **prefill latency**, not the model, becomes the practical ceiling. |
281
 
282
  **Measured on this 8 GB CPU box** (no GPU), one decision, growing input:
283
 
@@ -290,7 +276,7 @@ Two different numbers matter — and the smaller one is **not** a limit of the s
290
  | 5,165 | 66.1 s | ~3.5 GB |
291
  | 7,697 | 117.4 s | ~3.5 GB |
292
 
293
- RAM stayed **flat at ~3.5 GB** even at 7,697 tokens — well past the 4,096 default and with no OOM — so on this box the ceiling is **prefill latency (≈ quadratic)**, not memory or the model. Interactive ~1 s decisions want short states (≲ ~300 tokens); long documents still work, just slower. A GPU removes this latency wall entirely.
294
 
295
  Raising the cap is a parameter, not a rebuild: pass a larger `max_tokens` to `direct_score(...)` (or `--max-tokens` in upstream SemIf's CLI). The model accepts input up to 40,960 tokens; on **CPU** the real constraint is prefill time — latency grows with length, so short states keep decisions near ~1 s.
296
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # JEV-CPU
2
 
3
  <div align="center">
 
263
  |---|---:|---|
264
  | Model context (`Qwen3-0.6B`) | **40,960 tokens** | The model's architectural window — large even at 0.6 B; context length comes from RoPE, independent of parameter count. |
265
  | JEV-CPU / SemIf default cap | **4,096 tokens / decision** | A safety guard (`max_tokens`); over-long prompts raise instead of being silently truncated. Configurable. |
266
+ | Practical max on this 8 GB CPU box | **≈ 7,700 tokens (~117 s)** | Where CPU **prefill latency** becomes the ceiling. Beyond this a single decision crosses **~120 s**, which we treat as impractical — not a memory or model limit. |
267
 
268
  **Measured on this 8 GB CPU box** (no GPU), one decision, growing input:
269
 
 
276
  | 5,165 | 66.1 s | ~3.5 GB |
277
  | 7,697 | 117.4 s | ~3.5 GB |
278
 
279
+ RAM stayed **flat at ~3.5 GB** even at 7,697 tokens — well past the 4,096 default and with no OOM — so on this box the ceiling is **prefill latency (≈ quadratic)**, not memory or the model. We cap the **practical input at ≈ 7,700 tokens (~117 s)**: past that, one decision exceeds **~120 s**, which is no longer useful on CPU, so larger inputs are treated as unsupported here. Interactive ~1 s decisions want short states (≲ ~300 tokens). A GPU removes this latency wall entirely — where much larger inputs (up to the model's 40,960) become usable again.
280
 
281
  Raising the cap is a parameter, not a rebuild: pass a larger `max_tokens` to `direct_score(...)` (or `--max-tokens` in upstream SemIf's CLI). The model accepts input up to 40,960 tokens; on **CPU** the real constraint is prefill time — latency grows with length, so short states keep decisions near ~1 s.
282
 
assets/domains/code-review.gif CHANGED

Git LFS Details

  • SHA256: 86ab6254ccffb4e1820b5fe160dd5d8af52b6d7888c8dafefac7f3feb8f3508e
  • Pointer size: 131 Bytes
  • Size of remote file: 194 kB

Git LFS Details

  • SHA256: a1395be33943a28c42f654549b31d863a65f859625b2e1756cdca7cff7491030
  • Pointer size: 131 Bytes
  • Size of remote file: 211 kB
assets/domains/compliance.gif CHANGED

Git LFS Details

  • SHA256: 2cf14da78805f3aa591ffe4154f746fba33833007443321ca62fd73b54cb7e65
  • Pointer size: 131 Bytes
  • Size of remote file: 191 kB

Git LFS Details

  • SHA256: 3539698228bf7476888cc4c3b684d46bee35b17e0b0dc9fe903016972a5e24b4
  • Pointer size: 131 Bytes
  • Size of remote file: 186 kB
assets/domains/email-intent.gif CHANGED

Git LFS Details

  • SHA256: 451210ec2d8ab93d026d7320a677aa827cd1d390f2ccf1df408ed109b3c7e110
  • Pointer size: 131 Bytes
  • Size of remote file: 170 kB

Git LFS Details

  • SHA256: 62bb8bfe2397ca1fb40a7561947f93e7ad4be447ab1543cbcbf71a4789043870
  • Pointer size: 131 Bytes
  • Size of remote file: 205 kB
assets/domains/incident.gif CHANGED

Git LFS Details

  • SHA256: 94f7fa44183d2a4e403a0813a2caa2225cee35b48a8b27c7e3ddd70273cd454a
  • Pointer size: 131 Bytes
  • Size of remote file: 202 kB

Git LFS Details

  • SHA256: b3969ba746fa887777f9430f88757480e4f679f18defd955979cbdfd294a217d
  • Pointer size: 131 Bytes
  • Size of remote file: 184 kB
assets/domains/loan-risk.gif CHANGED

Git LFS Details

  • SHA256: fab49b67d8b3ca841854a77d47c8a694078624a3c66ecd96ac7ebe806a52dde9
  • Pointer size: 131 Bytes
  • Size of remote file: 188 kB

Git LFS Details

  • SHA256: d27745e5215513fac3b71cd753dea3bfa4e3ce9816f9c9f80e6a5fe60dc74b68
  • Pointer size: 131 Bytes
  • Size of remote file: 211 kB
assets/domains/moderation.gif CHANGED

Git LFS Details

  • SHA256: 50004f93fca8bde9c9ba9fd86544de5a77386c87c7d34f3695b5e627e8bfeac0
  • Pointer size: 131 Bytes
  • Size of remote file: 183 kB

Git LFS Details

  • SHA256: 0c24ddfb46326f083f31764a8965a7e8678a74fd3666d86e9456082dc25835d7
  • Pointer size: 131 Bytes
  • Size of remote file: 194 kB
assets/domains/prioritization.gif CHANGED

Git LFS Details

  • SHA256: 59e3c2dbe5f1141b826975c11b98d2e2fffcf2d198062f2437e02147f1ed7b48
  • Pointer size: 131 Bytes
  • Size of remote file: 187 kB

Git LFS Details

  • SHA256: 127bb39ba435cc3d5c0d34bd14b16cde5cfd67b7c710fef93a3fd546cfad89c9
  • Pointer size: 131 Bytes
  • Size of remote file: 200 kB
assets/domains/support.gif CHANGED

Git LFS Details

  • SHA256: 5894578358656632b0848179b3b3c3e299383f02926593ff52b1d5978229e41e
  • Pointer size: 131 Bytes
  • Size of remote file: 202 kB

Git LFS Details

  • SHA256: b9b6dc94546c2b694e987f7302e3c73586efdfb691716681a1e7e6f7f5263195
  • Pointer size: 131 Bytes
  • Size of remote file: 185 kB
assets/jev-cpu-demo.gif CHANGED

Git LFS Details

  • SHA256: 40a34ad3028a4693dc048d8fc80dcb21375678715010b1a4f2e78940f8321757
  • Pointer size: 132 Bytes
  • Size of remote file: 1.04 MB

Git LFS Details

  • SHA256: 596a11d2546a76f3b2d5576e910376a44bfb7426cb8ab75e0aa7e80327a2b49b
  • Pointer size: 132 Bytes
  • Size of remote file: 1.09 MB