DevQuasar/miromind-ai.MiroThinker-v1.5-30B-GGUF
Text Generation
•
31B
•
Updated
DevQuasar
community
Verified
AI & ML interests
Open-Source LLMs, Local AI Projects: https://pypi.org/project/llm-predictive-router/
Recent Activity
View all activity
csabakecskemeti
updated
a
model
1 minute ago
csabakecskemeti
published
a
model
25 minutes ago
csabakecskemeti
updated
a
model
about 17 hours ago
csabakecskemeti
published
a
model
about 17 hours ago
csabakecskemeti
updated
a
model
about 24 hours ago
csabakecskemeti
published
a
model
1 day ago
csabakecskemeti
updated
a
model
1 day ago
csabakecskemeti
published
a
model
1 day ago
csabakecskemeti
updated
a
collection
3 days ago
csabakecskemeti
updated
2
models
3 days ago
csabakecskemeti
posted
an
update
11 days ago
Post
3138
Just sharing a result of a homelab infrastructure experiment:
I've managed to setup a distributed inference infra at home using a DGX Spark (128GB unified gddr6) and a linux workstation with an RTX 6000 Pro (96GB gddr7) connected via 100Gbps RoCEv2. The model I've used (https://lnkd.in/gx6J7YuB) is about 140GB so could not fit either of the GPU. Full setup and tutorial soon on devquasar.com
Screen recording:
https://lnkd.in/gKM9H5GJ
I've managed to setup a distributed inference infra at home using a DGX Spark (128GB unified gddr6) and a linux workstation with an RTX 6000 Pro (96GB gddr7) connected via 100Gbps RoCEv2. The model I've used (https://lnkd.in/gx6J7YuB) is about 140GB so could not fit either of the GPU. Full setup and tutorial soon on devquasar.com
Screen recording:
https://lnkd.in/gKM9H5GJ
-
- 3 replies
csabakecskemeti
posted
an
update
about 1 month ago
Post
1262
FYI: Mistral.Ministral-3 dequantizer FP8->BF16
https://github.com/csabakecskemeti/ministral-3_dequantizer_fp8-bf16
(The instruct model weights are in FP8)
https://github.com/csabakecskemeti/ministral-3_dequantizer_fp8-bf16
(The instruct model weights are in FP8)
csabakecskemeti
posted
an
update
about 1 month ago
Post
2074
Looking for some help to test an INT8 Deepseek 3.2:
SGLang supports Channel wise INT8 quants on CPUs with AMX instructions (Xeon 5 and above AFAIK)
https://lmsys.org/blog/2025-07-14-intel-xeon-optimization/
Currently uploading an INT8 version of Deepseek 3.2 Speciale:
DevQuasar/deepseek-ai.DeepSeek-V3.2-Speciale-Channel-INT8
I cannot test this I'm on AMD
"AssertionError: W8A8Int8LinearMethod on CPU requires that CPU has AMX support"
(I assumed it can fall back to some non optimized kernel but seems not)
If anyone with the required resources (Intel Xeon 5/6 + ~768-1TB ram) can help to test this that would be awesome.
If you have hints how to make this work on AMD Threadripper 7000 Pro series please guide me.
Thanks all!
SGLang supports Channel wise INT8 quants on CPUs with AMX instructions (Xeon 5 and above AFAIK)
https://lmsys.org/blog/2025-07-14-intel-xeon-optimization/
Currently uploading an INT8 version of Deepseek 3.2 Speciale:
DevQuasar/deepseek-ai.DeepSeek-V3.2-Speciale-Channel-INT8
I cannot test this I'm on AMD
"AssertionError: W8A8Int8LinearMethod on CPU requires that CPU has AMX support"
(I assumed it can fall back to some non optimized kernel but seems not)
If anyone with the required resources (Intel Xeon 5/6 + ~768-1TB ram) can help to test this that would be awesome.
If you have hints how to make this work on AMD Threadripper 7000 Pro series please guide me.
Thanks all!
-
- 8 replies
csabakecskemeti
posted
an
update
about 2 months ago
Post
304
Recently there are so much activity on token efficient formats, I've also build a package (inspired by toon).
Deep-TOON
My goal was to token efficiently handle json structures with complex embeddings.
So this is what I've built on the weekend. Feel free try:
https://pypi.org/project/deep-toon/0.1.0/
Deep-TOON
My goal was to token efficiently handle json structures with complex embeddings.
So this is what I've built on the weekend. Feel free try:
https://pypi.org/project/deep-toon/0.1.0/
csabakecskemeti
posted
an
update
3 months ago
Post
2622
Christmas came early this year
- 3 replies
csabakecskemeti
posted
an
update
7 months ago
Post
3072
Has anyone ever backed up a model to a sequential tape drive, or I'm the world first? :D
Just played around with my retro PC that has got a tape drive—did it just because I can.
Just played around with my retro PC that has got a tape drive—did it just because I can.
- 5 replies
csabakecskemeti
posted
an
update
7 months ago
Post
476
Deepseek R1 0528 Q2 locally.
(I believe it has overthinking it a bit :) )
https://youtu.be/Iqu5s9aFaXA?si=QWZe293iTKf_3ELU
DevQuasar/deepseek-ai.DeepSeek-R1-0528-GGUF
(I believe it has overthinking it a bit :) )
https://youtu.be/Iqu5s9aFaXA?si=QWZe293iTKf_3ELU
DevQuasar/deepseek-ai.DeepSeek-R1-0528-GGUF
csabakecskemeti
posted
an
update
9 months ago
Post
2120
Local Llama4 Maverick Q2
https://youtu.be/4F8g_LThli0?si=MGba2SUTHt6xYw3T
Quants uploading now
Big thanks to @ngxson !
https://youtu.be/4F8g_LThli0?si=MGba2SUTHt6xYw3T
Quants uploading now
Big thanks to @ngxson !
csabakecskemeti
posted
an
update
9 months ago
Post
1751
Why the 'how many r's in strawberry' prompt "breaks" llama4? :D
Quants DevQuasar/meta-llama.Llama-4-Scout-17B-16E-Instruct-GGUF
Quants DevQuasar/meta-llama.Llama-4-Scout-17B-16E-Instruct-GGUF
- 3 replies