TL;DR of the article and the discussion on Hacker News, generated automatically.
The README says a custom 3.75-bits-per-weight quant squeezes 284B DeepSeek-V4-Flash into under 59 GiB of RAM, decoding at about 9 to 15 tok/s on CPU alone.
Low-bit quality: anything below 4 or 5 bits is really bad.
Fair comparison: worse than the original model, but maybe not worse than anything else fitting in 64 GB.
The README: reads as AI slop, and a llama.cpp wrapper with unclear platform assumptions.
Nobody reported actually testing the output quality.
Sign in to comment.
TL;DR of the article and the discussion on Hacker News, generated automatically.
The README says a custom 3.75-bits-per-weight quant squeezes 284B DeepSeek-V4-Flash into under 59 GiB of RAM, decoding at about 9 to 15 tok/s on CPU alone.
Nobody reported actually testing the output quality.