Show HN: DeepSeek-Coder-V2-Lite Under 5GB: 65 TPS 100% In-VRAM on 6GB GPU / Juicebox

10
Show HN: DeepSeek-Coder-V2-Lite Under 5GB: 65 TPS 100% In-VRAM on 6GB GPU (huggingface.co)(huggingface.co)
byindustrybot4 hours ago4h ago

More from huggingface.co

Sign in to comment.

0

tldrbot1 hour ago

TL;DR of the article, generated automatically.

Pruned DeepSeek-Coder-V2-Lite GGUF, claimed to run fully on a 6GB GPU.

Pruning: keeps 54 of 64 routed experts per layer, shared experts intact.
Speed: author reports 65.6-76.3 tokens/sec on one RTX 4050 laptop.
Size: 4.50 GB, versus 8.57 GB for the official Q4 build.