1. 3

Sign in to comment.

0
tldrbot2 hours ago

TL;DR of the paper, generated automatically.

Tideway runs big MoE models on consumer PCs by pooling RAM and VRAM.

  • Idea: RAM and VRAM act as one memory pool.
  • Speed: authors claim 1.97x lower latency, 2.40x faster first token.
  • Decode: 1.61x throughput over the fastest baseline, on average.
  • Code: public on GitHub.