Glyd – Run LLMs in 33% less GPU memory, bit for bit | Hacker News - But Better

Glyd – Run LLMs in 33% less GPU memory, bit for bit getglyd.com

Glyd stores open models' bf16 weights in about 11 bits instead of 16 and decodes them inside the GPU's matrix multiply: bit for bit, often faster.

3 points•surya-koritala•3 days ago•0 comments•

#bit #glyd #less #memory #llms #gpu #less gpu #run llms

0 comments

No comments yet.

Read the full thread on Hacker News →

Related stories

Show HN: Cross-platform governed memory for a fleet of agentsgithub.com
Hacker News · 1 points · 8 days ago
Show HN: StarSkirmish, an arena where LLMs create StarCraft Brood War botsstarskirmish.com
Hacker News · 3 points · 4 days ago
Show HN: SelMem – selective reconstructive memory for LLMsgithub.com
Hacker News · 1 points · 6 days ago
Show HN: Lossless-memory – a personal AI memory that never summarizesgithub.com
Hacker News · 67 points · 10 days ago
Sparse resources helped my GPU-driven renderer memory usagezino2201.substack.com
Hacker News · 1 points · 10 days ago
Forging 1024-bit RSA signatures in nearly SNFS timeeprint.iacr.org
Lobsters · 7 points · 7 days ago