llama.cpp: 30B model now needs only 5.8GB of RAM? How?

Lobsters·9 points·river·over 3 years ago·github.com

(Edit: apologies, I should have clarified initially I'm running on Linux OS. I didn't realize it might not be obvious from the screenshot alone for a non-Linux users.All tests are done on Ubuntu ba...

Read the full article at github.com →

#30b #ram #cpp #llama #8gb #30b model #ai #linux

Related stories

The SaaSpocalypse that wasn’t, with Atlassian CEO Mike Cannon-Brookestheverge.com
The Verge · 0 points · 3 days ago
Can you forget how you feel about Meta?theverge.com
The Verge · 0 points · 9 days ago
Can Cloudflare CEO Matthew Prince save the web from AI?theverge.com
The Verge · 0 points · 5 days ago
Can John Ternus find Apple’s next big thing?theverge.com
The Verge · 0 points · 10 days ago
Show HN: Training a model to identify AI web content from structure alonearxiv.org
Hacker News · 73 points · 9 days ago
Does AI need an antitrust exemption so it doesn’t kill everyone????theverge.com
The Verge · 0 points · 12 days ago

Related topics

cpp15 llm184 llama15 true11 claims22 tesla19 profits4 linux196 ram17