Jev-serve: Run latest qwen3.8, all LLM models, mlx-supported | Hacker News - But Better

Jev-serve: Run latest qwen3.8, all LLM models, mlx-supported github.com

Single logit inference runtime for all LLM models. Contribute to rreinold/jev-serve development by creating an account on GitHub.

2 points•rreinold2•8 days ago•2 comments•

#jev #llm #serve #llm models #jev serve #qwen3 #mlx #run latest

2 comments

allensallinger7 days ago

Any recommendations for which of the models to try on macs with 16 GB of mem, and is this running well with any of the smaller models?

rreinold26 days ago

Guideline is (TOTAL_RAM - 5GB overhead) = max size, in billions of parameters. So 16 - 5 = 11, so <11B. Qwen 3.8 9B is a good fit

Read the full thread on Hacker News →

Related stories

Jev-rs: a Rust crate to turn a LLM to serve jevgithub.com
Hacker News · 2 points · 8 days ago
Show HN: Jevopt: Making intelligent compiler optimisation decisions with Jevgithub.com
Hacker News · 1 points · 9 days ago
JEV assisted LLM Tradingdev.to
DEV Community · 2 points · 10 days ago
Show HN: Onesie – An expressive CLI for System One models like Jevgithub.com
Hacker News · 3 points · 3 days ago
Show HN: Jev.Store – A dedicated directory of useful things made on Jevjev.store
Hacker News · 1 points · 5 days ago
Show HN: Will Jev pull the lever in the trolley problem?gpu.studio
Hacker News · 7 points · 11 days ago