>1B tokens/minute/GPU by combining query planner and inference engine | Hacker News - But Better

>1B tokens/minute/GPU by combining query planner and inference engine modal.com

Maximizing perf on AI-SQL queries with the KV-optimal left-deep join

1 points•birdculture•4 days ago•0 comments•

#engine #gpu #tokens #query #inference #planner #combining #combining query

0 comments

No comments yet.

Related stories

>1B tokens/minute/GPU by combining query planner and inference enginemodal.com
Hacker News · 4 points · 4 days ago
Beyond Utilization: Energy-Conscious GPU Sharing for Inference Servingal.radbox.org
Hacker News · 2 points · about 22 hours ago
Launch HN: Magnitude (YC S25) – Self-optimizing inference engine for agentsgithub.com
Hacker News · 127 points · about 9 hours ago
Show HN: Reflex – a GGUF/CUDA inference engine tuned for cold-start latencygithub.com
Hacker News · 1 points · 7 days ago
NobodyWho is an inference engine that lets you run LLMs locally and efficientlygithub.com
Hacker News · 1 points · 8 days ago
Show HN: AI·rete·RAG – a Rete rule engine decides, RAG explains whyai-rete-rag.com
Hacker News · 44 points · 8 days ago