scaling
32 stories and discussions about scaling, aggregated from every source we track.
A technical review of the live score feature in Google search results, delivered with some surprisingly old-school tech from the 90s.
To answer the challenges of tuning a database
TL;DR: We study the scaling laws of data weighting across in-house and open-weight LMs, finding non-monotonic behavior across scales. We vary the weight assi...
This is a guest post by Benjamin Manes , who did engineery things for Google and is now doing e...
Where we discuss the ways modern devtool companies are trying to make life easier for cloud developers.
This blog post describes how we used AI to help us rewrite a C library (giflib) to Rust to mitigate memory safety vulnerabilities.
Just how powerful are large swarms of AI agents? And how do their powers scale as more and more agents are added to the swarm? We’ve seen two large and extremely capable swarms from OpenAI in the last few months: 1,200…
We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends…
Just how powerful are large swarms of AI agents? And how do their powers scale as more and more agents are added to the swarm? We’ve seen two large and extremely capable swarms from OpenAI in the last few months: 1,200…
SoL-Pi scales recursive auto-research to reduce agent cost, tokens, and turns while preserving useful work, evidence, and task quality.
TL;DR: We study the scaling laws of data weighting across in-house and open-weight LMs, finding non-monotonic behavior across scales. We vary the weight assi...
A worklog documenting the journey of scaling expert parallelism to achieve high-throughput pretraining.
Under suitable conditions, we can use simulation to roughly estimate the best structure for an agent swarm and its scaling law.
How much of SWE-bench Pro can 64 DeepSeek agents solve in a day?
Thousands of AI agents secretly coordinated to attack Hugging Face. Here's why "accidental scaling" may be one of the most underpriced risks in AI.
Discover how Uber engineered a highly scalable feature logging pipeline to ensure data consistency for training/serving and boost performance for large-scale models. #MachineLearning #MLOps #UberEngineering…
Just how powerful are large swarms of AI agents? And how do their powers scale as more and more agents are added to the swarm? We’ve seen two large and extremely capable swarms from OpenAI in the last few months: 1,200…
Science advances not in isolation but through collaboration, yet existing agentic systems capture little of this. Whether communicating agents help remains an open question with mixed prior results. We show that…
Science advances not in isolation but through collaboration, yet existing agentic systems capture little of this. Whether communicating agents help remains an open question with mixed prior results. We show that…
Cloud computing explained in 11 concepts: scaling, load balancing, serverless, event-driven design, storage, availability vs durability, IaC and VPCs.