A study of sequence weighting at scale | Hacker News - But Better

A study of sequence weighting at scale blog.janestreet.com

TL;DR: We study the scaling laws of data weighting across in-house and open-weight LMs, finding non-monotonic behavior across scales. We vary the weight assi...

2 points•eatonphil•10 days ago•0 comments•

#study #weighting #sequence #scale #sequence weighting #across #weight #scaling

0 comments

No comments yet.

Related stories

A study of sequence weighting at scaleblog.janestreet.com
Hacker News · 6 points · 9 days ago
A study of sequence weighting at scaleblog.janestreet.com
Lobsters · 2 points · 10 days ago
Scaling Laws for Neural Language Models (first "scaling laws" paper from 2020)arxiv.org
Hacker News · 2 points · 6 days ago
Most powerful obesity drug yet: People lost up to 25% of weight in trialarstechnica.com
Ars Technica · 0 points · 1 day ago
Scaling to 100k Usersalexpareto.com
Lobsters · 18 points · over 6 years ago
Rapid release at massive scale (Facebook)code.facebook.com
Lobsters · 2 points · about 9 years ago