A study of sequence weighting at scale | Hacker News - But Better

A study of sequence weighting at scale blog.janestreet.com

TL;DR: We study the scaling laws of data weighting across in-house and open-weight LMs, finding non-monotonic behavior across scales. We vary the weight assi...

15 points•pranitha_m•9 days ago•1 comment•

#study #weighting #sequence #scale #sequence weighting #across #weight #scaling

1 comment

gwern8 days ago

Double descent?

Read the full thread on Hacker News →

Related stories

A study of sequence weighting at scaleblog.janestreet.com
Hacker News · 2 points · 10 days ago
A study of sequence weighting at scaleblog.janestreet.com
Lobsters · 2 points · 10 days ago
Scaling Laws for Neural Language Models (first "scaling laws" paper from 2020)arxiv.org
Hacker News · 2 points · 6 days ago
Most powerful obesity drug yet: People lost up to 25% of weight in trialarstechnica.com
Ars Technica · 0 points · 1 day ago
Rapid release at massive scale (Facebook)code.facebook.com
Lobsters · 2 points · about 9 years ago
Scaling to 100k Usersalexpareto.com
Lobsters · 18 points · over 6 years ago