Scaling Laws for Neural Language Models (first "scaling laws" paper from 2020)

Scaling Laws for Neural Language Models (first "scaling laws" paper from 2020)arxiv.org

We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends…

2 points•thoughtpeddler•6 days ago•0 comments•

#scaling #laws #scaling laws #paper #neural #language models #neural language #neural language models

0 comments

No comments yet.

0 comments

Related stories