arXiv preprint · 2020
Scaling Laws for Neural Language Models
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Chess Chess, Ramesh Ramesh, Nick Joseph, Jared Kaplan +4 more
2discussions
Abstract
Empirical scaling laws for language model performance on cross-entropy loss as a function of model size, dataset size, and compute budget.