New Scaling Law from Meta
Meta has introduced a new scaling law that connects model capacity and data through a single interaction parameter. This could significantly change budgeting approaches for model pre-training.
A strong new work from Meta.
Typically, scaling laws assume that model size and data volume affect error independently of each other.
In this work, the authors introduce the Skaling law—a law that connects model capacity and data through a single interaction parameter.
An additional term reduces the average absolute percentage error by about 1.5 to 3 times for both interpolation and extrapolation.
The most notable improvements occur in regimes where data is scarce or the model is heavily overfitted. It is precisely in these areas that the standard Chinchilla and Kaplan laws begin to miss significantly.
Another important aspect. Along with a sparse grid of small runs, the method allows for extrapolating the entire picture with approximately 10 times lower computational costs than uniform enumeration.
Why does this matter?
Today, models are often trained and deployed far beyond the classical computationally optimal regime. If the scaling law remains accurate there, and it can be estimated on small runs, it directly changes how budgets for pre-training can be planned.
Why it matters
AnalysisThis new scaling law could significantly change model training approaches by reducing computational costs and improving accuracy. It opens up new possibilities for optimizing pre-training budgets.
Discuss in community
Share your questions and insights with developers
Related in Ecosystem
Practical guides, core terminology, and agent skills
Llama Family (Meta Llama)
A series of foundational open language models from Meta (Llama 3, 3.1, 3.3) that have become the industrial standard for the Open Weights ecosystem, local AI, and enterprise fine-tuning.
How to Build a Custom Design System in Claude Design: From Moodboard to Animated Mascot
Complete practical guide to building a brand design system in Claude Design: visual references, 4 diverging concepts, tokenization, stress-testing, and animated SVG mascot design.
Jev and System One AI: A Complete Practical Guide to the Decision Model from TypeSafe
A deep engineering breakdown of the flagship Jev model: System One architecture versus LLMs, Choice/Score/Noul primitives, request parallelism, confidence mathematics, video use cases, code, and an overview of local open-weight alternatives (Laya, GLiNER2.5, CLM-8B).
Architectural Erosion from Fast Coding
The unnoticed accumulation of hidden systemic contradictions and the degradation of architectural boundaries in a project due to patchy local code fixes by language models.