Every technology wave begins with a metric that makes progress look simple. During the dot-com era, it was eyeballs: more visitors, more registered users, more page views. These numbers were an important input factor for success, but they were incomplete and easy to game. It took a painful correction before the market began asking what kinds of usage actually created durable value and commercial success.

Frontier AI is entering a similar transition. For several years, progress could be described with a reassuringly simple formula of input factors: more parameters, more data, more compute. That formula has not stopped working. But the dominance of simple scaling laws is coming to an end, and the bottleneck is moving.

Wave One: Scaling the Internet

GPT-3, released by OpenAI in 2020, was the clearest early expression of the scaling thesis. It was dramatically larger than its predecessors, trained on a vast mixture of public web pages, books, code and other text, and it displayed capabilities that seemed to emerge simply from making the system bigger.

The approach was attractive because it was measurable. Labs could add more training data and compute, then harvest the resulting improvements. For a while, each new generation of models seemed to validate the same lesson: scale the inputs and intelligence would follow.

The Chinchilla scaling laws refined that formula in 2022. DeepMind showed that many large models had been undertrained relative to their size: a smaller model trained on substantially more data could outperform a much larger one using the same compute budget. The lesson was not that scale had failed, but that parameters, compute and training data had to grow in better proportion.

That adjustment exposed a practical constraint. High-quality human-generated text is finite, and much of the easily accessible public Internet has already been used. Estimates of when the industry will fully exhaust that resource vary, and labs can extend it through various tricks. But it is increasingly clear that adding more raw training data aggressively will not produce the same gains.

Wave Two: Teaching Models What People Want

A second development emerged in parallel: post-training. OpenAI's InstructGPT demonstrated that a relatively small model trained with human feedback could be preferred over a much larger base model. OpenAI turned that insight into a mass-market product with ChatGPT.

Reinforcement learning from human feedback, or RLHF, changed the meaning of training data. Instead of only asking models to predict the next word in an enormous corpus, labs asked people to compare outputs, identify better answers and demonstrate the behavior they wanted. Human judgment became part of the training signal.

This helped create an important new infrastructure category. Pioneers like Scale AI, which had originally built large-scale data operations for computer vision and autonomous vehicles, expanded into foundation-model data, post-training and evaluation. More recently, Handshake has used its university and professional network to connect frontier labs with people capable of producing and evaluating specialized work.

The rapid growth of these businesses is not an accident. It reflects a shift from abundant generic data toward scarce, task-specific judgment.

Wave Three: From Preference to Correctness

The first generation of human feedback mostly taught models to be helpful, coherent and safe. A non-specialist evaluator can usually tell whether an answer follows instructions, stays on topic or sounds reasonable.

The frontier has been moving into areas where sounding reasonable is not enough. A legal argument can be eloquent and still misunderstand the relevant jurisdiction. A financial model can look polished while containing a subtle conceptual error. A medical recommendation can be plausible but dangerous. A robotics plan can work in a simulation and fail when confronted with the physical world.

Some domains are easier to verify automatically. Code can be executed, mathematical answers can be checked and agents can be tested inside controlled environments. AI models can also evaluate other models, generate synthetic examples and perform a growing share of routine supervision.

But automated evaluation does not eliminate the need for human expertise. It moves it upstream. Experts increasingly define the tasks, write the rubrics, construct adversarial examples, identify failure modes and determine what a correct outcome should look like. They provide the standards against which automated systems can operate.

The scarce resource is therefore not simply human labor. It is trusted judgment: domain-specific knowledge converted into tasks, feedback, evaluations and evidence that can distinguish a plausible answer from a truly correct one.

A New Layer of AI Infrastructure

This creates room for several kinds of companies. Some will build full-stack data and evaluation platforms. Others will specialize in supplying verified experts into workflows that frontier labs and established data platforms already operate. What they share is the ability to turn fragmented human expertise into a reliable, scalable training signal.

Not every task requires the most senior available expert. A scalable system matches task difficulty to the right level of knowledge, language and professional experience, then combines automated screening, independent review and senior adjudication where needed. The infrastructure lies partly in knowing who is qualified to do what.

In the old data-labeling model, the objective was throughput: divide a simple task among a large workforce and deliver the results as cheaply as possible. In expert evaluation, the objective is reliability. A small number of carefully selected contributors may be more valuable than a huge undifferentiated crowd.

The strongest companies in this category will also become more than marketplaces. Each project can improve their understanding of which experts perform well, which rubrics produce consistent results, where models fail and how quality should be measured. Human work becomes structured knowledge; structured knowledge becomes reusable evaluation infrastructure.

The strongest position in this space won't belong to whoever processes the most tasks fastest. It will belong to whoever can reach deepest into a genuine pool of qualified people and hold their trust over years, not single projects — companies built around durable professional and academic networks, not generic labor marketplaces. That kind of relationship is a genuine moat, not just a go-to-market advantage, since it rests on reputations built over decades and cannot be assembled with capital alone. The businesses with the deepest, most credentialed expert networks are the ones best placed to compound that advantage, rather than get commoditized by the next wave of automation.

‍