Matthew N. Bernstein

Matthew N. Bernstein

Principal Machine Learning Scientist at Cellular Intelligence

AI makes quantity cheap, but quality is still expensive. The net effect is more human work

5 minute read

Published:

At the time of this writing (September 22, 2026), the quality gap between AI-generated output and the output produced by human experts has still not closed in most domains (coding and math being clear exceptions). In this post, I make two arguments. First, AI changes how we should plan projects: where project planning was once mostly about how much work could be produced in a given amount of time, we now have to think separately about quantity and quality. AI has made it extremely cheap to produce work at scale, but producing high-quality work remains very difficult. Second, every unit of AI-generated output still requires human effort to close the remaining quality gap. Because the growth in how much AI can produce is outpacing the rate at which that gap is shrinking, the amount of work that knowledge workers will have to do may only increase!

Before AI, project planning was mostly about quantity

In the old days (all of 8 months ago at the time of this writing), when someone went about planning a project, they would estimate how big of a project they could complete given the resources allocated to them. By “project” here, I mean any kind of knowledge-based product that an individual person or team of people seek to produce: a written article, a computer program, an image, etc. Importantly, the quality of work produced by most experts is often relatively stable. If you ask a professional software engineer, or team of engineers, to create a program, the question would rarely be, “How robust should the program be?” Or, “How clean should the code be?” Rather, the questions would be along the lines of, “What features should this product have?” If you ask a data scientist to perform a data analysis, the question would rarely be, “How correct should these conclusions be?” Instead, it would be more along the lines of, “What questions do we seek to answer?” Depending on how much time is allowed to this person or team, they would estimate what they could accomplish given the constraint.

Because quality, in most cases, is a relatively stable property of the output of human experts, the dimension most often considered during project planning was that of quantity. That is, the relationship between effort and quantity was, at a rough pass, linear: The more work I put in, the more stuff I can accomplish.

drawing

With AI, project planning is also about quality

At the time of this writing (September 22, 2026), the gap in quality between the output produced by an AI versus a human expert has still not closed in most domains (coding and math being clear exceptions). Based on my observation, this is because models still lack the nuanced judgement and taste of an expert human being. They often miss the point, write with unnatural syntax, or add unnecessary complexity. That is, without human guidance, much of what they produce is slop.

Unfortunately, closing this quality gap takes a lot of effort! I find this in my own work. As a machine learning scientist and computational biologist, I am now using AI to create analyses, write reports, and create presentations. In my personal experience, the quality of the output is somewhere in the ballpark range of 75% of what I am capable of producing. I find that it takes quite a lot of time to tweak the writing, to remove unnecessary features in its code, to clean its figures, etc.

The calculus to how we should approach work has completely changed. No longer is quantity the dimension along which we should plan, but rather, it is quality. Generating many projects at 75% quality is extremely cheap. But closing the quality gap is still quite expensive. This is illustrated in the schematic below:

drawing

The act of planning a project has become an optimization problem in two dimensions. Producing a large amount of high quality work is extremely expensive because the amount of effort and time it takes to close the quality gap for each unit of work is high. To make this concrete, one must decide whether to produce:

  • A lot of work at low quality
  • A medium amount of work at medium quality
  • A small amount of work at high quality

One must consider two dimensions!

The quality gap explains why humans have more work to do than ever

Moreover, this phenomenon explains why it feels to me like the amount of work we human knowledge workers have to do is higher than ever. For every unit of work produced by an AI, a human has to close the quality gap. Therefore, there are two competing forces: how much quantity AI can produce and how small the quantity gap becomes. The more quantity, the more units of work will require a human to close the quality gap. But the smaller the gap becomes, the less human work will be required per unit of AI output. I predict the net effect will be such that the amount of human work grows as illustrated in the plot below:

drawing


That is, no matter how small this quality gap becomes, as the quantity produced by AI becomes cheaper, the amount of work we humans have will increase anyway! The AIs will continue to generate output at ever increasing rates and we humans will have to close the quality gap in each unit of work produced by the AI fire hose.

Of course, I may be completely wrong about this. It may be that a week from now, a frontier AI lab releases a model that closes the quality gap so utterly that there truly is no quality gap to close. Quantity and quality become equally cheap and generated at will. Only time will tell.

Tags:

You May Also Enjoy

A framework for making sense of metrics in technical organizations

17 minute read

Published:

If you work in a quantitative or technical field, there is little doubt that you or your team has worked long and hard to define which metrics to measure and track. Using data-driven metrics is a critical practice for making rational decisions and deciphering truth in a complex and noisy world. However, as others have pointed out, an over-reliance on metrics can lead to poor outcomes. In an effort to better articulate the value and risks inherent in metrics, in this blog post, I will present a mental framework for thinking about metrics that has helped me reason about their value and risks with a bit more clarity.

Understanding attention

33 minute read

Published:

Attention is a type of layer in a neural network that is widely regarded to be one of the most important breakthroughs that enabled the development of modern AI systems and large language models. At its heart, attention is a mechanism for explicitly drawing relationships between items in a set. In natural language processing, the set being processed are words (or tokens) and attention enables the model to relate those words to one another even when those words lie far away from eachother in the body of text. In this blog post, we will step through the attention mechanism both mathematically and intuitively. We then present a minimal example of a neural network that uses attention to perform binary classification in a task that is not solveable using a naïve bag-of-words model.

Demystifying Euler’s number

16 minute read

Published:

Euler’s number $e := 2.71828\dots$ has, to me, always been a semi-mysterious number. While I understood many facts about $e$, I never felt I ever truly understood what it really was – it’s core essence so to speak. I believe that part of the reason for my confusion is that $e$ is often taught coming from two seemingly different perspectives: Either it is introduced in the context of compound interest or it is introduced in the context of calculus as being the base of the exponential function whose derivative is itself. Thanks to an excellent explanation by Grant Sanderson’s 3Blue1Brown video, I now better understand this constant and how these two perspectives relate to one another. In this blog post, I will attempt to describe, in my own words, my understanding of Euler’s number and expound on Sanderson’s explanation.

Reproducing kernel Hilbert spaces and the kernel trick

22 minute read

Published:

If you’re a practitioner of machine learning, then there is little doubt you have seen or used an algorithm that falls into the general category of kernel methods. The premier example of such methods is the support vector machine. When introduced to these algorithms, one is taught that one must provide the algorithm with a kernel function that, intuitively, computes a degree of “similarity” between the objects you are classifying. In practice, one can get pretty far with only this understanding; however, to understand these methods more deeply, one must understand a mathematical object called a reproducing kernel Hilbert space (RKHS). In this post, I will explain the definition of a RKHS and exactly how they produce the kernels used in kernel methods thereby laying a rigorous foundation for a deeper understanding of these methods.