Matthew N. Bernstein

Matthew N. Bernstein

Principal Machine Learning Scientist at Cellular Intelligence

Project planning in the early years of AI requires a shift in mindset

4 minute read

Published:

THIS POST IS CURRENTLY UNDER CONSTRUCTION

Not only is the rapid increase of AI capabilities changing _how we work, it is also changing how we think about work. In this post, I will highlight an important way in which I have changed my mindset when it comes to planning and prioritizing work. In the old days (all of 8 months ago at the time of this writing), the key question one needed to answer when it came to planning a project was how much could one produce given the available resource. In the age of AI, the calculus has changed; generating output is extremely cheap. But generating high quality output is still expensive. We now must consider two dimensions rather than one: quantity and quality. Any given project must be defined with respect to both dimensions simultaneously._

Introduction

Before AI, when planning a project, the key dimension to be considered was quantity

In the old days (all of 8 months ago at the time of this writing), when someone went about planning a project, they would estimate how big of a project they could complete given the resources allocated to them. By “project” here, I mean any kind of knowledge-based product that someone or some team needs to produce: an article, a computer program, an image, etc. For most professionals, or experts in their field, the quality of work is often relatively stable. Therefore, if you ask an expert to produce a project

If you ask a professional software engineer, or team of engineers, to create a program, the question would rarely be, “How robust should the program be?” Or, “How clean should the code be?” Rather, the questions would be more like, “How many features should this product have?” If you ask a data scientist to perform a data analysis, the question would rarely be, “How correct should these conclusions be?” Instead, it would be more along the lines of, “How many questions do we seek to answer?”

Because quality, in most cases, is a relatively stable property of the output of human experts, the dimension most often considered was that of quantity. That is, the relationship between effort and quantity was, at a rough pass, linear: The more work I put in, the more stuff I can accomplish. This is illustrated schematically below:

drawing

This mental model now requires revision in the age of AI.

In the age of AI, we now must consider a second dimension: quality

As of the time of this writing (September 13, 2026), AI models are capable enough to perform most knowledge work at a reasonable level; however, for the vast majority of tasks, models still lack the nuanced judgement and taste that an expert would possess. They miss the point, write with unnatural syntax, add features that are not necessary, or make assumptions that are not obvious, but that affect the result. That is, without human guidance, much of what they produce is slop.

Unfortunately, closing the gap between what an AI produces and what an expert deems to be at their acceptable level of quality still takes a lot of effort to close! I find this in my own work. As a machine learning scientist and computational biologist, I am now using AI to create analyses, write reports, and create presentations. In my personal experience, the quality of the output is somewhere in the ballpark range of 75% of what I would produce (outside of sheer of ability to write code and do math). I find that it takes quite a lot of time to tweak the writing, to remove unnecessary features in its code, to clean its figures, etc.

The calculus now has completely changed. No longer is quantity the dimension along which I plan, but more often than not, it is quality. Generating many, many projects at 75% quality is extremely cheap. But closing the quality gap is still quite expensive.

Tags:

You May Also Enjoy

A framework for making sense of metrics in technical organizations

17 minute read

Published:

If you work in a quantitative or technical field, there is little doubt that you or your team has worked long and hard to define which metrics to measure and track. Using data-driven metrics is a critical practice for making rational decisions and deciphering truth in a complex and noisy world. However, as others have pointed out, an over-reliance on metrics can lead to poor outcomes. In an effort to better articulate the value and risks inherent in metrics, in this blog post, I will present a mental framework for thinking about metrics that has helped me reason about their value and risks with a bit more clarity.

Understanding attention

33 minute read

Published:

Attention is a type of layer in a neural network that is widely regarded to be one of the most important breakthroughs that enabled the development of modern AI systems and large language models. At its heart, attention is a mechanism for explicitly drawing relationships between items in a set. In natural language processing, the set being processed are words (or tokens) and attention enables the model to relate those words to one another even when those words lie far away from eachother in the body of text. In this blog post, we will step through the attention mechanism both mathematically and intuitively. We then present a minimal example of a neural network that uses attention to perform binary classification in a task that is not solveable using a naïve bag-of-words model.

Demystifying Euler’s number

16 minute read

Published:

Euler’s number $e := 2.71828\dots$ has, to me, always been a semi-mysterious number. While I understood many facts about $e$, I never felt I ever truly understood what it really was – it’s core essence so to speak. I believe that part of the reason for my confusion is that $e$ is often taught coming from two seemingly different perspectives: Either it is introduced in the context of compound interest or it is introduced in the context of calculus as being the base of the exponential function whose derivative is itself. Thanks to an excellent explanation by Grant Sanderson’s 3Blue1Brown video, I now better understand this constant and how these two perspectives relate to one another. In this blog post, I will attempt to describe, in my own words, my understanding of Euler’s number and expound on Sanderson’s explanation.

Reproducing kernel Hilbert spaces and the kernel trick

22 minute read

Published:

If you’re a practitioner of machine learning, then there is little doubt you have seen or used an algorithm that falls into the general category of kernel methods. The premier example of such methods is the support vector machine. When introduced to these algorithms, one is taught that one must provide the algorithm with a kernel function that, intuitively, computes a degree of “similarity” between the objects you are classifying. In practice, one can get pretty far with only this understanding; however, to understand these methods more deeply, one must understand a mathematical object called a reproducing kernel Hilbert space (RKHS). In this post, I will explain the definition of a RKHS and exactly how they produce the kernels used in kernel methods thereby laying a rigorous foundation for a deeper understanding of these methods.