Good-enough AI and the commoditization of intelligence

The demand for the latest Nvidia GPUs seems insatiable. The AI race is driving computation at a massive scale. Will it continue?  Or is AI on a path to commoditization?

My prediction is that, for many real-world tasks, human-level intelligence is "good enough". That is, most jobs today are done by humans and humans have "bounded" intelligence. Thus, virtualizing jobs done by humans requires human-level intelligence and not super-human.

It seems completely plausible to me that we will relatively soon (if not already) have open-source language models that are "good enough" in that they replicate human-level understanding and linguistic performance. The data to train these is already there and they only need to be trained once.

A good-enough LLM is not a bad LLM. It's good enough to solve a business problem. Such a model may need fine tuning but this is relatively inexpensive. 

If this thesis is correct, it will drive down the demand for GPUs to train large models from scratch. 

Intel Inside

It is worth looking at other examples where "good enough" has changed industries. In the 1990's and early 2000's, everyone wanted a computer with "Intel Inside". Software was becoming more sophisticated at a rate that constantly pushed the boundaries of existing chips. We all needed more compute power.

This coincided with Intel driving Moore's law through feats of physics that consistently put more transistors on a chip. New software made computers sluggish and each new Intel processor was greeted with fanfare and wide press and public attention.

Then the computer caught up to the software and surpassed it. People mostly use their computers to browse the web, watch videos, read email, etc. The computational load of these tasks is bounded. As compute demand saturated, existing chips became "good enough". The pressure to upgrade subsided and the slogan "Intel Inside" lost its power.

Artificial good-enough intelligence (AGEI)

AI is, in multiple senses, the new computer. If human-level performance is good-enough, then the Intel moment will come where capacity meets demand. 

That doesn't mean that training specialized AI models and super-intelligent models will stop. This will continue but the majority of applications that people care about will be satisfied by "commodity AI". 

When we reach this point, the demand for training new models will decrease.  This will be matched by an increase in compute at inference time.

The future is inference

We will reach a tipping point where the incremental value of training large models decreases. If this coincides with good-enough performance, then demand for compute will shift to users.

Inference-time compute will get pushed out to the end user to reduce costs to providers. This may cause a new "Nvidia Inside" or "AMD Inside" phase where user devices (phones and laptops) need the latest generation of inference-oriented chips. 

Overall, I predict that the landscape for compute will shift significantly in the next few years. If the demand shifts from "big compute" to "edge compute" it will open up the hardware playing field in exciting ways.