The Five-Layer Cake of AI

5. Applications

Applications are the products people use to get work done with AI: an assistant in your email, a tool that sorts incoming claims, a chatbot on your website, or a platform that helps scientists design drugs.

“At the top are applications, where economic value is created.”

Software companies work here, and so does your own company whenever a team builds AI into a process or sets up an AI tool for its own work.

Your company competes in this layer. A competitor can rent the same four layers below you, so your advantage comes from the problems you choose, the data only you hold, and the way your people work. The AI + Human page shows how to put that material into the work.

4. Models

A model is the trained AI system that turns a request into an answer. A model learns patterns from a very large amount of example material, such as text, images or lab results.

“Language models are only one category.”

Huang lists models that work with biology, chemistry, physics, finance, medicine and the physical world, as well as with language.

OpenAI, Anthropic, Google and Meta build general-purpose language models. Other companies build models for one field, such as weather forecasting or drug design.

Models improve every few months, and the best model for a task can come from a different company next year. Prefer AI tools that let you change the model behind the tool without redesigning your process. The right model for a fraud check may not be a chatbot at all.

3. Infrastructure

Infrastructure is the buildings and equipment that hold the chips: the land, the power lines, the cooling, the construction and the cables that connect thousands of computers so they work as one. These buildings are called data centres. Most companies rent time in someone else’s data centre instead of owning one, and the companies that rent out that time are called cloud providers.

Huang calls these buildings AI factories and writes that the buildings are made for producing answers, not for storing files:

“They are not designed to store information. They are designed to manufacture intelligence.”

Cloud providers such as Microsoft, Amazon and Google work here, along with the builders, cooling suppliers and property firms that put up data centres.

The data centre’s location decides which country’s laws apply to the information you send to an AI tool. For a regulated firm, such as an insurer or a bank, ask every AI vendor where your data is processed and stored.

2. Chips

Chips are the processors that turn electricity into calculations. Most AI work runs on chips called graphics processing units, which do thousands of small calculations at the same time.

“Progress at the chip layer determines how fast AI can scale and how affordable intelligence becomes.”

NVIDIA, AMD and Google design AI chips. TSMC, a manufacturer in Taiwan, makes many of the most advanced chips for those designers.

If chip progress makes AI cheaper, as Huang expects, the price of the same AI task should fall during a long contract. Before you sign a multi-year AI contract, ask the vendor whether your price will fall as the vendor’s own costs fall.

1. Energy

Energy is the electricity that runs every computer in the layers above. AI uses a great deal of electricity because every answer takes a very large number of calculations.

“Energy is the first principle of AI infrastructure and the binding constraint on how much intelligence the system can produce.”

Power utilities and power plant builders work here, along with the grid operators and electricians who connect new data centres to the grid.

You will not buy power for AI directly, but the cost of power reaches you through every layer above. Huang writes that “every successful application pulls on every layer beneath it, all the way down to the power plant that keeps it alive.” A shortage of power or chips can raise your AI prices or slow a vendor’s plans to serve you.