Nine layers between electrons and tokens
Following an AI request from the application down to the power supply.
An AI assistant appears to answer from a box on a screen. Behind that box are a model, software that runs it, chips that execute the calculations, and a physical site that supplies power and cooling. A slow answer, an expensive service, and an unavailable model can each have a different cause. It helps to know where to look.
I use nine layers to separate those jobs. The boundaries are a convenience: a real system can combine several of them, and a company can operate across the whole map. This is a way to ask better questions about a system, rather than a complete account of the industry.
From a request to a calculation
L8: Applications. The application decides what the user can do: ask a question, edit a document, search a collection, or take an action through a connected tool. Its permissions, interface, and treatment of errors shape the experience alongside the model's capabilities.
L7: Serving. Serving software takes a request and runs the model. It manages resources and can group requests or reuse parts of earlier computation. The vLLM documentation provides a practical example. Throughput matters here, but a user may care more about how long the first answer takes. Those are different measures.
L6: Models. A model combines a computational design with parameters learned during training. Training choices and data affect what it can do. A product name alone does not reveal the model's architecture, training cost, or suitability for a particular task.
L5: Frameworks. Frameworks help developers describe and train models. They provide operations on arrays of numbers, calculate gradients, and manage execution on available hardware. The PyTorch introductory workflow follows these steps from data to a trained model.
L4: Kernels. A kernel is a program executed on the GPU. Libraries supply kernels for operations such as matrix multiplication; higher-level software calls them. Moving data efficiently can matter as much as performing arithmetic. The CUDA programming guide explains how that execution is organised.
The physical system
L3: Systems. A chip operates inside a larger arrangement of servers, network connections, storage, power distribution, and cooling. When a task uses many devices, useful work depends on how they exchange data and recover from failures.
L2: Chips. An accelerator's computing units need memory and connections that can keep them supplied with data. The balance depends on the workload. A headline calculation rate cannot, on its own, tell a buyer how quickly a complete application will run.
L1: Silicon manufacturing. Chip designs must become physical devices, then be packaged and tested. Foundries, equipment suppliers, and packaging processes belong here. TSMC's description of CoWoS illustrates the role of packaging in connecting components.
L0: Energy. All this equipment requires electricity and a site capable of supporting it. Power availability, cooling, and connection arrangements therefore belong in the same planning conversation as hardware. Their importance varies by location and project; there is no single bottleneck that explains every deployment.
What a vertical map leaves out
Data, evaluation, and operating practice cut across the layers. A model can run successfully on expensive hardware and still answer the wrong question. A useful demonstration may rely on documents that the eventual operator cannot access. A tool can complete an action that the user never authorised.
These problems require different evidence. Test the answers against the intended task, establish the rights and permissions attached to the data, and inspect what the application actually does. Capital, skills, regulation, and environmental constraints also affect which systems can be built and maintained.
Reading the dependency graph
The companion graph uses a deliberately simple rule: when a node fails, every node downstream of it is marked as affected. Selecting a memory or power scenario makes the connections visible.
A real supply disruption would be more complicated. Existing equipment, inventories, alternative suppliers, spare capacity, and recovery time could all change the outcome. A manufacturing interruption would not instantly switch off every installed chip. The graph omits those distinctions, so its counts must not be read as measured losses or probabilities.
For a practical decision, start with the application and work down only as far as the question requires. A slow document search might need better retrieval or serving. A proposed computing facility needs a much deeper account of equipment, utilities, and operation. The value of the map is in making those dependencies easier to examine.