Two rack announcements landed on the same day and they agree on something I did not expect. AMD introduced Helion, an agentic AI rack with a CPU-only configuration: 6th Gen EPYC processors, up to 32,768 cores in one rack, aimed at what AMD calls agent execution. NVIDIA published a Vera CPU rack specification through MGX at the OCP Global Summit in San Jose, also aimed at the agent rather than the model.
The proposal is that these racks run the agent, not the model. The model stays on separate inference hardware.
For two years, every agent question turned into a GPU question. How many, how fast, how many tokens per second per watt. The racks announced on October 12 are a different purchase order. The CPU-only Helion has no accelerators in it at all.
Where the agent actually runs
Watch one agent loop and time the steps. A model call is a request. Something has to compose it, pick a model, send it, wait, read the response, call the tool the model asked for, write the file, inspect the output, and choose the next move. That something is a process. It needs a core, memory, a filesystem and a socket. Mostly it needs to wait.
The GPU is busy during the model call. The work around it is CPU work. Orchestration, tool calls, sandboxes, file I/O, retries, timeouts. The vendors now say this in their own words. AMD says the CPU-only rack provides “dedicated capacity for agent execution and connections to separate model inference infrastructure.” NVIDIA says its Vera CPU rack is designed to support tool calling, orchestration and sandboxed execution.
The model moved out of the box. The agent stayed.
What AMD announced
Helion is an open blueprint based on an Open Compute Project spec design. AMD lists a few shapes. The CPU-only rack AMD proposes: 6th Gen EPYC, up to 32,768 cores, for hyperscale customers. An all-in-one rack that mixes EPYC nodes, Instinct MI350P GPU nodes and general purpose compute for enterprises. Cooling comes in liquid or air versions. AMD says Helion agentic rack configurations are in development with partners and planned for November 2026, and AMD plans to contribute the specification to OCP.
The all-in-one is the safe enterprise pitch. The CPU-only rack is the one that says something. AMD does not put up to 32,768 cores and no accelerators in a blueprint unless it believes a large share of agent compute is not matrix multiplication.
The other piece is the router. AMD says Helion combines open hardware and software with an integrated token router that directs agents’ model calls across frontier and open-source models. That is the whole documented claim.
My guess about the token router
Everything past that sentence is my guess. I think the router holds the per-step model choice, a cheap open model for a classification and a frontier model for the hard turn. It holds the retry and failover logic for when a provider goes slow. It holds the budget that stops a loop from burning a month of quota in an hour. Centralizing that would mean agents stop carrying provider credentials and every team stops rebuilding the same routing table. It would also put one component in front of every model call an agent makes. Fine place for a spend cap. Bad place for an outage.
What NVIDIA announced
NVIDIA’s entry is a design specification, not a shipping rack. The Vera CPU rack extends MGX, NVIDIA’s open rack architecture, beyond accelerated computing. It is built on the third-generation MGX rack design and aimed at the agentic era while Vera Rubin NVL72 keeps doing training and large-scale inference.
The MGX 1.2 update carries the plumbing. NVIDIA says it adds rack-scale blind-mate liquid cooling, a liquid-cooled 5,000A busbar, and support for rack deployments up to 250kW. More power per rack, in a standard other companies can build against.
The two announcements are not the same shape. AMD is publishing a blueprint. NVIDIA is publishing a specification. They land on the same thesis anyway. If agents are going to run continuously, the execution layer deserves hardware of its own instead of whatever cores are left over next to the GPUs.
The inference side got faster too
Microsoft published Maia 300. It is the other half of the picture. Microsoft says Maia 300 puts 96 accelerators in a 167 kW rack, each with eight HBM4 stacks, 288GB of memory and 18TB/s of bandwidth. That configuration, Microsoft says, delivers up to 7x the rack-scale token throughput and more than 2.5x the performance per watt of Maia 200.
That is the inference rack, and it got better at inference. It did not get better at orchestrating a sandbox, retrying a flaky tool, or holding a file handle open. The split holds.
The shape I think we are building
Put the two racks next to each other and the architecture has a seam down the middle. My guess at the wiring, since the vendors describe the parts and not the seam.
The CPU rack owns the loop. It plans, calls tools, runs sandboxes, touches the filesystem and holds state while it waits. The GPU rack takes a model call, runs prefill and decode, and hands tokens back. The router sits between them and decides which model answers each call. The split puts the agent on the CPU rack. The accelerator only finishes one sentence of the agent’s work.
Once you see that split, the cost model changes. Tokens per second stops being the number that describes an agent fleet. Cores, sockets and open connections start to. A slow model makes an agent late. The token bill still lands, and waiting ties up a sandbox, memory and a connection on the CPU rack. Waiting for a model does not mean a CPU core is busy. Running a tool does not reserve the inference rack either; it can serve other runs.
What I would watch
The CPU-only Helion shipping in November. AMD says configurations are planned then, which is not the same as promised.
Where the router policy lives. Open and customer-controlled makes the split real. A vendor console makes the split a subscription.
Whether the CPU rack comes with a credential story of its own. A dedicated rack full of long-lived sandboxes is also a dedicated place to accumulate over-permissioned agents. Cheap execution and loose permissions are a bad pair.
I have been sizing agent fleets in tokens per second for as long as I have been building them. The hardware announced this week suggests I should size them in cores and sockets, and treat tokens as a utility the execution layer plugs into. Agents are a CPU workload now. The GPU is where the sentence gets finished.