The CPU Era of AI: Why I Am Not Buying a GPU in 2026

The CPU era is real, but only for half my stack

I watched Zen van Riel's "The CPU Era of AI Has Begun" this week and nodded through most of it. The diagnosis is right. When ChatGPT shipped in 2022, the workload was effectively single-pass: tokenize, generate, return. Today, in a serious agentic loop (cursor-agent running a build, a Claude Code session shelling out to Playwright, a Telegram bridge orchestrating three MCP servers), the GPU is doing roughly one thing per turn. The CPU is doing everything else.

What the framing misses for a solo builder like me is that "CPU era" describes only the top half of the stack. The tool-call layer is genuinely on its way back to the CPU. The actual large language model is not. That part is still firmly GPU-gated, and right now it lives on someone else's hardware. I am OK with that. I am not buying a GPU this year, and I am going to spend the rest of this post explaining why, including a concrete test I ran on a $9 VPS that died exactly the way the trend says it should die.

This is a builder note, not a benchmark roundup. It is the view from one engineer's writing room, where the agent stack has to work on a laptop and a small server, not on a rack.

Two layers, two truths

The cleanest way to think about an agentic stack in 2026 is two layers, with two separate hardware curves.

The tool-call layer. Cursor-agent calling a shell script. Playwright driving a real browser to fill a form. n8n routing a webhook. A Python process parsing JSON, walking a database, writing a file. None of that touches a GPU. All of it is CPU plus a bit of disk and a bit of network. This is the layer the Intel and Nvidia data is screaming about. Intel called out a CPU to GPU ratio tightening from 1 to 8 toward 1 to 1 in agentic scenarios on the Q1 2026 earnings call. Arm puts the structural growth at roughly 4x more CPU cores per gigawatt as workloads shift from "LLM serving" to "agents doing work."

The LLM layer. A 7B, a 30B, or a 70B parameter model doing the actual thinking. That layer's bottleneck is not core count. It is VRAM, memory bandwidth, and the matrix-multiply throughput of whatever silicon is sitting under the model. This layer has not moved. If anything, the gap between "model that is smart enough to drive an agent" and "model that fits in 16 GB of consumer VRAM" got worse in the last twelve months, not better.

The reason this distinction matters: most of the "CPU era" talking points are correct, but the implicit conclusion is wrong. The conclusion people keep drawing is "buy more CPU." For an agency, the truer conclusion is: keep the tool-call layer on your own hardware, and keep paying someone else for the LLM layer until the economics of running it locally actually flip.

What the trend looks like from inside one engineer's writing room

My writing room is small, deliberately. The Telegram bridge that lets me drive cursor-agent from my phone is one Python process. The Cursor IDE sits on my Mac. The agent itself spawns shell commands, runs git, runs the test suite, calls MCP servers for browsing and search, and PATCHes posts into Payload. Most days, the bridge plus the dev stack plus the agent's tool calls peg one or two cores hard. None of it touches a local model. Every meaningful LLM call goes out over the network to Anthropic or OpenAI.

Read that again, because it is the actual operating reality of an agentic stack in 2026: the part of the system that is "yours" is mostly CPU. The part that costs the most per call is somebody else's GPU.

The asymmetry has consequences. It means when I am sizing hardware for the writing room, I am sizing for parallel agentic loops (one Cursor session, one cursor-agent run, one Playwright browser, two MCP servers, n8n's worker process, a Payload dev server, a Postgres), not for model inference. Every one of those processes is a CPU plus RAM plus disk story. None of them needs CUDA.

A lot of builder content I read talks about local LLMs like they are the default and the cloud is the special case. In my actual writing room, the opposite is true. The cloud LLM is the default. The local model is the experiment.

I tried to self-host an LLM on a $9 VPS

I wanted to know what happens when you push the second half of the trend (LLM moves to your hardware) onto a cheap cloud server. So I bought the Hostinger KVM 2 plan: 2 vCPU AMD EPYC cores, 8 GB RAM, 100 GB NVMe, 8 TB bandwidth, $8.99 per month on the promo. The kind of box thousands of solo builders run their entire side-project on.

Installed Ollama. Pulled gemma3n:e2b, Google's Gemma 3n effective-2B variant. The model is 5.6 GB on disk with a 32K context window. The "E2B" naming is the MatFormer trick: the raw model is closer to 4.46 B parameters but loads at an effective 2 B memory footprint by offloading low-utilization matrices. It is, by design, the smallest serious open model Google ships for "everyday devices."

The first run was an OOM kill. Box was at roughly 7.2 GB used (Ollama plus the model plus a bit of OS overhead) and Linux's OOM-killer fired the moment a second request hit. Added a 4 GB swap file, retried. This time it ran. Tokens streamed back. The model held a conversation. I could ask it to summarise a paragraph or write a small function and it would do it.

The catch was visible in a single number. Hard 1 concurrency. The moment I tried a second request in parallel, the box went swap-thrashing. Token throughput collapsed. Latency went from "slow but usable" to "I am going to make a coffee." For anything resembling production agentic load (where a single agent run can fire five tool calls in parallel and each one might want a model in the loop), this is unusable.

The point of running the test was not to publish a benchmark. The point was to feel where the wall is, with my own hands. The wall is not CPU. The CPU was bored. The wall is RAM and, by extension, the absence of dedicated VRAM. The smallest serious open model and the cheapest serious VPS cannot meet in the middle.

What "1 concurrency" actually means for an agent

This is the part that does not show up in benchmark posts. A modern agent does not make one model call at a time. A reasonable cursor-agent run on a single instruction might fan out into a planning call, a search-tool call, three "read this file" calls, a code-edit call, and a "summarise what we just did" call. Even if the model only sits in two or three of those, a 1-concurrency limit collapses the whole loop down to serial execution. The wall-clock time for the agent triples or quadruples.

I could throw more RAM at the box and buy a KVM 4 (16 GB) or KVM 8 (32 GB). I could quantize Gemma harder. I could pick a different model. None of those moves change the fundamental observation: at the price point and hardware shape where most solo builders actually operate, the LLM layer does not yet fit on your own machine in a way that supports parallel agentic work.

The honest read on this is not "VPS providers are stingy." It is "open models that are good enough for agent driving are still too large for the hardware most of us own."

The Mac M4 cheat code, and why it cheats

I run the same stack happily on a base M4 Mac (10 cores, 4P + 6E, 16 GB of LPDDR5X unified memory, 120 GB/s memory bandwidth). Same Ollama, same model class, comfortably alongside Cursor, the writing room, and a Postgres. The reason it works is not that the M4 has more compute than the VPS. It is the unified memory architecture.

On a traditional PC, model weights live in dedicated GPU VRAM, separate from system RAM, connected by a PCIe bus. Moving a tensor between the two crosses that bus at roughly 64 GB/s on PCIe 5.0. Apple Silicon collapses the split. The CPU cores, the GPU cores, and the Neural Engine all read and write the same physical memory at 120 GB/s on the base M4 (and much higher on the Max and Ultra variants). For LLM inference, every gigabyte of that 16 GB is effectively VRAM. No copy-to-device step. No bus tax.

A small but important nuance: Ollama, llama.cpp, and MLX all route LLM math through the GPU cores via Metal. They do not use the 38 TOPS Neural Engine, even though it sounds like the right hardware. The ANE is CoreML-exclusive and most LLM model architectures do not convert cleanly to CoreML, so the GPU path wins by default. If you were hoping the ANE was the secret sauce, it is not. The unified memory pool is.

What this gives me, as a builder, is a single $1,000-class machine that can sit on my desk and run a meaningful local model when I want one, and run cursor-agent against a cloud model when I want speed. The base M4 is the only consumer-grade box I have used where the LLM layer and the tool-call layer can comfortably co-exist on the same machine.

That is also the reason I am not panicking about buying a discrete GPU. The Mac already does the small-model job.

Why I am still not buying a GPU

State the position plainly so nobody has to guess. I am not buying a GPU for the agency this year. The reasons, in order:

1. GPUs that matter are too expensive to justify against current token prices. A used RTX 3090 with 24 GB of VRAM is the cheap end and still $700 to $900. A new 4090 or 5090 with 24 to 32 GB is $2,000 plus. Anything bigger (A6000, H100) is not a consumer purchase. At my current LLM bill, the GPU pays for itself in years, not months, even before electricity.

2. The model that fits in 16 to 24 GB of consumer VRAM is not the model that drives my agents. Cursor-agent and Claude Code are pinned to Anthropic's Sonnet / Opus class models for a reason. The 7 B to 13 B open models I can fit on a consumer GPU are good for narrow tasks, not for the multi-step reasoning that an agentic loop actually requires. The gap is closing, but it has not closed.

3. The cloud absorbs the GPU shortage problem for me. I get a stable API price; the providers eat the capex, the supply crunch, the data center deals, the cooling, the spare parts, the model-upgrade migrations. The cloud is doing exactly the thing the cloud is good at: turning a capital problem into an opex problem on someone else's balance sheet.

4. Self-hosting introduces an ops surface that does not help my customers. Model updates, quantization tuning, capacity planning, monitoring, recovery from OOMs, GPU driver drift. None of that ships product. The closest argument I covered in detail was the middleman tax on no-code AI SaaS. The same principle applies one level deeper: paying for managed infrastructure for the things that are not your core product is, more often than not, the right move.

5. The whole bet of agentic infrastructure (which is what FusionSync ships for event companies on top of WhatsApp and Instagram pipelines) is that the value lives in the orchestration, not in the raw model. I keep the orchestration on hardware I own. I let someone else own the model.

If you put those five together, "buy a GPU" is the wrong answer for almost every solo builder I know. The right answer is buy CPU, rent GPU. That is the same recommendation the CPU-era video lands on by accident; this is just the version that names the trade.

When I will buy one

There are conditions under which I flip. I want to be honest about them so nobody reads this as a religious position.

• A 7B-to-13B open model becomes good enough to replace Sonnet-class reasoning in a real agent. Not "good on benchmarks." Good in a 20-step cursor-agent run, with tool calls and retries, on real code.
• Unified memory becomes a 32 GB consumer baseline at the same price point as 16 GB today. Apple, Intel, or AMD; doesn't matter who ships it.
• Quantization stabilises at Q3 or Q2 without losing reasoning quality. Current Q4_K_M is the practical sweet spot. If that floor drops, the same model fits on less hardware.
• A customer with a hard data-residency requirement signs a contract that justifies a dedicated box. This is the most likely trigger near-term.
• Cloud token prices triple. Possible but not the base case. Token prices have been falling, not rising, on a per-task basis for two years.

Hit any one of these and I am back at the GPU page on Newegg. Until then, the box on my desk is enough.

FAQ

Why not just buy a used 3090 and call it a day? A 24 GB 3090 will run 7B to 13B models at decent speed. It will not run the 30B and 70B class models that actually compete with cloud-served Sonnet or GPT for agent reasoning, except at aggressive quantization that hurts quality. For the price of one used 3090 plus PSU upgrades plus electricity, I can pay Anthropic for years. The argument is not "consumer GPUs are bad." It is "consumer GPUs are good at small models that are not good enough yet."

Is Apple's Neural Engine the secret sauce for local LLMs? No. The ANE is impressive at 38 TOPS, but Ollama, llama.cpp, and MLX route LLM math through Metal on the GPU cores, not through the ANE. CoreML conversion does not support most LLM architectures cleanly. The unified memory pool is the actual cheat code, not the ANE.

Why does Hostinger KVM 2 collapse on a 5.6 GB model when it has 8 GB of RAM? Because the model is not the only thing in memory. Ollama needs working space, the OS needs roughly 0.5 to 1 GB, and any concurrent request loads its own context. By the time the second request lands, you are over the cliff. Adding swap rescues the box from crashing; it does not rescue throughput, because swap is disk and disk is slow.

Should I run my agent's model on the same VPS as the rest of my stack? For learning, yes. For production, no, not yet. Treat local-model experiments as experiments. Keep the agent's actual model on a managed endpoint until you have a real reason (cost, residency, latency) to move it. If you need a baseline, the Cursor and Claude Code-style writing room setup on a Mac plus a cloud model is the highest-leverage shape I know.

Will Intel or AMD's unified memory chips catch up to Apple? Probably, eventually. Intel's Lunar Lake and AMD's Strix Halo are pointed at the same architectural idea. None of them ship with Apple's memory bandwidth at consumer prices yet. The day they do is the day "CPU era of AI" becomes a literal statement instead of a half-true one.

Is "the CPU era of AI" overstated? Partly. The top half of the stack (orchestration, tool calls, parsing, RAG, browser automation) is genuinely a CPU story now, and that is a real, durable shift. The bottom half (LLM inference) is not a CPU story. Anyone arguing CPUs are about to displace GPUs for actual model work is selling something. The honest framing is: CPU and GPU both matter again, and the ratio has snapped back closer to 1 to 1 for agentic workloads from the 1 to 8 it was during the pure-training era.

The bottom line

The CPU era of AI is real, but it is half the story. The tool-call layer of any serious agentic stack is moving back to the CPU and will keep doing so. The LLM layer is still gated by GPU memory, bandwidth, and price. For a solo builder running an agency in 2026, the right answer is not to buy a GPU. It is to buy a machine that does the small-model and tool-call work well (a base M4 Mac is the cleanest single answer today), rent the LLM layer from Anthropic or OpenAI, and let the cloud providers eat the capex problem.

• Two layers, two trends. Tool calls trend toward CPU on your hardware. LLM inference stays on someone else's GPU until consumer VRAM catches up.
• A $9 Hostinger KVM 2 runs Gemma 3n E2B only with a swap-file rescue and hard 1 concurrency. CPU is fine; RAM and the lack of VRAM are the wall.
• A base M4 Mac with 16 GB unified memory works because the LLM weights and the OS share the same 120 GB/s memory pool. No PCIe tax.
• Most LLM runtimes route through Metal GPU cores, not the Neural Engine. The unified memory pool is the cheat code, not the ANE.
• I am not buying a GPU this year. Buy CPU, rent GPU. I flip the day a 13B open model is good enough to drive cursor-agent in production.

If you are running an inbound stack on top of WhatsApp and Instagram for a service business and you want to see the same "CPU for orchestration, cloud for model" architecture installed against real customer conversations, the next step is a free 7-day production pilot. You watch the model spend land in Anthropic's bill, not yours, while the orchestration layer runs on the boxes you already pay for.