The Future of AI Chips: Seven Hardware Trends After the GPU (2026 to 2030)
Short answer: the GPU is no longer the only serious way to run AI. Our review of public specifications and reports from 2023 to 2026 finds seven hardware directions now competing or complementing it: custom chips from the cloud giants, ever-larger multi-die GPUs with HBM4 memory, SRAM-based inference chips like Groq's LPU, wafer-scale processors like Cerebras', memory-centric designs like IBM's NorthPole, photonic chips that compute with light, and brain-inspired neuromorphic systems like Intel's Hala Point.
The first four are already shipping. Nvidia struck a licensing deal with Groq on Christmas Eve 2025, reported at about $20 billion. Cerebras raised about $5.55 billion in a May 2026 IPO. Google, Amazon and Microsoft custom chips are all reported on 3-nanometre-class processes. The last three remain mostly in research, but they target the same problem: AI is limited less by arithmetic than by moving data.
We reviewed 14 sources, including company specifications, peer-reviewed results reported by the press, IPO coverage and industry analysis, to explain each trend, the numbers behind it and how close it is to everyday use.
Key findings
- Memory is the battleground. Almost every new design changes where memory sits or how fast it moves. Nvidia's Rubin GPU moves to 288 GB of HBM4 at 22 TB/s, while Groq's LPU and Cerebras' wafer keep data in on-chip SRAM.
- Custom chips are now mainstream. Google's TPU v7 Ironwood, AWS Trainium3 and Microsoft Maia 200 are all reported on 3nm-class processes, with 144 GB to 216 GB of HBM each.
- GPU makers are absorbing alternatives. Nvidia now positions the Groq LPU as a complement to its GPUs for the decode stage of inference, and AWS says Trainium4 will support Nvidia's NVLink Fusion.
- Speed records come from SRAM. Artificial Analysis measured Cerebras at more than 2,200 tokens per second on GPT-OSS 120B, about 2.8 times the next GPU cloud, according to The Register.
- Efficiency breakthroughs are still in the lab. IBM reported NorthPole as 25 times more energy efficient than a comparable GPU on an image benchmark, and Intel reported Hala Point at 15 trillion operations per watt in early results.
How we reviewed the evidence
This is a review, not a benchmark. We gathered vendor specifications, conference and journal results as reported by technology media, IPO and deal coverage, and analyst summaries. Specifications use each vendor's own precision and sparsity assumptions, so peak numbers from different companies are not directly comparable. Where a figure comes from a source with a commercial interest, such as a GPU rental company, we note it.
Why chips are changing: the memory problem
Large language models generate text one token at a time. For each token, the chip must read a large share of the model's weights from memory. As we explained in The Memory Wall, this makes response speed depend mostly on memory bandwidth, not raw compute. Every trend in this review is, at heart, a different answer to one question: where should the memory go?
| Approach | Where the data lives | Trade-off |
|---|---|---|
| GPU and custom chips | HBM stacks beside the chip | Large capacity, limited bandwidth |
| LPU and wafer-scale | SRAM on the chip itself | Very high bandwidth, small capacity per chip |
| Memory-centric | Memory woven into the compute | Very efficient, but models must fit on chip |
| Photonic and neuromorphic | Different physics or computing models | Big efficiency promise, early stage |
Trend 1: Custom chips from the cloud giants
The biggest cloud companies now design their own AI accelerators and run much of their own AI on them. A June 2026 comparison by Spheron, a GPU rental company, summarises the current generation:
| Chip | Memory | Bandwidth |
|---|---|---|
| Google TPU v7 Ironwood | 192 GB HBM3E | 7.37 TB/s |
| Microsoft Maia 200 | 216 GB HBM3e | 7 TB/s |
| AWS Trainium3 | 144 GB HBM3E | 4.9 TB/s |
- Google Ironwood delivers 4.6 PFLOPS of FP8 compute per chip, slightly above the 4.5 PFLOPS cited for Nvidia's B200, and a 9,216-chip pod reaches 42.5 exaFLOPS, according to a RAISE Summit analysis. The same source reports that Anthropic committed in December 2025 to deploying up to one million TPU v7 chips, with more than 1 GW of capacity.
- AWS Trainium3 was announced on 2 December 2025 on a 3nm process. TechCrunch reports AWS's claims of more than 4 times the performance of Trainium2, 40% better energy efficiency, 144 chips per UltraServer and up to one million chips linked together.
- Microsoft Maia 200 is built on TSMC 3nm with native FP8 and FP4 support and serves Copilot and OpenAI traffic inside Azure.
- Meta's MTIA line runs Meta's ranking and recommendation workloads, with MTIA 300 in production and later versions in testing or on the roadmap.
Spheron also cites a projection that Nvidia's share of inference could fall from about 90% to 20 to 30% by 2028, mainly because hyperscalers move their internal workloads to their own chips. Treat that as one projection from a company in the GPU market, not a consensus.
Trend 2: Bigger GPUs built from multiple dies, with HBM4
GPUs are not standing still. Nvidia's Rubin, due in the second half of 2026, puts two reticle-sized GPU dies in one package and pairs them with 288 GB of HBM4 at 22 TB/s, according to specifications compiled by Glenn Lockwood from Nvidia's announcements. The headline inference figure is 50 PFLOPS of FP4 with sparsity. For comparison, Spheron lists Nvidia's B300 at 288 GB of HBM3e and 8 TB/s.
Joining several dies in one package, often called a chiplet approach, gets around the size limit of a single die. Faster HBM4 memory raises bandwidth. Together, they are the GPU's answer to the memory wall.
Trend 3: SRAM inference chips (the LPU)
Groq's Language Processing Unit takes the opposite approach: keep the model in fast SRAM on the chip instead of in external HBM. On Christmas Eve 2025, Nvidia agreed a licensing deal with Groq, reported at about $20 billion, and hired its leadership, including founder Jonathan Ross, a former Google TPU designer, according to SDxCentral. Groq remains independent and keeps running its GroqCloud service.
The same report describes the Groq 3 LPU as follows:
- 500 MB of SRAM per chip, and 128 GB across a 256-chip LPX rack.
- 22 to 150 TB/s of bandwidth per chip, according to Nvidia's Ian Buck, far above a GPU's memory bandwidth for its capacity.
- Positioned as a complement to GPUs: GPUs hold large models in HBM, while LPUs speed up the token-by-token decode stage. Nvidia expects the LPU in the second half of 2026.
Trend 4: Wafer-scale processors
Cerebras builds one chip from an entire silicon wafer instead of cutting it into hundreds of dies. TweakTown reported the WSE-3 as 57 times larger than the largest GPU, with 4 trillion transistors. The Register lists it on TSMC's 5nm process with 44 GB of on-chip SRAM and 21 PB/s of memory bandwidth.
The payoff is speed. Artificial Analysis measured Cerebras at more than 2,200 tokens per second on GPT-OSS 120B, about 2.8 times faster than the next GPU cloud, as reported by The Register. Cerebras listed on Nasdaq on 14 May 2026, raising about $5.55 billion and closing its first day valued at more than $66 billion. Named inference customers include AWS, Meta, Mistral AI, Notion and Perplexity, and The Register reports OpenAI was added as a customer.
Trend 5: Memory-centric chips
IBM's NorthPole, published in Science in 2023, removes external memory altogether. AI Business reports that all of the chip's memory sits on the chip itself, avoiding the constant shuttling between memory and processor known as the von Neumann bottleneck. Built on a 12nm process with 22 billion transistors, NorthPole was reported to be 25 times more energy efficient and to have 22 times lower latency than a comparable 12nm GPU on the ResNet-50 image benchmark.
The catch is capacity. NorthPole cannot reach external memory, so larger networks must be split across several chips. That limits it, for now, to models that fit, which is why this approach is better suited to edge devices and specialised inference than to the largest language models.
Trend 6: Photonic computing
Photonic chips use light instead of electrons for some of the computation, especially matrix multiplication, which dominates AI workloads. RAISE Summit reports that a Lightmatter system demonstrated in January 2025 ran ResNet and BERT models at 65.5 trillion operations per second while drawing 79.6 W.
The promise is lower energy per operation and less heat. The open questions are precision, converting between light and electronic signals, and manufacturing at scale. Light already carries data between servers through optical fibre, and computing with light is the next step researchers are pursuing.
Trend 7: Neuromorphic, brain-inspired chips
Neuromorphic chips imitate how neurons fire, computing only when something changes. In April 2024, Intel unveiled Hala Point, the largest neuromorphic system reported so far. Live Science reports it combines 1,152 Loihi 2 processors with 140,544 cores, simulating 1.15 billion neurons and 128 billion synapses. Early results showed 15 trillion operations per watt, and Intel claims up to 50 times faster and 100 times more energy-efficient AI workloads than CPU and GPU systems, figures that come from a preprint not yet peer reviewed.
Earlier systems show the efficiency idea: IBM's TrueNorth ran 1 million neurons on about 70 milliwatts for image classification, according to RAISE Summit. Neuromorphic chips are strongest for always-on sensing and event-driven tasks. They do not yet run today's large language models.
How close is each trend to everyday use?
| Trend | Example | Stage (2026) |
|---|---|---|
| Custom cloud chips | TPU Ironwood, Trainium3, Maia 200 | Deployed at scale |
| Multi-die GPUs with HBM4 | Nvidia Rubin | Shipping from 2H 2026 |
| SRAM inference chips | Groq LPU | In service, entering Nvidia systems |
| Wafer-scale | Cerebras WSE-3 | In service, publicly listed company |
| Memory-centric | IBM NorthPole | Research and early products |
| Photonic computing | Lightmatter | Demonstrations |
| Neuromorphic | Intel Hala Point | Research systems |
What this means for India
- Design, not fabrication, is the realistic entry point. As our review of Made in India chips found, India's first fab at Dholera will start mainly at 90nm in 2028, while the leading AI chips in this review use 3nm to 5nm processes. C-DAC's GPU programme targets specialisation around 2029.
- Specialised inference chips are a better fit than general GPUs. A smaller, inference-focused design, the direction Groq and NorthPole took, is more achievable for a new chip programme than competing with Nvidia's flagship GPUs.
- Developers benefit first. More hardware choice means more providers, faster responses and lower prices per token. Indian developers can use models served on GPUs, TPUs, LPUs and wafer-scale chips through APIs today, without buying any hardware.
- Energy matters more in India. With data centre power demand expected to rise sharply, as we found in India's AI Data Centres in 2026, chips that do more work per watt will shape where and how AI runs in the country.
Frequently asked questions
What will replace GPUs for AI?
No single chip is replacing GPUs. The sources we reviewed show GPUs evolving, with multi-die designs and HBM4, while custom chips such as TPUs and Trainium, SRAM-based LPUs and wafer-scale processors take over specific workloads, especially inference.
What is the difference between a TPU and a GPU?
A GPU is a general-purpose parallel processor that Nvidia and others sell widely. A TPU is Google's custom AI accelerator, designed for its own models and offered through Google Cloud. Google's TPU v7 Ironwood delivers 4.6 PFLOPS of FP8 compute per chip, with 192 GB of HBM3E memory.
What is an LPU?
An LPU, or Language Processing Unit, is Groq's inference chip. It keeps model data in on-chip SRAM rather than external HBM, giving 22 to 150 TB/s of bandwidth per chip. Nvidia licensed Groq's technology in December 2025 and plans to pair LPUs with GPUs.
Why is Cerebras so fast?
Cerebras uses a whole silicon wafer as one chip, with 44 GB of on-chip SRAM and 21 PB/s of memory bandwidth. Artificial Analysis measured it at more than 2,200 tokens per second on GPT-OSS 120B.
What is photonic computing in AI?
Photonic computing uses light to perform calculations such as matrix multiplication. A Lightmatter system ran ResNet and BERT at 65.5 trillion operations per second on 79.6 W, according to reports, but the technology is still at the demonstration stage.
What is a neuromorphic chip?
A neuromorphic chip imitates how biological neurons fire, computing only when inputs change. Intel's Hala Point system simulates 1.15 billion neurons using 1,152 Loihi 2 chips. It is used for research and is not yet used for large language models.
Will India make its own AI chips?
India is designing them. C-DAC's GPU programme targets specialisation around 2029, but production of advanced AI chips would happen abroad, because India's first fab will start mainly on 90nm technology in 2028.
Limits of this review
- We did not benchmark any hardware. All figures come from vendors, researchers and the press, as listed below.
- Peak figures are not comparable across vendors. Precision, sparsity and system size differ between specifications.
- Some figures come from interested parties. The chip comparison and market-share projection come from a GPU rental company, and Hala Point's efficiency claims come from a preprint.
- Deal values are reported, not confirmed. The roughly $20 billion Nvidia and Groq figure is a reported estimate.
References
- The post-GPU era: the silicon horizon, RAISE Summit, February 2026
- Hyperscaler custom AI chips in 2026, Spheron, June 2026, updated October 2026
- Amazon releases an impressive new AI chip and teases an Nvidia-friendly roadmap, TechCrunch, 2 December 2025
- Nvidia Rubin (R200), Glenn K. Lockwood, April 2026
- Nvidia bets big on bandwidth with Groq 3 LPU to complement GPUs, SDxCentral, 2026
- Nvidia's $20 billion Groq deal, Fintool
- Cerebras' wafer-scale AI bet delivers blockbuster IPO, The Register, 15 May 2026
- Cerebras WSE-3: 57x bigger than the largest GPU with 4 trillion transistors, TweakTown
- IBM's new AI chip mimics the human brain and outperforms GPUs, AI Business, 26 October 2023
- Intel unveils largest-ever AI neuromorphic computer, Live Science, 18 April 2024
- Intel's Hala Point has 1.15 billion neurons, ZDNet
- Meta's MTIA chip lineup joins hyperscaler push, Tom's Hardware
- India can build indigenous GPU by 2029, says C-DAC Bengaluru chief, Analytics India Magazine, August 2026
- Tata's India fab reportedly to launch at 90nm, TrendForce, 20 July 2026