Micron: AI Growth Now Hits a Memory Cooling Wall, Not Chips

Micron Technology's president and chief technology and product officer, Scott J. DeBoer, delivered the keynote "From Silicon Innovation to System Intelligence: Architecting the Next Wave of AI" at the SEMICON Taiwan 2026 Master Forum and Global Ecosystem Summit, held September 2 at the Taipei Nangang Exhibition Center. His central claim set the tone for the day: AI is now expanding at the speed of memory. Years of engineering gains have squeezed more efficiency out of every bit and more bandwidth out of fixed capacity, DeBoer said, but that still falls well short of what global data-center buildouts demand — memory, he warned, will remain severely undersupplied for years to come.

Bandwidth, capacity, cooling: AI's iron triangle

DeBoer broke memory systems into three dimensions using a car analogy he said he had settled on only that morning. Bandwidth is the engine's horsepower, the first priority that must keep rising to deliver the compute gains people now take for granted. Capacity is the fuel tank: when bandwidth is scarce, systems don't need much of it, but as bandwidth climbs, capacity must grow to match it in a self-reinforcing loop. The third leg, cooling, is the tires — no matter how big the engine or how full the tank, a car with flat tires doesn't move. He said thermal management has changed dramatically over the past year: as partners push bandwidth ever higher together, getting heat out of the system has become the factor that ultimately limits how fast that bandwidth can run. All three, he said, must arrive at once — that is what "AI expanding at the speed of memory" really means.

Memory's own version of Moore's Law, DeBoer argued, is still on an exponential curve. Over his 30 years in the industry, DRAM has moved through 15 process nodes, with growth increasingly coming from architectural reinvention rather than simple geometric shrinking — and several more such shifts lie ahead. Cutting-edge DRAM density is now so high that more than 1,500 DRAM capacitors fit within the width of a single human hair. Scaled at that pace, he said, a 1995 Porsche with 270 horsepower, a $60,000 price tag and 23 miles per gallon would today deliver nearly 3 million horsepower, cost $5, and get 23,000 miles per gallon.

Three unresolved bottlenecks in advanced packaging

• Advanced packaging under lateral pressure: pushing ever more data into logic chips through a fixed number of square millimeters of I/O connections keeps raising the pressure on interconnect density. Wafer-to-wafer bonding, already standard in NAND, must extend to DRAM — but manufacturability and process control still need major work.
• Hybrid bonding's manufacturing maturity — the real bottleneck: hybrid bonding is still used at relatively small scale industry-wide, and die-to-wafer processes, stacking, metrology and bonding physics must all mature considerably before reliable bonding at very high volumes is achievable. High-volume manufacturing capability clearly lags today.
• Cooling moving from an afterthought to structural design: heat generation across modern logic is uneven, so DRAM must now be engineered per customer and application, with physical cooling paths built directly into dies and 3D stacks to move heat from the bottom logic layer up to the cooling interface. DeBoer called this the fundamental challenge for HBM4E, HBM5, HBM5E and vertical-memory bandwidth roadmaps over the next four to five years.

In data centers, HBM sits at the top of the performance pyramid, but the whole system leans on more than HBM alone: high-density DDR and low-power DRAM modules now play a bigger, more complex role moving models in and out of memory and absorbing the huge memory demands of KV caching. High-performance NAND SSDs, dismissed by many as a poor business just four or five years ago, have become an unexpectedly large driver of data-center demand. At the edge, robotics and automotive applications also need high bandwidth, but under very different power and cost constraints — adding further pressure to a DRAM shortage that data-center buildouts have already made severe. Micron has publicly committed roughly $250 billion over the next decade to expand fabs in the United States, Taiwan, Singapore and Japan, but DeBoer stressed that his talk wasn't about that expansion: the most urgent near-term task is squeezing more out of existing capacity through innovation and efficiency.

Memory sets the price of an AI token

Many system bottlenecks won't change soon, DeBoer said, so most of the room to optimize sits in memory bandwidth and how efficiently it gets used. HBM bandwidth gains — from HBM3E to the coming HBM4 and HBM4E — have been the main driver behind falling AI "tokenomics" costs in recent years, a curve that extends to phones, post-phone devices and automotive applications too. The next step is vertical HBM, stacking memory directly on top of the logic chip for far higher bandwidth at lower capacity, suited to bandwidth-hungry inference workloads — though it still depends on hybrid bonding maturing further. Moving memory physically closer to compute, cutting resistance and routing losses, also lowers the power cost between HBM and logic. On optical interconnect, DeBoer said he is less focused on rack-to-rack links than on chip-to-chip optical connections between HBM and GPUs — a harder problem with a longer timeline. Even brute-force packaging — more HBM, more GPUs, bigger systems — lowers token costs. DRAM itself is also heading toward 3D structures; unlike 3D NAND, it won't double capacity in one leap, but it should be enough to sustain a decade of bit growth.

Micron is also expanding its AI-focused venture fund to get in earlier on promising startups, and in late August announced Micron Research Labs, a $10 billion, decade-long commitment to work with universities, national laboratories and startups on research with a longer time horizon than the company has ever funded. DeBoer summed up Micron's bet as running the full chain — wafer fabrication, collaboration with equipment makers, and system-level co-design with customers. Capacity, he said, will eventually catch up; the immediate task is to get everything possible out of what already exists.

Cooling is like the tires: however big the engine or however full the tank, if the tires have no air, the car isn't going anywhere.

Three memory giants, one diagnosis

DeBoer's warning wasn't an isolated claim. According to the Taiwanese technology outlet DigiTimes, executives from Samsung Electronics and SK Hynix joined Micron on the same SEMICON Taiwan 2026 stage in agreeing that traditional interposer-based 2.5D HBM is approaching a performance ceiling, and that the industry is shifting toward 3D vertical stacking and wafer-level bonding to cut transmission power and unlock further bandwidth. In a separate report, DigiTimes said Micron is exploring "tightly coupled DRAM" architectures that could deliver more than ten times HBM's bandwidth at substantially lower energy per bit — evidence that the widening gap between processor performance and memory bandwidth is now an industry-wide constraint on how far AI compute can scale, not a claim unique to Micron.

For companies anywhere that depend on renting or buying AI compute — cloud customers, GPU-server buyers, AI startups building on rented infrastructure — a "severely undersupplied" memory market isn't an abstract warning. It shows up as longer lead times and higher prices for HBM-equipped AI servers and GPU modules whenever memory spot prices tighten. Locking in supply agreements early and building cost flexibility into budgets is now more realistic than ordering on short notice. And because cooling has become a structural bottleneck in its own right, thermal and power planning for data centers and edge AI hardware now matters as much as the chips themselves.