NVIDIA takes the memory controller off your custom chip and puts it in the DRAM
NVHBM offers NVLink Fusion partners 30 percent more bandwidth and 15 percent less power, in exchange for NVIDIA owning the interface between their compute and their memory.
The Engineer · Build desk
What happened
- NVIDIA has added NVHBM, a custom high-bandwidth memory base die validated with memory vendors, to the NVLink Fusion program for partners building their own accelerators.
- The memory controller moves off the customer's compute die into the DRAM stack, and partners integrate a smaller NVIDIA-supplied PHY instead.
- NVIDIA claims up to 30 percent more memory bandwidth per stack than standard HBM4e.
Why it matters
- What it limits constraint The block a custom accelerator team no longer designs is the one joining compute to data, so the boundary of its own chip is now specified by the vendor it competes with.
- What it costs cost Staying independent means paying the memory qualification bill NVIDIA describes as a bottleneck, while rivals inside the program get it absorbed.
- What it sets up precedent A memory spec authored by an accelerator vendor, second-sourced across DRAM makers but sold only to fabric members, becomes a plausible default route for custom AI chips.
Next story · Build · Aug 31
MediaTek will wire custom XPUs into NVIDIA's scale-up fabric as a prevalidated blockConfirmed3 publishers
A custom accelerator team that takes this deal stops designing its own memory controller and starts consuming NVIDIA's, delivered inside the DRAM stack [3]. That single relocation is where all three advertised numbers come from, which means they do not come apart: the 15 percent power saving [5] and the freed die area [17] arrive only if you also give up the controller and integrate the PHY NVIDIA hands you [3].
The reward for making that trade is published twice, at two sizes. NVIDIA's newsroom says NVHBM frees up to 25 percent more area on the XPU compute die against standard HBM4E [17]. Tom's Hardware reports the company claiming the freed package real estate is worth up to 30 percent more compute on the primary die [10]. That is five points of spread on the one figure a silicon architect would budget against [15].
NVIDIA's own framing of the area problem lists what a designer has to allocate: matrix engines, vector units, on-chip SRAM, cache hierarchy, control logic, memory interfaces, network-on-chip and scale-up connectivity [21]. Two of those eight are now things you buy from NVIDIA rather than design [19]. And the coupling extends past the block itself: NVHBM is built on the same technology NVIDIA says it will use for its future GPUs [18], and NVLink Fusion is offered with each generation of its rack-scale architecture [6]. A partner already taking NVLink chiplets, switches and MGX racks [7] has now added its memory interface to the list of things that move when NVIDIA's roadmap moves.
The offer is not empty. NVIDIA's developer post is candid that qualifying leading memory, package integration and validation can become a bottleneck for custom accelerator programs [20], and the company says it is establishing one standard NVHBM implementation available from multiple memory providers, cutting the effort to qualify across suppliers [8]. That is real schedule risk lifted off a customer. It is also a part that only NVLink Fusion customers can get [11].
What does not exist yet is silicon. Tom's Hardware is explicit that these are reasons for prospective partners to consider, not benefits arriving in the Rubin rack systems already in production [12]. Annapurna Labs is first to work on NVHBM [13], Trainium4 will support the NVLink Fusion scale-up interface so that Amazon chips and NVIDIA GPUs share a common rack architecture [14], and Nafea Bshara of Annapurna says only that he looks forward to a collaboration "to benefit future AWS infrastructure designs" [16]. No part number and no date.
So the question in front of an XPU program has narrowed. It is no longer whether you can build your own accelerator instead of buying NVIDIA's. It is how much of the package you still author once the scale-up fabric, the rack and now the memory interface are drawn by the vendor you were building silicon to route around [9].
What to watch
- Whether NVIDIA names a second NVHBM partner, and which silicon generation it is aimed at.
- Which memory vendors publicly commit to building the NVHBM base die, and whether they may sell equivalent parts outside NVLink Fusion.
- Whether NVHBM appears on a Trainium generation after Trainium4, with dates and bandwidth figures attached.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence34
- Adoption14
- Hype gap+38
- Incentives88
- Confidence66
Perspective Coverage
3 publishers- Builder
- Builder 45%
- Operator
- Operator 29%
- Investor
- Investor 26%
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
NVIDIA expanded NVLink Fusion with NVHBM, a next-generation custom HBM base-die technology for XPUs, to be validated and offered by leading memory partners and extended to NVLink Fusion customers.
- [2]
Traditional HBM architectures place the memory controller on the XPU die, consuming silicon area that could otherwise be dedicated to compute.
- [3]
NVHBM moves the memory controller into the base die of the HBM stack and provides a smaller custom PHY that NVLink Fusion customers integrate into their own designs.
- [4]
NVIDIA says NVHBM delivers up to 30 percent more memory bandwidth per stack compared with standard HBM4e.
- [5]
NVIDIA says NVHBM delivers 15 percent lower HBM power consumption compared with standard HBM4E.
- [6]
NVLink Fusion is offered with each generation of NVIDIA's rack-scale system architecture.
- [7]
NVLink Fusion partners can access NVIDIA NVLink chiplets, NVLink-C2C, NVLink Switches and NVIDIA MGX systems and racks, plus an ecosystem of CPU partners, ASIC designers and system manufacturers.
- [8]
NVIDIA is establishing a standard NVHBM implementation available from multiple memory providers, reducing the engineering effort required to integrate and qualify memory across multiple suppliers.
- [9]
NVLink Fusion gives partners the building blocks to connect custom chips into the NVLink scale-up domain used to join many separate processors into a single coherent system, such as the Vera Rubin NVL72 rack-scale accelerator.
- [10]
Tom's Hardware reports Nvidia saying the approach frees package real estate that can be used for up to 30 percent more compute on the primary silicon die.
- [11]
NVHBM is not an HBM replacement; it is a building block Nvidia is only offering to its custom silicon partners.
- [12]
Tom's Hardware states that NVHBM and NVLink Fusion are, as of now, reasons for prospective partners to consider, not benefits that will materialize in the Rubin rack-scale systems already in production.
- [13]
Amazon's Annapurna Labs will be the first to work on NVHBM, as part of its broader NVLink Fusion collaboration with NVIDIA.
- [14]
Annapurna Labs will support NVLink Fusion with its next-generation Trainium chips starting with Trainium4, which would allow Amazon chips and NVIDIA GPUs to work together with a common rack-scale architecture.
- [15]
The two published figures for the compute-die gain differ by 5 percentage points: 30 percent more compute die as reported by Tom's Hardware versus up to 25 percent more freed area in NVIDIA's newsroom post.
- [16]
Nafea Bshara, vice president of Annapurna Labs at Amazon: "NVHBM represents a new architectural approach to advancing high-bandwidth memory performance and efficiency... We look forward to this technology collaboration to benefit future AWS infrastructure designs."
- [17]
NVIDIA's newsroom says NVHBM frees up to 25 percent more area on the XPU compute die compared with standard HBM4E.
- [18]
NVHBM is built on the same technology that NVIDIA will use for its future GPUs.
- [19]
Two of the eight blocks in NVIDIA's own area-allocation list, the memory interface and scale-up connectivity, are supplied by NVIDIA under NVHBM and NVLink Fusion rather than designed by the partner.
- [20]
NVIDIA's developer post states that qualifying leading memory technology, package integration and validation can become a bottleneck for custom accelerator programs.
- [21]
NVIDIA lists the blocks a custom accelerator designer must allocate area to: matrix engines, vector units, on-chip SRAM, cache hierarchy, control logic, memory interfaces, network-on-chip and scale-up connectivity.
Sources
2 independent publishers whose own reporting we read for this story.
- developer.nvidia.comNVIDIA NVLink Fusion Brings NVHBM to Next-Generation AI Infrastructure
1 article · August 26, 2026
- nvidianews.nvidia.comNVIDIA NVLink Fusion Expands With NVHBM Custom High-Bandwidth Memory
1 article · August 26, 2026
- tomshardware.comNvidia custom 'NVHBM' promises 30% higher bandwidth, 15% lower power than commodity HBM4e — custom base die and PHY will be available to NVLink Fusion partners
1 article · August 26, 2026
Topics and entities
Topics
- Platform Lock-In
- Custom AI Accelerators
- Data Center Power Efficiency
- Advanced Chip Packaging
- High-Bandwidth Memory
- Rack-Scale Interconnect and Scale-Up Fabrics