AIThis post was created with the assistance of artificial intelligence (AI).

TL;DR

Modern AI accelerators combine compute dies, high-bandwidth memory stacks and dense package wiring in one module. Vendor specifications show rapid growth in memory bandwidth and package complexity, while SK hynix says demand for memory is expected to exceed supply through 2030.

Thorsten Meyer AI published an explainer on November 22 showing how an AI accelerator package combines compute dies, stacked high-bandwidth memory (HBM), a silicon interposer, a substrate and cooling hardware. The report matters because AI system performance and supply depend on more than the processors themselves: memory bandwidth and advanced packaging are also central, and memory makers say demand remains ahead of supply.

Buying for a business?Offer from Amazon

Get business pricing on monitors, keyboards and dev gear

  • Business-only prices and quantity discounts
  • Tax-exempt purchasing
  • Multiple users, one account, clear invoices
As an affiliate, we earn on qualifying purchases.

The explainer’s 3D model follows vendor documents for layer order and selected component counts, while keeping dimensions schematic. From the bottom up, a package substrate carries power and signals between the accelerator package and its module. Above it, a silicon interposer provides dense connections between compute dies and nearby memory stacks. This wiring layer can carry connections that would be impractical on a conventional circuit board.

At the center are the compute dies, which perform the calculations used by AI models. The accompanying source material describes Nvidia’s Rubin GPU as having two reticle-limited compute dies and 336 billion transistors. AMD’s MI350 Series takes a different approach: AMD says it integrates eight vertically stacked accelerator complex dies with two I/O dies; the MI355X has 185 billion transistors. These are vendor specifications, and the designs should not be treated as interchangeable measures of performance.

HBM stacks sit beside the compute dies. They contain layers of DRAM connected vertically by through-silicon vias, or TSVs, so data can move between memory and processor across many connections. The article cites eight 12-high stacks for AMD’s MI355X. It also describes a cold plate above the package for liquid cooling. For an AMD system, the source says eight OAM accelerator modules connect through a universal baseboard; that system-level arrangement is distinct from the layers inside one package.

At a glance
reportWhen: Published November 22, 2026; includes a…
The developmentA November 22, 2026, explainer from Thorsten Meyer AI uses a 3D model and vendor specifications to show the layers inside AI accelerators and the manufacturing constraints around them.
Four 3D stills titled One rack, zoomed in four steps. Step 1, rack: 72 GPUs and 36 CPUs in 18 compute trays, with 9 NVLink switch trays. Step 2, compute tray: 1 RU, 2 CPUs and 4 GPUs, liquid-cooled through cold plates. Step 3, package: two compute dies and eight HBM stacks on a silicon interposer, one stack in orange. Step 4, HBM stack: 12 DRAM layers on a base die, joined by vertical wires (TSVs), shown as a cutaway with the layer heights drawn 10 times taller. Schematic 3D models.
From rack to memory stack in four steps. Counts from NVIDIA's DGX GB user guide (p. 9) and AMD's CDNA 4 white paper (p. 2); rack outline from the OCP listing; sizes schematic. 3D models: Thorsten Meyer AI.

Why Memory and Packaging Matter

AI accelerators need to move large volumes of model data between memory and compute. The source reports that Nvidia’s listed memory bandwidth increased from 900 GB/s for V100 to 19.2 TB/s for Rubin. Those figures describe bandwidth specifications for different products; they do not, on their own, establish how a particular AI workload will perform. They do show why memory access has become a major part of accelerator design.

The packaging that connects memory and compute is itself a manufacturing challenge. TSMC describes CoWoS-S as using a large silicon interposer to accommodate logic dies and HBM. The company says the technology can support interposers up to about 2,700 square millimeters, or 3.3 times a reticle. Bigger packages require other approaches: the source says CoWoS-L joins silicon bridges in a carrier layer.

These constraints affect how many complete accelerators can be built and delivered. The explainer cites SK hynix’s expectation that demand will exceed supply through 2030. That is a company forecast, not an independently verified guarantee of shortages in every market. For businesses, the practical implication is that access to computing capacity may depend on memory and packaging supply as well as demand for GPU compute.

Products Worth Considering

LG 27GS60QC-B Ultragear 27-inch Curved Gaming Monitor (QHD, 180Hz, 1ms, 1000R)

LG 27GS60QC-B Ultragear 27-inch Curved Gaming Monitor (QHD, 180Hz, 1ms, 1000R)

Bose TV Speaker Compact Soundbar with Bluetooth and HDMI-ARC, Black

Bose TV Speaker Compact Soundbar with Bluetooth and HDMI-ARC, Black

As an affiliate, we earn on qualifying purchases.

From Silicon Dies to HBM Stacks

A “reticle” is the maximum area a chipmaking scanner can print in a single exposure. The source says TSMC reported 3.5-reticle CoWoS-L packages in volume production since 2024. On October 15, 2026, TSMC’s chief executive said the company was in “real world production of 5x reticle size” and working on 9x and 14x. Those statements describe TSMC’s packaging roadmap and production claims, not the size of every accelerator package.

HBM capacity and stack height have also advanced. The article cites SK hynix describing 12 layers of 3 GB DRAM chips at the same thickness as an earlier eight-layer product, with each chip 40% thinner than before. Samsung has said its 12-high HBM3E reached a seven-micrometer gap between chips. The JEDEC HBM4 standard, published in April 2025, allows stacks with four, eight, 12 or 16 layers and up to 64 GB per stack.

““large silicon interposer area to accommodate various functional top dies, including logic chiplets, with high-bandwidth memory (HBM) cubes stacked over it””

— TSMC

What the Specifications Leave Open

The source does not provide millimeter measurements for the dies, package or HBM stacks; its model uses schematic sizes. It also does not quantify how much packaging capacity is available across suppliers or how allocation may vary among customers. SK hynix’s supply outlook is a forecast, and the material does not establish the scale or duration of shortages for each type of memory.

Specifications alone do not show real-world performance across different AI workloads, system configurations or software. The source gives bandwidth and transistor counts but does not offer a controlled comparison of the cited Nvidia and AMD products. It also does not provide a specific price outlook or explain how much packaging constraints contribute to accelerator prices.

Packaging Capacity and Supply Outlook

The developments to watch are whether packaging suppliers expand production and whether memory makers’ supply forecasts change as new capacity comes online. TSMC’s stated work on larger CoWoS-L packages and AMD’s plans for CoWoS-L-based products point to continued investment in advanced packaging, but the source does not give a delivery timetable for every design. Further vendor disclosures on production volumes, shipping schedules and memory availability would help clarify how these manufacturing limits affect buyers.

Key Questions

What parts make up an AI accelerator package?

A typical package described in the source combines compute dies, HBM stacks, a silicon interposer and a package substrate. A cooling plate sits above the package in the illustrated setup.

What is HBM?

High-bandwidth memory is DRAM arranged in vertical stacks. Through-silicon vias connect the layers so data can move through the stack.

What does the silicon interposer do?

The interposer provides dense wiring between the compute dies and HBM. TSMC’s CoWoS technologies are cited as examples of packaging that uses silicon interposers or silicon bridges.

Does the article show the physical dimensions of the chips?

No. Its 3D model uses schematic sizes because the source says the vendors do not publish the relevant measurements in millimeters.

How long might memory supply remain tight?

SK hynix expects demand to exceed supply through 2030, according to the source. That is the company’s forecast; the article does not quantify shortages across suppliers or products.

Source: Thorsten Meyer AI

You May Also Like

The Local-First Agentic Operator

A single operator, leveraging agentic AI, now builds and manages multiple complex products that traditionally required organizations, emphasizing local-first, provider-agnostic principles.

True 4K Gaming PC (2026)

A new gaming PC designed to deliver true 4K performance is scheduled for release in 2026, promising significant advancements in gaming hardware.

Maximize AI Performance With A Local Document Pipeline

A new architecture enables efficient, scalable local AI document processing, emphasizing simplicity, maintainability, and data governance.

DeepSWE – The benchmark that made the models spread out again

DeepSWE, released May 26, 2026, reveals wider gaps among AI coding models, exposing flaws in previous benchmarks and redefining model performance evaluation.