South Korean memory giant SK Hynix has announced it has begun the mass production of the world’s first 12-layer HBM3E, featuring a total memory capacity of 36GB, a huge increase from the previous 24GB capacity in the 8-layer configuration.
This new design was made possible by reducing the thickness of each DRAM chip by 40%, allowing more layers to be stacked while maintaining the same overall size. The company plans to start volume shipments by the end of 2024.
The HBM3E memory supports a bandwidth of 9600 MT/s, translating to an effective speed of 1.22 TB/s if used in an eight-stack configuration. The improvement makes it ideal for handling LLMs and AI workloads that require both speed and high capacity. The ability to process more data at faster rates enables AI models to run more efficiently.
Nvidia and AMD hardware
For advanced memory stacking, SK Hynix employs innovative packaging technologies, including Through Silicon Via (TSV) and the Mass Reflow Molded Underfill (MR-MUF) process. These methods are essential for maintaining the structural integrity and heat dissipation required for stable, high-performance operation in the new HBM3E. The improvements in heat dissipation performance are particularly important for maintaining reliability during intensive AI processing tasks.
In addition to its increased speed and capacity, the HBM3E is designed to offer enhanced stability, with SK Hynix’s proprietary packaging processes ensuring minimal warpage during stacking. The company’s MR-MUF technology allows for better management of internal pressure, reducing the chances of mechanical failures and ensuring long-term durability.
Early sampling for this 12-layer HBM3E product began in March 2024, with Nvidia’s Blackwell Ultra GPUs and AMD’s Instinct MI325X accelerators expected to be among the first to use this enhanced memory, taking advantage of up to 288GB of HBM3E to support complex AI computations. SK Hynix recently rejected a $374 million advanced payment from an unknown company to ensure it could provide Nvidia with enough HMB for its in-demand AI hardware.
“SK Hynix has once again broken through technological limits demonstrating our industry leadership in AI memory,” said Justin Kim, President (Head of AI Infra) at SK Hynix. “We will continue our position as the No.1 global AI memory provider as we steadily prepare next-generation memory products to overcome the challenges of the AI era.”
SK Hynix and Kioxia could build lucrative HBM chips for Nvidia and AMDSK Hynix teams up with TSMC to advance HBM developmentA glimpse at what the future of memory and storage could look like