Nana Wu
At the prestigious Hot Chips 2026 conference, Samsung Electronics unveiled a transformative advancement in semiconductor architecture: the industry’s first LPDDR5X-PIM (Processing-in-Memory) technology. By integrating computational logic directly into low-power memory, Samsung is poised to dismantle the "memory wall"—the persistent bottleneck that has long restricted the performance of AI inference tasks in mobile, client, and edge computing environments.
This development marks a significant shift in how memory architecture is viewed, moving away from traditional passive storage toward a more active, collaborative role in processing. By enabling basic mathematical operations to occur directly within the DRAM cells, Samsung is effectively reducing the constant, energy-draining data movement between the processor and memory, promising a new era of efficiency for on-device artificial intelligence.
The Core Concept: What is Processing-in-Memory (PIM)?
To understand the magnitude of Samsung’s announcement, one must first understand the fundamental limitation of current computing architectures. In traditional systems, data must be fetched from the memory (DRAM), sent across the bus to the Central Processing Unit (CPU) or Graphics Processing Unit (GPU), processed, and then written back to the memory. This constant back-and-forth transit consumes substantial power and latency, creating a performance ceiling for AI applications that require massive amounts of data throughput.
Processing-in-Memory (PIM) fundamentally changes this paradigm. By placing small, specialized logic circuits alongside the DRAM cells, Samsung allows the memory to perform basic arithmetic operations—specifically multiply-accumulate (MAC) operations—locally. Instead of sending raw data to the processor to be multiplied by a weight and then sent back, the calculation happens where the data resides.
Samsung first introduced the world to this concept in 2021, successfully piloting the technology through High Bandwidth Memory (HBM) stacks in high-end AMD accelerators. However, while HBM is exceptionally powerful, it is also prohibitively expensive and power-hungry, making it unsuitable for the mass-market mobile and client devices that dominate the consumer landscape. LPDDR5X-PIM serves as the bridge, bringing the efficiency gains of HBM-PIM to the ubiquitous low-power memory standard.
A Chronology of Innovation: From Concept to Validation
The journey to LPDDR5X-PIM has been a calculated, multi-year progression:
- 2021: Samsung publicly demonstrates the initial potential of PIM, showcasing integration within HBM to accelerate data-intensive workloads.
- 2023: Samsung officially introduces the concept of LPDDR5X-PIM at industry forums, signaling its intent to bring this technology to the mobile and edge sectors.
- 2026 (Hot Chips): Samsung moves beyond theoretical discussion, presenting a fully validated, functional product. The presentation confirmed that the technology is ready for real-world implementation, with the company simultaneously revealing plans for future LPDDR6X-PIM iterations, aiming for a JEDEC standardization specification by the end of the year.
This timeline highlights Samsung’s strategic patience. By first proving the technology in the extreme environment of server-grade HBM, they refined the logic required for PIM before shrinking it down for the more space-constrained and power-sensitive LPDDR5X format.
Architectural Breakthroughs: Enabling Localized Intelligence
The architecture behind LPDDR5X-PIM is a masterclass in efficiency. In previous HBM-PIM implementations, Samsung was forced to remove certain memory banks to make physical room for the additional logic. With the new LPDDR5X-PIM, the company has successfully integrated the logic without such compromises.
Each memory bank now features its own dedicated PIM circuitry. The data flow is structured to feed into parallel MAC trees, drawing from a source register file and a scale register file. Once the calculation is complete, the results—whether in integer or floating-point format—are written directly to a vector register file (VRF).
A significant technical hurdle was managing the reordering of data, which is a standard procedure in conventional DRAM controllers. Samsung solved this by implementing "Address Align Mode" (AAM). AAM acts as a translator, mapping DRAM addresses to MAC instructions. By assigning the VRF and source register addresses based on Row/Column (RA/CA) addresses rather than a fixed instruction register, Samsung allows the system to switch seamlessly between a standard "single-bank" mode (traditional DRAM behavior) and a "multi-bank" mode (PIM-accelerated behavior).
Performance Metrics and Benchmarks
During their Hot Chips presentation, Samsung provided compelling data on the performance benefits of their new architecture. Testing was conducted using an Llama 3.1 model with 8 billion parameters, run on an edge AI accelerator.
The results were stark:
- Model Run Time: A 2.28x speedup compared to standard, non-PIM LPDDR5X.
- Throughput: An increase of 3.01x in tokens per second (TPS).
- Bandwidth: While standard LPDDR5X-9600 offers a peak bandwidth of 76.8 GB/s, the PIM-enabled variant effectively increases this eightfold to 614 GB/s in compute-intensive tasks, primarily by eliminating the latency inherent in off-chip data transfers.
While some attendees at Hot Chips noted minor discrepancies in output compared to conventional processing, Samsung emphasized that ongoing optimization efforts are focused on refining accuracy. The company remains confident that these performance gains are consistent and represent a major leap forward for edge AI, where bandwidth and latency are the primary constraints.
Official Responses and Power Consumption Implications
A primary concern for any new memory technology is power efficiency. In an era where mobile devices are expected to maintain long battery life, adding logic to memory might seem counterintuitive. However, Samsung’s engineers argue the opposite.
Karam Hwang, a key presenter at the event, addressed these concerns directly. While he acknowledged that the peak power consumption during a PIM operation is "much higher" due to the bursty nature of the on-chip calculations, the total system power consumption is lower. The reduction in energy required to move data back and forth between the memory and the processor significantly outweighs the incremental power used by the PIM logic.
"We’re not seeing a significant power increase," Hwang stated, noting that the overall energy budget of an AI-enabled device would actually benefit from the reduced workload on the host processor. By keeping the "heavy lifting" of matrix multiplication local to the memory bank, the host SoC can operate more efficiently, leading to a net gain in power savings.
Implications for the Tech Industry
The implications of Samsung’s LPDDR5X-PIM are far-reaching. By bringing this technology to the LPDDR standard, Samsung is effectively democratizing AI acceleration.
1. The Proliferation of Edge AI
Until now, sophisticated AI inference required substantial dedicated hardware or cloud-based processing. With PIM-enabled LPDDR5X, mobile devices, laptops, and IoT appliances can perform complex AI tasks locally. This is a game-changer for privacy, as data does not need to be sent to a cloud server, and for performance, as the device becomes responsive even without an active internet connection.
2. Easing the Cost of HBM
Micron and other industry analysts have recently warned that the demand for HBM is creating a "silicon gap," where supply cannot keep pace with the needs of the AI industry. By providing a high-performance alternative for less demanding tasks, Samsung is helping to alleviate the pressure on the HBM supply chain, allowing the most expensive memory to be reserved for the most critical data-center applications.
3. New Hardware Partnerships
The industry is already shifting to accommodate this. We have seen the emergence of SOCAMM2 serviceable modules, which allow manufacturers like Nvidia to utilize LPDDR5X in specialized CPU configurations. Similarly, Intel’s "Crescent Island" AI accelerator project underscores the industry’s desire to use LPDDR5X as the backbone for next-generation, high-efficiency AI engines.
Conclusion: A Paradigm Shift
Samsung’s debut of LPDDR5X-PIM at Hot Chips 2026 is not merely a product launch; it is a fundamental redirection of how we define memory. By moving from a passive storage medium to an active computing partner, DRAM is entering its most innovative phase in decades.
As we look toward the potential standardization of LPDDR6X-PIM, it is clear that the future of artificial intelligence will not be decided solely by the speed of our processors, but by the intelligence of the memory that feeds them. For developers, manufacturers, and consumers alike, this advancement promises a future where our devices are faster, more efficient, and significantly more capable of handling the demands of the AI-driven world.
Nila Kartika Wati
Evan Lee Salim
