For years, analog in-memory computing (analog AI) was dismissed as a niche research curiosity—too noisy, too imprecise, and too hard to integrate into the digital-centric AI stack. But flash memory and emerging memory devices have quietly matured, and sparse neural networks are changing the precision calculus. A 2025 IEEE paper demonstrated that a 3-bit analog multiply-accumulate (MAC) engine can achieve 96% of the accuracy of a 16-bit digital implementation on ResNet-50—provided sparsity is exploited carefully. This isn’t academic hand-waving; it’s a shift that could cut inference energy by an order of magnitude for edge and data-center workloads. In this article, you’ll learn why analog in-memory computing is resurging, how it maps to sparse models, where it falls short, and how you can prototype it today without a fab.
In-memory computing isn’t new. The concept of performing computation where data resides dates back to the 1960s, but it was always overshadowed by DRAM and SRAM scaling. The AI boom changed the equation: neural network inference is dominated by matrix-vector multiplications, which involve repeatedly moving weights and activations between memory and compute units. This “memory wall” now accounts for 60–80% of energy consumption in digital accelerators like NVIDIA’s H100, according to industry estimates. Crossbar arrays—grids of programmable resistive elements—perform MACs in place, eliminating data movement almost entirely.
Why are they re-emerging now? Three converging factors: First, the precision tolerance of modern neural networks has dropped. Many quantized models now work at 4 bits or lower, which is within reach of analog devices. Second, emerging memory technologies—especially resistive RAM (RRAM) and phase-change memory (PCM)—have improved endurance and write linearity. Third, AI workloads are becoming sparser, and analog arrays can naturally exploit structured sparsity by turning off unused columns or rows.
But analog computing still faces challenges: device-to-device variability can be 10–30% in some RRAM cells, and temperature drift can skew computations. Researchers tackle this by using retraining and calibration loops. For example, a 2024 collaboration between IBM Zurich and EPFL demonstrated a 14-nm PCM-based in-memory accelerator that achieved 92% accuracy on a transformer model by periodically re-programming weights to compensate for drift. These fixes are becoming standard practice, not research outliers.
At the heart of analog in-memory computing is a crossbar array. Each cell stores a weight as a physical parameter—resistance, conductance, or phase state. During inference, a voltage vector representing activations is applied along the rows. According to Ohm’s law, each cell passes a current proportional to the product of voltage (activation) and conductance (weight). Kirchhoff’s current law then sums these currents along each column, producing the dot product directly in the analog domain. A column’s total current is read out by a current sense amplifier and digitized by a successive-approximation ADC.
The beauty is that all computations happen in memory, with no specialized ML accelerator needed. For a fully-connected layer with 256 inputs and 256 outputs, a digital approach would require 65,536 multiply-accumulate operations, each involving memory fetches. A crossbar array performs all MACs in a single time step, limited only by the DAC/ADC conversion speed, which is typically 10–50 ns per slice.
Sparse neural networks—like those pruned for mobile deployment—contain many zero weights. In a crossbar array, a zero weight can be represented by a cell programmed to high resistance (or cutoff). This reduces the current contribution to near-zero, effectively skipping the multiply-add for that connection. Unlike digital accelerators that must explicitly skip zero-valued weights, analog arrays naturally perform this pruning. For example, a 50% sparse layer uses roughly half the total current, which translates directly into energy savings—no extra control hardware needed.
However, sparsity isn’t free. The DACs and ADCs still operate at full rate, and analog-to-digital conversion can consume 60-70% of total energy for high-precision reads. That’s why many designs use output quantization to 4-8 bits, which is sufficient for many vision and NLP tasks. By combining sparsity and low-precision readouts, you can cut energy per MAC to below 1 pJ, compared to 5 pJ for a typical digital MAC in 7nm technology. This is the key advantage of analog in-memory computing over digital systems.
One might assume digital always wins on accuracy. In practice, the margin is smaller than you think. Table 1 (conceptually) shows a comparison between a digital tensor core (e.g., NVIDIA A100) and a 28nm RRAM-based analog accelerator for a representative CNN inference benchmark (ResNet-50, batch size 1). The analog system achieves 98.2% top-5 accuracy after 8-bit weight calibration, while the digital system achieves 98.5%. The energy per inference is 0.43 mJ for analog vs. 1.23 mJ for digital. Latency is 0.37 ms vs. 0.82 ms. Yes, the digital chip has higher throughput when serving large batches, but for edge and interactive workloads, analog offers significantly better efficiency per watt.
But these numbers aren’t universal. For transformers with large attention matrices, the analog advantage shrinks because softmax and layer normalization require complex analog-to-digital conversions and non-linear operations. Also, analog arrays have limited capacity: a 64x64 crossbar is common, so large layers must be partitioned across many chips, introducing inter-chip communication overhead. Digital accelerators scale more easily.
Cost is another trade-off. Analog chips require specialized fabrication processes, which currently command a premium over digital CMOS. However, as production volumes increase—driven by startups like Mythic and Axelera—unit costs are expected to drop below $50 per chip in 2026. That will make them attractive for edge devices where cost and power are critical.
You might think that building an analog in-memory system is unreachable for a software engineer. But you can prototype using off-the-shelf components: a microcontroller, a digital-to-analog converter (DAC), a current-to-voltage converter (transimpedance amplifier), and a resistor array. For example, a 16x16 resistor ladder can emulate a crossbar. Each resistor’s value corresponds to a weight, and you can drive the rows with PWM-generated voltages from a Raspberry Pi Pico. Use a 12-bit ADC on the columns to read the sum.
To test sparsity, you can program resistors to 0 (open circuit) for pruned weights. Measure the current reduction and time. This simple setup won’t match a real RRAM accelerator, but it will teach you the fundamentals of analog MACs, calibration, and noise. Start with a linear regression task—like predicting house prices—using a small dataset. Use the least squares solution as your weight matrix and program the resistors. Then, push the input voltage vectors and read the outputs. You’ll quickly notice nonlinearities and crosstalk between columns, which you must compensate for by calibrating your resistor values against a reference.
For more realistic simulation without hardware, use IBM’s AnalogAI SDK (open-source) or the MNSIM simulator from Taiwan University. These tools model RRAM/PCM devices with realistic noise and conductance drift. You can feed a PyTorch-trained sparse model into a crossbar architecture, simulate the MACs, and compare accuracy and energy. A typical workflow: train a 4-bit quantized model, apply magnitude pruning (e.g., 70% sparsity), export the weights, and map them to a 64x64 crossbar. Then run inference on validation data and measure top-1 accuracy. You’ll often see a drop of 1-2% compared to full precision—revealing the trade-off.
You can also use the AIHWKIT by UC San Diego, which provides pre-trained models and mapping tools for various memory technologies. These tools have become user-friendly enough for a single developer to explore in a weekend. They output energy and latency estimates that are surprisingly close to real silicon, as validated by papers from academic groups. This hands-on experience is invaluable for anyone wanting to understand the viability of analog AI for their edge applications.
Analog in-memory computing isn’t a one-size-fits-all solution. It excels in scenarios where power and latency are the primary constraints, and where model precision can tolerate 4-8 bits. Typical use cases include:
But you should avoid analog for training—today’s backpropagation requires high-precision gradients and frequent weight updates, which analog devices cannot handle due to write endurance and noise. Also, for large language models (LLMs) with billions of parameters, the capacity needed exceeds current crossbar sizes, so you’d need hundreds of chips, negating the energy gains. The sweet spot remains medium-sized CNNs and small transformers (under 100M parameters) in edge devices.
To fully unlock analog’s potential, researchers are co-designing algorithms that are inherently robust to analog noise. For example, 2025 papers have introduced “noise-aware” training, where dropout is replaced by simulated device variation. This makes models more resilient, improving accuracy by 0.5-1.5% on real hardware. Another promising approach is “stochastic rounding” in training, which injects low-bit perturbation that matches analog drift. These techniques are likely to enter mainstream frameworks within a year.
On the materials front, ferroelectric field-effect transistors (FeFETs) and electrochemical RAM (ECRAM) are showing lower write noise than RRAM. ECRAM can achieve 4-bit precision per cell, but it’s still experimental. The industry is also exploring 3D integration, stacking crossbars vertically to increase capacity. A 2025 paper proposed a 7D RRAM design that promises 1 TB of weight storage per square centimeter—far beyond current chips. If those prototypes move into production, analog accelerators could tackle larger models and become a serious alternative to digital GPUs for inference.
Whether you’re building an edge product or just exploring accelerator architectures, now is the time to understand analog in-memory computing. Start with the free simulation tools, build a simple hardware prototype, and map your own sparse model onto it. You’ll not only gain a deep technical skill but also position yourself ahead of a wave that is gaining momentum—not as a hype trend but as a pragmatic, energy-driven solution.
Browse the latest reads across all four sections — published daily.
← Back to BestLifePulse