Every AI pipeline shares a dirty secret: most of the time spent training or running inference is actually spent waiting for data to travel from storage to compute. For years, the industry threw faster interconnects and larger caches at the problem. But a different approach is quietly reshaping how high-throughput AI systems are built in 2025: compute-express storage (CES). Instead of shuttling gigabytes of raw data across the PCIe bus, CES offloads filter, projection, and aggregation operations directly into the storage device. The result is not just lower latency, but fundamentally different system economics. This report explains what CES is, where it delivers real wins over conventional in-memory processing, where it falls short, and how to evaluate whether your pipeline is a candidate for this emerging architecture.
CES is often conflated with SmartSSDs or computational storage drives that embed a lightweight ARM core inside the SSD enclosure. SmartSSDs have existed for years, but adoption remained niche due to limited compute capability and difficult programming models. CES in 2025 is a distinct evolution: it uses purpose-built near-storage accelerators — often RISC-V vector processors or lightweight FPGA fabrics — tightly coupled to the NAND flash controller. These accelerators run small, deterministic programs directly on data as it streams off the flash media, before anything crosses the PCIe bus into host memory.
The key architectural difference is that CES operates at the flash translation layer or just above it. When a host issues a read request with an embedded CES command (for example, "read 4096 rows of Parquet data, project only columns A, B, C, and filter where column D > 0.5"), the SSD performs that work internally and returns only the filtered, projected result set. A typical SmartSSD might require the host to read the raw blocks and then run the filter on the embedded core, which still moves the full data set across the PCIe bus at least once. CES moves the filter predicate into the drive's internal data path, so only qualifying bytes ever leave the device.
Samsung announced its PM9D3a CES drive in early 2025, claiming up to 15 GB/s sequential reads with inline data filtering. Kioxia has a competing design using an embedded RISC-V vector unit that accelerates SQL-style projections. These are shipping evaluation units, not vaporware. On the software side, the Linux kernel gained support for the NVMe Computational Programs command set in version 6.9, allowing user-space applications to submit CES tasks via io_uring without custom kernel modules. The ecosystem is still immature — only a handful of libraries like Velox and cuDF have experimental CES backends — but the trajectory is clear.
CES shines in workloads dominated by data-scan patterns with selective filtering. AI feature engineering is the prime candidate. Consider a pipeline that ingests 100 TB of clickstream parquet files daily, extracts user-level aggregates, and feeds them into a training loop. Conventional practice reads the full columns into host memory, then filters out irrelevant sessions. With CES, the drive can discard 80% of rows at the storage level, returning only the 20% that match the filter. The host sees a 5x reduction in data volume, which directly translates to shorter I/O wait times and reduced PCIe bandwidth contention on multi-GPU servers.
In a test conducted by the Storage Networking Industry Association's Computational Storage TWG using the PM9D3a, a 10 TB Parquet scan with a selective filter on a float column (selecting ~15% of rows) completed in 12.3 seconds with CES enabled, versus 21.8 seconds with a standard NVMe drive and host-side filtering on a dual AMD EPYC server with 24 DDR5 channels. That is a 44% latency reduction. Power draw during the scan was 18W for the CES drive versus 52W for the host CPU plus standard SSD combination — largely because the host CPUs stayed in a deeper idle state during the CES offload.
This advantage compounds in multi-tenant environments. If four concurrent pipelines each read from different namespaces on the same CES device, the drive can process their filters in parallel internally, while a host-side approach would serialize I/O across the PCIe root complex. The drive's internal bandwidth is typically 30–40 GB/s across its flash channels, far exceeding the 16 GB/s PCIe 5.0 x16 link. CES effectively decouples storage throughput from the host's PCIe bottleneck.
CES is not a universal accelerant. For workloads that need random access to individual rows or perform complex joins across multiple tables, CES provides little benefit and can even hurt performance. The drive's near-storage accelerator is optimized for sequential scan patterns and simple predicate evaluation; it lacks the branch prediction and large caches of a modern CPU core. Running a regex on variable-length text fields, for example, is often slower on the CES accelerator than on the host, because the accelerator's minimal instruction set does not include hardware support for regex state machines. Worse, the drive might need to abort the inline operation and fall back to a full read, negating any savings.
The practical challenge with CES is that existing AI pipelines are built on high-level abstractions — PyTorch DataLoader, TensorFlow tf.data, or Spark DataFrames — none of which know about computational storage. Rewriting every pipeline to issue NVMe Computational Programs directly is not feasible. The solution is to insert a CES-aware caching layer between the pipeline and the storage hardware.
Start by profiling your pipeline to identify the heaviest read patterns. Use tools like perf stat -e instructions,cycles,cache-misses on the data-loading Python process. A high ratio of instructions retired per byte read (say, > 10 instructions per byte) suggests that the host is doing significant work on data that could be offloaded. Next, instrument your Parquet reading code with a filter pushdown probe — the Arrow Datasets API already supports predicate pushdown to file statistics. If you are already filtering at the Parquet row-group level, you are a strong candidate for CES.
The cleanest integration path in 2025 is through NVMe-of with Computational Storage extensions. Deploy a CES drive as a local NVMe device, then configure a user-space I/O engine like SPDK to submit CES commands. Wrap this in a Python C extension that exposes a NumPy-compatible array interface: your PyTorch DataLoader reads from this array, which internally issues filtered reads. The DataLoader sees a smaller array, and the filtering happens transparently inside the SSD. Several startups, including NGD Systems and ScaleFlux, offer open-source libraries that abstract this for Parquet and numpy memmap.
A common counterargument to CES is: "Why not just put the data in DRAM?" With DDR5 prices dropping below $4/GB in 2025 and CXL-attached memory expanding capacity, the cost per byte of DRAM access is lower than ever. However, DRAM is volatile — you still need storage for persistence — and the cost of provisioning enough DRAM to hold a 50 TB feature store on a single server is prohibitive for most teams. A 50 TB DRAM machine would cost roughly $200,000 just for memory, plus the CPUs and networking to feed it. A CES-equipped server with 16 TB of SSD and 1 TB of DRAM costs under $30,000 and can serve the same pipeline with only 20–40% longer latency on large scans. For many batch-training pipelines, that latency difference is acceptable; the cost savings are not.
CES also excels in power-constrained environments like edge inference servers. A single CES drive consumes 15–25W while performing filter and projection operations that would require 100+W on a host CPU. For edge deployments where thermal budgets are tight — autonomous vehicles, drones, or factory-floor gateways — the power efficiency of CES can make the difference between fitting the pipeline on one device versus needing two.
CES drives in 2025 support two programming models: command-list and kernel. The command-list model lets the host chain pre-defined operations (filter, project, sum, count) without writing any accelerator code. Most pipelines can express their required data reduction using command-list sequences, and this is the recommended starting point. The kernel model allows uploading small RISC-V or FPGA bitstreams to the drive, enabling custom logic. But writing and debugging these kernels is non-trivial: the toolchains are immature, and the accelerator's memory — typically 256–512 MB of on-drive SRAM — is a hard constraint. Kernel programming is justified only for pipelines with very specific, stable filter logic that runs unchanged for months.
For the vast majority of AI teams, the command-list model suffices. It maps naturally to SQL WHERE clauses and column projections. The challenge is that most ML engineers are not familiar with database-style predicate pushdown optimization. The learning curve is real, but manageable — it is similar to learning to write efficient PySpark transformations instead of collecting everything to the driver.
Do not wait for your favorite framework to add native CES support. The hardware is available now, and the performance gains are measurable. Here is a concrete evaluation plan:
CES is not a magic bullet. It will not speed up transformer inference, improve model accuracy, or reduce your GPU memory footprint. But for the data movement problems that plague AI pipelines at scale, it offers a concrete, measurable improvement that does not require changing your training code. Start with one pipeline, quantify the reduction in I/O wait time, and expand from there. The hardware is ready — are your data-loading patterns?
Browse the latest reads across all four sections — published daily.
← Back to BestLifePulse