Proxics: an efficient programming model for far memory accelerators
Proxics proposes OS abstractions for near-data processing accelerators, showing real hardware gains — but the hardware ecosystem is still nascent.
The Thesis
Near-data processing (NDP) — placing compute cores physically adjacent to memory rather than shipping data across a bus to the CPU — promises to relieve the bandwidth bottlenecks that increasingly limit data-intensive workloads. The emerging CXL standard, which allows memory pools to be disaggregated and shared across servers, has revived interest in NDP hardware. The problem is that no clean, portable programming model exists for these devices: today, each vendor ships bespoke, low-level interfaces. Proxics proposes mapping familiar OS abstractions — virtual processors and inter-process communication channels similar to Unix pipes — onto NDP hardware in a lightweight way that avoids the bandwidth and compute overhead those abstractions normally carry. The authors demonstrate the approach on real hardware across three workload classes: bulk memory operations, in-memory databases, and graph analytics.
Catalyst
The CXL (Compute Express Link) interconnect standard reached version 3.0 in 2022 and is now being implemented in shipping server hardware from major vendors, making disaggregated memory pools a near-term data-center reality rather than a research concept. NDP accelerator silicon is beginning to appear from startups and research labs, but without a portable software layer, each device requires hand-tuned code. The combination of available hardware to run experiments on and the absence of any widely adopted programming model creates a timely gap this paper addresses.
What's New
Prior NDP proposals — including academic systems like PIM-enabled instructions from Samsung and various UPMEM-based frameworks — typically exposed raw memory-side compute through custom intrinsics or vendor-specific APIs, with no shared abstraction layer. Some proposals used shared-memory IPC (inter-process communication, where processes exchange data through a common memory region), which defeats the purpose of NDP by consuming the very bandwidth NDP is meant to conserve. Proxics replaces shared-memory IPC with channel-based communication compiled into the interconnect protocol itself, and replaces heavyweight OS processes with lightweight virtual processors whose scheduling overhead is minimized, demonstrating that familiar abstractions need not impose familiar costs.
The Counter
The core bet here is that a portable OS-level abstraction will become the standard interface for NDP accelerators — but history suggests hardware vendors resist converging on software abstractions they did not design, especially when differentiation lives in the programming model. CUDA's dominance in GPU computing took over a decade and required Nvidia to own both hardware and software simultaneously; no single vendor has that position in the CXL NDP space today. More fundamentally, the workloads where NDP shines — in-memory databases, graph traversal, bulk copies — are also the workloads cloud providers are aggressively moving to custom ASIC or FPGA solutions that bypass general-purpose NDP entirely. The paper also does not compare against these custom-accelerator alternatives, only against CPU-only baselines, which is a flattering but incomplete comparison. Finally, the lightweight-process abstraction requires compiler cooperation that currently exists only in research prototypes, and getting compiler toolchain vendors to adopt a new lowering pass for each new NDP device is a significant coordination problem the paper does not address.
Longs
None listed.
Shorts
- Vendors of RDMA (Remote Direct Memory Access) network accelerators whose value proposition overlaps with low-latency CXL-based NDP if the latter proves cheaper per-bandwidth-unit
- Companies selling proprietary NDP SDKs that would be displaced by a portable open abstraction layer
Enablers (Picks & Shovels)
- CXL Consortium open standard (enables interoperable far-memory hardware)
- UPMEM DPU hardware (used as a real NDP platform in the paper's experiments)
- LLVM compiler infrastructure (Proxics exploits compilation to implement lightweight abstractions)
Private Watchlist
- UPMEM (makes DRAM with embedded processors; directly relevant hardware target)
- MemVerge (CXL memory software stack)
- Enfabrica (CXL interconnect silicon)
The Paper
The use of disaggregated or far memory systems such as CXL memory pools has renewed interest in Near-Data Processing (NDP): situating cores close to memory to reduce bandwidth requirements to and from the CPU. Hardware designs for such accelerators are appearing, but there lack clean, portable OS abstractions for programming them. We propose a programming model for NDP devices based on familiar OS abstractions: virtual processors (processes) and inter-process communication channels (like Unix pipes). While appealing from a user perspective, a naive implementation of such abstractions is inappropriate for NDP accelerators: the paucity of processing power in some hardware designs makes classical processes overly heavyweight, and IPC based on shared buffers makes no sense in a system designed to reduce memory bandwidth. Accordingly, we show how to implement these abstractions in a lightweight and efficient manner by exploiting compilation and interconnect protocols. We demonstrate them with a real hardware platform runing applications with a range of memory access patterns, including bulk memory operations, in-memory databases and graph applications. Crucially, we show not only the benefits over CPU-only implementations, but also the critical importance of efficient, low-latency communication channels between CPU and NDP accelerators, a feature largely neglected in existing proposals.