GPU Acceleration of Sparse Fully Homomorphic Encrypted DNNs
FHE-encrypted neural nets just got 3x faster on AMD GPUs — privacy-preserving AI inference is no longer purely theoretical.
The Thesis
This paper shows that sparse matrix multiplication in fully homomorphic encryption can be pushed from cubic to semi-linear time complexity on AMD GPUs, hitting 3x speedups over CPU baselines using the open-source FIDESlib. That's not production-ready, but it moves FHE-based private inference from 'interesting footnote' to 'engineering problem with a clear roadmap.' The company that cracks low-latency FHE inference at scale owns the privacy-preserving AI infrastructure layer — a market with serious regulatory tailwinds in healthcare, finance, and government.
Catalyst
FIDESlib, a GPU-native open-source FHE library, only recently became available, giving researchers a real hardware-optimized substrate instead of CPU-bound reference implementations. AMD's ROCm ecosystem maturation is also critical — this work runs on AMD GPUs specifically, diversifying FHE acceleration beyond the NVIDIA monoculture.
What's New
Prior FHE matmul implementations ran in cubic time complexity on CPUs; this work exploits sparsity in both ciphertext operands to hit semi-linear complexity on GPU. The baseline comparison is against CPU-side FHE matmul, not optimized plaintext GPU inference, so the absolute gap to unencrypted computation remains enormous.
The Counter
A 3x speedup sounds exciting until you realize you're comparing against a deliberately slow baseline. FHE inference on even small networks is still thousands of times slower than plaintext GPU inference — this paper doesn't tell you how many thousands. Semi-linear complexity is theoretically lovely, but the constant factors in FHE bootstrapping are brutal; complexity class improvements often don't translate to wall-clock wins at real model sizes. More practically, the entire framework depends on sparsity in the ciphertext operands. Real production DNNs, especially transformers, aren't sparse by default, meaning you'd need to co-design the model and the encryption scheme — a constraint that most ML teams will simply refuse to accept. And building this on AMD's ROCm rather than CUDA is a principled but commercially awkward choice given where enterprise GPU fleets actually live. This is solid academic infrastructure work, not a near-term product.
Longs
- AMD
- NVDA
Shorts
- Incumbent secure multi-party computation vendors whose MPC-based approaches compete directly with FHE for private ML inference contracts
- CPU-centric FHE startups whose performance moats erode as GPU libraries mature
Enablers (Picks & Shovels)
- AMD ROCm ecosystem
- FIDESlib (open-source)
- cloud providers offering confidential computing (Azure, GCP)
Private Watchlist
- Zama
- Optalysys
- Duality Technologies
- Cornami
The Paper
Fully homomorphic encryption (FHE) has recently attracted significant attention as both a cryptographic primitive and a systems challenge. Given the latest advances in accelerated computing, FHE presents a promising opportunity for progress, with applications ranging from machine learning to information security. We target the most computationally intensive operation in deep neural networks from a hardware perspective, matrix multiplication (matmul), and adapt it for execution on AMD GPUs. We propose a new optimized method that improves the runtime and complexity of ciphertext matmul by using FIDESlib, a recent open-source FHE library designed specifically for GPUs. By exploiting sparsity in both operands, our sparse matmul implementation outperforms its CPU counterpart by up to $3.0\times$ and reduces the time complexity from cubic to semi-linear, demonstrating an improvement over existing FHE matmul implementations.