AgenTEE: Confidential LLM Agent Execution on Edge Devices
AgenTEE runs AI agents inside hardware-isolated enclaves on ARM edge devices, with under 5.15% overhead — but real-world adoption depends on ARM CCA reaching production hardware.
The Thesis
AI agents running on edge devices — phones, industrial controllers, personal computers — handle sensitive data like health records and private messages, yet today's software stacks offer weak isolation between the agent, the model, and third-party tools it calls. AgenTEE proposes placing each piece of an agent pipeline into its own confidential virtual machine, a hardware-enforced sandbox that even the device owner cannot easily inspect or tamper with. This matters most in regulated industries — healthcare, finance, defense — where processing sensitive data on a local device is preferable to sending it to a cloud, but where the device itself may be compromised or adversarially owned. The catch is that the whole system depends on ARM CCA (Confidential Compute Architecture), a hardware extension that is still maturing and not yet widely available in production devices. The paper demonstrates near-native performance in a prototype, but it is an early-stage systems result, not a deployed product.
Catalyst
ARM CCA was formally specified in 2021 and has only recently become available in prototype silicon and emulation environments, making it possible to build a real prototype rather than a purely theoretical design. Simultaneously, the explosion of LLM-based agents that call external APIs and tools has dramatically widened the attack surface on edge devices, creating urgent demand for isolation primitives that did not exist at scale two years ago.
What's New
Prior confidential computing work on edge devices, such as ARM TrustZone-based TEEs (Trusted Execution Environments — small, hardware-isolated regions of a chip), was designed for simple, narrow tasks like key storage or biometric verification. Those approaches cannot fit a full LLM inference engine or a multi-step agent runtime. AgenTEE is the first system, according to the authors, to decompose a complete agent pipeline — runtime, model inference, and third-party tools — into separately attested confidential VMs that communicate over verifiable channels, using ARM CCA rather than the older TrustZone model.
The Counter
The entire architecture rests on ARM CCA, a hardware feature that is not yet in the phones or laptops people actually use. Publishing a prototype that requires non-production silicon is not the same as solving the problem — it is a proof of concept that could sit on a shelf for years while the hardware ecosystem catches up, or never catches up if ARM and its partners deprioritize CCA in cost-sensitive consumer segments. Even granting that CCA ships widely, the security model has a fundamental gap: confidential VMs protect data at rest and in use inside the enclave, but they cannot stop a malicious tool from returning a carefully crafted response that manipulates the agent's reasoning. Prompt injection — where a third-party service tricks the agent into taking unintended actions — is arguably the dominant real-world attack vector today, and AgenTEE's isolation boundaries do not address it. Finally, the performance overhead figure is almost certainly optimistic. Multi-cVM coordination, attestation handshakes, and encrypted inter-VM channels add latency with every tool call; in a real agentic loop that makes dozens of API calls, those costs compound in ways a single-benchmark number does not capture.
Longs
- ARM Holdings (ARM) — ARM CCA is the hardware foundation; broader adoption of CCA-capable silicon directly expands ARM's premium IP licensing opportunity
Shorts
- Cloud AI inference providers (AWS Bedrock, Azure AI, Google Vertex) — if sensitive agent workloads migrate to attested edge devices, the case for sending data to cloud inference APIs weakens for privacy-sensitive verticals
- Existing mobile TEE vendors relying solely on TrustZone — AgenTEE's architecture suggests TrustZone is too constrained for agent workloads, potentially making their integration work obsolete if CCA takes over
Enablers (Picks & Shovels)
- ARM Confidential Compute Architecture (CCA) silicon and firmware stack
- Open-source inference engines such as llama.cpp and MLC-LLM, which AgenTEE would need to run inside a cVM
- Remote attestation infrastructure — services that verify a cVM's integrity before sharing secrets with it
Private Watchlist
- Edgeless Systems — builds confidential computing infrastructure and tooling that could complement or integrate with CCA-based deployments
- Anjuna Security — specializes in confidential VM deployment, directly adjacent to the multi-cVM architecture proposed here
The Paper
Large Language Model (LLM) agents provide powerful automation capabilities, but they also create a substantially broader attack surface than traditional applications due to their tight integration with non-deterministic models and third-party services. While current deployments primarily rely on cloud-hosted services, emerging designs increasingly execute agents directly on edge devices to reduce latency and enhance user privacy. However, securely hosting such complex agent pipelines on edge devices remains challenging. These deployments must protect proprietary assets (e.g., system prompts and model weights) and sensitive runtime state on heterogeneous platforms that are vulnerable to software attacks and potentially controlled by malicious users. To address these challenges, we present AgenTEE, a system for deploying confidential agent pipelines on edge devices. AgenTEE places the agent runtime, inference engine, and third-party applications into independently attested confidential virtual machines (cVMs) and mediates their interaction through explicit, verifiable communication channels. Built on Arm Confidential Compute Architecture (CCA), a recent extension to Arm platforms, AgenTEE enforces strong system-level isolation of sensitive assets and runtime state. Our evaluation shows that such multi-cVMs system is practical, achieving near-native performance with less than 5.15% runtime overhead compared to commodity OS multi-process deployments.