ProbeLogits: Kernel-Level LLM Inference Primitives for AI-Native Operating Systems
Kernel-level LLM logit probing hits F1=0.98 on OS action safety — no training required.
The Thesis
ProbeLogits embeds LLM inference directly into an OS kernel, reading raw logit distributions before text is generated to classify agent actions as safe or dangerous with zero learned parameters. On a 260-prompt OS action benchmark it hits F1=0.980 with perfect precision, and on 1,000 real ToxicChat conversations it reaches 89% of Llama Guard 3's performance — a supervised model — while adding no fine-tuning overhead. If this holds at scale, safety enforcement moves below the application layer, making it structurally harder to jailbreak than any prompt-filter running in userspace.
Catalyst
7B models at 4-bit quantization now run a full forward pass in 65ms on commodity hardware — fast enough to gate every agent action without killing latency. The maturation of WASM sandboxing as an agent execution primitive created a natural choke point that a kernel can own.
What's New
Prior safety classifiers like Llama Guard 3 require supervised fine-tuning on labeled datasets and operate at the application layer, where a sufficiently motivated agent or prompt injection can route around them. ProbeLogits reads logits inside the kernel before any output token is committed, sitting below the WASM sandbox boundary — a fundamentally different trust model.
The Counter
This is a single-author preprint benchmarked almost entirely on data the author generated. A 260-prompt test set is not a rigorous evaluation — it's a demonstration. The ToxicChat numbers are more credible but still show a real gap versus supervised baselines, and the paper's framing of '89% of Llama Guard 3' papers over the fact that Llama Guard 3 was trained specifically for this task while ProbeLogits gets to cherry-pick α post-hoc. More fundamentally, the threat model assumes adversaries are constrained by the WASM sandbox and kernel host functions — a determined attacker targeting the inference kernel itself, the KV cache state, or the model weights has a much larger attack surface than this framing acknowledges. The KV cache fork/checkpoint idea is genuinely interesting but is a research curiosity, not a shipping feature. Anima OS has zero production users. Calling this an 'AI-native OS' primitive when it runs on one person's bare-metal x86 project is a significant leap.
Longs
- NVDA
- AMD
- MSFT
- GOOGL
Shorts
- Application-layer content moderation vendors whose moat is classifier training data
- LlamaGuard/Meta's supervised safety pipeline if zero-parameter approaches close the gap
- Cloud-based LLM API safety wrappers that charge per-call for policy enforcement
Enablers (Picks & Shovels)
- NVDA (GPU compute for 4-bit inference)
- Hugging Face (7B model ecosystem)
- Bytecode Alliance (WASM runtime standards)
- AMD (ROCm inference stack)
Private Watchlist
- Anima OS (the paper's own system)
- Weights & Biases
- Robust Intelligence
- Protect AI
The Paper
An OS kernel that runs LLM inference internally can read logit distributions before any text is generated -- and act on them as a governance primitive. I present ProbeLogits, a kernel-level operation that performs a single forward pass and reads specific token logits to classify agent actions as safe or dangerous, with zero learned parameters. On a 260-prompt OS action benchmark (9 categories including adversarial attacks), ProbeLogits achieves F1=0.980, Precision=1.000, and Recall=0.960 using a general-purpose 7B model at 4-bit quantization. On ToxicChat (1,000 human-annotated real conversations), it achieves F1=0.790 at default calibration strength $α$=1.0, improving to F1=0.837 at $α$=0.5 -- 89% of Llama Guard 3's F1~0.939 with zero learned parameters. A key design contribution is the calibration strength $α$, which serves as a deployment-time policy knob rather than a learned hyperparameter. By adjusting $α$, the OS can enforce strict policies for privileged operations ($α\geq 0.8$, maximizing recall) or relaxed policies for conversational agents ($α$=0.5, maximizing precision). Contextual calibration improves accuracy from 64.8% to 97.3% on the custom benchmark. I implement ProbeLogits within Anima OS, a bare-metal x86_64 OS written in 80,400 lines of Rust. Because agent actions must pass through kernel-mediated host functions, ProbeLogits enforcement operates below the WASM sandbox boundary, making it significantly harder to circumvent than application-layer classifiers. Each classification costs 65ms on 7B -- fast enough for per-action governance. I also show that treating KV cache as process state enables checkpoint, restore, and fork operations analogous to traditional process management. To my knowledge, no prior system exposes LLM logit vectors as OS-level governance primitives.