Toward Zero-Egress Psychiatric AI: On-Device LLM Deployment for Privacy-Preserving Mental Health Decision Support
Three small LLMs running entirely on a phone could support psychiatric triage in prisons and remote clinics — but real-world validation is still missing.
The Thesis
This paper proposes running a trio of small, compressed language models entirely on a mobile device to provide psychiatric decision support without ever sending patient data to a server. The target settings — military units, correctional facilities, and remote clinics — are places where the stigma and legal risk of data exposure actively suppress help-seeking. The approach uses quantization (shrinking model weights so they fit on phone memory) and ensemble consensus (having three models vote on a diagnosis) to approximate the accuracy of a cloud-based system. The catch is significant: the evaluation is preliminary, lacks a formal clinical benchmark, and the paper does not report head-to-head accuracy numbers against established diagnostic gold standards. Until independent clinicians validate the outputs in real settings, this remains a promising architecture in search of evidence.
Catalyst
Three open-source models — Gemma, Phi-3.5-mini, and Qwen2 — reached a quality threshold in 2024 where they can run in quantized form on consumer mobile hardware without requiring data center GPUs. Simultaneously, regulatory pressure around patient data sovereignty (HIPAA, VA data policies, correctional system compliance requirements) has made cloud-based psychiatric AI politically and legally difficult to deploy in exactly the high-need environments the paper targets.
What's New
The authors' own earlier work built a cloud-based consortium of fine-tuned LLMs for psychiatric diagnosis, which required patient data to leave the device entirely. Other on-device AI health tools have generally used narrow classifiers or rule-based systems, not instruction-following language models capable of DSM-5-aligned differential diagnosis. This paper re-architects that cloud pipeline for fully local execution, adding an on-device orchestration layer that coordinates the three models and resolves disagreements through consensus — a design the authors claim preserves diagnostic accuracy while eliminating data egress.
The Counter
The paper's most important claim — that on-device accuracy is 'comparable' to the server-side predecessor — is supported by no published numbers. We are asked to take this on faith from the same authors who built the system, with no external validation, no clinical trial, and no comparison to the structured clinical interviews that psychiatrists actually use as a diagnostic standard. The three models chosen (Gemma, Phi-3.5-mini, Qwen2) are capable general-purpose models, but they were not designed for psychiatric diagnosis, and fine-tuning on unspecified datasets introduces unknown biases that could be dangerous in a population already at elevated suicide risk. Even if the accuracy were proven acceptable, the regulatory path for an LLM-based psychiatric decision support tool — particularly one deployed in correctional settings where patients have limited ability to refuse or seek alternatives — is long, expensive, and uncertain. The privacy argument is real, but existing solutions such as federated learning or on-premise server deployment could achieve similar privacy guarantees without the accuracy trade-offs of running compressed models on a phone. This paper describes a system design, not a validated clinical tool.
Longs
None listed.
Shorts
- Eleos Health and similar cloud-based behavioral health AI documentation vendors, whose core value proposition depends on server-side inference and whose pipeline architecture is precisely what this paper proposes to replace
- Credentialing and liability intermediaries who profit from the gatekeeping role between AI psychiatric tools and clinicians, which a self-screening on-device tool could partially bypass
Enablers (Picks & Shovels)
- Hugging Face (hosts and versions Gemma, Phi-3.5-mini, Qwen2 — the exact models used)
- llama.cpp and MLC LLM (open-source runtimes that make quantized LLM inference feasible on mobile CPUs)
- Google's MediaPipe / AI Edge (on-device ML deployment framework for Android/iOS)
- Apple Core ML (on-device inference pipeline for iOS deployment)
Private Watchlist
- Osmind (psychiatric EHR with treatment-resistant focus)
- Headspace Health (digital mental health, correctional and employer contracts)
- Talkiatry (telepsychiatry, potential interest in offline-capable tools)
The Paper
Privacy represents one of the most critical yet underaddressed barriers to AI adoption in mental healthcare -- particularly in high-sensitivity operational environments such as military, correctional, and remote healthcare settings, where the risk of patient data exposure can deter help-seeking behavior entirely. Existing AI-enabled psychiatric decision support systems predominantly rely on cloud-based inference pipelines, requiring sensitive patient data to leave the device and traverse external servers, creating unacceptable privacy and security risks in these contexts. In this paper, we propose a zero-egress, on-device AI platform for privacy-preserving psychiatric decision support, deployed as a cross-platform mobile application. The proposed system extends our prior work on fine-tuned LLM consortiums for psychiatric diagnosis standardization by fundamentally re-architecting the inference pipeline for fully local execution -- ensuring that no patient data is transmitted to, processed by, or stored on any external server at any stage. The platform integrates a consortium of three lightweight, fine-tuned, and quantized open-source LLMs -- Gemma, Phi-3.5-mini, and Qwen2 -- selected for their compact architectures and proven efficiency on resource-constrained mobile hardware. An on-device orchestration layer coordinates ensemble inference and consensus-based diagnostic reasoning, producing DSM-5-aligned assessments for conditions. The platform is designed to assist clinicians with differential diagnosis and evidence-linked symptom mapping, as well as to support patient-facing self-screening with appropriate clinical safeguards. Initial evaluation demonstrates that the proposed zero-egress deployment achieves diagnostic accuracy comparable to its server-side predecessor while sustaining real-time inference latency on commodity mobile hardware.