Compliant But Unsatisfactory: The Gap Between Auditing Standards and Practices for Probabilistic Genotyping Software
DNA software sends people to prison — and its audits are toothless theater dressed up as rigor.
The Thesis
Probabilistic genotyping software (PGS) is embedded in U.S. criminal courts, but the primary audit standard governing it (ASB 018) is so vague that compliant audits can sidestep the standard's own stated goals — including setting limits on when the software should and shouldn't be used. Five real audit reports confirm this isn't theoretical. The governance gap creates both legal liability for software vendors and a policy opening for better-designed competitors or third-party auditors.
Catalyst
AI governance mandates are accelerating across federal and state agencies in 2025-2026, putting audit standards under scrutiny that didn't exist two years ago. Courts are also facing an uptick in Daubert challenges to PGS outputs, raising the cost of weak audits for everyone in the chain.
What's New
Prior critiques of PGS (e.g., work on TrueAllele and STRmix) targeted the software itself. This paper targets the audit infrastructure, showing that even 'passing' audits under ASB 018 can leave core safety questions unanswered — a layer up from software criticism.
The Counter
This paper diagnoses a governance process problem, not a software failure — and process problems in forensic standards get fixed on decade-long timescales, not investment cycles. ASB 018 is an OSAC-affiliated standard; revising it requires consensus from forensic labs, prosecutors, defense bar, and vendors, all of whom have conflicting incentives. Even if a tighter standard passes, enforcement is voluntary: crime labs choose which standards to follow. The commercial PGS market is also tiny and largely government-contract-driven, meaning margin pressure from 'better auditors' is nearly nonexistent. Finally, the paper's call for more rigorous audits could paradoxically entrench incumbents like Cybergenetics, who have the resources to pass harder audits while locking out open-source alternatives.
Longs
None listed.
Shorts
- Cybergenetics (TrueAllele) — proprietary black-box model most exposed to tightened disclosure requirements
- STRmix — dominant market share means most to lose from stricter use-boundary mandates
- Crime labs that have built workflows around current PGS without documented failure-mode restrictions
Enablers (Picks & Shovels)
- Legal tech firms building court-admissibility compliance tooling
- NIST (standards infrastructure)
- Academic forensic science labs positioned to offer independent audits
Private Watchlist
- Cybergenetics (TrueAllele)
- STRmix (ESR Group)
- Forensic Bioinformatics
- AI audit/assurance firms entering forensic verticals
The Paper
AI governance efforts increasingly rely on audit standards: agreed-upon practices for conducting audits. However, poorly designed standards can hide and lend credibility to inadequate systems. We explore how an audit standard's design influences its effectiveness through a case study of ASB 018, a standard for auditing probabilistic genotyping software -- software that the U.S. criminal legal system increasingly uses to analyze DNA samples. Through qualitative analysis of ASB 018 and five audit reports, we identify numerous gaps between the standard's desired outcomes and the auditing practices it enables. For instance, ASB 018 envisions that compliant audits establish restrictions on software use based on observed failures. However, audits can comply without establishing such boundaries. We connect these gaps to the design of the standard's requirements such as vague language and undefined terms. We conclude with recommendations for designing audit standards and evaluating their effectiveness.