Building Better Environments for Autonomous Cyber Defence
A cross-sector workshop produced the first consolidated playbook for building AI training environments that teach software agents to defend computer networks autonomously.
The Thesis
Teaching an AI agent to defend a computer network requires a realistic practice environment — and until now, no single resource told builders how to make one well. This paper synthesizes lessons from researchers, government officials, and industry practitioners into a two-part contribution: a framework for mapping the gap between simulated cyber environments and real networks, and a set of best-practice guidelines for training and evaluating autonomous defence agents. The target use cases include corporate networks, government systems, and critical infrastructure such as power grids and water treatment facilities. The catch is that this is a design guide, not a deployed system — it produces no benchmark numbers and proves no agent works better than a human.
Catalyst
Reinforcement learning for cyber defence (RL-ACD) has accumulated several years of published research, but practitioners kept rediscovering the same pitfalls in isolation. The November 2025 workshop was the first structured attempt to pool that dispersed tradecraft into a shared framework, timed to a moment when government interest in autonomous cyber operations has materially increased — particularly following high-profile intrusions into critical infrastructure networks.
What's New
Prior RL-ACD work — including environments such as CybORG, CAGE, and NASIM — each made individual design choices without a unifying theory of what a good environment even looks like. Those projects documented their own assumptions but did not produce cross-cutting guidance on how simulation fidelity, action space design, and agent evaluation interact. This paper provides the first consolidated framework that explicitly maps the interface between a simulated training environment and a real operational network, giving future builders a shared vocabulary and checklist.
The Counter
This paper is a workshop summary dressed as a research contribution — it offers no new algorithm, no benchmark, and no empirical result. The 'framework' for decomposing environment-to-real-system interfaces is a taxonomy, not a validated methodology; we have no evidence that following these guidelines actually produces better-trained agents or safer deployed systems. Reinforcement learning agents have repeatedly failed to transfer from simulation to real environments in robotics and gaming, and cyber networks are far more heterogeneous and adversarially adaptive than either domain. The authors themselves note that the tradecraft they are capturing was previously undocumented, which is an admission that the field is still pre-paradigmatic. Practitioners reading this for actionable guidance will find useful checklists but no proof that the checklist items predict success. The paper also cannot resolve the fundamental tension it identifies: the more realistic a training environment, the harder it is to share safely with the research community.
Longs
- PANW (Palo Alto Networks) — autonomous SOC products are a stated roadmap item
- CRWD (CrowdStrike) — AI-native endpoint defence, direct overlap with ACD agent deployment
- CACI International (CACI) — government cyber and defence IT contracts
- BOTZ (Global Robotics & AI ETF) — broad AI-in-defence exposure
- Leidos (LDOS) — critical infrastructure cybersecurity for US government clients
Shorts
- Traditional SIEM vendors (Splunk, IBM QRadar) — if autonomous agents replace human-in-the-loop alert triage, demand for analyst-facing dashboards shrinks
- Managed Security Service Providers (MSSPs) — ACD agents are explicitly designed to automate the human analyst role that MSSPs sell
Enablers (Picks & Shovels)
- CybORG simulation environment (open-source RL cyber gym, University of Adelaide / TTCP CAGE)
- CAGE Challenge series (public benchmark challenges for ACD agents)
- OpenAI Gymnasium (standard RL environment interface that ACD environments wrap)
- MITRE ATT&CK framework (structured adversary behaviour taxonomy used to design realistic attack scenarios)
Private Watchlist
- Dstl (UK Defence Science and Technology Laboratory) — co-originator of CAGE challenge environments
- Morpheus Labs — RL-based autonomous security research
- Rebellion Defense — AI for defence cyber operations
- Protected Harbor — critical infrastructure network defence
Resources
The Paper
In November 2025, the authors ran a workshop on the topic of what makes a good reinforcement learning (RL) environment for autonomous cyber defence (ACD). This paper details the knowledge shared by participants both during the workshop and shortly afterwards by contributing herein. The workshop participants come from academia, industry, and government, and have extensive hands-on experience designing and working with RL and cyber environments. While there is now a sizeable body of literature describing work in RL for ACD, there is nevertheless a great deal of tradecraft, domain knowledge, and common hazards which are not detailed comprehensively in a single resource. With a specific focus on building better environments to train and evaluate autonomous RL agents in network defence scenarios, including government and critical infrastructure networks, the contributions of this work are twofold: (1) a framework for decomposing the interface between RL cyber environments and real systems, and (2) guidelines on current best practice for RL-based ACD environment development and agent evaluation, based on the key findings from our workshop.