CIAO - Code In Architecture Out - Automated Software Architecture Documentation with Large Language Models
An LLM workflow can auto-generate standards-compliant architecture docs from a GitHub repo in minutes — developers who tested it found it broadly useful but imperfect.
The Thesis
Software projects routinely suffer from missing or outdated architecture documentation — the kind of high-level map that explains how a system's parts fit together. CIAO is a structured workflow that feeds a GitHub repository into a large language model and gets back a formatted architectural document, following established standards like the C4 model (a widely-used notation for describing software systems at different zoom levels) and ISO/IEC/IEEE 42010. In a study with 22 developers reviewing docs generated for their own codebases, most rated the output as comprehensible, valuable, and broadly accurate. The catch is real: diagram quality was uneven, high-level context modeling was weak, and deployment views fell short — exactly the parts that require understanding intent and infrastructure, not just code structure. The method is cheap and fast, which matters more than perfect output if the alternative is no documentation at all.
Catalyst
Modern LLMs have crossed a capability threshold where they can parse large, multi-file codebases and produce coherent prose summaries — something that was not reliably possible even two years ago. Simultaneously, code-hosting platforms like GitHub have become the de facto canonical source of truth for software projects, making repository-as-input a practical and standardized entry point. The cost of running frontier models via API has dropped enough that generating a full architectural document costs only a few dollars, making the economics viable for individual projects and teams.
What's New
Earlier LLM-based documentation tools — such as Copilot's docstring generation or tools built around code summarization — focused on local artifacts: individual functions, classes, or modules. They did not attempt to synthesize a coherent, system-level view of how components interact. CIAO instead defines a structured pipeline that maps repository contents onto a multi-view architectural template derived from recognized industry standards, forcing the model to reason at the system level rather than line by line. The authors claim this standards-grounding produces documentation that is both more complete and more directly usable by developers who need to onboard to a project.
The Counter
Twenty-two developers is a small, self-selected sample — and critically, these are people who already contributed to the repos being documented, so they bring favorable priors and contextual knowledge that helped them overlook gaps. The paper openly admits failures in the most important parts of architecture documentation: deployment views (how the system actually runs in production) and high-level context modeling (what the system does and who it serves). These are precisely the sections a new team member needs most. A document that accurately summarizes code structure but misses runtime topology and business context is arguably a false comfort — it looks like documentation but may mislead. The evaluation also relies on developer perception rather than a controlled test of whether the docs actually helped someone unfamiliar with the codebase get up to speed. Finally, any workflow that treats code as the source of truth will systematically miss architectural decisions that live in Slack threads, ADRs (architecture decision records), and people's heads — and will be confidently wrong about legacy systems where code no longer reflects original intent.
Longs
- MSFT — GitHub Copilot platform, natural integration point for repo-level doc generation
- GTLB (GitLab) — competing DevOps platform that would adopt or build similar capability
- PSTG (PagerDuty, adjacent) — incident response tooling that depends on accurate architecture context
- IGV (iShares Expanded Tech-Software ETF) — broad exposure to developer tooling sector
Shorts
- Confluence and Notion — if auto-generated architecture docs become good enough, the manual wiki-writing workflow these tools support loses urgency
- Traditional software architecture consultants and tech writers — the paper's explicit value proposition is replacing expensive manual documentation work
- Swimm and similar doc-as-code startups — their differentiation erodes if GitHub-native LLM pipelines produce equivalent output at near-zero marginal cost
Enablers (Picks & Shovels)
- OpenAI and Anthropic APIs — the underlying LLM inference that makes per-repo generation affordable
- GitHub REST API — provides structured access to repository contents, the paper's input layer
- C4 model tooling (Structurizr) — the diagramming standard the output targets
- arXiv / academic benchmarks for software documentation quality — needed to validate future iterations
Private Watchlist
- Swimlane (process automation, adjacent)
- Sourcegraph — code intelligence platform directly relevant to repository-level analysis
- Tabnine — AI coding assistant with enterprise doc features
- Mintlify — developer documentation startup with LLM-assisted features
- Continue.dev — open-source AI coding assistant that could integrate this workflow
Resources
The Paper
Software architecture documentation is essential for system comprehension, yet it is often unavailable or incomplete. While recent LLM-based techniques can generate documentation from code, they typically address local artifacts rather than producing coherent, system-level architectural descriptions. This paper presents a structured process for automatically generating system-level architectural documentation directly from GitHub repositories using Large Language Models. The process, called CIAO (Code In Architecture Out), defines an LLM-based workflow that takes a repository as input and produces system-level architectural documentation following a template derived from ISO/IEC/IEEE 42010, SEI Views \& Beyond, and the C4 model. The resulting documentation can be directly added to the target repository. We evaluated the process through a study with 22 developers, each reviewing the documentation generated for a repository they had contributed to. The evaluation shows that developers generally perceive the produced documentation as valuable, comprehensible, and broadly accurate with respect to the source code, while also highlighting limitations in diagram quality, high-level context modeling, and deployment views. We also assessed the operational cost of the process, finding that generating a complete architectural document requires only a few minutes and is inexpensive to run. Overall, the results indicate that a structured, standards-oriented approach can effectively guide LLMs in producing system-level architectural documentation that is both usable and cost-effective.