Towards Counterfactual Explanation and Assertion Inference for CPS Debugging
DeCaF auto-generates minimal input changes that turn failing CPS tests into passing ones — then explains why, in logic engineers can read.
The Thesis
Cyber-physical systems (CPS) — software that controls physical hardware like cars, aircraft, or power grids — are notoriously hard to debug because failures often emerge from precise timing interactions between continuous sensor signals and discrete control logic. When a simulation test fails, engineers typically know something went wrong but not which input values, or at what exact moment, caused the problem. DeCaF addresses this by generating 'counterfactual' inputs — the smallest possible changes to the original failing test that would have made it pass — and then expressing those changes as logical rules engineers can inspect and reuse. The catch is that the approach is validated on only three case studies, and the quality of the generated assertions depends heavily on which combination of algorithms is chosen.
Catalyst
Large-scale simulation-based testing of CPS — using tools like Simulink and model-in-the-loop frameworks — has become standard practice in automotive and aerospace, producing enormous volumes of failing test cases that human engineers cannot manually triage. Simultaneously, counterfactual explanation methods from machine learning have matured enough to be adapted to time-series signal data, making this cross-pollination tractable today in a way it was not five years ago.
What's New
Earlier CPS debugging tools — such as fault localization techniques built on spectrum-based analysis or delta debugging — can narrow a failure down to a model component or a line of code, but they say nothing about the input conditions that triggered the fault. DeCaF shifts the focus from 'where did it break' to 'what input change would have prevented the break,' and then generalizes those changes into human-readable logical assertions (if-then predicates over signal values and timing), something prior localization tools do not produce.
The Counter
Three case studies is a thin empirical foundation for a framework aimed at industrial-scale CPS debugging — it is entirely unclear whether DeCaF's best algorithm combination generalizes beyond the specific Simulink models tested. The paper acknowledges that success rate and causal precision trade off against each other depending on which generator-model pairing is used, which means engineers deploying this tool face a tuning problem without clear guidance. Counterfactual generation for continuous time-series signals is computationally expensive, and the paper does not report runtimes or scalability to models with hundreds of inputs — a common situation in automotive or avionics CPS. Finally, the inferred logical assertions are only as interpretable as the underlying signal features chosen; in complex systems, those predicates may still be too abstract to map onto physical engineering decisions.
Longs
- ANSYS (ANSS) — simulation and verification toolchain for CPS engineering
- PTC Inc. (PTC) — model-based systems engineering and digital twin platforms
- Cadence Design Systems (CDNS) — hardware-software co-verification tools
- LDOS/Leidos (LDOS) — defense CPS integration and validation
Shorts
- Manual test triage consultancies — if assertion inference automates what senior engineers currently do by hand
- Spectrum-based fault localization tool vendors — DeCaF's input-signal framing makes component-level localization alone look incomplete
Enablers (Picks & Shovels)
- Simulink / MathWorks — dominant CPS modeling environment where such failures are generated
- scikit-learn — open-source library providing Random Forest and KD-Tree primitives used in DeCaF
- Weka / M5 model trees — decision-tree regression used for assertion inference in the paper
- Signal Temporal Logic (STL) toolboxes — formal specification language underlying CPS property checking
Private Watchlist
- Galois Inc. — formal methods and CPS assurance for defense
- BedRock Systems — safety-critical CPS software verification
- Motional — autonomous vehicle CPS validation at scale
Resources
The Paper
Verification and validation of cyber-physical systems (CPS) via large-scale simulation often surface failures that are hard to interpret, especially when triggered by interactions between continuous and discrete behaviors at specific events or times. Existing debugging techniques can localize anomalies to specific model components, but they provide little insight into the input-signal values and timing conditions that trigger violations, or the minimal, precisely timed changes that could have prevented the failure. In this article, we introduce DeCaF, a counterfactual-guided explanation and assertion-based characterization framework for CPS debugging. Given a failing test input, DeCaF generates counterfactual changes to the input signals that transform the test from failing to passing. These changes are designed to be minimal, necessary, and sufficient to precisely restore correctness. Then, it infers assertions as logical predicates over inputs that generalize recovery conditions in an interpretable form engineers can reason about, without requiring access to internal model details. Our approach combines three counterfactual generators with two causal models, and infers success assertions. Across three CPS case studies, DeCaF achieves its best success rate with KD-Tree Nearest Neighbors combined with M5 model tree, while Genetic Algorithm combined with Random Forest provides the strongest balance between success and causal precision.