ANGELA
Analytical Network for Gated Evidence Logic and Assessment
A Gated Architecture for Evidence-Bounded Analysis
George Powell
Abstract
Multi-step analyses can move beyond what available evidence actually supports when missing measurements are substituted, unsupported relationships are inferred, or later evidence is allowed to compensate for earlier evidentiary gaps. This study introduces ANGELA (Analytical Network for Gated Evidence Logic and Assessment), a gated evidence-governance architecture designed to make the boundary between evidence and inference explicit and enforce that boundary throughout an analysis. ANGELA receives structured evidence through predefined Tickets and evaluates it sequentially through gates. Advancement occurs only when the evidence satisfies the requirement of the current gate; otherwise, the analysis stops.
ANGELA was empirically tested using Federal Aviation Administration runway-incursion evidence across five sequential Tickets examining challenged-control outcomes, severity, operational pathways, upstream differentiation, and stop-instruction conditions. The Tickets produced different analytical paths and outcomes, including a challenged-control failure proportion of 13.80%, statistically significant differentiation among operational pathways, qualified associations, and evidentiary stops where required comparisons were not identifiable. Across all five Tickets, no violation of ANGELA’s governing rules was observed. Qualifying evidence permitted advancement, unsupported interpretations remained bounded, and missing evidence did not generate inferred bridges.
The FAA test therefore moved ANGELA’s invariance from a design claim to an observed empirical result within a single domain. These findings do not establish cross-domain generalizability or validate the scientific design of every Ticket. They demonstrate that, under the conditions tested, ANGELA consistently advanced when warranted, qualified when required, and stopped when the evidence stopped, providing an empirical basis for subsequent cross-domain validation.
1. Introduction — The Problem ANGELA Addresses
Complex analyses often require evidence to support a sequence of connected steps. An observation supports one conclusion, that conclusion becomes the basis for another question, and the analysis continues toward a downstream result. The difficulty is that evidence supporting one step does not automatically support the next. At some point in that progression, an analysis may move beyond what the available evidence has actually established (Rae & Alexander, 2017; Goerlandt et al., 2017).
Multi-step analyses do not necessarily contain explicit rules specifying what evidence must be present before each successive analytical transition is permitted. When those requirements are unspecified, decisions about whether the evidence is sufficient to continue necessarily remain part of analytical judgment. Related measures may be considered sufficiently comparable, approximations may be accepted, or relationships may be inferred when direct measurement is unavailable. These are not defects in analysis or failures of the analyst. They are decisions the analytical process leaves unresolved when evidentiary progression has not been specified in advance (Rae & Alexander, 2017; Bani-Mustafa et al., 2020).
That creates a reproducibility problem at the level of the analytical process. The same evidence and technically valid calculations can support different analytical paths when the requirements governing progression have not been explicitly established. Consistency in calculation therefore does not by itself establish consistency in evidentiary progression. A method is also needed for specifying whether the evidence required for each successive analytical step is present before that step is allowed to occur (Goerlandt et al., 2017; Bani-Mustafa et al., 2020).
The problem addressed in this study is therefore not simply how to analyze evidence, but how to govern its progression through an analysis. The central question is:
Can an analytical architecture make the boundary between evidence and inference explicit—and enforce that boundary consistently?
2. ANGELA: Architecture and Governance
2.1 What is ANGELA?
ANGELA—the Analytical Network for Gated Evidence Logic and Assessment—is an evidence-governance architecture designed to determine how far available evidence is entitled to carry an analysis. Its purpose is not to produce a predetermined answer, but to govern the progression from evidence to conclusion by requiring each analytical step to be supported before the next is permitted.
ANGELA operates through a defined sequence:
Ticket → Structured Evidence → Gate → Test → Pass/Stop → Next Gate → Permitted Output
The process begins with a Ticket that establishes the evidentiary requirements and their order. Once populated, the structured evidence is evaluated at the appropriate gate. The gate tests whether the predetermined requirement has been satisfied. If it has, ANGELA permits the analysis to advance. If it has not, ANGELA stops.
The stop rule is fundamental to the architecture. When required evidence is absent, incomplete, incompatible, or otherwise fails the predetermined requirement, ANGELA does not fill the missing space with an assumption, substitute a related measurement, or extrapolate across the unsupported transition.
In this way, ANGELA does not leave the permissible extent of analytical progression unspecified. It determines how far the available evidence permits the analysis to travel.
2.2 The Ticket and Structured Evidence
The Ticket is the structure through which evidence is presented to ANGELA. It is an ordered container established in advance by the architects of the Ticket. Those architects determine what evidence fields are required, the sequence in which those fields appear, and the conditions associated with them.
The Ticket itself is not structured evidence. It provides the designated places where evidence must go. The distinction is important: the Ticket establishes the structure and order, while the evidence supplied to those positions provides the content.
Once the required evidence has been placed into its designated positions, the populated Ticket provides the structured evidence that ANGELA accepts as input. ANGELA does not determine the order after seeing the evidence, rearrange the fields, or redesign the Ticket during the analysis. It receives the evidence in the structure established by the Ticket's architects and evaluates it accordingly.
The relationship can therefore be expressed simply:
Ticket = where the evidence goes and in what order.
Structured evidence = what ANGELA receives and evaluates.
2.3 Gates, Advancement, and the Stop Rule
ANGELA evaluates structured evidence sequentially through a series of gates. Each gate represents a defined evidentiary requirement that must be satisfied before the analysis is permitted to continue. Advancement therefore depends on the evidence presented at the current gate, not on the overall strength of the Ticket or on evidence that may appear later.
The advancement rule is:
Current Gate → Required Evidence → Requirement Satisfied → Advance
When the required evidence satisfies the predetermined requirement, ANGELA permits movement to the next gate. Evidence assigned to a later position cannot be pulled forward to compensate for weakness at the current gate.
The corresponding stop rule is:
Current Gate → Required Evidence → Requirement Not Satisfied → Stop
When the required evidence is absent, insufficient, incompatible, or otherwise fails the gate's requirement, ANGELA stops. It does not search ahead for stronger evidence, substitute a related variable, infer the missing connection, or relax the requirement so that the analysis can continue.
A stop therefore identifies an empirical boundary. It marks the point beyond which the supplied evidence no longer authorizes further progression. A stop does not represent a failure of the architecture.
Stopping when the evidence stops is ANGELA functioning as designed.
2.4 Invariance
ANGELA’s governance logic remains invariant across runs. The evidence presented to the architecture may change. Ticket contents may change. The path through the gates may change, and different evidence may produce different stopping points and outputs. None of those differences changes the rules ANGELA applies.
The same requirements for advancement and stopping apply regardless of whether following those requirements produces a completed pathway, a qualified result, or an early stop. ANGELA does not alter its rules to accommodate the particular evidence presented or to favor completion of the analysis.
The governing principle is therefore:
Different evidence. Different Tickets. Different paths. Same rules.
2.5 Permitted Outputs
ANGELA’s output is determined by how far the evidence is permitted to travel through the gated architecture. Because different Tickets may satisfy different numbers of gates, ANGELA does not require every analysis to produce the same type of endpoint.
A gate-pass records that the evidence satisfied the current requirement and authorized advancement. A gate-stop records that the requirement was not satisfied and identifies the point at which further progression became unsupported. A completed path occurs when the evidence satisfies every required gate in the Ticket. An evidentiary state report preserves the path actually traveled, including the gates passed and, when applicable, the point at which the analysis stopped.
These are bounded outputs. ANGELA reports what the supplied evidence permits at the point reached rather than extending the result beyond that evidentiary boundary.
The output is therefore not simply an answer. It is an answer with its evidentiary boundary preserved.
ANGELA operates within the broader tradition of systems-oriented safety analysis while addressing a narrower problem: governance of evidentiary progression within an analytical chain (Rasmussen, 1997; Leveson, 2004).
3. Empirical Test: FAA Runway-Incursion Evidence
ANGELA was tested using FAA runway-incursion evidence through five sequential Tickets. The purpose of the test was to determine whether ANGELA would maintain its governing rules as the evidentiary questions and conditions changed from one Ticket to the next. The five Tickets therefore represented separate runs through the same architecture rather than five different versions of ANGELA. (Federal Aviation Administration [FAA], n.d.-a; FAA Aviation Safety Information Analysis and Sharing [ASIAS], n.d.).
For every run, the requirements remained fixed. ANGELA was required to accept the structured evidence presented by the Ticket, follow its prescribed order, apply the gate rules, advance when the evidence satisfied the current requirement, stop when it did not, and report only what the evidence permitted.
The evidence presented across the five Tickets could change, as could the path taken and the resulting output. The governing rules could not.
The FAA test therefore placed the architecture under a straightforward requirement:
Different evidence could produce different results. It could not produce different rules.
4. Results — Five FAA Tickets
4.1 Ticket 1 — Challenged Control
Ticket 1 asked a foundational question: given an observable implemented control challenge, how often did the control fail? From 29,851 FAA runway-incursion records, 4,465 events met the eligibility requirements for a challenged-control event with an identifiable outcome. Of these, 3,849 were classified Held and 616 Failed, producing:
P(Failed | Challenged) = 13.80%.
ANGELA permitted that conditional transition probability because both the transition-specific denominator and corresponding Held/Failed outcomes were identifiable. The result was confined to the transition actually measured. The 13.80% failure proportion was not reinterpreted as an accident probability, catastrophe probability, or probability of a later downstream state.
4.2 Ticket 2 — Severity
Ticket 2 asked whether challenged-control outcome was associated with independently recorded FAA severity. The analysis included 4,368 events with both an identifiable Held/Failed outcome and usable FAA severity information; 97 events with missing severity remained missing rather than being recoded. (FAA, 2013/2021; FAA, n.d.-a).
The analysis detected an association:
χ²(4) = 22.713, p = .000144; Cramér’s V = .072
An exact-style check produced p = .000268.
ANGELA permitted the observed association to stand but did not convert it into the stronger claim that control failure meant greater severity. Native FAA severity categories were preserved, and missing observations were not reassigned to create a more complete comparison.
4.3 Ticket 3 — Operational Pathways
Ticket 3 asked whether the aggregate challenged-control transition was composed of distinguishable operational pathways. The analysis separated three principal single-challenge pathways while retaining the same Held/Failed outcome definition.
The observed failure proportions were:
Go-around: 11.82%
Stop instruction: 48.39%
Cancelled takeoff: 15.90%
The pathways differed substantially:
χ²(2) = 324.07, p < 10⁻⁷⁰; Cramér’s V = .283.
The aggregate challenged-control transition therefore contained meaningful pathway heterogeneity. In particular, the stop-instruction pathway exhibited a substantially higher observed failure proportion than either go-around or cancelled takeoff.
4.4 Ticket 4 — Upstream Differentiation
Ticket 4 moved upstream and asked whether prespecified conditions could distinguish the operational pathways identified in Ticket 3. Two variables were frozen before calculation: INCDNT_TYPE_FAA_CODE and ACFT_1_FLTCNDT_CODE. Other potentially useful variables were excluded from this first analysis rather than introduced after the results were known.
Both frozen upstream fields differentiated the operational pathways. Incident type produced:
χ²(10) = 64.307, p = 5.50 × 10⁻¹⁰; Cramér’s V = .089
The aircraft operational-category field produced:
χ²(18) = 51.183, p = .0000499; Cramér’s V = .080.
ANGELA retained the native categories and existing missingness and did not collapse categories merely to obtain statistical significance. The Ticket therefore supported upstream differentiation while preserving the evidentiary structure established before calculation.
4.5 Ticket 5 — Stop-Instruction Follow-up
Ticket 5 entered the pathway with the highest observed failure proportion and asked whether four prespecified narrative conditions could distinguish Held from Failed outcomes within stop-instruction challenges.
The analysis contained 341 stop-instruction challenges: 176 Held and 165 Failed. Four conditions were frozen before outcome comparison:
progression, boundary position, acknowledgment/readback, and repetition.
The narrative conditions were coded without consulting the Held/Failed outcome. Silence remained Unknown rather than being interpreted as evidence of absence. Only after the condition fields had been created were the outcomes joined for comparison.
This Ticket produced both testable and untestable conditions. Where the evidence supported legitimate contrasting categories, analysis could proceed. Where it did not, ANGELA stopped. In particular, progression and repetition were NOT IDENTIFIABLE under the frozen rules because the evidence did not establish the contrasting categories required for the proposed comparisons.
ANGELA did not manufacture those missing categories, reinterpret silence as evidence, or modify the coding rules after observing the outcome.
Those were genuine evidentiary stops, not repaired analyses.
5. Did ANGELA Obey Its Own Rules?
Across all five FAA Tickets, no violation of ANGELA’s evidence-governance rules was observed. The evidence and analytical conditions changed from one Ticket to the next, but the rules governing advancement and stopping did not.
When the evidence satisfied the requirement of the current gate, ANGELA permitted the analysis to advance. When the required evidence did not satisfy that requirement, ANGELA stopped. Missing information was not converted into evidence, unsupported categories were not manufactured, and evidence appearing later in an analysis was not used to bridge an unsupported earlier transition.
The five Tickets provided different opportunities for these rules to operate. Ticket 1 permitted calculation of a defined conditional transition probability but did not permit that probability to be extended to accident or catastrophe. Ticket 2 permitted an association with FAA severity but not a stronger interpretation that failure meant greater severity. Tickets 3 and 4 permitted additional analyses because the required evidence was available. Ticket 5 demonstrated the stop rule directly when progression and repetition could not support legitimate comparisons under the predetermined coding rules.
Across the five runs, the observed behavior was therefore consistent:
Qualifying evidence → Advance
Nonqualifying evidence → Stop
Missing evidence → No invented bridge
Within the five-ticket FAA test, ANGELA obeyed its own rules.
6. Discussion — What FAA Demonstrated
Before the FAA test, ANGELA existed as an architecture with defined operating rules. Its invariance was therefore a design claim: the architecture was intended to apply the same evidence-governance rules regardless of the evidence presented, but that behavior had not yet been demonstrated across repeated analyses using real-world evidence.
The five FAA Tickets changed the status of that claim. Across the sequence, the evidentiary questions changed, the populations and variables changed, the statistical results changed, and the permitted endpoints changed. ANGELA’s governing rules did not. (Goerlandt et al., 2017; Paiva & Tewari, 2022).
The importance of the FAA test therefore does not rest on every Ticket reaching a completed analytical endpoint. In fact, requiring every Ticket to succeed would undermine the purpose of the architecture. ANGELA must be capable of permitting an analysis when the evidence supports it and refusing further progression when it does not.
That distinction was observable across the five runs. ANGELA permitted calculation of a challenged-control failure proportion while preventing that value from being interpreted as a probability of a different downstream event. It permitted a statistical association with FAA severity while withholding a stronger unsupported interpretation. It permitted decomposition into operational pathways and subsequent upstream differentiation when the necessary evidence was available. Finally, when the evidence in Ticket 5 could not establish legitimate contrasting categories for progression and repetition, ANGELA stopped rather than altering the evidentiary rules to complete those analyses.
6.1 What ANGELA Added
The FAA evidence could have been analyzed without ANGELA. The Tickets could have been constructed, and the same statistical procedures could have been performed. What would have been absent was a fixed mechanism governing how far the evidence was permitted to carry the analysis.
Without that constraint, decisions about whether one measure was sufficiently comparable to another, whether later evidence could compensate for weakness earlier in the chain, whether a related measure could substitute for unavailable evidence, or whether reasoning could continue beyond direct evidentiary support would remain matters of analytical judgment. Such judgments are not evidence of misconduct or deficient analysis. They are decisions that must be made when the analytical process itself does not specify the requirements governing progression (Rae & Alexander, 2017; Bani-Mustafa et al., 2020).
ANGELA made those requirements explicit. It required the same advancement and stopping rules to operate regardless of whether applying those rules produced advancement, a qualified finding, or a stop. Its contribution was therefore not the creation of the FAA evidence or the statistical methods applied to it. It was the reproducible separation between what the evidence supported and what the predefined analytical process permitted to be inferred from it.
The scientific result concerning ANGELA is therefore not that every Ticket succeeded. The result is that ANGELA behaved according to the same evidence rules when Tickets succeeded, when the evidence supported only a qualified finding, and when the evidence required a stop.
Before FAA, invariant evidence governance was something ANGELA was designed to do.
Across Tickets 1–5, it became something ANGELA was observed to do.
7. Limitations — What ANGELA Has Not Demonstrated
The FAA test provides an initial empirical demonstration of ANGELA’s operation, but its conclusions remain bounded by the scope of that test. All five Tickets were evaluated within a single domain using FAA runway-incursion evidence. The results therefore demonstrate that ANGELA maintained its governing rules across different questions, evidence conditions, analytical paths, and stopping points within that domain. They do not establish that the architecture will maintain the same invariance when applied to entirely different domains with different evidence structures and newly designed Tickets.
Cross-domain generalizability therefore remains untested. A stronger validation would require ANGELA to encounter a new domain, accept a Ticket constructed for that domain, process different forms of structured evidence, and still apply the same rules governing advancement, qualification, and stopping. Until such testing occurs, the defensible claim is limited to the FAA demonstration.
A second limitation concerns the external validity of the Ticket itself. ANGELA governs whether supplied evidence satisfies the requirements established by a Ticket; it does not guarantee that those requirements represent a scientifically valid model of the underlying phenomenon. A poorly conceived Ticket can therefore be executed correctly by ANGELA while still producing a scientifically weak or inappropriate analysis.
This distinction places an important boundary around the present findings:
Correct execution does not validate an incorrect Ticket.
The FAA test demonstrates that ANGELA can preserve its evidence-governance rules when operating a defined sequence of Tickets. It does not yet demonstrate cross-domain invariance, nor does it remove the responsibility of Ticket architects to construct scientifically defensible evidentiary requirements.
8. Conclusion
ANGELA operated as a gated evidence-governance architecture across five FAA tests while maintaining the same governing rules from one Ticket to the next. The evidence, analytical questions, permitted findings, and stopping points changed across the sequence, but the conditions governing advancement and stopping remained invariant.
When the required evidence was present, ANGELA permitted the analysis to advance. When the evidence supported a finding but not a stronger interpretation, the resulting claim remained bounded. When the evidence was insufficient to support further progression, ANGELA stopped rather than filling the gap with assumptions, substitutions, or manufactured inferences.
The FAA test therefore established more than whether individual aviation analyses could be completed. It provided an empirical test of ANGELA itself. Across five different evidentiary conditions, the architecture exhibited the behavior it was designed to enforce:
Advance when warranted. Qualify when required. Stop when the evidence stops.
The FAA results do not establish cross-domain validity. They establish that ANGELA survived its initial empirical challenge while preserving its governing rules, providing the basis for subsequent testing with different Tickets, different evidence, and different domains.
References
Bani-Mustafa, T., Zeng, Z., Zio, E., & Vasseur, D. (2020). A practical approach for evaluating the strength of knowledge supporting risk assessment models. Safety Science, 124, 104596. https://doi.org/10.1016/j.ssci.2019.104596
Federal Aviation Administration. (2013/2021). FAA Order 7050.1B, Runway Safety Program, with Changes 1 and 2 (Appendix B: Runway Incursion Severity Classification). https://www.faa.gov/regulations_policies/orders_notices/index.cfm/go/document.information/documentid/1022546
Federal Aviation Administration. (n.d.-a). Runway incursions. https://www.faa.gov/airports/runway_safety/resources/runway_incursions
Federal Aviation Administration, Aviation Safety Information Analysis and Sharing. (n.d.). Runway Safety Office Runway Incursion (RWS) database. https://extranet.asias.faa.gov/apex/f?p=100:28
Goerlandt, F., Khakzad, N., & Reniers, G. (2017). Validity and validation of safety-related quantitative risk analysis: A review. Safety Science, 99, 127–139. https://doi.org/10.1016/j.ssci.2016.08.023
Leveson, N. G. (2004). A new accident model for engineering safer systems. Safety Science, 42(4), 237–270. https://doi.org/10.1016/S0925-7535(03)00047-X
Paiva, A. R., & Tewari, A. (2022). Methodology for testing and evaluation of safety analytics approaches. Safety Science, 152, 105737.
Rae, A., & Alexander, R. D. (2017). Forecasts or fortune-telling: When are expert judgements of safety risk valid? Safety Science, 99, 156–165. https://doi.org/10.1016/j.ssci.2017.02.018
Rasmussen, J. (1997). Risk management in a dynamic society: A modelling problem. Safety Science, 27(2–3), 183–213. https://doi.org/10.1016/S0925-7535(97)00052-0
