Machine Answerability Without Human Scapegoating

Machine Answerability Without Human Scapegoating

Executive summary

The claim under review contains a compelling diagnosis but an overextended conclusion.

The diagnosis is strong: placing a nominal human “in the loop” does not create meaningful responsibility when that person lacks the time, information, authority, alternatives, or cognitive capacity to evaluate a machine recommendation. Human-factors research calls attention to automation complacency and “out-of-the-loop” performance degradation, while Madeleine Elish’s concept of the “moral crumple zone” describes how the nearest operator may absorb blame despite having little practical control over the system. Evulgare’s public materials accurately target this problem with the proposition that “a human click is not liability transfer.” citeturn22search2turn17search2turn1view0turn3view3

The narrower proposition—that software can make machine decisions technically answerable—is feasible. A sufficiently engineered system can preserve the evidence it received, model and software versions, uncertainty estimates, authorization state, interface information, machine transformations, human actions, execution records, and outcomes. This can materially improve causal attribution, incident reconstruction, legal review, command oversight, and the ability to distinguish operator error from design, data, policy, authorization, integration, or organizational failure. Evulgare’s strongest public contribution is its treatment of accountability as an evidence-and-provenance architecture rather than as a generated explanation or ordinary event log. citeturn2view0turn3view0turn3view1turn4view0turn11search2

The broader proposition—that the machine can therefore replace human legal or moral responsibility—is not supported. Under the legal and policy frameworks reviewed, obligations attach to states, commanders, operators, developers, manufacturers, procuring authorities, and other natural or legal persons. International discussions expressly preserve human responsibility across the weapon-system life cycle. Software may be causally attributable and evidentially answerable, but it cannot presently bear criminal intent, command duties, punishment, compensation obligations, professional discipline, or a duty to remedy victims. citeturn18view1turn20search5turn20search9turn20search3turn19view8

Removing humans from split-second execution can nevertheless be defensible in a tightly bounded class of cases. The defensible model is not “the machine becomes the accountable person.” It is delegated, constrained machine execution under prior human authorization, with verified operating limits, runtime safety monitors, deterministic authority checks, fail-safe or abstention behavior, complete decision provenance, independent certification, and retained human and institutional responsibility upstream and downstream. The U.S. Department of Defense’s current directive similarly requires appropriate human judgment, realistic testing against adaptive adversaries, auditability, explainability, cybersecurity, verification and validation, and termination or additional operator input when a system cannot remain within its intended constraints. citeturn19view0turn19view1turn19view2

Evulgare’s public evidence does not yet establish that it has achieved this operationally. Its website presents a coherent platform model, ten synthetic browser laboratories, and a twelve-report research library. The materials expressly characterize the labs as fictional, abstract, non-operational, and disconnected from targeting, firing, payload, force-authorization, or weapon endpoints. The principal accountability report describes a W3C-PROV-oriented provenance design, replay, cryptographic integrity, uncertainty and authority records, and dynamic responsibility assignment, but the public materials reviewed do not show an independent safety evaluation, peer-reviewed validation, deployed lethal-system integration, accredited certification, red-team results, or measured real-time performance. citeturn2view1turn2view2turn4view0

Bottom line: software can eliminate much of the evidentiary ambiguity that enables scapegoating, and it can reveal when a supposed human decision was merely ceremonial. It cannot eliminate human and institutional accountability. The sound policy objective is therefore not to “make the machine liable,” but to make the entire socio-technical decision chain reconstructable while assigning responsibility to actors who had actual control, authority, knowledge, and capacity to prevent or remedy harm.

What Evulgare publicly offers

Evulgare presents itself as an “assurance layer” for consequential machine decisions. Its homepage states that when decisions occur in milliseconds, the nearest human should not automatically carry the burden of explaining them. The platform’s stated purpose is to preserve what the machine knew, why it acted, what authority existed, what the human saw, and where a failure originated. Its recurring design propositions include “capability is not authority,” “confidence is not truth,” “a log is not provenance,” “generated explanation is not evidence,” and “a human click is not liability transfer.” citeturn1view0turn2view3

The public platform architecture is organized around a chain of evidence and decision transformations:

Evidence → observation → transformation → model → uncertainty → interpretation → policy → authority → interface → human judgment → decision → outcome → provenance → assurance.

Evulgare describes each consequential decision as a graph rather than as a single log entry. The graph may include evidence sources, contradictory evidence, assumptions, gaps, defeaters, model and configuration versions, policy rules, authorization state, interface presentation, reviewers, outcomes, and later changes that invalidate earlier claims. This is architecturally more useful than a conventional chronological log because it seeks to represent not only what happened but also the dependencies supporting the decision. citeturn2view0turn3view2

The public product is divided into related capabilities:

Evulgare Labs publicly lists ten interactive browser demonstrations covering accountability traces, assurance graphs, provenance replay, human judgment, authority boundaries, uncertainty, federated trust, resilience, change impact, and the distinction among kill chains, kill webs, and assurance. The site expressly states that these demonstrations are synthetic and fictional and contain no operational targeting, firing, payload, force-authorization, or weapon-system endpoints. Consequently, they demonstrate interaction and information architecture, not weapon-system safety or operational effectiveness. citeturn2view1

The research library publicly lists twelve reports addressing distributed kill-web architecture, delegated authority, assurance and verification, resilience, uncertainty, implementation, decision provenance, acquisition, meaningful human judgment, governance, and simulator requirements. The library assigns stable report identifiers and SHA-256 values, which can assist document integrity and version identification. Integrity hashes, however, establish that a particular artifact has not changed; they do not establish that the artifact’s factual claims, evidence, or safety conclusions are true. Evulgare itself explicitly recognizes this distinction. citeturn2view2turn2view3turn3view1

The most directly relevant public report, EVR-0007, proposes a provenance model using W3C PROV concepts and an extended decision-provenance schema. It describes append-only JSON-LD records containing identifiers, actor and system roles, trusted time, input hashes, transformations, software and model versions, output hashes, authority state, human-interface telemetry, dependencies, evidence status, subsequent changes, review status, remedies, and residual unknowns. It also separates the canonical history from counterfactual or replay branches and proposes propagating changes through a directed graph of assurance dependencies. citeturn4view0turn11search2

That design is conceptually credible. W3C PROV provides a standardized way to represent entities, activities, agents, derivation, attribution, and association. Extending it to decisions, authority, uncertainty, interface state, and operational outcomes is a reasonable basis for forensic reconstruction. The most significant feature is not the graph technology itself but the insistence that missing evidence and unresolved contradictions remain explicit rather than being silently replaced by a fluent explanation. citeturn11search2turn4view0turn2view3

Several limitations remain visible from the public record.

First, Evulgare carefully states that its accountability product supports technical answerability, not automated liability. This is legally prudent and should remain a foundational design constraint: the product should produce evidence for investigators and adjudicators, not issue a machine-generated “blame score.” citeturn3view0

Second, the architecture is only as reliable as its evidence acquisition. An append-only graph can faithfully preserve false sensor data, spoofed identity, incorrect timestamps, omitted messages, compromised model outputs, or a policy rule that was itself unlawful. Evulgare acknowledges that hashes prove integrity rather than truth and that replayability does not establish correctness. citeturn2view3turn3view1

Third, the public reports and labs are self-published product materials. EVR-0007 itself notes that inclusion in the research library does not independently verify all supplied framing or citations. In the materials reviewed, there is no public demonstration of independent certification, operational deployment, accredited safety testing, adversarial evaluation, measured worst-case latency, or successful integration with a real command-and-control or weapon system. That absence does not prove those capabilities do not exist privately, but it means they cannot be credited as established from the public evidence. citeturn2view1turn2view2turn4view0

The defensible characterization is therefore: Evulgare publicly presents a sophisticated accountability and decision-provenance design, not yet public proof of a certified machine-accountable lethal decision system.

Literature and conceptual baseline

The accountability problem becomes clearer when four different concepts are separated.

Causal attribution asks which component, action, data item, configuration, or organizational decision contributed to an outcome. Machines can be subjects of causal attribution.

Answerability asks whether a decision can be reconstructed and reasons or evidence can be produced for review. Software can materially improve answerability.

Responsibility may refer to role obligations, moral agency, legal culpability, command duty, or responsibility for prevention and remediation. These do not automatically follow from causal contribution.

Accountability is an institutional relationship in which an actor must explain conduct to a forum that can judge it and impose consequences or require remedy. Current legal systems generally require a human, state, corporation, agency, commander, or other recognized legal actor at this stage.

Andreas Matthias’s foundational “responsibility gap” argument is that learning or autonomous systems may behave in ways that were not sufficiently foreseeable or controllable for traditional blame attribution. Robert Sparrow’s analysis of autonomous weapons similarly examines the designer, commander, and machine as possible loci of responsibility and argues that a lack of a satisfactory bearer of responsibility creates an ethical objection to deployment. citeturn9search0turn19view8

Later work has refined rather than eliminated the problem. Santoni de Sio and van den Hoven propose “tracking” and “tracing” conditions: a system should track relevant human reasons and values, while it must remain possible to trace system behavior to people who understand their role and possess the capacity to act responsibly. This suggests that accountability does not require a human to perform every millisecond-level control action; it requires meaningful human control over purposes, constraints, deployment, monitoring, and remediation. citeturn10search0turn10search17

Elish’s “moral crumple zone” identifies the converse danger: an operator may be formally designated as responsible while system design, organizational incentives, automation, and temporal constraints leave that operator with little actual control. Her analysis directly supports Evulgare’s objection to treating the final human confirmation as dispositive evidence of responsibility. citeturn17search2

Human-factors evidence also complicates simplistic “human-on-the-loop” solutions. Endsley and Kiris found that automation can reduce situation awareness and impair a supervisor’s ability to resume control after system failure. An operator who passively monitors a highly reliable system may be least prepared to intervene during the rare, rapidly unfolding event for which intervention matters most. citeturn22search2

The commonly used control categories are therefore descriptive, not sufficient assurance standards:

• Human-in-the-loop ordinarily means that a human must affirmatively authorize or perform a critical action.
• Human-on-the-loop ordinarily means that the system acts while a human supervises and may intervene.
• Human-out-of-the-loop means that no human participates in the immediate execution or intervention cycle.

The labels do not reveal whether the human had adequate evidence, whether intervention was technically possible, whether the machine remained within a legally reviewed envelope, or who was responsible for the system’s design and deployment. Scholarly analysis has consequently urged more precise distinctions among control, oversight, responsibility, and accountability. citeturn10search10turn3view3

Policy frameworks converge on several principles but differ on how much autonomy should be permitted.

The U.S. Department of Defense requires autonomous and semi-autonomous weapon systems to allow commanders and operators to exercise appropriate levels of human judgment over the use of force. Its directive calls for transparent, auditable and explainable technologies and data sources, understandable interfaces, clear activation and deactivation procedures, rigorous verification and validation, realistic testing against adaptive adversaries, cyber resilience, anti-tamper measures, and continued testing following system changes. It permits bounded autonomy rather than requiring a human approval for every individual machine action. citeturn19view0turn19view1turn19view2

The ICRC adopts a more restrictive position. It defines autonomous weapon systems as systems that select and apply force without human intervention after activation and emphasizes that users may not know the particular target, place, or time of force application. It recommends legally binding prohibitions on unpredictable autonomous weapons and systems designed or used to apply force against persons, together with strict restrictions on other systems. citeturn20search5turn20search17turn20search9

The United Nations Convention on Certain Conventional Weapons process has affirmed that human responsibility can be exercised across the weapon-system life cycle and through human-machine interaction. The same process recognizes potential benefits such as precision and reduced human error, while recording concerns about unpredictability, bias, hacking, interference, escalation, civilian harm, and whether development-stage human involvement is sufficient in variable operational environments. citeturn18view1turn19view3

NATO’s responsible-AI principles include lawfulness, responsibility and accountability, explainability and traceability, reliability, governability, and bias mitigation. NIST’s AI Risk Management Framework emphasizes governance, mapping context and risk, measurement, and risk management. OECD and UNESCO frameworks likewise call for lifecycle accountability, traceability, documentation, auditability, human oversight, impact assessment, and mechanisms for challenge and remedy. These frameworks support Evulgare-like evidence architectures, but they do not treat technical explanation as a replacement for an accountable institution. citeturn7search3turn8search1turn7search5turn7search2

The EU AI Act offers extensive civilian high-risk-AI concepts—including risk management, records, documentation, human oversight, transparency, incident reporting, and conformity assessment—but expressly excludes systems used exclusively for military, defense, or national-security purposes. Its requirements can inform engineering practice by analogy, but the Act is not itself a general regulatory regime for lethal military AI. citeturn21view0turn21view1turn21view2

These sources support a central distinction:

Meaningful human control need not mean human motor control at the last millisecond. It means that identifiable humans and institutions retain informed, reviewable control over the system’s objectives, legal envelope, target and environmental constraints, deployment conditions, authority, testing, suspension, and remediation.

Technical feasibility and architecture

A machine can be made substantially more answerable, but doing so requires more than explainable AI. The necessary system is an integrated safety, security, authorization, provenance, and governance architecture.

The authority plane must represent who authorized the mission, which legal and operational review applies, what functions are delegated, the permitted time and geographic area, allowed target or object classes, environmental conditions, collateral-risk limits, system modes, expiration conditions, and who can suspend or revoke authority. Authority should be machine-verifiable and cryptographically bound to the exact software, model, configuration, platform, mission envelope, and period of validity.

The evidence plane must preserve sensor inputs or legally appropriate evidentiary extracts, source identity, calibration state, timestamps, transformations, fusion operations, missing data, contradictory evidence, uncertainty, provenance, and data-quality assessments. A model output without the inputs and transformations that produced it is not an adequate accountability record.

The decision plane must record the model and software versions, rules and policy versions, inference output, confidence and calibration information, alternative hypotheses, threshold crossings, uncertainty, abstention logic, and the exact transition from recommendation to authorized action.

The human-machine plane must record what information was displayed, what was hidden or unavailable, alert timing, interface mode, operator workload proxies, alternatives presented, review duration, requests for additional evidence, attempted interventions, and whether an intervention could physically have affected the outcome. This is necessary to distinguish independent judgment from a rubber-stamp click.

The runtime-assurance plane must independently monitor invariant safety properties and the approved operating envelope. NASA’s “simplex” runtime-assurance pattern uses an advanced but less trusted controller under the supervision of a verified monitor, with transfer to a trusted reversionary controller when a safety property is threatened. A lethal system would require an analogous independent monitor whose trusted computing base is smaller, simpler, separately tested, and unable to be bypassed by the primary autonomy stack. citeturn19view4

The forensic and remedy plane must preserve execution, effects, post-action sensor observations, communications, faults, overrides, injuries or damage, later-discovered evidence, investigator annotations, corrective actions, and the status of affected assurance claims. It must support independent custody, disclosure rules, victim or oversight access where legally appropriate, and a process for suspending similar systems while the incident is investigated.

Comparative architecture

Explainability methods have distinct roles. Intrinsically interpretable rules, state machines, temporal logic, and formally specified safety constraints are preferable for authorization and runtime gates because their behavior can be analyzed in advance. LIME and SHAP can help describe which features influenced a particular model prediction, but LIME is a local surrogate and SHAP is a feature-attribution framework; neither by itself proves that the input was true, the model was lawful, the action was causally justified, or the result complied with humanitarian law. citeturn22search0turn22search1

For lethal decisions, a generated natural-language rationale should therefore be treated as a view over evidence, not as evidence. It should cite immutable event identifiers, expose uncertainty and contradictory information, and be reproducible from the underlying record. A fluent after-the-fact explanation that cannot be tied to the actual inference, model version, policy state, and evidence available at decision time risks becoming “explanation laundering.” Evulgare’s distinction between generated explanation and preserved provenance is sound. citeturn2view3turn3view1

Formal verification is necessary but necessarily partial. Engineers can verify properties such as authorization-token validity, state-transition correctness, geographic or temporal bounds, collision constraints, fail-safe transitions, message authentication, and absence of specified classes of software error. They cannot generally prove that open-world perception will correctly interpret every person, object, intent, surrender indication, civilian pattern, or adversarial deception. Verification claims must therefore specify exactly what was proven, under which assumptions, and which components remain unverified.

Runtime performance introduces further constraints. The safety and authority checks that can block an action must have known worst-case execution times and must not depend on remote services that may be unavailable or delayed. Logging should use preallocated local buffers, trusted hardware-backed timestamps, monotonic sequence numbers, authenticated event records, and nonblocking or bounded-time writes; replication, indexing, and full graph construction can occur asynchronously so long as the primary record cannot be silently lost or altered. A system that sacrifices its evidence record whenever computation or communications are stressed will fail precisely when accountability matters most.

The machine should default to abstention, safe state, or bounded fallback when evidence is incomplete, clocks disagree, authority is expired, sensors conflict beyond certified limits, model confidence is uncalibrated, the environment is outside the approved domain, the runtime monitor fails, or the provenance recorder cannot establish a trustworthy decision identity. DoD policy similarly requires termination or additional operator input when a system cannot complete an engagement within approved constraints. citeturn19view0

A proposed accountable machine decision chain is:

flowchart TD
    A[Political and command authorization] --> B[Legal review and rules of engagement]
    B --> C[Certified mission envelope]
    C --> D[Signed authority token bound to platform, software, model, time and scope]

    D --> E[Evidence and sensor acquisition]
    E --> F[Provenance, calibration and data-quality checks]
    F --> G[Perception, fusion and alternative hypotheses]
    G --> H[Uncertainty and out-of-distribution assessment]
    H --> I[Policy and authority gate]

    I --> J{Within certified envelope?}
    J -- No or unknown --> K[Abstain, safe state, terminate or request human review]
    J -- Yes --> L[Independent runtime-assurance monitor]

    L --> M{Safety invariants satisfied?}
    M -- No --> K
    M -- Yes --> N{Meaningful human review feasible and required?}

    N -- Yes --> O[Present evidence, uncertainty, alternatives and consequences]
    O --> P{Human authorizes?}
    P -- No --> K
    P -- Yes --> Q[Bounded execution]

    N -- No; prior delegation permits machine-speed action --> Q

    Q --> R[Record action, timing, system state and effects]
    R --> S[Capture outcome and subsequent evidence]
    S --> T[Seal tamper-evident provenance package]
    T --> U[Independent investigation and legal review]
    U --> V[Remedy, sanctions, lessons and assurance updates]
    V --> W{Change invalidates certification?}
    W -- Yes --> X[Suspend, retest and recertify]
    W -- No --> Y[Continue under monitored authorization]

This flow shifts human judgment to the stages at which it can be meaningful—authorization, legal review, envelope definition, deployment, exception handling, investigation, and remedy—without pretending that a person can reason through a complex engagement in a sub-second confirmation window. It also preserves the possibility of real-time human judgment where the operational window actually permits it. The design is a synthesis of DoD requirements, runtime-assurance research, provenance standards, meaningful-human-control scholarship, and Evulgare’s public architecture. citeturn19view0turn19view2turn19view4turn11search2turn10search0turn4view0

Major failure modes include sensor spoofing, adversarial examples, data poisoning, model extraction, compromised updates, prompt or interface manipulation, supply-chain substitution, authority-token theft or replay, clock manipulation, selective log deletion, denial of logging, model drift, unmodeled environmental changes, cascading multi-agent interactions, unsafe fallback controllers, automation bias, and collusion among components that are falsely assumed to be independent. NIST’s adversarial-machine-learning taxonomy specifically covers evasion, poisoning, privacy and other lifecycle attacks and warns that mitigations have limitations. citeturn16search3turn16search7

Accordingly, the provenance system must not share all trust roots with the autonomy system it is intended to audit. Keys, clocks, secure boot measurements, software attestations, and event sealing should be independently anchored. Software-supply-chain attestations such as SLSA- or in-toto-style provenance can help establish which source, build process, dependencies, and artifacts produced the deployed binary, although they cannot prove that the resulting behavior is safe or lawful. citeturn12search15turn12search1turn12search4

Legal and ethical implications

International humanitarian law applies to the use of autonomous and AI-enabled weapons just as it applies to other means and methods of warfare. Relevant obligations include distinction, proportionality, precautions in attack, and prohibitions on weapons or attacks that cannot be directed or limited as required. A technical accountability system can provide evidence relevant to these duties, but it cannot redefine them. citeturn20search1turn20search9turn20search25

Article 36 of Additional Protocol I requires state parties to determine whether a new weapon, means, or method of warfare would be prohibited in some or all circumstances. The ICRC recommends rigorous, multidisciplinary reviews involving legal, military, technical, medical, and environmental expertise. For adaptive or frequently updated software, review cannot sensibly be a one-time approval of a product name; it must cover specific versions, operating envelopes, training and validation data, configurations, intended uses, failure responses, and later modifications. citeturn20search2turn20search6turn20search14

Machine provenance would be valuable to an Article 36 process in at least four ways. It could demonstrate that the fielded configuration matches the reviewed configuration; establish which assumptions and operating limits supported approval; show whether the system remained within those limits; and identify changes that should have triggered a new review. Evulgare’s change-impact and continuous-assurance concepts align well with this need, although an assurance graph would still require independent legal and technical validation. citeturn2view0turn3view2turn19view2

State responsibility is not displaced merely because no individual can be shown to have intended a particular machine outcome. A state may remain responsible for an internationally wrongful act attributable to it, while separate questions arise concerning the criminal responsibility of individuals. Technical uncertainty may make individual culpability difficult to prove, but it does not transform the machine into the responsible state or extinguish duties to cease violations, investigate, make reparation, or prevent recurrence.

Command responsibility also remains pertinent. Article 28 of the Rome Statute addresses responsibility of military commanders and other superiors under specified knowledge, control, prevention, repression, and reporting conditions. Autonomous systems may complicate proof of what a commander knew or should have known, but comprehensive monitoring and provenance may strengthen rather than weaken the evidentiary basis for evaluating those questions. A system designed to hide uncertainty or aggregate away warnings could, conversely, obstruct accountability. citeturn20search3turn20search7

Individual criminal responsibility presents a harder attribution problem. Prosecutors may need to determine whether a commander, developer, operator, or other actor possessed the required mental state; whether the harmful behavior was foreseeable; whether the actor had effective control; whether warnings were available; whether deployment was reckless; and whether causal links can be established beyond technical complexity. Software evidence can clarify these matters but cannot decide them automatically.

Civil and product liability varies significantly by jurisdiction. Potential defendants could include manufacturers, software suppliers, system integrators, contractors, deployers, procuring authorities, or operators, subject to sovereign immunity, combatant-activities doctrines, procurement terms, classified-evidence rules, product-defect standards, negligence rules, and statutory exclusions. Because the user specified no jurisdiction, no universal liability rule can be stated. The EU AI Act is particularly instructive as a caution: although it supplies useful high-risk governance concepts, it excludes systems used exclusively for military, defense, or national-security purposes. citeturn21view0turn21view1

The idea of making the machine itself legally liable encounters fundamental problems:

For these reasons, Evulgare’s formulation of “technical answerability, not automated liability” is the legally defensible one. citeturn3view0

The ethical issue is not whether all immediate execution must be human. Humans routinely delegate tightly specified actions to machines when machine response is faster, more accurate, or more consistent. The ethical threshold is whether the delegation itself is reasoned and bounded, whether the system remains predictable enough for the context, whether human dignity and humanitarian protections are preserved, whether meaningful control exists across the lifecycle, and whether victims retain a route to investigation and remedy.

A human-in-the-loop requirement may become unethical when it serves principally to manufacture a blame recipient. Conversely, removing the human without imposing stringent design and institutional obligations may create an accountability vacuum. The ethically preferable arrangement is responsibility matched to actual control: designers answer for foreseeable design and verification choices; data and model owners for provenance and known limitations; commanders for authorization and deployment; operators for actions genuinely within their control; organizations for training and safety culture; states for lawful use and remedy; and investigators or courts for final attribution.

Case studies

The historical record shows that automation can both expose systemic responsibility and create tempting scapegoats.

Therac-25 is a classic example of accountability failure in a safety-critical computerized system. Between 1985 and 1987, six known accidents involved massive radiation overdoses, deaths, and serious injuries. Leveson and Turner’s investigation emphasized that these were system accidents arising from complex interactions among software, hardware, design, development, operation, manufacturer practices, regulation, and communication—not events that could responsibly be reduced to a single careless operator. citeturn18view3turn19view5

The lesson for autonomous weapons is that removing hardware interlocks or independent safety layers because software appears reliable can produce catastrophic common-mode failures. It also demonstrates why opaque error messages, inadequate incident sharing, poor hazard analysis, and excessive confidence in software are accountability problems. An Evulgare-style evidence graph could have accelerated recognition that apparently separate operator incidents shared a common technical and organizational pattern, but provenance alone would not have prevented the accidents without independent safety constraints and institutional willingness to act.

The Uber automated-driving fatality in Tempe illustrates the moral-crumple-zone problem. The automated system detected the pedestrian several seconds before impact but repeatedly misclassified her and did not correctly predict her path. Its design precluded emergency braking in the relevant condition and relied on the safety operator to intervene; Uber had also disabled the vehicle’s built-in forward-collision warning and automatic emergency-braking functions during operation of its developmental system. citeturn23view3

The NTSB identified the operator’s prolonged distraction as the probable cause, but it also identified inadequate company risk assessment, ineffective operator oversight, inadequate safeguards against automation complacency, insufficient safety redundancy, an inadequate safety culture, and deficient state oversight as contributing factors. The investigation therefore avoided treating the final human failure as the complete causal explanation. citeturn23view0turn23view1turn23view2

This case supports both sides of the present thesis. Better system logging and reconstruction can reveal that a human did not originate the machine’s perception failure or safety architecture. Yet the presence of detailed records does not make the vehicle morally or legally responsible; the records instead allow responsibility to be distributed among the operator, designers, company governance, testing regime, and regulator according to their actual contributions and duties.

The NTSB model itself is relevant. Its reports state that accident investigations are fact-finding proceedings intended to improve safety rather than to assign legal fault or blame. This institutional separation enables technical investigation to build a reliable causal account without prematurely collapsing that account into liability adjudication. Evulgare’s proposed separation between technical answerability and legal liability follows a similar logic. citeturn19view7turn3view0

The May 2010 Flash Crash shows both machine-driven cascade risk and the value of high-resolution audit data. An automated execution program, algorithmic trading, market structure, and interactions among different classes of traders contributed to a rapid market collapse and recovery. Researchers used audit-trail data that timestamped and sequenced trades, identified associated accounts, and distinguished sides of transactions to reconstruct the event at machine timescales. citeturn18view4turn19view6

That audit trail reduced the temptation to attribute the entire event to a single trader, algorithm, or category such as high-frequency trading. The empirical analysis found that high-frequency traders did not simply “cause” the event in isolation; their behavior interacted with other market activity and amplified dynamics during stress. The lesson is that machine-speed systems require machine-speed evidence. Human recollection and conventional organizational logs are inadequate for reconstructing cascades that unfold faster than any person can observe. citeturn18view4

Across these cases, the recurring pattern is:

None of these examples supports replacing human accountability with machine liability. They support replacing simplistic blame with evidence-based system accountability.

Recommendations and verdict

A credible “answerable machine” product should be assessed against a concrete assurance case rather than against the fluency of its dashboards or explanations.

For Evulgare specifically, the highest-priority next step is independent evidence. A persuasive public validation program would include a precise threat model; a complete public data schema; reference implementations of trusted logging and authority enforcement; reproducible synthetic scenarios; fault-injection and adversarial test results; worst-case latency and storage measurements; formal specifications for critical gates; comparison with ordinary logs and existing provenance systems; usability studies measuring whether humans actually exercise independent judgment; and independent evaluation by safety engineers, human-factors specialists, international-humanitarian-law experts, cybersecurity assessors, and incident investigators.

Its provenance recorder should be designed as a separately trusted subsystem. It should use hardware-rooted identity where feasible, authenticated time, monotonic counters, append-only event sealing, redundant local storage, delayed external anchoring, independent key custody, and explicit records for missing or degraded evidence. The product should never imply that a cryptographic hash validates the truth of the underlying observation. citeturn3view1turn2view3

Its authority model should use machine-readable delegation objects rather than unstructured text alone. Each object should identify the issuing authority, legal and policy basis, allowed functions, prohibited functions, operating domain, temporal validity, required human role, revocation conditions, software and model configuration, and maximum permissible consequences. The execution controller must reject ambiguous, conflicting, expired, or unverifiable authority.

Its human-judgment module should go beyond recording a click. It should generate affirmative evidence of meaningful control: what evidence and contradictions were shown, how long they were available, whether the operator could seek more information, whether alternatives were presented, whether an abstention option existed, whether intervention remained physically effective, and whether the person had training and authority appropriate to the decision. citeturn3view3turn22search2

Its explanation system should maintain three distinct outputs:

1. A forensic account of actual inputs, transformations, versions and events.
2. An assurance account of which requirements and evidence support the claim that the system was fit for the approved use.
3. A human-readable explanation derived from and linked to the first two.

Only the first two should be authoritative. Natural-language summaries should be clearly marked as generated views and should never overwrite the underlying record.

Certification should be conditional, scoped and perishable. It should identify the exact software, model, policy, data lineage, hardware, operating domain, mission class and runtime monitor to which it applies. Every certificate should have expiration, revocation and automatic invalidation rules. This follows both DoD’s requirement for continued testing after changes and Evulgare’s own continuous-assurance philosophy. citeturn19view2turn3view2

Policy should prohibit institutions from using nominal human participation as a liability shield. Where a human lacks the information or time required for independent judgment, the governing documents should say explicitly that the person is a supervisor, exception handler, or witness—not the sole decision-maker. Responsibility should be allocated ex ante among command, acquisition, development, integration, testing, deployment, operation and oversight authorities.

At the same time, policy should reject the fiction that an autonomous system becomes the sole accountable subject. Every deployment should identify a responsible state or legal entity, a named command authority, a technical authority, a safety authority, an incident-investigation forum and a remedy mechanism. No machine action should be legally “ownerless.”

For systems capable of applying lethal force against persons, the strongest precautionary policy is the ICRC’s proposed prohibition or strict restriction, particularly where targets, places, timing, or effects cannot be sufficiently predicted and constrained. At minimum, systems should not be deployed where open-ended machine learning, uncontrolled self-modification, unbounded target profiles, unreliable communications, civilian density, surrender or incapacitation judgments, or context-dependent proportionality assessments exceed validated machine and human-control capabilities. citeturn20search17turn20search9

The final assessment is therefore qualified:

The anti-scapegoating claim is persuasive. A human who receives a machine recommendation too late to evaluate it, through an interface that conceals uncertainty or alternatives, should not automatically be treated as the responsible decision-maker. Software can expose that mismatch and prevent institutions from laundering responsibility through a final click.

The machine-accountability claim is persuasive only in an evidentiary sense. Machines can be made answerable as technical systems: their state, evidence, authority, processing, uncertainty and actions can be reconstructed and challenged.

The substitution claim is unsound. Technical answerability cannot replace legal, moral or political accountability. It makes human and institutional accountability more accurate.

Removing humans from the immediate execution loop can be legitimate only as bounded delegation. Human responsibility moves to authorization, design, verification, deployment, supervision, suspension, investigation and remedy. It does not disappear.

The strongest formulation of the thesis is therefore:

Humans need not be placed in impossible split-second approval roles merely to provide someone to blame. Machines should be engineered to produce complete, tamper-evident evidence of their decisions, while law and governance assign responsibility to the humans and institutions that actually designed, authorized, controlled, deployed, monitored and could have prevented or remedied the outcome.