Your App Logs Are Not an AI System of Record

Application stdout is not an audit trail. It is a debugging convenience that evaporates under retention policies and log sampling. High-risk AI in clinical and administrative workflows must support traceability of function — not just uptime of the API. When legal discovery or regulatory supervision asks which model version produced a score, on what inputs, and whether a responsible person verified the output, application logs rarely answer all three. A decision ledger is the engineering artefact that closes that gap — separate from your application log stream.

Problem: assistive AI without a decision ledger

The EU AI Act Article 12 requires high-risk systems to technically enable automatic recording of events over their lifetime, including inputs that matched, reference data checked, and natural persons who verified results. The ICO’s March 2026 AI and biometrics strategy update reinforces the same expectation for UK deployers: transparency and robust safeguards for automated decision-making, with ADM guidance forthcoming. Implement once in a ledger schema; map obligations from both frameworks to the same append-only events.

Decision flow: agent, model, ledger, clinician override

sequenceDiagram
    participant Agent as Triage Agent
    participant Model as Risk Model v2.3
    participant Ledger as Decision Ledger
    participant UI as Clinician UI
    participant Clin as Responsible Clinician

    Agent->>Model: Score request (presentation bundle)
    Model-->>Agent: acuity=low, confidence=0.81
    Agent->>Ledger: Append DecisionEvent (hash, inputRef, modelVersion)
    Ledger-->>Agent: eventId + chain hash
    Agent->>UI: Suggest low-acuity pathway
    UI->>Clin: Present score + override control
    alt Clinician agrees
        Clin->>Ledger: Append ReviewEvent (reviewerId, affirmed)
    else Clinician overrides
        Clin->>Ledger: Append OverrideEvent (reviewerId, newPathway, rationale)
    end
    Ledger-->>UI: Ack immutable record

Figure 1: Assistive triage with immutable ledger append before UI presentation and explicit clinician attestation.

Decision event schema

Each ledger entry is an append-only event. Hash chaining is optional but recommended for tamper-evidence. Store large clinical payloads by reference, not inline.

{
  "$schema": "https://blackflow.co.uk/schemas/decision-event/v1",
  "type": "object",
  "required": [
    "eventId", "eventType", "occurredAt", "hash",
    "inputRef", "modelVersion", "systemId", "subjectRef"
  ],
  "properties": {
    "eventId": { "type": "string", "format": "uuid" },
    "eventType": {
      "type": "string",
      "enum": ["DecisionProposed", "ClinicianAffirmed", "ClinicianOverride", "ModelRetired"]
    },
    "occurredAt": { "type": "string", "format": "date-time" },
    "hash": {
      "type": "string",
      "description": "SHA-256 of canonical payload + previousHash"
    },
    "previousHash": { "type": "string" },
    "inputRef": {
      "type": "string",
      "description": "URI to immutable input bundle (object store, FHIR Composition)"
    },
    "modelVersion": {
      "type": "string",
      "description": "Semantic version + build id"
    },
    "referenceDataRef": {
      "type": "string",
      "description": "Snapshot id of guidelines/knowledge base checked"
    },
    "outputSummary": {
      "type": "object",
      "properties": {
        "acuity": { "type": "string" },
        "confidence": { "type": "number" }
      }
    },
    "reviewerId": {
      "type": "string",
      "description": "OIDC sub of responsible clinician — required on review events"
    },
    "overrideRationale": { "type": "string", "maxLength": 2000 },
    "subjectRef": { "type": "string", "description": "Patient pseudonymised key" },
    "systemId": { "type": "string", "description": "EU AI Act Annex III registration id when applicable" }
  }
}

Agent orchestration without ledger bypass paths

Agent frameworks encourage tool loops: call model, call database, call UI. Each step emits unstructured logs. The failure mode is a shortcut path — emergency override or demo mode — that skips persistence when the ledger store is slow. Code review must treat any such branch as a severity-1 defect. Circuit-breaker patterns belong on model inference (fallback to manual triage), not on ledger writes.

Model version pinning is non-negotiable. modelVersion must include build id, not a floating latest tag. Blue-green model deploys run parallel inference in shadow until override rates match baseline; ledger events tag shadow runs until promotion. Retiring a model emits ModelRetired — consumers must reject inference requests against retired versions at the gateway.

Why vendors treat application logs as compliance

Most AI platforms ship CloudWatch or Application Insights integration and call it governance. That fails Article 12 record-keeping on three specifics: logs are sampled and rotated; they do not record who verified a result; they conflate infrastructure noise with decision provenance. The Act explicitly requires recording the period of each use, reference databases checked, input data that led to a match, and identification of natural persons involved in verification — for systems in Annex III point 1(a). Application logs rarely persist all four with tamper-resistant semantics.

Field	Application log	Decision ledger
Retention default	Short-lived, often sampled	Lifecycle of system plus regulatory minimum
Model version	Sometimes in debug field	Required on every DecisionProposed event
Input payload	Truncated or unstructured	Immutable inputRef with hash linkage
Clinician verifier	Not captured	reviewerId on Affirmed / Override events
Tamper evidence	None — admins can delete streams	Append-only store, hash chain, WORM or legal hold
Query for incident	Grep across shards, incomplete	eventId + subjectRef + time range — deterministic
Regulatory export	Manual scrape, format varies	Schema-versioned export by systemId

Architecture rule: No assistive AI output reaches a patient-facing pathway until a DecisionProposed event is persisted with inputRef, modelVersion, and hash — clinician override is a second event, never an in-place edit.

ICO and EU AI Act alignment matrix

UK deployers face parallel expectations. The ICO strategy update commits to ADM guidance and foundation-model scrutiny; the EU AI Act Article 12 applies if your system is high-risk in the Union market. Map obligations once, implement once in the ledger schema.

• Period of use (Art 12(3)(a)): occurredAt on every event; session start/end derived from DecisionProposed timestamps per subjectRef.
• Reference database (Art 12(3)(b)): referenceDataRef snapshot id — never a live guideline URL without version hash.
• Input that matched (Art 12(3)(c)): inputRef plus outputSummary — sufficient to reconstruct the decision without storing full PHI in the ledger row.
• Verifier identity (Art 12(3)(d)): reviewerId on Affirmed and Override — maps to accountable clinician per Article 14 human oversight.
• ICO transparency: UI displays modelVersion and confidence before action; ledger proves what was shown.

Implementable checklist

1. Provision append-only ledger store (dedicated table with INSERT-only grants, or object store with Object Lock).
2. Wrap model invocation: return only after ledger append succeeds; fail closed on ledger outage.
3. Bind UI override controls to ReviewEvent API — disable auto-apply until reviewerId present.
4. Snapshot reference guidelines at inference time; store referenceDataRef, not live URL.
5. Expose regulator export job: filter by systemId, date range, subjectRef; include hash chain verification report.
6. Integration test from isolated network: simulate model response, assert ledger row before HTTP 200 to UI.

Routing table: which components own which consistency class

Component	Consistency	Ledger obligation
Triage agent (orchestration)	Strong — no UI without append	Emits DecisionProposed
Risk model endpoint	Stateless inference	Referenced by modelVersion only
Decision ledger store	Append-only, sequential per subjectRef	Source of truth for audits
Clinician UI	Read-after-ledger-ack	Emits Affirmed or Override
Application log stack	Best-effort, sampled	Supplementary — never primary evidence
Analytics warehouse	Eventual — batch ETL	Read replica of ledger, not substitute

Post-market monitoring under Article 72 expects lifecycle performance tracking. Feed ledger aggregates into monitoring: override rate by modelVersion, confidence drift, time-to-clinician-review. Ledger bypass in production is P1 — compliance breach, not degraded performance. Assistive AI without a decision ledger is a demo; the ledger survives court, supervision, and post-market scrutiny.