An enterprise buyer asks what your agent decided on 14 March, which tools it called, and who approved the wire transfer. Your application logs rotated at fourteen days. The demo ledger in Postgres has no export path. The audit fails not because the agent misbehaved, but because you treated compliance as logging configuration.
Logs answer uptime; ledgers answer accountability
Operational logs are high-volume, short-retention, grep-oriented. A decision ledger is append-only, policy-retained, and keyed to tenant, actor, tool, and outcome. Regulated agentic SaaS needs both, with different access controls and different replay semantics. Confusing them produces either bankrupting storage bills or empty hands at audit time.
High-risk AI system documentation expectations under the EU framework — see Article 12 on record-keeping — run to months, not the default retention on stdout collectors. NIST’s AI Risk Management Framework likewise treats traceability as governance infrastructure, not an afterthought on CloudWatch.
Retention and replay flow
sequenceDiagram
participant A as Agent runtime
participant L as Decision ledger (hot)
participant C as Cold archive (WORM)
participant R as Replay sandbox
A->>L: append DecisionEvent (hash chain)
L->>L: replicate to follower
Note over L: hot tier 90d query SLA < 2s
L->>C: nightly export by tenant partition
Note over C: 7y regulatory tier immutable
R->>C: read-only slice by caseId + date
R->>R: deterministic re-run tools (stubbed side effects)
R-->>Auditor: diff original vs replay hash
Caption: Figure 1 — append hot, archive cold, replay isolated with stubbed side effects.
Decision event schema (minimal audit unit)
{
"eventId": "018f3a2e-7c4b-7d9a-b1e2-3f4a5b6c7d8e",
"tenantId": "t_acme_health",
"sessionId": "sess_20260314_093012",
"timestamp": "2026-03-14T09:30:45.123Z",
"actor": { "type": "agent", "model": "gpt-4.1", "version": "2026-03-01" },
"promptHash": "sha256:abc…",
"toolCalls": [
{
"name": "transfer_funds",
"inputHash": "sha256:def…",
"approval": { "by": "user:jdoe", "at": "2026-03-14T09:30:44.900Z" },
"outcome": "success",
"externalRef": "txn_88421"
}
],
"prevEventHash": "sha256:…",
"eventHash": "sha256:…"
}
Store hashes of prompts and tool inputs when raw text cannot live in the ledger for privacy. Pair with encrypted object storage for full payload when legal holds require it. Replay uses stored inputs or hashed reconstructions depending on tier.
Why vendors sell “audit logs” that fail audits
Generic observability stacks optimise for cardinality and cost, not immutability or legal hold. Export is an afterthought. Tamper evidence is absent. Multi-tenant isolation is a tag, not a partition key. Agent frameworks emit trace spans that expire with the collector retention policy. Spans prove latency; they do not prove who approved a side effect unless you model approval as a first-class event.
Our system-of-record framing for agentic products is in AI decision ledger as system of record. Retention and replay are the operational half of that design once append-only ingest works.
Event class × retention × replay mode
| Event class | Hot retention | Cold retention | Replay mode |
|---|---|---|---|
| Tool side effect (financial) | 365 days online query | 7 years WORM | Stub external APIs; assert hash match |
| Human approval gate | 365 days | 7 years | Read-only timeline export PDF |
| Model routing / prompt | 90 days | 1 year hashed | Deterministic re-run if inputs stored |
| Diagnostic trace | 14 days | None | Not replayed for audit |
Architecture rule: Logs are operational; a decision ledger is legal and product truth — different retention, different access controls, and different replay contracts.
Implementable checklist
1. Append ledger writes fail closed if hash chain breaks; alert on tamper detection.
2. Partition exports by tenant for DSAR and legal hold without scanning full table.
3. Replay sandbox network-isolated; tool stubs return recorded responses or safe fakes.
4. Separate RBAC: SREs read diagnostics; compliance reads ledger export role only.
5. Document maximum replay drift allowed (model non-determinism) vs hard fail.
6. Run quarterly restore drill from cold archive; measure time-to-first-query.
Cold storage tiers and legal hold
Hot tier stays in queryable Postgres or equivalent with row-level tenant isolation. Warm tier compresses JSON to object storage partitioned by tenantId/year/month with server-side encryption and bucket policies denying delete without break-glass role. Cold tier uses WORM or object lock for years seven through regulatory maximum.
Legal hold pins specific eventId ranges regardless of TTL job. Hold metadata lives in a separate table auditors can list without reading full payloads. Release requires compliance ticket id logged in ledger admin events.
Cost control: store prompt hashes by default; store full prompt text only when product tier or regulation demands. Most DSAR exports need decision outcomes and approvals, not every token of chain-of-thought diagnostic logging.
Replay fidelity and model non-determinism
Deterministic replay works for tool routing, approval gates, and rule engines. It fails for generative steps where temperature > 0. Document replay classes: Type A events replay bit-identical; Type B events replay structural path only with tolerance band on model output hash; Type C events are audit record only, no regeneration.
Sandbox replay connects read-only to cold archive, mounts stub adapters for payment, email, and CRM tools, and writes results to ephemeral storage never merged into production ledger. Auditor sees side-by-side: original eventHash, replay run id, diff summary.
Quarterly drill: pick random tenant week, export, replay, produce PDF timeline under four hours wall clock. Miss the SLA in staging, not during regulator request.
Export formats should include JSON Lines for engineering replay and signed PDF summary for legal counsel who will never open a terminal. Same source partition feeds both; do not maintain parallel spreadsheets.
Multi-tenant isolation on read and export
Every query against hot ledger enforces tenantId from authenticated context, not from client-supplied filter alone. Export jobs assume role with tenant scope; cross-tenant export requires break-glass role with mutual approval logged as its own ledger admin event.
Replay sandboxes spin per tenant namespace with synthetic credentials. Never point replay tooling at production APIs even with “read-only” keys; stub adapters prevent the classic audit tool that accidentally triggered a live refund.
Retention jobs deleting hot rows after archive must verify object storage checksum before DELETE. Fail closed if export missing; never delete hot copy because cold upload “probably succeeded.”
Contract with observability vendors
Ship traces to Datadog or similar with 14-day retention for latency debugging. Export ledger events to the compliance store with 7-year policy. Do not point auditors at trace UI expecting ledger completeness. Link trace_id in ledger optional field for engineer convenience only.
Agent frameworks that log “tool call finished” to stdout are not a substitute for signed append events. Wrap vendor SDK callbacks to emit ledger rows at approval boundaries, not only at completion.
Ops gates and sign-off
SLO: hot ledger query p95 under two seconds for ninety-day window; cold export job completes within four hours for single-tenant year slice. Legal hold freezes deletion regardless of default TTL. Compliance signs retention matrix; engineering signs replay stub catalogue; jointly sign after drill produces auditor-ready export for a simulated case file.
Regulated agentic SaaS on .NET needs ledger retention engineered like billing, not like debug logging. Talk to BlackFlow about export, replay, and tamper-evident storage that survives the question “what did the agent do last quarter?”
