All writing

Why an AI workflow needs an evidence ledger

Published

A PostgreSQL-backed execution model that pins workflow versions, records side effects, and keeps human review and release evidence auditable.

An AI workflow can return a convincing answer and still be impossible to operate. A production system must answer a different set of questions: which workflow version ran, what the model was allowed to do, where a human intervened, and which side effects actually happened.

I built a public AI Operations Deployment Platform around that evidence problem. The first product path handles vendor and subcontractor compliance documents. The useful part is not the document label. It is the boundary between a model's judgment and the durable record that an operator can inspect later.

make the published version the audit anchor

The executor does not read a mutable workflow definition. A publish operation validates the graph, model configuration, and output schema, then creates an immutable version with a hash. An execution copies that version hash onto its own record.

This gives every run a stable answer to "what code and configuration did this use?" A replay can pin the original version or select another published version from the same workflow family. The replay is always a dry run, so it cannot create an external-write receipt.

The graph also owns control flow. A model can make a bounded judgment inside one node, but it does not choose the next top-level node. That keeps workflow order, authorization, and side effects in application code instead of prompt text.

let PostgreSQL own the durable state

The runtime is a PostgreSQL-backed state machine inside a FastAPI monolith. Each execution step is durable before a worker claims it. Workers use row locks with SKIP LOCKED, and a stale claim can return to the queue after a timeout.

The database also holds the safety keys:

These rules turn a worker crash or a retried HTTP request into a known state transition. They do not make an external provider reliable, but they make the boundary visible and repeatable.

separate human decisions from side effects

Human review is a durable state, not a comment attached to a failed run. When a step reaches waiting_human, the operator can record a decision and resume it. A connector configured as an external writer performs its write only after that transition.

This distinction matters for compliance work. The system can explain an observed requirement and route an exception. It must not silently turn an uncertain extraction into an unreviewed approval.

make inspection and replay safe by default

An operator needs more than a final status. The inspector returns the execution timeline, tool arguments, provider traces, and correlation IDs, but it redacts secret-like and PII fields before the response leaves the server. The business timeline stays in the execution tables; optional OpenTelemetry or Langfuse spans add provider detail without creating a second source of truth.

Replay uses the same evidence path with dry_run=true. The timeline records the selected version and the disabled side-effect boundary. That makes a regression test useful without turning a debugging action into a duplicate customer write.

turn release criteria into executable evidence

The release gate is a contract, not a sentence in a README. The repository includes focused checks for workflow publishing, runtime recovery, evaluation drift, value-ledger reconciliation, governance redaction, backup and restore, image security, and a browser role tour.

The final CI run completed two jobs. The first ran Python, frontend, harness, evaluation, dependency, SBOM, and static package checks. The second started an ephemeral PostgreSQL stack, applied migrations, ran integration tests, exercised recovery and launch acceptance, and uploaded image evidence. The launch fixture is synthetic. It proves that the gate works; it is not customer acceptance.

The result is a more useful definition of "the model worked." It means the system can show which version ran, which decisions were made, which writes were acknowledged, and whether the release evidence satisfies its ship or hold rules. That ledger is what lets an AI workflow move from a successful demo to an operable product.