Peter Scheffer's Blog

Software Factory designer and builder

What Problem Does Write-Once-Read-Many Storage Solve in a Software Factory?

WORM storage concept diagram showing append-only ledger with provenance records and tamper-evident audit trail

A software factory makes decisions without a human in the loop. Those decisions need to be auditable after the fact. But auditability is not enough if the record can be edited or deleted after the fact. Write-Once-Read-Many (WORM) storage solves this by making the provenance trail tamper-evident: once a record is written, it can never be changed. Only appended to. Trust in an autonomous system requires that kind of immutability, because the alternative is trusting that nobody will ever tamper with the logs.

The Accountability Problem

When a human makes a decision, you can ask them later why they did it. They might remember, or they might not. But at least there is a person who was there. In an autonomous software factory, no human is present when the decision is made. The factory evaluates a change, runs it through the gate engine, and promotes or rejects it based on policy. If you cannot reconstruct exactly what happened and why, you cannot hold the system accountable.

This is discussion problem 4: the accountability gap. The factory's answer is provenance: a permanent record of every state transition, every gate decision, every artifact move. But a provenance system is only useful if its records cannot be altered. If someone can go back and change what happened, then the accountability trail is worthless. The record says one thing happened, but you have no way to verify that it was not rewritten after the fact.

How WORM Storage Works

WORM stands for Write-Once-Read-Many. The contract is simple: you can write a record once, read it any number of times, but you can never update or delete it. The only write operation is append. No edits. No deletes. No backdating. Each record is permanent.

In the Fable Software Factory, the provenance store is defined as a WORM audit log. The ProvenanceRecord artifact captures every transition: who or what actor made the decision, which gate fired and with what verdict, and why. The port contract (ports/telemetry.py) specifies this durability contract. The in-memory reference implementation in adapters/fakes.py models it as an append-only list. The real production backend, which is tracked as DECISION[D-002], would use something like object-lock S3, an append-only ledger, or a cryptographic signing scheme that chains records together.

The key design choice is that the WORM property is baked into the port contract, not left as an operational preference. Every adapter must implement append-only semantics. There is no configuration flag to turn immutability off.

What Immutability Enables

Immutability is the foundation for several factory capabilities that would be weak or meaningless with mutable storage.

Audit trails that hold up to scrutiny. An auditor can examine the provenance store and know that what they are reading is what was recorded at the time. No records have been deleted. No timestamps have been changed. The trail is intact. This is the difference between a system that logs and a system that provides evidence.

Escaped defect analysis. When a defect escapes to production, the eval loop harvests that incident as a regression test case. It replays the frozen execution context and scores the candidate component against it. But this only works if the provenance record of what happened during the original incident is still there, unmodified, exactly as it was recorded. WORM storage guarantees that.

Regulatory compliance. For industries with hard record-keeping requirements (finance, healthcare, defense), WORM storage is often a regulatory mandate, not a design preference. The factory needs to operate in those environments eventually. Baking WORM into the provenance port means the factory can meet those requirements without an architectural retrofit.

The Provenance Connection

Provenance and WORM storage are tightly linked. Provenance is the decision trail: a sequence of records that answer who did what, when, and why. WORM storage is the implementation choice that makes that trail trustworthy.

Every ProvenanceRecord in the factory links back to its originating spec through ArtifactMeta.parent_ids. This means any artifact can be walked back to its root cause. But that chain of links is only as strong as the storage that holds it. If any link in the chain can be edited, the entire trail is suspect. WORM storage locks every link in place.

The factory's architecture diagram labels the provenance store as a WORM audit log precisely because the immutability is not incidental. It is the defining property that makes accountability real rather than cosmetic.

Real-World Implications for Audits

Consider a real scenario. A production incident occurs. An investigation begins. The team asks: which component version was live at the time? Which gate approved it? What evidence did the gate see? What was the risk score?

With a mutable log, the answer might be uncertain. Maybe someone cleaned up old records. Maybe a rollback overwrote the state. Maybe a logging pipeline dropped entries and nobody noticed. With a WORM provenance store, the records are still there exactly as they were when the events happened. There is no ambiguity. The investigator reads the append-only log and sees the unaltered timeline.

This matters for post-incident reviews. It matters for regulatory audits. It matters for any situation where the factory's decisions need to be justified after the fact. The WORM contract means the factory can be held accountable for its actions even though no human was present when those actions were taken. That is the entire point of the provenance system, and WORM storage is what makes it work.

Ready to put these ideas into practice?

Book a free 30-minute consultation to discuss how AI-driven delivery engineering can transform your organisation.

Book a Strategy Call