What Is Provenance in Software Factory Design?

Autonomous software factories promise to compress months of engineering work into hours. But with autonomy comes a critical question: how do you know *why* a decision was made? Provenance is the permanent, append-only record of every decision, who made it, which policy gate fired, and what the outcome was. It's the accountability layer that transforms a black box into a trustworthy system.

What Is Provenance?

In the context of software factories, provenance is far more than a log file. It's a structured, tamper-evident audit trail that records every state transition: every artifact promotion, every gate decision, every policy applied. Each ProvenanceRecord captures who or what actor triggered the transition, which gate evaluated it, the gate's decision and reasoning, and a link back to the parent specification. This creates an unbroken chain of custody from the initial requirement all the way through to the deployed change.

The key word is append-only. Unlike a mutable log, provenance lives in a WORM (Write-Once-Read-Many) store. Once written, a record cannot be edited or deleted; only new records can be appended. This tamper-evidence is what separates provenance from a convenient fiction. You're not just trusting that the record was made; you're mathematically certain it hasn't been altered.

Solving the Accountability Gap

Without provenance, autonomous systems create an accountability vacuum. A feature ships. A bug slips through. The question "who signed off on this?" or "which test was supposed to catch that?" has no answer. No decision was ever recorded. Defects disappear into a void, and the factory has no way to learn from them.

Provenance solves this by making every decision visible and traceable. When an escaped defect (a bug that made it to production) is discovered, the factory can walk the provenance chain backwards: from the deployed change to the gate that approved it, back through the evidence bundle that informed the gate, through the test oracle that failed to catch it, all the way to the original specification. This traceability is the foundation of learning. Each escaped defect becomes a permanent regression test case, harvested automatically from the provenance trail.

This is discussion problem 4 from the Software Factory design: the missing accountability trail. Provenance doesn't eliminate mistakes. It ensures they're never silent.

The Artifact Lineage

Provenance works because every artifact in the factory carries lineage metadata: ArtifactMeta.parent_ids. A change-set knows what spec it came from. A verdict knows what plan it evaluated. A promotion decision knows what component version it tested. Every artifact can be walked backwards to its origin.

Combine this with the append-only provenance store, and you have something powerful: a complete, verifiable history of how any piece of code came to exist. Not just that it was deployed, but why, by whom, under what conditions, and what evidence supported the decision. This history is unforgeable. It's cryptographically bound by the structure of the records themselves.

Practical Enforcement

In the factory architecture, provenance isn't optional. Every state transition (from spec intake through planning, building, testing, gating, and promotion) writes a ProvenanceRecord. The control plane, which orchestrates the whole workflow, is deliberately kept thin: it sequences steps and records state, but it never makes decisions. Decision-making is the gate engine's job, and every gate decision is provenance-logged.

What makes this practical rather than bureaucratic is automation. The factory harvests these records to build regression corpora automatically. New eval cases are generated from escaped defects without manual triage. Policy violations are caught at gate time and routed to the right human. The provenance trail becomes the fuel for continuous improvement, not just an audit artifact that sits unread.

Trust Through Traceability

Autonomous software factories succeed or fail on trust. Teams won't grant autonomy to systems they don't understand. Regulators won't approve systems they can't audit. Provenance solves both problems. It answers "how did this happen?" with unforgeable evidence, and it does so automatically, at scale, without requiring manual investigation.

This is why provenance is foundational to the Software Factory design. It's the thread that runs through every decision, every gate, every promotion. Without it, you have autonomy but no accountability. With it, you have a system you can actually trust.

Conclusion

Provenance is the permanent, append-only decision trail that answers the hardest question autonomous systems face: why was this decision made? By recording every state transition, every gate decision, and every actor involved, provenance transforms the accountability gap from a blind spot into a competitive advantage. It enables learning from defects, speeds regulatory compliance, and builds the foundation for earned autonomy. In earned autonomy, systems prove their reliability through measured evidence, not blind faith. In dark factories, provenance isn't a luxury. It's the difference between a system you can trust and one you can't.