Compliance as Code: Evidence Should Be Generated, Not Collected

A few years ago, I watched a team spend two weeks pulling audit evidence from screenshots, export files, ticket comments, and Slack threads. Nothing was wrong with the controls. The problem was that the evidence was a late reconstruction of work the system had already done.

That model falls apart quickly. People miss artifacts. Screenshots drift away from the build they were meant to prove. Logs rotate. Someone edits a spreadsheet and the trail turns into a guess.

Why Manual Evidence Collection Breaks Down

Manual evidence collection fails for the same reason manual configuration fails: it relies on memory and discipline. Auditors want proof that a control ran, that it produced a result, and that the result maps to a specific system version. A folder full of screenshots rarely proves all three.

The failure mode is usually not malicious. It is ambiguity.

• Which commit produced this test result?
• Which environment ran the deployment?
• Was the policy check executed before or after release approval?
• Who changed the evidence after the fact?

If you cannot answer those questions automatically, you end up rebuilding them every audit cycle.

What Compliance as Code Actually Means

Compliance as code is not just “put the policy in YAML.” It means controls, checks, and evidence generation are part of the delivery system itself.

The useful shift is simple: evidence should be a byproduct of running the system, not a separate project.

That gives you three practical properties:

• Repeatability: the same pipeline step produces the same kind of evidence every run.
• Traceability: the evidence links to commit, build, environment, and approver.
• Tamper resistance: evidence is written once, signed, and stored outside the mutable app path.

If a control matters, it should leave a machine-readable trace.

Evidence Should Come From the Pipeline

The cleanest evidence is generated at the same layer that enforces the control. That usually means build, test, deploy, and policy gates.

Build, test, and deployment artifacts

A release pipeline already knows things auditors care about:

• the Git commit SHA
• the build number
• the test suite result
• the container digest
• the deployment target
• the person or service account that approved the run

If you capture those values as structured output, you do not need to rebuild them from screenshots later.

Policy checks as executable controls

A control that exists only in a document is easy to ignore. A control that runs as code can fail the pipeline.

Example checks:

• block deployment if unit tests fail
• require signed artifacts before publish
• reject builds without dependency scans
• verify that production deploys require approval

That is the real value of compliance as code: the control and the proof come from the same execution path.

A Practical JavaScript Example for Evidence Generation

I usually start with a JSON evidence record emitted by the pipeline and signed before storage.

Capturing test results and build metadata

const evidence = {
  pipelineId: process.env.PIPELINE_ID,
  commitSha: process.env.GIT_COMMIT_SHA,
  buildNumber: process.env.BUILD_NUMBER,
  environment: process.env.DEPLOY_ENV,
  tests: {
    passed: 128,
    failed: 0
  },
  policyChecks: [
    { name: "artifact-signed", result: "pass" },
    { name: "approval-required", result: "pass" }
  ],
  generatedAt: new Date().toISOString()
};

const payload = JSON.stringify(evidence, null, 2);
const digest = createHash("sha256").update(payload).digest("hex");

const { privateKey, publicKey } = generateKeyPairSync("ed25519");
const signature = sign(null, Buffer.from(payload), privateKey).toString("base64");

await fs.writeFile(
  `./evidence/${evidence.pipelineId}.json`,
  JSON.stringify({ evidence, digest, signature, publicKey: publicKey.export({ type: "spki", format: "pem" }) }, null, 2)
);

This is not a full compliance system. It is the minimum shape of trustworthy evidence: data, digest, signature, and provenance.

Storing signed, queryable evidence

Signed files are good, but searchable evidence is better. Put the JSON into an append-only store or an object bucket with versioning enabled. Index the same record into a query layer so you can answer audit questions without opening files one by one.

A simple record should be queryable by:

• commit SHA
• service name
• control ID
• date range
• deployment environment

That makes evidence operational instead of ceremonial.

Common Failure Modes and How to Avoid Them

Mutable logs and missing provenance

If evidence lives in the same system it is supposed to prove, it can be edited after the fact. Mutable database rows, shared drive exports, and overwritten log files all weaken the story.

Use:

• append-only storage
• versioned object buckets
• signed records
• separate retention from the application database

Also keep provenance fields mandatory. A control result without commit SHA or pipeline ID is just a note.

Controls that live only in spreadsheets

Spreadsheets are fine for planning. They are weak as evidence. They drift, they invite manual updates, and they do not tell you whether a control actually ran.

If the spreadsheet is the source of truth, the control is probably not automated.

How to Prove the Evidence Is Trustworthy

Auditors usually care about three things: integrity, origin, and completeness.

You can prove integrity by hashing the payload and signing it with a key controlled by the pipeline. You can prove origin by linking the record to a specific build and identity. You can improve completeness by generating evidence from every release, not only from exceptional cases.

A good test is to try to break the trail safely:

1. Re-run the same pipeline and compare outputs.
2. Change the commit SHA and confirm the signature no longer verifies.
3. Remove one control result and verify the report fails validation.
4. Query the evidence store for one deployment and confirm it returns all linked artifacts.

If any of those steps are hard to do, the evidence is probably too manual.

Conclusion

Compliance as code works when evidence is treated like a build artifact. Generate it in the pipeline, sign it, store it immutably, and make it queryable. That removes the guesswork from audits and makes controls easier to defend because the proof was created at the same time the control ran.

The goal is not more paperwork. The goal is less interpretation.