Compliance as Code: Automating Audit Evidence Collection

Compliance programs fail when they live in spreadsheets. The evidence is always out of date, the screenshots are always from last quarter, and the control descriptions always diverge from what the systems actually do. Engineers spend weeks before an audit scrambling to produce evidence that should have been collecting itself all along.

Compliance as code treats compliance requirements the same way you treat infrastructure — as code that is version-controlled, tested, and continuously validated.

---

The Compliance Gap

Traditional compliance:
  Policy document (Word) ──→ Manual implementation ──→ Manual evidence ──→ Audit
       (outdated)              (inconsistent)           (scrambled)       (painful)

Compliance as code:
  Policy-as-code (Git) ──→ Automated enforcement ──→ Automated evidence ──→ Audit
     (version-controlled)   (continuous)              (always current)     (smooth)

Traditional	Compliance as Code
Evidence collected weeks before audit	Evidence collected continuously
Policies in Word documents	Policies in version-controlled code
Manual checks once per audit cycle	Automated checks every hour/day
Screenshots as evidence	API-generated reports with timestamps
”We believe this control works"	"Here is the automated test result from 2 hours ago”

---

Policy-as-Code Frameworks

Open Policy Agent (OPA)

# Rego policy: enforce encryption at rest for all S3 buckets
package aws.s3

deny[msg] {
    bucket := input.resource.aws_s3_bucket[name]
    not bucket.server_side_encryption_configuration
    msg := sprintf("S3 bucket '%s' must have encryption enabled", [name])
}

deny[msg] {
    bucket := input.resource.aws_s3_bucket[name]
    bucket.acl == "public-read"
    msg := sprintf("S3 bucket '%s' must not be publicly readable", [name])
}

deny[msg] {
    bucket := input.resource.aws_s3_bucket[name]
    not bucket.versioning[_].enabled
    msg := sprintf("S3 bucket '%s' must have versioning enabled", [name])
}

Terraform Sentinel / Checkov

# Checkov: scan Terraform plans for compliance violations
# checkov -d ./terraform --framework terraform

# Custom check: ensure all databases are encrypted
from checkov.terraform.checks.resource.base_resource_check import BaseResourceCheck
from checkov.common.models.enums import CheckResult, CheckCategories

class RDSEncryption(BaseResourceCheck):
    def __init__(self):
        name = "Ensure RDS instance has encryption enabled"
        id = "CUSTOM_RDS_001"
        supported_resources = ['aws_db_instance']
        categories = [CheckCategories.ENCRYPTION]
        super().__init__(name=name, id=id,
                        categories=categories,
                        supported_resources=supported_resources)

    def scan_resource_conf(self, conf):
        if conf.get('storage_encrypted', [False])[0]:
            return CheckResult.PASSED
        return CheckResult.FAILED

---

Automated Evidence Collection

Evidence Types and Automation

Compliance Control	Evidence Needed	Automation Method
Access reviews	List of users and permissions	API query IAM/AD, generate report weekly
Encryption at rest	Database and storage config	Cloud config scan, export settings
Backup verification	Backup logs and restoration tests	Automated restore test, log collection
Change management	All changes tracked and approved	Git history + PR approval records
Vulnerability scanning	Scan results with remediation timeline	CI/CD scan output, ticket tracking
Incident response	Incident logs and post-mortems	PagerDuty/OpsGenie export, wiki audit
Network segmentation	Firewall rules and network policies	Export security groups, network policies

# Automated evidence collection script
import json
from datetime import datetime, timedelta

class ComplianceEvidenceCollector:
    """Collect and store compliance evidence automatically."""

    def collect_access_review(self):
        """SOC 2 CC6.1: Logical access security."""
        users = self.iam_client.list_users()
        evidence = {
            "control": "CC6.1",
            "description": "Logical access review",
            "collected_at": datetime.utcnow().isoformat(),
            "data": {
                "total_users": len(users),
                "users_with_mfa": sum(1 for u in users if u['mfa_enabled']),
                "inactive_users": [u for u in users
                                   if u['last_login'] < datetime.utcnow() - timedelta(days=90)],
                "admin_users": [u for u in users if 'admin' in u['groups']],
            },
            "findings": []
        }

        # Flag findings
        if evidence["data"]["users_with_mfa"] < evidence["data"]["total_users"]:
            evidence["findings"].append({
                "severity": "HIGH",
                "message": f"{evidence['data']['total_users'] - evidence['data']['users_with_mfa']} users without MFA"
            })

        for user in evidence["data"]["inactive_users"]:
            evidence["findings"].append({
                "severity": "MEDIUM",
                "message": f"User {user['username']} inactive for > 90 days"
            })

        self.store_evidence(evidence)
        return evidence

    def collect_encryption_evidence(self):
        """SOC 2 CC6.7: Encryption at rest and in transit."""
        databases = self.rds_client.describe_instances()
        evidence = {
            "control": "CC6.7",
            "collected_at": datetime.utcnow().isoformat(),
            "data": {
                "databases": [{
                    "identifier": db['id'],
                    "encrypted": db['storage_encrypted'],
                    "encryption_key": db.get('kms_key_id', 'default'),
                    "ssl_enforced": db.get('ssl_required', False)
                } for db in databases]
            }
        }
        self.store_evidence(evidence)
        return evidence

---

Continuous Compliance Monitoring

┌──────────────────────────────────────────────────┐
│  CONTINUOUS COMPLIANCE PIPELINE                   │
├──────────────────────────────────────────────────┤
│                                                    │
│  Every commit:                                     │
│  ├─ Policy-as-code check (Terraform → OPA/Checkov)│
│  ├─ Vulnerability scan (Trivy, Snyk)              │
│  └─ Block merge if compliance check fails          │
│                                                    │
│  Every day:                                        │
│  ├─ Cloud configuration scan (AWS Config, Scout)   │
│  ├─ Access review automation                       │
│  └─ Evidence snapshot stored to compliance vault    │
│                                                    │
│  Every week:                                       │
│  ├─ Compliance dashboard updated                   │
│  ├─ Findings triaged and assigned                  │
│  └─ SLA tracking on open findings                  │
│                                                    │
│  Every quarter:                                    │
│  ├─ Full evidence package generated                │
│  ├─ Control effectiveness review                   │
│  └─ Policy updates based on findings               │
│                                                    │
└──────────────────────────────────────────────────┘

---

Audit Preparation Workflow

Phase	Duration	Activities
6 months before	Ongoing	Continuous evidence collection running, dashboard green
3 months before	1 week	Review evidence completeness, fill gaps, fix findings
1 month before	2-3 days	Generate evidence package, dry-run with internal audit
During audit	1-2 weeks	Answer auditor questions, provide additional evidence on request
After audit	1 week	Address audit findings, update controls, improve automation

---

Implementation Checklist

• Define compliance controls as code (OPA, Checkov, or custom policies)
• Integrate policy checks into CI/CD pipeline — block non-compliant deployments
• Automate evidence collection for top 10 audit controls
• Set up daily cloud configuration scanning (AWS Config, Scout Suite)
• Build compliance dashboard showing control status and open findings
• Automate access reviews: weekly user/permission reports with anomaly flags
• Store all evidence in a versioned, immutable evidence vault
• Run quarterly internal audits against your evidence package
• Track finding remediation with SLAs: critical (7 days), high (30 days)
• Generate audit-ready evidence packages with one command