GitOps Secrets Management

GitOps declares that git is the source of truth for all infrastructure and application configuration. But secrets — database passwords, API keys, TLS certificates — cannot be stored in git. This creates a tension: how do you maintain the GitOps principle of everything-in-git while keeping secrets secure? The solution is encrypted references in git that are decrypted at deployment time.

Secrets Management Approaches

Option 1: Sealed Secrets (Bitnami)
  How: Encrypt secrets with cluster's public key, store encrypted in git
  Decrypt: Sealed Secrets controller in cluster decrypts at deploy time
  Pros: Simple, native Kubernetes, secrets versioned in git
  Cons: Cluster-specific encryption, manual rotation, no external vault
  
  Workflow:
    Developer → kubeseal encrypt → Commit SealedSecret to git
    ArgoCD syncs → Sealed Secrets controller decrypts → Kubernetes Secret created

Option 2: External Secrets Operator (ESO)
  How: Store secrets in external vault, reference them from Kubernetes
  Decrypt: ESO fetches secrets from vault, creates Kubernetes Secrets
  Pros: Centralized vault, auto-rotation, multi-cluster
  Cons: Vault dependency, network access required
  
  Supported backends: AWS Secrets Manager, HashiCorp Vault, 
                      Azure Key Vault, GCP Secret Manager

Option 3: SOPS (Mozilla)
  How: Encrypt specific values in YAML/JSON files
  Decrypt: CI/CD pipeline decrypts with KMS key at deploy time
  Pros: Partial encryption (metadata visible, values encrypted)
  Cons: KMS key management, CI/CD must have decrypt access

Recommendation:
  Small teams / single cluster → Sealed Secrets
  Multi-cluster / enterprise → External Secrets Operator
  CI/CD focused (non-Kubernetes) → SOPS

External Secrets Operator Example

# Step 1: Define how to connect to the secret store
apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  name: aws-secrets-manager
spec:
  provider:
    aws:
      service: SecretsManager
      region: us-east-1
      auth:
        jwt:
          serviceAccountRef:
            name: external-secrets-sa
            namespace: external-secrets

---
# Step 2: Define which secrets to fetch
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: database-credentials
  namespace: production
spec:
  refreshInterval: 1h  # Auto-refresh every hour
  
  secretStoreRef:
    name: aws-secrets-manager
    kind: ClusterSecretStore
  
  target:
    name: database-credentials  # Kubernetes Secret name
    creationPolicy: Owner
  
  data:
    - secretKey: DB_HOST
      remoteRef:
        key: production/database
        property: host
    - secretKey: DB_PASSWORD
      remoteRef:
        key: production/database
        property: password
    - secretKey: DB_USERNAME
      remoteRef:
        key: production/database
        property: username

Anti-Patterns

Secrets management in GitOps is about maintaining two seemingly contradictory principles: everything in git (declarative, auditable) and secrets never in git (secure, least-privilege). The solution is always indirection — store a reference in git, resolve it at deployment time.