GitOps Automation Pipeline

GitOps is the practice of using Git as the single source of truth for declarative infrastructure and applications. Instead of running commands to change infrastructure, you commit desired state to Git. An automated reconciliation loop detects the difference between desired state and actual state, and corrects it. The result: auditable, reversible, and automated infrastructure delivery.

---

GitOps Reconciliation Loop

Developer commits desired state to Git
     │
     ▼
Git Repository (Source of Truth)
     │
     ▼
GitOps Operator (ArgoCD/Flux) detects change
     │
     ▼
Compare: Desired State (Git) vs Actual State (Cluster)
     │
     ├── Match → No action needed ✓
     │
     └── Drift detected → Reconcile
           │
           ▼
         Apply changes to cluster
           │
           ▼
         Verify health checks
           │
           ├── Healthy → Update status, notify ✓
           │
           └── Unhealthy → Auto-rollback, alert ✗

This loop runs continuously (every 3-5 minutes)
No human intervention needed for normal operations

---

ArgoCD Application

# ArgoCD Application manifest
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: production-api
  namespace: argocd
spec:
  project: production
  source:
    repoURL: https://github.com/org/k8s-manifests.git
    targetRevision: main
    path: environments/production/api
  destination:
    server: https://kubernetes.default.svc
    namespace: api
  syncPolicy:
    automated:
      prune: true        # Remove resources deleted from Git
      selfHeal: true     # Fix manual changes (drift)
      allowEmpty: false  # Prevent accidental deletion of all resources
    syncOptions:
      - CreateNamespace=true
      - PruneLast=true   # Delete old resources after new ones are healthy
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m

---

Repository Structure

k8s-manifests/
├── base/                          # Shared base manifests
│   ├── api/
│   │   ├── deployment.yaml
│   │   ├── service.yaml
│   │   ├── hpa.yaml
│   │   └── kustomization.yaml
│   └── worker/
│       ├── deployment.yaml
│       └── kustomization.yaml
│
├── environments/
│   ├── staging/
│   │   ├── api/
│   │   │   ├── kustomization.yaml    # Patches for staging
│   │   │   └── replicas-patch.yaml   # replicas: 1
│   │   └── worker/
│   │       └── kustomization.yaml
│   │
│   └── production/
│       ├── api/
│       │   ├── kustomization.yaml    # Patches for production
│       │   ├── replicas-patch.yaml   # replicas: 5
│       │   └── resources-patch.yaml  # Higher CPU/memory limits
│       └── worker/
│           └── kustomization.yaml
│
├── platform/                      # Cluster-level resources
│   ├── monitoring/
│   ├── ingress/
│   └── cert-manager/
│
└── scripts/
    ├── promote.sh                 # Promote staging → production
    └── rollback.sh               # Rollback to previous commit

---

Drift Detection

class DriftDetector:
    """Detect and alert on infrastructure drift."""
    
    def check_drift(self, app_name: str):
        """Compare Git state to cluster state."""
        git_state = self.get_git_manifests(app_name)
        cluster_state = self.get_cluster_state(app_name)
        
        drifts = []
        for resource in git_state:
            actual = cluster_state.get(resource.key)
            if actual is None:
                drifts.append({
                    "type": "missing",
                    "resource": resource.key,
                    "action": "Resource in Git but not in cluster",
                })
            elif resource.spec != actual.spec:
                drifts.append({
                    "type": "modified",
                    "resource": resource.key,
                    "action": "Manual change detected, will revert",
                    "diff": self.diff(resource.spec, actual.spec),
                })
        
        if drifts:
            self.alert(f"Drift detected in {app_name}", drifts)
            if self.auto_heal:
                self.reconcile(app_name)  # Force sync from Git
        
        return drifts

---

Anti-Patterns

Anti-Pattern	Consequence	Fix
Manual kubectl apply	No audit trail, drift-prone	All changes through Git commits
No drift detection	Manual changes go unnoticed	selfHeal enabled, drift alerts
Secrets in Git repo	Security breach	External Secrets Operator, sealed-secrets
No promotion workflow	Changes skip staging	PR-based promotion: staging → production
Single repo for everything	Blast radius too large	Separate repos per team or domain

GitOps eliminates the “what changed?” question. Every change is a Git commit with an author, timestamp, review, and the ability to revert. When your production environment is defined in Git, your change management process is your Git workflow.