Enterprise Secrets Management: HashiCorp Vault, Kubernetes ESO, and SOPS

Secrets (API tokens, database passwords, TLS private keys, SSH keys) require centralized lifecycle governance, encryption-at-rest, dynamic generation, automated rotation, and strict access auditing. This guide covers HashiCorp Vault, dynamic database credentials, Kubernetes External Secrets Operator (ESO), and GitOps-friendly encryption with SOPS.

⚡ Quick Dive

Secrets Management Approaches Comparison

Solution Architecture Storage Location Rotation GitOps Friendly?
HashiCorp Vault Centralized API server with secret engines Encrypted backend (Raft, Consul) ⚡ Automated / Dynamic ✅ (Via ESO or Agent)
AWS Secrets Manager Managed Cloud Key-Value Store AWS Cloud Automated Lambda ✅ (Via ESO)
SOPS + age Client-side file encryption Encrypted in Git repo Manual / Scripted ⚡ Native GitOps
Sealed Secrets Asymmetric K8s Controller Encrypted in Git repo Cluster-bound key ⚡ Native GitOps
External Secrets (ESO) K8s Operator syncing external Vault/AWS Syncs to native K8s Secret Synced from upstream ⚡ Native GitOps

📖 Extended Guide

1. HashiCorp Vault: Dynamic Secrets Architecture

Static secrets (long-lived passwords) present permanent risk when leaked. Dynamic Secrets are generated on-demand with short Time-To-Live (TTL) leases and automatically revoked upon expiration.

[ App / Pod ] ── (Requests DB Access) ──► [ HashiCorp Vault ]
                                                   │
                                                   ▼ (Generates unique user & password)
                                          [ PostgreSQL Database ]
                                                   │
[ App receives: 'v-app-usr-8x2', lease 1h ] ◄──────┘
(After 1 hour, Vault automatically drops the user from PostgreSQL)

Vault CLI Quick Commands:

# Read static secret
vault kv get -mount=secret app/config

# Generate dynamic database credentials
vault read database/creds/read-only-role

# Revoke lease immediately
vault lease revoke database/creds/read-only-role/7f8a9...

2. Kubernetes External Secrets Operator (ESO)

In GitOps workflows, raw Kubernetes Secrets cannot be committed to Git. The External Secrets Operator (ESO) reads encrypted metadata from external Vault/AWS and generates native Kubernetes Secrets inside the cluster:

[ Git Repository ]
└── ExternalSecret YAML (Contains ONLY secret names & references)
          │ (Committed safely to Git)
          ▼
┌──────────────────────────────────────────────────────────┐
│ Kubernetes Cluster                                       │
│   [ External Secrets Operator ]                          │
│          ▲                                               │
│          │ (Fetches actual secret payload over mTLS)     │
│   [ HashiCorp Vault ]                                    │
│          │                                               │
│          ▼ (Creates inside cluster)                      │
│   [ Native Kubernetes Secret: 'db-credentials' ]         │
└──────────────────────────────────────────────────────────┘

ExternalSecret Resource Manifest:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: payment-db-secret
  namespace: production
spec:
  refreshInterval: "1h"
  secretStoreRef:
    name: vault-backend
    kind: ClusterSecretStore
  target:
    name: payment-db-credentials # Name of created K8s Secret
  data:
    - secretKey: password
      remoteRef:
        key: secret/data/production/database
        property: password

3. Encrypted Secrets in Git with SOPS and age

SOPS (Secrets OPerationS) encrypts values inside YAML/JSON files while leaving the keys in plain text (allowing diffs and Git merges):

# 1. Generate an age keypair
age-keygen -o ~/.config/sops/age/keys.txt

# 2. Encrypt secrets.yaml with public key
sops --encrypt --age $(cat ~/.config/sops/age/keys.txt | grep "public key:" | cut -d: -f2 | tr -d ' ') \
  secrets.raw.yaml > secrets.enc.yaml

# 3. Edit encrypted file seamlessly (decrypts in memory, re-encrypts on save)
sops secrets.enc.yaml
⌂ All Guides