Mozilla SOPS: GitOps-Native Secret Encryption That Actually Works

Abstract

Mozilla SOPS (Secrets OPerationS) solves a fundamental challenge in GitOps: how to safely commit secrets to version control while maintaining developer productivity. Unlike cloud-native secret stores, SOPS encrypts files directly in Git repositories, preserving YAML/JSON structure while protecting sensitive values. This guide covers practical implementation patterns including age encryption, AWS Lambda integration, AWS CDK workflows, AWS SAM patterns, and CI/CD automation for serverless deployments across GitHub Actions, GitLab CI, and Jenkins.

The GitOps Secret Management Challenge

Working with Infrastructure as Code creates an immediate problem. You need to version-control your serverless configuration files, Terraform variables, and environment configs. But those files contain database passwords, API keys, and service credentials. The moment you commit secrets to Git, you’ve created a security vulnerability.

Traditional solutions create friction. HashiCorp Vault requires running infrastructure and API calls at deployment time. AWS Secrets Manager costs $0.40 per secret per month and adds runtime API calls to your Lambda functions. AWS Systems Manager Parameter Store is free but still requires runtime fetching. Each approach pulls secrets out of your GitOps workflow and into external systems.

SOPS takes a different approach. It encrypts files directly in your Git repository, keeping secrets versioned alongside your code. When you change an API key and update your Lambda function, both changes go in the same commit. When you roll back, both roll back together. Your Git history becomes your audit trail.

Understanding SOPS Architecture

SOPS uses envelope encryption. When you encrypt a file, SOPS generates a random 256-bit data key and encrypts your file content with AES256-GCM. Then it encrypts that data key with one or more master keys (AWS KMS, age, PGP, GCP KMS, or Azure Key Vault) and stores the encrypted data key in the file’s metadata. Age encrypts the SOPS data key using X25519 + ChaCha20-Poly1305.

For structured formats like YAML and JSON, SOPS only encrypts values, not keys. This keeps your file structure visible for code reviews and allows tools to parse the schema even when values are encrypted.

Here’s what an encrypted YAML file looks like:

database:
  host: ENC[AES256_GCM,data:Zm9vYmFy,iv:abc123,tag:def456,type:str]
  port: ENC[AES256_GCM,data:NTQzMg==,iv:xyz789,tag:uvw012,type:int]
  password: ENC[AES256_GCM,data:c3VwZXI=,iv:secret,tag:hash,type:str]
sops:
  kms:
    - arn: arn:aws:kms:us-east-1:123456789012:key/abc-123
      created_at: '2025-12-17T10:00:00Z'
      enc: AQICAHh...encrypted_data_key...
  age:
    - recipient: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
      enc: |
        -----BEGIN AGE ENCRYPTED FILE-----
        YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBl...
        -----END AGE ENCRYPTED FILE-----
  version: 3.11.0

You can still see the database configuration structure. You know there’s a host, port, and password field. But the actual values are encrypted. Git diff shows which fields changed, not just that “the encrypted blob changed.”

Installation and Setup

Installation varies by platform but takes less than five minutes:

# macOS with Homebrew
brew install sops

# Linux - download latest release
wget https://github.com/getsops/sops/releases/download/v3.11.0/sops-v3.11.0.linux.amd64
chmod +x sops-v3.11.0.linux.amd64
sudo mv sops-v3.11.0.linux.amd64 /usr/local/bin/sops

# Verify installation
sops --version

For container environments, use the official image:

FROM mozilla/sops:v3.11.0

# Copy your encrypted files
COPY secrets.enc.yaml /app/

# Decrypt at runtime
CMD ["sops", "--decrypt", "/app/secrets.enc.yaml"]

Age: The Modern Encryption Choice

SOPS supports multiple key management systems. For team environments, age (pronounced like “h-age”) has become the recommended choice over PGP.

Age public keys are 62 characters, private keys are 74 characters. PGP keys are 4096 characters. You can copy and paste an age public key in Slack. PGP keys break across lines. Age uses modern cryptography (X25519 + ChaCha20-Poly1305). PGP comes with decades of complexity from the GPG keyring system.

Generate an age key pair:

# Install age (v1.2.0 or later)
brew install age  # macOS
apt install age  # Ubuntu

# Generate key pair
age-keygen -o ~/.config/sops/age/keys.txt

# Output shows public key
# Public key: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p

The private key is stored in ~/.config/sops/age/keys.txt. The public key is what you share with team members and configure in SOPS.

To encrypt a file with age:

# Set your private key location
export SOPS_AGE_KEY_FILE=$HOME/.config/sops/age/keys.txt

# Encrypt using public key
sops --age age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p \
  --encrypt secrets.yaml > secrets.enc.yaml

# Decrypt (uses private key from SOPS_AGE_KEY_FILE)
sops --decrypt secrets.enc.yaml > secrets.yaml

For team distribution, each person generates their own age key and shares their public key. You configure SOPS to encrypt with all team members’ public keys. Anyone with their private key can decrypt.

The .sops.yaml Configuration File

Creating a .sops.yaml file in your repository root eliminates manual key management. SOPS reads this file to determine which keys to use based on file paths.

Here’s a production-ready configuration:

creation_rules:
  # Development - simple age keys for all developers
  - path_regex: \.dev\.yaml$
    age: >-
      age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p,
      age1cy0su9fwf8gzkdqh3r4r6xgc92fp8jqrjp4fvd4ak6vd3mc0jjpqnhymkw

  # Staging - AWS KMS for testing production flow
  - path_regex: \.staging\.yaml$
    kms: arn:aws:kms:us-west-2:111111111111:key/staging-key-id
    age: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p

  # Production - multiple KMS keys with redundancy
  - path_regex: \.prod\.yaml$
    key_groups:
      - kms:
          - arn: arn:aws:kms:us-east-1:222222222222:key/prod-key-id
            role: arn:aws:iam::222222222222:role/sops-decrypt-role
          - arn: arn:aws:kms:eu-west-1:222222222222:key/prod-key-eu
        age:
          - age1yx3z8r0hnzjy9wh6fq5gldq3p7hxg6nfkz5vgqcdqhsj8tqxj8xq8w6qur

  # Serverless secrets - AWS KMS
  - path_regex: serverless/.*\.yaml$
    kms: arn:aws:kms:us-east-1:222222222222:key/serverless-key-id

  # Terraform variables - for infrastructure
  - path_regex: terraform/.*\.tfvars$
    kms: arn:aws:kms:us-east-1:222222222222:key/terraform-key-id

  # Default fallback for unmatched files
  - age: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p

Now encryption becomes automatic:

# SOPS reads .sops.yaml to find keys
sops --encrypt config.dev.yaml > config.dev.enc.yaml  # Uses age keys
sops --encrypt config.staging.yaml > config.staging.enc.yaml  # Uses AWS KMS
sops --encrypt config.prod.yaml > config.prod.enc.yaml  # Uses KMS + age

The path-based rules eliminate human error. Developers don’t need to remember which keys to use for which environment.

AWS KMS Integration

For production environments, AWS KMS provides centralized key management with IAM-based access control and audit logging through CloudTrail.

Create a KMS key:

# Create KMS key for SOPS
aws kms create-key \
  --description "SOPS encryption key for production" \
  --key-usage ENCRYPT_DECRYPT

# Create alias for easier reference
aws kms create-alias \
  --alias-name alias/sops-production \
  --target-key-id <key-id-from-previous-command>

# Get the key ARN
aws kms describe-key --key-id alias/sops-production

Configure SOPS to use KMS:

export SOPS_KMS_ARN="arn:aws:kms:us-east-1:123456789012:key/abc-123-def"

# Encrypt file
sops --kms $SOPS_KMS_ARN --encrypt secrets.yaml > secrets.enc.yaml

For CI/CD environments, use IAM roles instead of storing credentials:

# GitHub Actions with OIDC
- name: Configure AWS Credentials
  uses: aws-actions/configure-aws-credentials@v5
  with:
    role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsSOPS
    aws-region: us-east-1

- name: Decrypt with KMS
  run: sops --decrypt secrets.enc.yaml > secrets.yaml

The IAM role needs KMS decrypt permissions:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "kms:Decrypt",
      "kms:DescribeKey"
    ],
    "Resource": "arn:aws:kms:us-east-1:123456789012:key/abc-123"
  }]
}

Multi-Account AWS Setup

Enterprise environments rarely operate within a single AWS account. Development, staging, and production environments run in separate accounts for security isolation and blast radius containment. SOPS supports this architecture through environment-specific KMS keys and cross-account IAM permissions.

Architecture Overview

A typical multi-account setup separates environments into distinct AWS accounts, each with dedicated KMS keys. This prevents developers from accidentally accessing production secrets and provides clear security boundaries.

This architecture enforces several security principles. Developers need explicit cross-account role assumption to access each environment. KMS keys are account-local, so compromising one environment doesn’t expose others. CloudTrail logs in each account provide independent audit trails.

KMS Keys per Environment Account

Each AWS account maintains its own KMS key. The .sops.yaml configuration maps file paths to account-specific KMS keys.

# .sops.yaml - Multi-account configuration
creation_rules:
  # Development account secrets
  - path_regex: secrets/dev/.*\.yaml$
    kms: arn:aws:kms:eu-central-1:111111111111:key/aaaaaaaa-dev-1111-1111-111111111111

  # Staging account secrets
  - path_regex: secrets/staging/.*\.yaml$
    kms: arn:aws:kms:eu-central-1:222222222222:key/bbbbbbbb-stg-2222-2222-222222222222

  # Production account secrets
  - path_regex: secrets/prod/.*\.yaml$
    kms: arn:aws:kms:eu-central-1:333333333333:key/cccccccc-prd-3333-3333-333333333333

  # Fallback for development with age
  - age: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p

Directory structure mirrors the account separation:

secrets/
├── dev/
│  ├── database.yaml  # Encrypted with dev account KMS
│  └── api-keys.yaml
├── staging/
│  ├── database.yaml  # Encrypted with staging account KMS
│  └── api-keys.yaml
└── prod/
    ├── database.yaml  # Encrypted with prod account KMS
    └── api-keys.yaml

When you encrypt a file in secrets/prod/, SOPS automatically uses the production account KMS key. No manual key selection required.

Cross-Account KMS Access

For CI/CD pipelines to decrypt secrets across accounts, KMS key policies must allow cross-account access. This requires configuration in both the KMS key policy and IAM role permissions.

KMS key policy in the production account (333333333333):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Enable IAM User Permissions",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::333333333333:root"
      },
      "Action": "kms:*",
      "Resource": "*"
    },
    {
      "Sid": "Allow CI/CD account to decrypt",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::444444444444:role/GitHubActionsDeployRole"
      },
      "Action": [
        "kms:Decrypt",
        "kms:DescribeKey"
      ],
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "kms:ViaService": [
            "secretsmanager.eu-central-1.amazonaws.com",
            "lambda.eu-central-1.amazonaws.com"
          ]
        }
      }
    }
  ]
}

The condition restricts KMS usage to specific AWS services, preventing direct key access outside of legitimate deployment contexts.

IAM role in the CI/CD account (444444444444):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt",
        "kms:DescribeKey"
      ],
      "Resource": [
        "arn:aws:kms:eu-central-1:111111111111:key/aaaaaaaa-dev-1111-1111-111111111111",
        "arn:aws:kms:eu-central-1:222222222222:key/bbbbbbbb-stg-2222-2222-222222222222",
        "arn:aws:kms:eu-central-1:333333333333:key/cccccccc-prd-3333-3333-333333333333"
      ]
    },
    {
      "Effect": "Allow",
      "Action": "sts:AssumeRole",
      "Resource": [
        "arn:aws:iam::111111111111:role/LambdaDeployRole",
        "arn:aws:iam::222222222222:role/LambdaDeployRole",
        "arn:aws:iam::333333333333:role/LambdaDeployRole"
      ]
    }
  ]
}

This IAM policy allows the CI/CD role to both decrypt with KMS keys and assume deployment roles in target accounts.

CI/CD with Role Assumption

GitHub Actions workflows assume different roles for different environments. The AWS credentials action supports role chaining for multi-account deployments.

name: Multi-Account Lambda Deploy
on:
  push:
    branches: [main]

jobs:
  deploy-dev:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS Credentials for Dev
        uses: aws-actions/configure-aws-credentials@v5
        with:
          role-to-assume: arn:aws:iam::111111111111:role/GitHubActionsDeployRole
          aws-region: eu-central-1

      - name: Install SOPS
        run: |
          wget -q https://github.com/getsops/sops/releases/download/v3.11.0/sops-v3.11.0.linux.amd64
          chmod +x sops-v3.11.0.linux.amd64
          sudo mv sops-v3.11.0.linux.amd64 /usr/local/bin/sops

      - name: Decrypt Dev Secrets
        run: sops --decrypt secrets/dev/database.yaml > /tmp/secrets.yaml

      - name: Deploy to Dev Lambda
        run: |
          # Deployment commands use /tmp/secrets.yaml
          serverless deploy --stage dev

  deploy-prod:
    runs-on: ubuntu-latest
    needs: [deploy-dev]
    environment: production
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS Credentials for Prod
        uses: aws-actions/configure-aws-credentials@v5
        with:
          role-to-assume: arn:aws:iam::333333333333:role/GitHubActionsDeployRole
          aws-region: eu-central-1

      - name: Install SOPS
        run: |
          wget -q https://github.com/getsops/sops/releases/download/v3.11.0/sops-v3.11.0.linux.amd64
          chmod +x sops-v3.11.0.linux.amd64
          sudo mv sops-v3.11.0.linux.amd64 /usr/local/bin/sops

      - name: Decrypt Prod Secrets
        run: sops --decrypt secrets/prod/database.yaml > /tmp/secrets.yaml

      - name: Deploy to Prod Lambda
        run: |
          # Deployment commands use /tmp/secrets.yaml
          serverless deploy --stage prod

The workflow separates deployment jobs by environment. Each job assumes the appropriate account role before decrypting secrets. The needs dependency ensures dev deploys before production. The environment: production adds manual approval gates.

Developer Workflow with AWS Profiles

Developers working locally need AWS profile configurations for each account. The ~/.aws/config file defines role assumption chains.

# ~/.aws/config
[profile dev]
role_arn = arn:aws:iam::111111111111:role/DeveloperRole
source_profile = default
region = eu-central-1
output = json

[profile staging]
role_arn = arn:aws:iam::222222222222:role/DeveloperRole
source_profile = default
region = eu-central-1
output = json

[profile prod]
role_arn = arn:aws:iam::333333333333:role/DeveloperRole
source_profile = default
region = eu-central-1
output = json
mfa_serial = arn:aws:iam::444444444444:mfa/ayhan.sipahi

Production access requires MFA. When a developer runs SOPS commands for production secrets, AWS prompts for an MFA token.

Encrypting secrets for different environments:

# Encrypt dev secret
AWS_PROFILE=dev sops --encrypt secrets/dev/database.yaml > secrets/dev/database.enc.yaml

# Encrypt staging secret
AWS_PROFILE=staging sops --encrypt secrets/staging/database.yaml > secrets/staging/database.enc.yaml

# Encrypt prod secret (prompts for MFA)
AWS_PROFILE=prod sops --encrypt secrets/prod/database.yaml > secrets/prod/database.enc.yaml

Decrypting for local testing:

# Decrypt dev secrets locally
AWS_PROFILE=dev sops --decrypt secrets/dev/database.enc.yaml > .env.dev

# Decrypt staging (with staging role)
AWS_PROFILE=staging sops --decrypt secrets/staging/database.enc.yaml > .env.staging

The profile selection happens through the AWS_PROFILE environment variable. SOPS automatically uses the correct KMS key based on file path and assumes the appropriate role based on the active profile.

Security Considerations

Multi-account SOPS deployments introduce several security requirements that must be enforced through IAM policies and organizational controls.

Principle of Least Privilege: Developers should access only the environments they actively work with. A junior developer working on development environments shouldn’t have production KMS decrypt permissions. Role policies should reflect this segregation.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt",
        "kms:DescribeKey"
      ],
      "Resource": [
        "arn:aws:kms:eu-central-1:111111111111:key/aaaaaaaa-dev-1111-1111-111111111111"
      ]
    },
    {
      "Effect": "Deny",
      "Action": [
        "kms:Decrypt",
        "kms:DescribeKey"
      ],
      "Resource": [
        "arn:aws:kms:eu-central-1:333333333333:key/cccccccc-prd-3333-3333-333333333333"
      ]
    }
  ]
}

The explicit deny ensures even if higher-level policies grant access, this developer cannot decrypt production secrets.

Audit Logging with CloudTrail: Each AWS account should have CloudTrail enabled with logs shipped to a centralized security account. This creates an immutable audit trail of all KMS operations.

{
  "eventVersion": "1.08",
  "userIdentity": {
    "type": "AssumedRole",
    "principalId": "AROAEXAMPLE:ayhan.sipahi",
    "arn": "arn:aws:sts::333333333333:assumed-role/DeveloperRole/ayhan.sipahi"
  },
  "eventTime": "2025-12-17T10:30:00Z",
  "eventSource": "kms.amazonaws.com",
  "eventName": "Decrypt",
  "requestParameters": {
    "keyId": "arn:aws:kms:eu-central-1:333333333333:key/cccccccc-prd-3333-3333-333333333333"
  },
  "responseElements": null,
  "requestID": "abc-123-def-456",
  "resources": [{
    "accountId": "333333333333",
    "type": "AWS::KMS::Key",
    "ARN": "arn:aws:kms:eu-central-1:333333333333:key/cccccccc-prd-3333-3333-333333333333"
  }]
}

CloudTrail logs show who accessed which KMS key at what time. This enables detection of unauthorized access attempts or compliance audits.

Key Policies vs IAM Policies: Use both for defense in depth. KMS key policies define who can use the key at the resource level. IAM policies define what the identity can do. Both must allow the operation for it to succeed.

A production KMS key should have a restrictive key policy that only allows specific deployment roles, even if broader IAM policies exist elsewhere. This prevents privilege escalation through IAM policy changes alone.

Break-Glass Procedures: Even with MFA and restrictive policies, emergencies require rapid production access. Maintain an emergency access role with time-limited credentials and automatic alerting when used.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::333333333333:role/EmergencyBreakGlassRole"
      },
      "Action": [
        "kms:Decrypt",
        "kms:DescribeKey"
      ],
      "Resource": "*",
      "Condition": {
        "DateGreaterThan": {
          "aws:CurrentTime": "2025-12-17T00:00:00Z"
        },
        "DateLessThan": {
          "aws:CurrentTime": "2025-12-18T00:00:00Z"
        }
      }
    }
  ]
}

This policy allows emergency access but only within a time window. When the role is assumed, CloudWatch Events trigger alerts to security teams and management.

AWS Lambda Integration with SOPS

Lambda functions need secrets at runtime, but you want to version-control those secrets alongside your function code. SOPS enables this by decrypting secrets during deployment, not at runtime.

AWS CDK Integration

AWS CDK provides the most elegant integration with SOPS. CDK can decrypt secrets at synthesis time and inject them directly into Lambda environment variables, SSM parameters, or Secrets Manager. This approach keeps your infrastructure code clean while maintaining GitOps practices.

Create encrypted secrets:

# secrets/prod.enc.yaml
database:
  host: prod-db.example.com
  username: admin
  password: super_secret_password
stripe:
  secret_key: sk_live_abc123
  webhook_secret: whsec_xyz789
redis:
  host: prod-redis.example.com
  port: 6379

Encrypt the file:

sops --encrypt secrets/prod.yaml > secrets/prod.enc.yaml

Pattern 1: Direct Environment Variable Injection

The simplest pattern decrypts SOPS at synthesis time and injects values as Lambda environment variables:

// lib/lambda-stack.ts
import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import { execSync } from 'child_process';
import * as yaml from 'js-yaml';

export class LambdaStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Decrypt SOPS file at synthesis time
    const decrypted = execSync('sops --decrypt secrets/prod.enc.yaml', {
      encoding: 'utf-8'
    });
    const secrets = yaml.load(decrypted) as any;

    // Create Lambda with decrypted secrets
    new lambda.Function(this, 'ApiFunction', {
      runtime: lambda.Runtime.NODEJS_20_X,
      handler: 'index.handler',
      code: lambda.Code.fromAsset('src'),
      environment: {
        DB_HOST: secrets.database.host,
        DB_USERNAME: secrets.database.username,
        DB_PASSWORD: secrets.database.password,
        STRIPE_SECRET_KEY: secrets.stripe.secret_key,
        REDIS_HOST: secrets.redis.host,
      },
    });
  }
}

Pattern 2: SSM Parameter Store Population

A more flexible pattern uses SOPS to populate SSM Parameter Store, then references parameters in Lambda:

// lib/lambda-stack.ts
import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as ssm from 'aws-cdk-lib/aws-ssm';
import { execSync } from 'child_process';
import * as yaml from 'js-yaml';

export class LambdaStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const decrypted = execSync('sops --decrypt secrets/prod.enc.yaml', {
      encoding: 'utf-8'
    });
    const secrets = yaml.load(decrypted) as any;

    // Store secrets in SSM Parameter Store
    const dbPassword = new ssm.StringParameter(this, 'DbPassword', {
      parameterName: '/prod/database/password',
      stringValue: secrets.database.password,
      tier: ssm.ParameterTier.ADVANCED,
      description: 'Production database password',
    });

    const stripeKey = new ssm.StringParameter(this, 'StripeKey', {
      parameterName: '/prod/stripe/secret_key',
      stringValue: secrets.stripe.secret_key,
      tier: ssm.ParameterTier.ADVANCED,
    });

    // Create Lambda referencing SSM parameters
    const fn = new lambda.Function(this, 'ApiFunction', {
      runtime: lambda.Runtime.NODEJS_20_X,
      handler: 'index.handler',
      code: lambda.Code.fromAsset('src'),
      environment: {
        DB_HOST: secrets.database.host,
        DB_PASSWORD_PARAM: dbPassword.parameterName,
        STRIPE_KEY_PARAM: stripeKey.parameterName,
      },
    });

    // Grant read permissions
    dbPassword.grantRead(fn);
    stripeKey.grantRead(fn);
  }
}

Your Lambda code fetches parameters at runtime:

import { SSMClient, GetParameterCommand } from '@aws-sdk/client-ssm';

const ssm = new SSMClient({});

async function getSecret(paramName: string): Promise<string> {
  const response = await ssm.send(
    new GetParameterCommand({
      Name: paramName,
      WithDecryption: true,
    })
  );
  return response.Parameter!.Value!;
}

export async function handler() {
  const dbPassword = await getSecret(process.env.DB_PASSWORD_PARAM!);
  const stripeKey = await getSecret(process.env.STRIPE_KEY_PARAM!);
  // Use secrets...
}

Pattern 3: Using cdk-sops-secrets Construct

The cdk-sops-secrets npm package provides a higher-level construct:

npm install cdk-sops-secrets

import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import { SopsSecret } from 'cdk-sops-secrets';

export class LambdaStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Load SOPS secrets as CDK construct
    const secrets = new SopsSecret(this, 'Secrets', {
      sopsFilePath: 'secrets/prod.enc.yaml',
      kmsKey: 'alias/sops-production',
    });

    const fn = new lambda.Function(this, 'ApiFunction', {
      runtime: lambda.Runtime.NODEJS_20_X,
      handler: 'index.handler',
      code: lambda.Code.fromAsset('src'),
      environment: {
        DB_HOST: secrets.getString('database.host'),
        DB_PASSWORD: secrets.getString('database.password'),
        STRIPE_SECRET_KEY: secrets.getString('stripe.secret_key'),
      },
    });
  }
}

Pattern 4: Multi-Stack Secret Sharing

For complex applications, share SOPS secrets across multiple stacks:

// lib/secrets-stack.ts
import * as cdk from 'aws-cdk-lib';
import * as ssm from 'aws-cdk-lib/aws-ssm';
import { execSync } from 'child_process';
import * as yaml from 'js-yaml';

export class SecretsStack extends cdk.Stack {
  public readonly dbPasswordParam: ssm.IStringParameter;
  public readonly stripeKeyParam: ssm.IStringParameter;

  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const decrypted = execSync('sops --decrypt secrets/prod.enc.yaml', {
      encoding: 'utf-8'
    });
    const secrets = yaml.load(decrypted) as any;

    this.dbPasswordParam = new ssm.StringParameter(this, 'DbPassword', {
      parameterName: '/prod/database/password',
      stringValue: secrets.database.password,
    });

    this.stripeKeyParam = new ssm.StringParameter(this, 'StripeKey', {
      parameterName: '/prod/stripe/secret_key',
      stringValue: secrets.stripe.secret_key,
    });
  }
}

// lib/api-stack.ts
import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';

interface ApiStackProps extends cdk.StackProps {
  dbPasswordParam: cdk.aws_ssm.IStringParameter;
  stripeKeyParam: cdk.aws_ssm.IStringParameter;
}

export class ApiStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props: ApiStackProps) {
    super(scope, id, props);

    const fn = new lambda.Function(this, 'ApiFunction', {
      runtime: lambda.Runtime.NODEJS_20_X,
      handler: 'index.handler',
      code: lambda.Code.fromAsset('src'),
      environment: {
        DB_PASSWORD_PARAM: props.dbPasswordParam.parameterName,
        STRIPE_KEY_PARAM: props.stripeKeyParam.parameterName,
      },
    });

    props.dbPasswordParam.grantRead(fn);
    props.stripeKeyParam.grantRead(fn);
  }
}

// bin/app.ts
import * as cdk from 'aws-cdk-lib';
import { SecretsStack } from '../lib/secrets-stack';
import { ApiStack } from '../lib/api-stack';

const app = new cdk.App();

const secretsStack = new SecretsStack(app, 'SecretsStack');
new ApiStack(app, 'ApiStack', {
  dbPasswordParam: secretsStack.dbPasswordParam,
  stripeKeyParam: secretsStack.stripeKeyParam,
});

Synthesize and deploy:

# CDK decrypts SOPS during synthesis
cdk synth
cdk deploy --all

AWS SAM Integration (Brief)

For teams using AWS SAM, decrypt SOPS files during deployment and populate Secrets Manager:

# Decrypt and populate SSM/Secrets Manager
sops exec-file secrets/prod.enc.yaml 'aws secretsmanager create-secret \
  --name prod/database \
  --secret-string file://{}'

# Then deploy SAM
sam deploy --stack-name my-api --capabilities CAPABILITY_IAM

Local Development Workflow

For local testing with CDK, decrypt secrets temporarily:

# Decrypt for local development
sops --decrypt secrets/dev.enc.yaml > .env.local

# Run locally with decrypted secrets
npm run dev

# Clean up
rm .env.local

Or use SOPS exec mode to run commands with decrypted environment:

sops exec-env secrets/dev.enc.yaml 'npm run dev'

For CDK synthesis locally, SOPS decrypts automatically:

# CDK synth with SOPS decryption
cdk synth

# Deploy specific stack
cdk deploy ApiStack

SSM Parameter Store vs SOPS Comparison

Use SOPS when:

• Secrets change with code deployments
• You want Git-based audit trails
• Secrets are static (API keys, OAuth credentials)
• Team collaboration on secrets is important
• Cost optimization is priority

Use SSM Parameter Store when:

• Secrets rotate independently of deployments
• Multiple services share the same secrets
• You need AWS-native secret rotation
• Runtime secret updates without redeployment
• Cross-region secret replication needed

Hybrid approach:

// CDK: Some secrets from SOPS, others from SSM
const sopsSecrets = loadSopsSecrets('secrets/prod.enc.yaml');

new lambda.Function(this, 'ApiFunction', {
  environment: {
    // Static secrets from SOPS
    STRIPE_PUBLIC_KEY: sopsSecrets.stripe.public_key,
    OAUTH_CLIENT_ID: sopsSecrets.oauth.client_id,

    // Dynamic secrets from SSM
    DB_PASSWORD: cdk.aws_ssm.StringParameter.valueForStringParameter(
      this, '/prod/database/password'
    ),
  },
});

Terraform Integration

The Terraform SOPS provider enables reading encrypted variable files while keeping your state file clean.

Configure the provider:

terraform {
  required_providers {
    sops = {
      source  = "carlpett/sops"
      version = "~> 1.3"
    }
  }
}

provider "sops" {}

Create an encrypted variables file:

# secrets.enc.yaml
database:
  username: postgres_admin
  password: super_secret_password
  host: prod-db.example.com
  port: 5432

aws:
  access_key: AKIAIOSFODNN7EXAMPLE
  secret_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Encrypt it:

sops --encrypt secrets.yaml > secrets.enc.yaml
git add secrets.enc.yaml

Reference in Terraform:

data "sops_file" "secrets" {
  source_file = "secrets.enc.yaml"
}

resource "aws_db_instance" "main" {
  identifier  = "production-db"
  engine  = "postgres"
  instance_class = "db.t3.medium"

  username = data.sops_file.secrets.data["database.username"]
  password = data.sops_file.secrets.data["database.password"]

  lifecycle {
    ignore_changes = [password]
  }
}

output "database_endpoint" {
  value  = aws_db_instance.main.endpoint
  sensitive = true
}

The password never appears in your Terraform state file in plaintext because we use ignore_changes. For initial creation, SOPS decrypts the value. For subsequent applies, Terraform ignores password changes.

A better pattern is using SOPS to populate AWS Secrets Manager, then referencing the secret ARN:

resource "aws_secretsmanager_secret" "db_password" {
  name = "prod/database/password"
}

resource "aws_secretsmanager_secret_version" "db_password" {
  secret_id  = aws_secretsmanager_secret.db_password.id
  secret_string = data.sops_file.secrets.data["database.password"]
}

resource "aws_ecs_task_definition" "app" {
  container_definitions = jsonencode([{
    secrets = [{
      name  = "DB_PASSWORD"
      valueFrom = aws_secretsmanager_secret.db_password.arn
    }]
  }])
}

Now your application runtime fetches secrets from Secrets Manager, but the initial secret values are version-controlled with SOPS.

CI/CD Integration Patterns for AWS CDK

GitHub Actions with CDK

The recommended pattern for CDK deployments with SOPS. CDK automatically decrypts during synthesis:

name: Deploy CDK Lambda with SOPS
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v5
        with:
          role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsCDK
          aws-region: us-east-1

      - name: Install SOPS
        run: |
          wget -q https://github.com/getsops/sops/releases/download/v3.11.0/sops-v3.11.0.linux.amd64
          chmod +x sops-v3.11.0.linux.amd64
          sudo mv sops-v3.11.0.linux.amd64 /usr/local/bin/sops

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: CDK Synth (SOPS decrypts during synthesis)
        run: npx cdk synth

      - name: CDK Deploy
        run: npx cdk deploy --all --require-approval never

CDK code decrypts SOPS during synthesis, so no explicit decrypt step needed in CI/CD.

The IAM role needs CDK deployment permissions plus KMS decrypt:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt",
        "kms:DescribeKey"
      ],
      "Resource": "arn:aws:kms:us-east-1:123456789012:key/abc-123"
    },
    {
      "Effect": "Allow",
      "Action": [
        "cloudformation:*",
        "lambda:*",
        "iam:*",
        "s3:*",
        "ssm:*"
      ],
      "Resource": "*"
    }
  ]
}

Multi-Environment CDK Deployment

Deploy to different environments with environment-specific secrets:

name: Multi-Environment CDK Deploy
on:
  push:
    branches: [main, develop]

jobs:
  deploy-dev:
    if: github.ref == 'refs/heads/develop'
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v5
        with:
          role-to-assume: arn:aws:iam::111111111111:role/GitHubActionsCDK
          aws-region: us-east-1

      - name: Install SOPS
        run: |
          wget -q https://github.com/getsops/sops/releases/download/v3.11.0/sops-v3.11.0.linux.amd64
          chmod +x sops-v3.11.0.linux.amd64
          sudo mv sops-v3.11.0.linux.amd64 /usr/local/bin/sops

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: CDK Deploy Dev
        run: npx cdk deploy DevStack --require-approval never

  deploy-prod:
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment: production
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v5
        with:
          role-to-assume: arn:aws:iam::222222222222:role/GitHubActionsCDK
          aws-region: us-east-1

      - name: Install SOPS
        run: |
          wget -q https://github.com/getsops/sops/releases/download/v3.11.0/sops-v3.11.0.linux.amd64
          chmod +x sops-v3.11.0.linux.amd64
          sudo mv sops-v3.11.0.linux.amd64 /usr/local/bin/sops

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: CDK Deploy Prod
        run: npx cdk deploy ProdStack --require-approval never

GitHub Actions with AWS SAM (Brief)

For teams using AWS SAM:

name: Deploy SAM with SOPS
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v5
        with:
          role-to-assume: arn:aws:iam::123456789012:role/GitHubActionsSAM
          aws-region: us-east-1

      - name: Install SOPS
        run: |
          wget -q https://github.com/getsops/sops/releases/download/v3.11.0/sops-v3.11.0.linux.amd64
          chmod +x sops-v3.11.0.linux.amd64
          sudo mv sops-v3.11.0.linux.amd64 /usr/local/bin/sops

      - name: Setup SAM CLI
        uses: aws-actions/setup-sam@v2

      - name: Decrypt and populate Secrets Manager
        run: |
          sops exec-file secrets/prod.enc.yaml \
            'aws secretsmanager put-secret-value \
              --secret-id prod/app-secrets \
              --secret-string file://{}'

      - name: SAM Build & Deploy
        run: |
          sam build
          sam deploy --stack-name my-lambda-app --no-confirm-changeset

GitLab CI

GitLab CI uses similar patterns:

variables:
  SOPS_VERSION: "3.11.0"

stages:
  - decrypt
  - deploy

decrypt-secrets:
  stage: decrypt
  image: alpine:latest
  before_script:
    - apk add --no-cache wget
    - wget -q https://github.com/getsops/sops/releases/download/v${SOPS_VERSION}/sops-v${SOPS_VERSION}.linux.amd64
    - chmod +x sops-v${SOPS_VERSION}.linux.amd64
    - mv sops-v${SOPS_VERSION}.linux.amd64 /usr/local/bin/sops
  script:
    - mkdir -p ~/.config/sops/age
    - echo "$SOPS_AGE_KEY" > ~/.config/sops/age/keys.txt
    - chmod 600 ~/.config/sops/age/keys.txt
    - sops --decrypt secrets.enc.yaml > secrets.yaml
  artifacts:
    paths:
      - secrets.yaml
    expire_in: 10 minutes

The decrypted secrets artifact is available to subsequent stages but expires after 10 minutes.

Developer Experience and IDE Integration

Editing encrypted files manually would be painful. SOPS provides an edit mode that handles encryption transparently.

Set your editor:

export EDITOR="code --wait"  # VS Code
# or
export EDITOR="vim"  # Vim

Edit an encrypted file:

sops secrets.enc.yaml

SOPS decrypts the file, opens it in your editor, waits for you to save and close, then re-encrypts with updated values. You never see the encrypted content during editing.

For VS Code, install the SOPS extension:

// .vscode/settings.json
{
  "sops.enable": true,
  "sops.defaults": {
    "age": "age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p"
  },
  "files.associations": {
    "*.enc.yaml": "yaml",
    "*.enc.json": "json"
  }
}

The extension automatically decrypts files when you open them in VS Code and re-encrypts on save.

For meaningful Git diffs, configure a custom differ:

# .gitattributes
*.enc.yaml diff=sopsdiffer
*.enc.json diff=sopsdiffer

# .git/config or ~/.gitconfig
[diff "sopsdiffer"]
  textconv = sops --decrypt

Now git diff secrets.enc.yaml shows the actual value changes, not encrypted blob differences.

Pre-commit Hooks and Validation

Prevent committing decrypted secrets with pre-commit hooks:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/yuvipanda/pre-commit-hook-ensure-sops
    rev: v1.1
    hooks:
      - id: sops-encryption
        files: (secrets|prod).*\.(yaml|json)$

This hook verifies that files matching the pattern are encrypted before allowing the commit.

Add custom validation:

# .git/hooks/pre-commit
#!/bin/bash

# Check for unencrypted sensitive patterns
FORBIDDEN_PATTERNS="password|api_key|secret_token"

for file in $(git diff --cached --name-only); do
  if [[ $file =~ \.(yaml|json)$ ]] && [[ ! $file =~ \.enc\. ]]; then
    if grep -qiE "$FORBIDDEN_PATTERNS" "$file"; then
      echo "ERROR: Possible unencrypted secret in $file"
      echo "Did you mean to commit ${file%.yaml}.enc.yaml?"
      exit 1
    fi
  fi
done

# Verify encrypted files have SOPS metadata
for file in $(git diff --cached --name-only | grep '\.enc\.'); do
  if ! grep -q "^sops:" "$file"; then
    echo "ERROR: $file missing SOPS metadata"
    exit 1
  fi
done

The hook prevents both accidentally committing plaintext secrets and committing files that claim to be encrypted but aren’t.

Key Rotation Strategies

Age keys should rotate every 90 days. Automating this process prevents it from being forgotten.

Generate a new age key:

age-keygen -o new-key.txt
OLD_KEY=$(grep "public key:" old-key.txt | cut -d' ' -f3)
NEW_KEY=$(grep "public key:" new-key.txt | cut -d' ' -f3)

Add the new key to all encrypted files:

find . -name "*.enc.yaml" -type f | while read file; do
  sops --add-age "$NEW_KEY" "$file"
done

Rotate the data keys (generates new random keys):

find . -name "*.enc.yaml" -type f | while read file; do
  sops --rotate --in-place "$file"
done

Remove the old key:

find . -name "*.enc.yaml" -type f | while read file; do
  sops --rm-age "$OLD_KEY" "$file"
done

Update .sops.yaml:

sed -i "s/$OLD_KEY/$NEW_KEY/g" .sops.yaml

Commit and distribute the new private key to your team through a secure channel (password manager, encrypted email, secure messaging).

For KMS keys, the process is similar but involves creating a new KMS key, adding it to files, rotating, and removing the old KMS key ARN.

Multi-Team Access with Key Groups

Production environments often require multiple teams to access secrets. SOPS supports this through key groups and Shamir’s Secret Sharing.

# .sops.yaml
keys:
  platform: &platform
    - &platform_admin age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
    - &platform_sre age1cy0su9fwf8gzkdqh3r4r6xgc92fp8jqrjp4fvd4ak6vd3mc0jjpqnhymkw

  security: &security
    - &sec_team age1yx3z8r0hnzjy9wh6fq5gldq3p7hxg6nfkz5vgqcdqhsj8tqxj8xq8w6qur

creation_rules:
  - path_regex: prod/.*\.enc\.yaml$
    key_groups:
      - age:
          - *platform_admin
          - *platform_sre
      - age:
          - *sec_team
    shamir_threshold: 2

With shamir_threshold: 2, decryption requires keys from 2 of the 3 groups. This implements separation of duties. A platform engineer can’t decrypt production secrets alone. Neither can the security team. But any two groups together can decrypt.

The data key is split into fragments using Shamir’s Secret Sharing. Fragment 1 is encrypted for the platform team, fragment 2 for the security team, and fragment 3 for backup. Any 2 fragments can reconstruct the complete data key.

Cost Comparison and Trade-offs

For a scenario with 100 secrets:

SOPS with AWS KMS:

• KMS keys: 3 × $1/month =$3
• API calls: ~1,000 decryptions/month = $0.03
• Git storage: $0 (existing repository)
• Total: $3.03/month

AWS Secrets Manager:

• Secrets: 100 × $0.40/month =$40
• API calls: 10,000/month × $0.05/10k =$0.05
• Total: $40.05/month

Savings: $37/month (92% reduction)

But cost isn’t the only consideration. Secrets Manager provides automated rotation, while SOPS requires scripting. Secrets Manager has built-in audit logs through CloudTrail. SOPS relies on Git history.

SOPS wins for static secrets (API keys, OAuth credentials, database connection strings that change infrequently). Secrets Manager wins for dynamic secrets (database passwords that rotate weekly, service credentials with automated renewal).

A hybrid approach works well:

• Development and staging: SOPS with age keys
• Production static secrets: SOPS with KMS
• Production dynamic secrets: AWS Secrets Manager
• Database root passwords: Secrets Manager
• Third-party API keys: SOPS

Common Pitfalls and Solutions

Pitfall: Committing Decrypted Files

Add to .gitignore:

secrets.yaml
config/production.yaml
*.decrypted.yaml

# Allow encrypted files
!*.enc.yaml

Pitfall: Losing Age Private Keys

Store backups in multiple locations:

• Password manager (1Password, LastPass)
• Encrypted USB drive in physical safe
• Emergency recovery key stored offline

Create an emergency key and add it to all production secrets:

age-keygen -o emergency-key.txt
AGE_PUBLIC_KEY=$(grep "public key:" emergency-key.txt | cut -d' ' -f3)

find prod/ -name "*.enc.yaml" | while read file; do
  sops --add-age "$AGE_PUBLIC_KEY" "$file"
done

Pitfall: KMS Permission Issues

The error “AccessDeniedException” when decrypting usually means IAM permissions are wrong. Verify:

aws sts get-caller-identity  # Confirm assumed role
aws kms describe-key --key-id $KMS_KEY_ID  # Test KMS access
sops --decrypt --verbose secrets.enc.yaml  # See detailed error

Ensure your IAM role has both kms:Decrypt and kms:DescribeKey permissions for the KMS key.

Pitfall: Git Merge Conflicts

When two developers edit the same encrypted file simultaneously, Git creates a merge conflict with encrypted blobs.

Note: For normal editing, sops secrets.enc.yaml decrypts the file, opens it in your editor, and automatically re-encrypts when you save and exit. But for merge conflicts, you need to compare two different versions, so the manual decrypt → merge → re-encrypt workflow is required.

Resolving requires:

# Decrypt both versions
sops --decrypt secrets.enc.yaml > mine.yaml
git show origin/main:secrets.enc.yaml | sops --decrypt /dev/stdin > theirs.yaml

# Merge manually
vimdiff mine.yaml theirs.yaml

# Save merged version
mv merged.yaml secrets.yaml

# Re-encrypt
sops --encrypt secrets.yaml > secrets.enc.yaml
git add secrets.enc.yaml

Better: communicate when editing shared secrets, or split large files into smaller domain-specific files to reduce collision probability.

Key Takeaways

SOPS enables GitOps workflows for serverless without external secret dependencies. Secrets are versioned with Lambda code, deployed together, and rolled back together. Your Git history becomes your audit trail.

Age encryption provides a modern, simple alternative to PGP. The keys are short enough to share in chat. The tooling is minimal. The onboarding time is under 30 minutes.

The .sops.yaml configuration file eliminates manual key management. Path-based rules automatically select the right keys for each environment. Developers encrypt files without knowing KMS ARNs or remembering which keys to use.

For Lambda deployments, SOPS decrypts at build/deploy time, not at runtime. This eliminates cold start overhead from fetching secrets. Environment variables are baked into function configuration during deployment.

AWS CDK provides the most elegant SOPS integration. CDK decrypts secrets during synthesis and supports multiple patterns: direct environment variable injection, SSM Parameter Store population, the cdk-sops-secrets construct, and multi-stack secret sharing. This keeps infrastructure code clean while maintaining GitOps practices.

AWS SAM also integrates well for teams preferring CloudFormation templates. SAM deployments can populate Secrets Manager from SOPS files, combining version-controlled secrets with AWS-native rotation.

For production environments, combining KMS with age provides both centralized management and emergency recovery. KMS handles the primary encryption with IAM-based access control. Age provides a backup decryption path if KMS is unavailable.

The cost savings compared to AWS Secrets Manager are significant for static secrets. SOPS works best for configuration that changes with CDK deploys (API keys, OAuth credentials). Use SSM Parameter Store or Secrets Manager for dynamic secrets that rotate independently (database passwords).

A hybrid approach maximizes value: SOPS for static secrets, SSM for dynamic secrets. This balances cost optimization with operational flexibility.

Pre-commit hooks and validation are mandatory, not optional. Without automated checks, someone will eventually commit a decrypted file. Set up guardrails before your team starts using SOPS in production.