2025-11-18
Breaking Through CloudFormation's 500 Resource Barrier: Practical Strategies for Large-Scale Infrastructure
Exploring proven strategies to overcome CloudFormation's 500 resource limit using nested stacks, cross-stack references, SSM Parameter Store, and microstack architecture with real TypeScript CDK examples and decision frameworks.
Abstract
AWS CloudFormation’s 500 resource limit per stack is a hard constraint that teams frequently encounter when building production-grade infrastructure. Working with this limit has taught me that the choice between nested stacks, cross-stack references, SSM Parameter Store, and microstack architecture depends on operational preferences, deployment patterns, and team structure. This post explores five strategies with complete TypeScript CDK examples, decision frameworks, and lessons learned from refactoring infrastructure deployments that exceeded this limit.
Understanding the 500 Resource Limit
CloudFormation restricts each stack to a maximum of 500 resources; a hard limit that can’t be increased through service quotas. This constraint exists due to CloudFormation’s internal processing requirements: dependency graph complexity, rollback operation management, and state synchronization overhead all increase exponentially with resource count.
When Teams Hit This Limit
Serverless Microservices: A single Lambda function creates 8-12 CloudFormation resources:
// Single Lambda function creates multiple resources
const userHandler = new NodejsFunction(this, 'UserHandler', {
entry: 'src/handlers/user.ts'
});
// Creates:
// - AWS::Lambda::Function (1)
// - AWS::IAM::Role (1)
// - AWS::IAM::Policy (1-2)
// - AWS::Logs::LogGroup (1)
// - AWS::Lambda::Version (1)
// If with DLQ: AWS::SQS::Queue (1)
// If with alarms: AWS::CloudWatch::Alarm (2-4)
// Total: 8-12 resources per Lambda
// 60 Lambda functions = 480-720 resources just for compute layer
// Add DynamoDB tables, API Gateway, SQS queues, EventBridge rules = limit exceeded
Production vs Development Disparity: Development environments with 200 resources work fine, but production adds redundancy, monitoring, and multi-AZ configurations:
// Development: 178 resources
const devStack = {
lambdas: 20, // 160 resources
tables: 5, // 5 resources
queues: 3, // 3 resources
apis: 2, // 10 resources
total: 178
};
// Production: 505 resources (EXCEEDS LIMIT)
const prodStack = {
lambdas: 20, // 160 resources
tables: 5, // 5 resources
queues: 3, // 3 resources
apis: 2, // 10 resources
alarms: 100, // 100 resources (5 per Lambda)
dashboards: 5, // 5 resources
backupPlans: 8, // 16 resources
kmsKeys: 3, // 6 resources
multiAzResources: 40, // HA redundancy
total: 505 // LIMIT EXCEEDED
};
Tracking Resource Count
Monitor your resource count proactively before hitting the limit:
# CDK - Count resources before deployment
cdk synth -j | jq '.Resources | length'
# For specific stack in multi-stack app
cdk synth YourStackName -j | jq '.Resources | length'
# CLI - Count existing stack resources
aws cloudformation describe-stack-resources --stack-name MyStack \
--query "StackResources[].ResourceType" --output text | \
tr "\t" "\n" | sort | uniq -c | sort -nr
# Example output:
# 142 AWS::Lambda::Function
# 85 AWS::IAM::Role
# 78 AWS::Logs::LogGroup
# 42 AWS::CloudWatch::Alarm
# 28 AWS::DynamoDB::Table
# 15 AWS::SQS::Queue
# ---
# 390 Total
Strategy 0: Resource Consolidation - Reduce Before Splitting
Before splitting stacks, consolidate resources to reduce the total count. This should be your first step; splitting stacks adds operational complexity, so avoid it if consolidation gets you under the limit.
When to Use Consolidation
- As a first step before considering stack splitting
- When you have many similar resources that could be shared
- Before hitting the 500 resource limit (proactive optimization)
- To reduce operational overhead and costs
Pattern 1: Shared IAM Roles Instead of Per-Function Roles
// BEFORE: Each Lambda gets own role
// 10 Lambdas = 10 functions + 10 roles + 10+ policies = 30+ resources
const userHandler = new NodejsFunction(this, 'UserHandler', {
entry: 'src/handlers/user.ts',
// CDK creates dedicated role automatically
});
const orderHandler = new NodejsFunction(this, 'OrderHandler', {
entry: 'src/handlers/order.ts',
// Another dedicated role created
});
// AFTER: Shared execution role
// 10 Lambdas = 10 functions + 1 role + 1 policy = 12 resources
// Savings: 18 resources (60% reduction)
const sharedLambdaRole = new iam.Role(this, 'SharedLambdaExecutionRole', {
assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'),
managedPolicies: [
iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaVPCAccessExecutionRole'),
iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'),
],
});
// Add permissions for all tables/resources at once
sharedLambdaRole.addToPolicy(new iam.PolicyStatement({
actions: [
'dynamodb:GetItem',
'dynamodb:PutItem',
'dynamodb:UpdateItem',
'dynamodb:DeleteItem',
'dynamodb:Query',
'dynamodb:Scan',
],
resources: ['arn:aws:dynamodb:*:*:table/*'],
}));
sharedLambdaRole.addToPolicy(new iam.PolicyStatement({
actions: ['sqs:SendMessage', 'sqs:ReceiveMessage', 'sqs:DeleteMessage'],
resources: ['arn:aws:sqs:*:*:*'],
}));
// Reuse role for all functions
const userHandler = new NodejsFunction(this, 'UserHandler', {
entry: 'src/handlers/user.ts',
role: sharedLambdaRole,
});
const orderHandler = new NodejsFunction(this, 'OrderHandler', {
entry: 'src/handlers/order.ts',
role: sharedLambdaRole,
});
Pattern 2: Shared Security Groups
// BEFORE: Each Lambda in VPC gets own security group
// 20 Lambdas = 20 security groups
const userHandlerSG = new ec2.SecurityGroup(this, 'UserHandlerSG', {
vpc,
description: 'Security group for user handler',
});
// AFTER: Shared security group for all Lambda functions
// 20 Lambdas = 1 security group
// Savings: 19 resources
const lambdaSecurityGroup = new ec2.SecurityGroup(this, 'LambdaSecurityGroup', {
vpc,
description: 'Shared security group for all Lambda functions',
allowAllOutbound: true,
});
lambdaSecurityGroup.addIngressRule(
albSecurityGroup,
ec2.Port.tcp(443),
'Allow HTTPS from ALB'
);
const lambdaDefaults = {
vpc,
securityGroups: [lambdaSecurityGroup],
};
Pattern 3: Aggregate CloudWatch Alarms
// BEFORE: Individual alarm per Lambda
// 20 Lambdas × 3 alarms (errors, duration, throttles) = 60 alarms
// AFTER: Composite alarms with metric math
// 20 Lambdas = 1 aggregate alarm
// Savings: 19 resources (for error alarms)
const allLambdaErrors = new cloudwatch.MathExpression({
expression: 'SUM([m1, m2, m3, m4, m5])',
usingMetrics: {
m1: userHandler.metricErrors(),
m2: orderHandler.metricErrors(),
m3: paymentHandler.metricErrors(),
// ... up to 10 metrics per expression
},
});
const aggregatedAlarm = new cloudwatch.Alarm(this, 'AllLambdaErrors', {
metric: allLambdaErrors,
threshold: 50,
evaluationPeriods: 2,
alarmName: 'aggregate-lambda-errors',
alarmDescription: 'Total errors across all Lambda functions',
});
// Trade-off: Less granular alerting, but fewer resources
Consolidation Impact Example
Original Infrastructure:
- 50 Lambda functions: 50 resources
- 50 IAM roles: 50 resources
- 50 IAM policies: 50 resources
- 50 Log groups: 50 resources (auto-created)
- 50 Security groups: 50 resources
- 150 CloudWatch alarms (3 per Lambda): 150 resources
Total: 400 resources
After Consolidation:
- 50 Lambda functions: 50 resources
- 1 shared IAM role: 1 resource
- 1 shared IAM policy: 1 resource
- 50 Log groups: 50 resources (can't consolidate for Lambda)
- 1 shared Security group: 1 resource
- 10 aggregate CloudWatch alarms: 10 resources
Total: 113 resources
Savings: 287 resources (72% reduction!)
Trade-offs of Consolidation
Advantages:
- Significant resource count reduction (typically 50-70% possible)
- Simpler IAM management (fewer roles to audit)
- Faster deployments (fewer resources to create/update)
- Reduced CloudFormation template size
Disadvantages:
- Security: Shared roles have broader permissions (least privilege violations)
- Blast radius: Role change affects all resources using it
- Debugging: Harder to trace issues to specific functions
- Compliance: May violate separation of concerns requirements
- Rollback: Can’t independently rollback permissions for one service
Decision Framework for Consolidation
// HIGH CONSOLIDATION: Development/staging environments
const devEnvironment = {
sharedRoles: true, // Reduce costs, faster deployments
sharedSecurityGroups: true, // Simpler management
aggregateAlarms: true, // Less critical monitoring
};
// MODERATE CONSOLIDATION: Production environments
const prodEnvironment = {
sharedRoles: 'same-service', // Only within service boundary
sharedSecurityGroups: true, // Network isolation still maintained
aggregateAlarms: false, // Granular alerting critical
};
// NO CONSOLIDATION: Highly regulated environments
const regulatedEnvironment = {
sharedRoles: false, // Audit trail per function
sharedSecurityGroups: false, // Network segmentation
aggregateAlarms: false, // Individual compliance monitoring
};
Best Practice: Start with consolidation before stack splitting. If you can reduce from 600 to 400 resources through consolidation, you may not need to split stacks at all.
Strategy 1: Nested Stacks - The Official Solution
Nested stacks allow a parent stack to contain multiple child stacks, each with up to 500 resources. The nested stack counts as a single resource in the parent stack.
When to Use
- Infrastructure logically divides into distinct domains (networking, compute, storage, monitoring)
- All resources deploy and rollback together as a unit
- Single deployment operation is preferred
- Team manages infrastructure from centralized control point
CDK Implementation
// lib/stacks/nested/networking-stack.ts
import { NestedStack, NestedStackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
export class NetworkingNestedStack extends NestedStack {
public readonly vpc: ec2.IVpc;
constructor(scope: Construct, id: string, props?: NestedStackProps) {
super(scope, id, props);
this.vpc = new ec2.Vpc(this, 'ApplicationVpc', {
ipAddresses: ec2.IpAddresses.cidr('10.0.0.0/16'),
maxAzs: 3,
natGateways: 3,
subnetConfiguration: [
{
name: 'Public',
subnetType: ec2.SubnetType.PUBLIC,
cidrMask: 24,
},
{
name: 'Private',
subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
cidrMask: 24,
},
{
name: 'Isolated',
subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
cidrMask: 24,
},
],
});
new ec2.FlowLog(this, 'FlowLog', {
resourceType: ec2.FlowLogResourceType.fromVpc(this.vpc),
destination: ec2.FlowLogDestination.toCloudWatchLogs(),
});
}
}
// lib/stacks/nested/storage-stack.ts
import { NestedStack, NestedStackProps, RemovalPolicy } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
export interface StorageNestedStackProps extends NestedStackProps {
readonly environment: string;
}
export class StorageNestedStack extends NestedStack {
public readonly userTable: dynamodb.ITable;
public readonly orderTable: dynamodb.ITable;
constructor(scope: Construct, id: string, props: StorageNestedStackProps) {
super(scope, id, props);
const isProd = props.environment === 'prod';
this.userTable = new dynamodb.Table(this, 'UserTable', {
partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING },
billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
encryption: dynamodb.TableEncryption.AWS_MANAGED,
pointInTimeRecovery: isProd,
removalPolicy: isProd ? RemovalPolicy.RETAIN : RemovalPolicy.DESTROY,
stream: dynamodb.StreamViewType.NEW_AND_OLD_IMAGES,
});
this.orderTable = new dynamodb.Table(this, 'OrderTable', {
partitionKey: { name: 'orderId', type: dynamodb.AttributeType.STRING },
sortKey: { name: 'timestamp', type: dynamodb.AttributeType.NUMBER },
billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
encryption: dynamodb.TableEncryption.AWS_MANAGED,
pointInTimeRecovery: isProd,
removalPolicy: isProd ? RemovalPolicy.RETAIN : RemovalPolicy.DESTROY,
});
this.orderTable.addGlobalSecondaryIndex({
indexName: 'UserOrderIndex',
partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING },
sortKey: { name: 'timestamp', type: dynamodb.AttributeType.NUMBER },
});
}
}
// lib/stacks/nested/compute-stack.ts
import { NestedStack, NestedStackProps, Duration } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as nodejs from 'aws-cdk-lib/aws-lambda-nodejs';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
import * as logs from 'aws-cdk-lib/aws-logs';
export interface ComputeNestedStackProps extends NestedStackProps {
readonly vpc: ec2.IVpc;
readonly userTable: dynamodb.ITable;
readonly orderTable: dynamodb.ITable;
}
export class ComputeNestedStack extends NestedStack {
public readonly userHandler: nodejs.NodejsFunction;
public readonly orderHandler: nodejs.NodejsFunction;
constructor(scope: Construct, id: string, props: ComputeNestedStackProps) {
super(scope, id, props);
const lambdaDefaults = {
runtime: lambda.Runtime.NODEJS_22_X,
timeout: Duration.seconds(30),
memorySize: 1024,
tracing: lambda.Tracing.ACTIVE,
logRetention: logs.RetentionDays.ONE_WEEK,
vpc: props.vpc,
vpcSubnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS },
bundling: {
minify: true,
sourceMap: true,
externalModules: ['@aws-sdk/*'],
},
};
this.userHandler = new nodejs.NodejsFunction(this, 'UserHandler', {
...lambdaDefaults,
entry: 'src/handlers/user.ts',
environment: {
USER_TABLE_NAME: props.userTable.tableName,
},
});
props.userTable.grantReadWriteData(this.userHandler);
this.orderHandler = new nodejs.NodejsFunction(this, 'OrderHandler', {
...lambdaDefaults,
entry: 'src/handlers/order.ts',
environment: {
ORDER_TABLE_NAME: props.orderTable.tableName,
USER_TABLE_NAME: props.userTable.tableName,
},
});
props.orderTable.grantReadWriteData(this.orderHandler);
props.userTable.grantReadData(this.orderHandler);
}
}
// lib/stacks/parent-stack.ts
import { Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { NetworkingNestedStack } from './nested/networking-stack';
import { StorageNestedStack } from './nested/storage-stack';
import { ComputeNestedStack } from './nested/compute-stack';
export interface ParentStackProps extends StackProps {
readonly environment: string;
}
export class ParentStack extends Stack {
constructor(scope: Construct, id: string, props: ParentStackProps) {
super(scope, id, props);
const networkingStack = new NetworkingNestedStack(this, 'Networking');
const storageStack = new StorageNestedStack(this, 'Storage', {
environment: props.environment,
});
const computeStack = new ComputeNestedStack(this, 'Compute', {
vpc: networkingStack.vpc,
userTable: storageStack.userTable,
orderTable: storageStack.orderTable,
});
}
}
Deployment Process
# Deploy everything as single operation
cdk deploy ProductionStack
# CloudFormation creates:
# 1. ProductionStack (parent)
# 2. ProductionStack-Networking (nested)
# 3. ProductionStack-Storage (nested)
# 4. ProductionStack-Compute (nested)
# Rollback behavior:
# - If any nested stack fails, entire parent stack rolls back
# - All resources created/updated atomically
Advantages and Limitations
Advantages:
- Single deployment operation
- Atomic rollback - all or nothing
- Logical organization by domain
- Each nested stack has 500 resource budget
- Parent stack only counts nested stacks (3 resources in example)
Limitations:
-
Changesets Become Opaque: CloudFormation changeset shows only parent-level changes, not what changed inside nested stacks. Developers can’t see actual table/Lambda changes without diving into nested stacks.
-
Drift Detection Complexity: Requires checking each nested stack separately with custom scripts to aggregate results.
-
Nested Stack Update Failures Create Stuck States: If a nested stack update fails and gets stuck waiting for resource deletion, the entire parent stack waits, blocking all deployments.
-
2500 Resource Operation Limit: Even with nested stacks, single deployment operation limited to 2500 resources total.
-
Not Independently Deployable: Cannot deploy individual nested stacks; must always deploy through parent.
Strategy 2: Cross-Stack References - Independent Deployment
Multiple independent stacks with explicit export/import of outputs allow different teams to manage different infrastructure components.
When to Use
- Teams want to deploy infrastructure components independently
- Different lifecycle for components (networking rarely changes, compute changes frequently)
- Multiple teams manage different parts of infrastructure
- Need to share resources across multiple consuming stacks
CDK Implementation
// lib/stacks/network-stack.ts
import { Stack, StackProps, CfnOutput } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
export class NetworkStack extends Stack {
public readonly vpc: ec2.IVpc;
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
this.vpc = new ec2.Vpc(this, 'AppVpc', {
ipAddresses: ec2.IpAddresses.cidr('10.0.0.0/16'),
maxAzs: 3,
natGateways: 3,
});
// Export VPC ID for cross-stack reference
new CfnOutput(this, 'VpcId', {
value: this.vpc.vpcId,
exportName: 'AppVpcId',
description: 'Application VPC ID',
});
new CfnOutput(this, 'PrivateSubnetIds', {
value: this.vpc.privateSubnets.map(s => s.subnetId).join(','),
exportName: 'AppVpcPrivateSubnetIds',
description: 'Private subnet IDs',
});
}
}
// lib/stacks/storage-stack.ts
import { Stack, StackProps, CfnOutput, RemovalPolicy } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
export class StorageStack extends Stack {
public readonly userTable: dynamodb.ITable;
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
this.userTable = new dynamodb.Table(this, 'UserTable', {
partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING },
billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
encryption: dynamodb.TableEncryption.AWS_MANAGED,
pointInTimeRecovery: true,
removalPolicy: RemovalPolicy.RETAIN,
});
new CfnOutput(this, 'UserTableName', {
value: this.userTable.tableName,
exportName: 'UserTableName',
description: 'User DynamoDB table name',
});
new CfnOutput(this, 'UserTableArn', {
value: this.userTable.tableArn,
exportName: 'UserTableArn',
description: 'User DynamoDB table ARN',
});
}
}
// lib/stacks/compute-stack.ts
import { Stack, StackProps, Fn, Duration } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as nodejs from 'aws-cdk-lib/aws-lambda-nodejs';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as iam from 'aws-cdk-lib/aws-iam';
export class ComputeStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
// Import VPC from NetworkStack using cross-stack reference
const vpcId = Fn.importValue('AppVpcId');
const vpc = ec2.Vpc.fromLookup(this, 'ImportedVpc', { vpcId });
const userTableName = Fn.importValue('UserTableName');
const userTableArn = Fn.importValue('UserTableArn');
const userHandler = new nodejs.NodejsFunction(this, 'UserHandler', {
runtime: lambda.Runtime.NODEJS_22_X,
entry: 'src/handlers/user.ts',
timeout: Duration.seconds(30),
vpc: vpc,
environment: {
USER_TABLE_NAME: userTableName,
},
});
userHandler.addToRolePolicy(new iam.PolicyStatement({
actions: [
'dynamodb:GetItem',
'dynamodb:PutItem',
'dynamodb:UpdateItem',
'dynamodb:DeleteItem',
'dynamodb:Query',
],
resources: [userTableArn],
}));
}
}
// bin/app.ts
import * as cdk from 'aws-cdk-lib';
import { NetworkStack } from '../lib/stacks/network-stack';
import { StorageStack } from '../lib/stacks/storage-stack';
import { ComputeStack } from '../lib/stacks/compute-stack';
const app = new cdk.App();
const env = {
account: process.env.CDK_DEFAULT_ACCOUNT,
region: process.env.CDK_DEFAULT_REGION,
};
const networkStack = new NetworkStack(app, 'NetworkStack', { env });
const storageStack = new StorageStack(app, 'StorageStack', { env });
const computeStack = new ComputeStack(app, 'ComputeStack', { env });
computeStack.addDependency(networkStack);
computeStack.addDependency(storageStack);
app.synth();
Deployment Process
# Deploy stacks independently
cdk deploy NetworkStack
cdk deploy StorageStack
cdk deploy ComputeStack
# Update compute stack without touching network/storage
cdk deploy ComputeStack
# List all stacks
cdk list
# Output:
# NetworkStack
# StorageStack
# ComputeStack
Critical Limitation - Export Update Lock
The most significant limitation: Cannot update or delete export while it’s imported by another stack.
# Attempt to update NetworkStack that changes VPC:
cdk deploy NetworkStack
# CloudFormation Error:
# Export AppVpcId cannot be updated as it is in use by ComputeStack
# Solution requires:
# 1. Delete ComputeStack (DOWNTIME!)
# 2. Update NetworkStack
# 3. Recreate ComputeStack
Trade-off Summary
- Pro: Independent deployment
- Pro: Team autonomy
- Con: Export changes require deleting consuming stacks
- Con: More complex dependency management
- Con: Harder to ensure atomic updates across related infrastructure
Strategy 3: SSM Parameter Store - Loose Coupling
Use AWS Systems Manager Parameter Store to share values between stacks, avoiding hard cross-stack references.
When to Use
- Need flexibility to update shared values without stack dependencies
- Want to decouple provider and consumer stacks
- Multiple stacks consume same values
- Cross-region deployments (parameters can be replicated)
CDK Implementation
// lib/stacks/network-stack-ssm.ts
import { Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as ssm from 'aws-cdk-lib/aws-ssm';
export class NetworkStackSSM extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
const vpc = new ec2.Vpc(this, 'AppVpc', {
ipAddresses: ec2.IpAddresses.cidr('10.0.0.0/16'),
maxAzs: 3,
});
// Store VPC ID in Parameter Store instead of exporting
new ssm.StringParameter(this, 'VpcIdParameter', {
parameterName: '/app/network/vpc-id',
stringValue: vpc.vpcId,
description: 'Application VPC ID',
tier: ssm.ParameterTier.STANDARD,
});
new ssm.StringParameter(this, 'PrivateSubnetIdsParameter', {
parameterName: '/app/network/private-subnet-ids',
stringValue: vpc.privateSubnets.map(s => s.subnetId).join(','),
description: 'Private subnet IDs',
});
}
}
// lib/stacks/storage-stack-ssm.ts
import { Stack, StackProps, RemovalPolicy } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
import * as ssm from 'aws-cdk-lib/aws-ssm';
export class StorageStackSSM extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
const userTable = new dynamodb.Table(this, 'UserTable', {
partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING },
billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
encryption: dynamodb.TableEncryption.AWS_MANAGED,
pointInTimeRecovery: true,
removalPolicy: RemovalPolicy.RETAIN,
});
new ssm.StringParameter(this, 'UserTableNameParameter', {
parameterName: '/app/storage/user-table-name',
stringValue: userTable.tableName,
description: 'User table name',
});
new ssm.StringParameter(this, 'UserTableArnParameter', {
parameterName: '/app/storage/user-table-arn',
stringValue: userTable.tableArn,
description: 'User table ARN',
});
}
}
// lib/stacks/compute-stack-ssm.ts
import { Stack, StackProps, Duration } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as nodejs from 'aws-cdk-lib/aws-lambda-nodejs';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as ssm from 'aws-cdk-lib/aws-ssm';
import * as iam from 'aws-cdk-lib/aws-iam';
export class ComputeStackSSM extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
// Method 1: Read parameter at synthesis time (valueFromLookup)
// Pro: Type-safe, early validation
// Con: Requires parameter to exist before synth, cached in cdk.context.json
const vpcId = ssm.StringParameter.valueFromLookup(this, '/app/network/vpc-id');
// Method 2: Read parameter at deployment time (valueForStringParameter)
// Pro: Always uses latest value, no caching
// Con: Value not known at synth time, less type-safe
const userTableName = ssm.StringParameter.valueForStringParameter(
this,
'/app/storage/user-table-name'
);
const userTableArn = ssm.StringParameter.valueForStringParameter(
this,
'/app/storage/user-table-arn'
);
const vpc = ec2.Vpc.fromLookup(this, 'ImportedVpc', { vpcId });
const userHandler = new nodejs.NodejsFunction(this, 'UserHandler', {
runtime: lambda.Runtime.NODEJS_22_X,
entry: 'src/handlers/user.ts',
timeout: Duration.seconds(30),
vpc: vpc,
environment: {
USER_TABLE_NAME: userTableName,
},
});
userHandler.addToRolePolicy(new iam.PolicyStatement({
actions: ['dynamodb:GetItem', 'dynamodb:PutItem', 'dynamodb:UpdateItem'],
resources: [userTableArn],
}));
userHandler.addToRolePolicy(new iam.PolicyStatement({
actions: ['ssm:GetParameter', 'ssm:GetParameters'],
resources: [`arn:aws:ssm:${this.region}:${this.account}:parameter/app/*`],
}));
}
}
Deployment Process
# Deploy in any order (though logical order recommended)
cdk deploy NetworkStackSSM
cdk deploy StorageStackSSM
cdk deploy ComputeStackSSM
# Update NetworkStack VPC without affecting ComputeStack
cdk deploy NetworkStackSSM
# Parameter value updated, no export lock issues
# ComputeStack can be redeployed later to pick up new VPC
Advantages and Trade-offs
Advantages:
- No cross-stack export locks
- Update provider stack without affecting consumers
- Multiple stacks can read same parameters
- Cross-region replication possible
- Can use versioned parameters for rollback
Trade-offs:
valueFromLookupcaches incdk.context.json- can become stalevalueForStringParameterresolves at deploy time - less type-safe- Runtime parameter reads add Lambda execution time
- Need IAM permissions for SSM read access
- Parameters must exist before deployment (or use default values)
Strategy 4: Multiple Independent Stacks - Microservices Pattern
Single CDK app creates multiple independent stacks, logically organized but with no coupling.
When to Use
- Microservices architecture - each service is independent stack
- Different deployment schedules for services
- Team ownership by service
- Each service under 500 resources
- Want mono-repo organization with deployment flexibility
CDK Implementation
// lib/constructs/service-stack.ts
import { Stack, StackProps, Duration, RemovalPolicy } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as nodejs from 'aws-cdk-lib/aws-lambda-nodejs';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
import * as apigateway from 'aws-cdk-lib/aws-apigateway';
import * as logs from 'aws-cdk-lib/aws-logs';
import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch';
export interface ServiceStackProps extends StackProps {
readonly serviceName: string;
readonly stage: string;
}
export class ServiceStack extends Stack {
public readonly api: apigateway.RestApi;
public readonly handler: nodejs.NodejsFunction;
public readonly table: dynamodb.Table;
constructor(scope: Construct, id: string, props: ServiceStackProps) {
super(scope, id, props);
const isProd = props.stage === 'prod';
this.table = new dynamodb.Table(this, 'Table', {
tableName: `${props.serviceName}-${props.stage}`,
partitionKey: { name: 'id', type: dynamodb.AttributeType.STRING },
billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
encryption: dynamodb.TableEncryption.AWS_MANAGED,
pointInTimeRecovery: isProd,
removalPolicy: isProd ? RemovalPolicy.RETAIN : RemovalPolicy.DESTROY,
});
this.handler = new nodejs.NodejsFunction(this, 'Handler', {
functionName: `${props.serviceName}-handler-${props.stage}`,
runtime: lambda.Runtime.NODEJS_22_X,
entry: `src/services/${props.serviceName}/handler.ts`,
timeout: Duration.seconds(30),
memorySize: 1024,
tracing: lambda.Tracing.ACTIVE,
logRetention: logs.RetentionDays.ONE_WEEK,
environment: {
TABLE_NAME: this.table.tableName,
SERVICE_NAME: props.serviceName,
STAGE: props.stage,
},
bundling: {
minify: true,
sourceMap: true,
externalModules: ['@aws-sdk/*'],
},
});
this.table.grantReadWriteData(this.handler);
this.api = new apigateway.RestApi(this, 'Api', {
restApiName: `${props.serviceName}-api-${props.stage}`,
deployOptions: {
stageName: props.stage,
tracingEnabled: true,
loggingLevel: apigateway.MethodLoggingLevel.INFO,
metricsEnabled: true,
},
});
const integration = new apigateway.LambdaIntegration(this.handler);
this.api.root.addMethod('ANY', integration);
const resource = this.api.root.addResource('{proxy+}');
resource.addMethod('ANY', integration);
new cloudwatch.Alarm(this, 'ErrorAlarm', {
metric: this.handler.metricErrors(),
threshold: 10,
evaluationPeriods: 2,
alarmName: `${props.serviceName}-errors-${props.stage}`,
});
}
}
// bin/app.ts
import * as cdk from 'aws-cdk-lib';
import { ServiceStack } from '../lib/constructs/service-stack';
const app = new cdk.App();
const stage = app.node.tryGetContext('stage') || 'dev';
const env = {
account: process.env.CDK_DEFAULT_ACCOUNT,
region: process.env.CDK_DEFAULT_REGION,
};
const services = [
'user-service',
'order-service',
'payment-service',
'inventory-service',
'notification-service',
];
services.forEach(serviceName => {
new ServiceStack(app, `${serviceName}-${stage}`, {
serviceName,
stage,
env,
stackName: `${serviceName}-${stage}`,
});
});
app.synth();
Deployment Options
# List all stacks
cdk list
# Output:
# user-service-prod
# order-service-prod
# payment-service-prod
# inventory-service-prod
# notification-service-prod
# Deploy all services
cdk deploy --all
# Deploy specific service
cdk deploy user-service-prod
# Deploy multiple specific services
cdk deploy user-service-prod order-service-prod
Trade-offs
Advantages:
- Complete deployment independence
- Each service team owns their stack
- Deploy frequently without affecting other services
- Scale development across teams
- Easy to add new services
- Clear service boundaries
Disadvantages:
- No shared infrastructure (VPC, networking duplicated if not using SSM/lookups)
- Need service discovery mechanism (SSM, EventBridge, service mesh)
- More stacks to manage (5 services = 5 stacks)
- Need orchestration for multi-service deployments
Decision Framework
Choose your strategy based on operational requirements and team structure:
Choose Nested Stacks When:
- Infrastructure deployed as single unit
- Atomic rollback important
- Logical domain separation (network/compute/storage)
- Single team manages all infrastructure
- Deployment frequency: Low to medium
Choose Cross-Stack References When:
- Different lifecycle for infrastructure layers
- Networking changes rarely, compute changes frequently
- Different teams own different layers
- Can tolerate export update complexity
- Deployment frequency: Medium
Choose SSM Parameter Store When:
- Need maximum deployment flexibility
- Update infrastructure without rigid dependencies
- Cross-region deployments
- Multiple consumers of same values
- Deployment frequency: High
Choose Multiple Independent Stacks When:
- Microservices architecture
- Team autonomy critical
- Services < 500 resources each
- Event-driven communication
- Deployment frequency: Very high (per service)
Common Pitfalls and Solutions
Pitfall 1: Not Monitoring Resource Count Proactively
Implement CI/CD check that fails build if stack exceeds threshold:
#!/bin/bash
# .github/workflows/cdk-check.sh
MAX_RESOURCES=450
for stack in $(cdk list); do
resource_count=$(cdk synth $stack -j | jq '.Resources | length')
echo "$stack: $resource_count resources"
if [ $resource_count -gt $MAX_RESOURCES ]; then
echo "ERROR: $stack exceeds $MAX_RESOURCES resources ($resource_count)"
exit 1
fi
done
GitHub Actions Integration:
# .github/workflows/cdk-validation.yml
name: CDK Validation
on: [push, pull_request]
jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
- run: npm ci
- run: npm run build
- run: bash .github/workflows/cdk-check.sh
Pitfall 2: Cross-Stack Export Lock During Emergency
Problem: Critical production issue requires networking change, but cross-stack export prevents update.
Solution: Use SSM Parameter Store for infrastructure likely to change:
// Use Cross-Stack Export: STABLE, rarely changing
// - AWS Account ID
// - Region
// - Root DNS zone ID
// Use SSM Parameter: MAY CHANGE without downtime requirement
// - VPC ID (might change due to networking redesign)
// - Subnet IDs (might change due to IP range expansion)
// - Database endpoints (might change due to migration)
Pitfall 3: Nested Stack Dependency Cycles
Keep nested stacks in clear hierarchy. Resources in child stack should never reference parent stack resources.
// Parent stack manages dependencies
class ParentStack extends Stack {
constructor(scope, id, props) {
super(scope, id, props);
const network = new NetworkStack(this, 'Network');
const compute = new ComputeStack(this, 'Compute', {
vpc: network.vpc, // One-way dependency
});
// Parent manages connections between nested stacks
compute.lambda.connections.allowFrom(network.vpc);
}
}
Pitfall 4: Not Testing Rollback Behavior
Test rollback in development by intentionally causing failures:
// Create intentional failure resource for testing
const testFailure = process.env.TEST_ROLLBACK === 'true';
if (testFailure) {
new lambda.Function(this, 'FailureTest', {
runtime: lambda.Runtime.NODEJS_22_X,
handler: 'index.handler',
code: lambda.Code.fromInline('INVALID CODE'), // Causes deployment failure
});
}
// Deploy with TEST_ROLLBACK=true to test rollback
// TEST_ROLLBACK=true cdk deploy
Key Takeaways
-
500 Resource Limit is Hard: No service quota increase available. Plan architecture accordingly from the start.
-
Start with Consolidation: Reduce resource count by typically 50-70% before splitting stacks. Shared IAM roles, security groups, and aggregate alarms significantly reduce resource counts.
-
Nested Stacks Trade Operational Complexity for Simplicity: Single deployment operation, but changesets become opaque and drift detection requires custom tooling.
-
Cross-Stack References Create Export Locks: Cannot update exported values without deleting consuming stacks. Reserve for truly stable resources.
-
SSM Parameter Store Provides Maximum Flexibility: Loose coupling allows independent deployment and updates. Best for values that may change.
-
Multiple Independent Stacks Best for Microservices: Each service under 500 resources, deployed independently. Requires event-driven communication patterns.
-
Monitor Resource Count Proactively: CI/CD checks should fail if approaching 450 resources. Don’t wait for production deployment failure.
-
Hybrid Approach is Most Common: Combine strategies based on infrastructure stability and change frequency. Stable foundations use cross-stack exports; variable applications use SSM.
-
Test Rollback Behavior: Intentionally cause failures in development to understand rollback behavior before production issues occur.
-
Choose Strategy Based on Deployment Frequency: Low frequency → Nested Stacks; Medium → Cross-Stack; High → SSM; Very High → Multiple Independent Stacks.
Working with CloudFormation’s resource limit has taught me that the right strategy depends on your team’s deployment patterns and operational preferences. Start with consolidation, monitor proactively, and choose the approach that matches your infrastructure stability and change frequency.
Related posts
A CDK guide for deploying a minimal Strands agent on AgentCore Runtime — parameterized stack, arm64 build, deploy and invoke, and the IAM and Marketplace prerequisites you need before the first call.
A comprehensive technical guide to choosing and implementing AWS edge computing solutions for global applications with practical examples and cost optimization strategies.
A comprehensive technical guide comparing AWS Secrets Manager and Systems Manager Parameter Store, demonstrating when to use each service with real-world implementation patterns.
A production-focused guide to implementing feature flags in distributed systems, comparing LaunchDarkly, Unleash, and AWS AppConfig with working examples for gradual rollouts, A/B testing, and managing technical debt.
A comprehensive guide to building scalable real-time APIs with AWS AppSync, covering JavaScript resolvers, subscription filtering, caching strategies, and infrastructure as code patterns.