AWS Lambda: Functions, Event Sources, Layers, and Serverless Patterns

Lambda's value proposition is narrow but real: you write a function, AWS runs it in response to events, and you pay only for the milliseconds of execution time. No instances, no clusters, no capacity planning. When the event rate is zero, cost is zero.

The trade-offs are real too: cold starts add latency on the first invocation after a function is idle, stateless execution constrains what you can build, and at high sustained throughput, Lambda's per-invocation pricing exceeds EC2 or EKS. Lambda is not a universal compute layer — it's the right tool for event-driven, bursty, or infrequent workloads.

Lambda Execution Model

Invocation Types

Lambda supports three invocation types:

Synchronous (request/response): the caller waits for the response. API Gateway → Lambda is synchronous. The caller receives the function's return value directly.

Asynchronous (event): Lambda queues the event and returns immediately to the caller. S3 event notifications, SNS, and EventBridge use asynchronous invocation. Lambda retries on failure (up to 2 retries by default) and can send failed events to a dead-letter queue.

Poll-based (event source mapping): Lambda polls the event source (SQS, Kinesis, DynamoDB Streams, MSK) and invokes the function in batches. Lambda manages the polling loop.

1# Invoke synchronously
2aws lambda invoke \
3  --function-name payments-processor \
4  --payload '{"orderId": "ord-123", "amount": 99.99}' \
5  --cli-binary-format raw-in-base64-out \
6  response.json
7
8# Invoke asynchronously (--invocation-type Event)
9aws lambda invoke \
10  --function-name payments-processor \
11  --invocation-type Event \
12  --payload '{"orderId": "ord-123"}' \
13  --cli-binary-format raw-in-base64-out \
14  /dev/null

Execution Environment and Cold Starts

Lambda runs functions in execution environments — lightweight VMs managed by AWS. On first invocation (or after a period of inactivity), Lambda initializes a new execution environment: downloads the code package, starts the runtime, and runs the initialization code outside the handler. This is the cold start.

Subsequent invocations reuse the warm execution environment, skipping initialization. The same environment can be reused for multiple sequential invocations, which is why you can cache database connections or SDK clients outside the handler.

1import boto3
2import psycopg2
3import os
4
5# Runs once per execution environment (on cold start)
6# Reused across warm invocations
7dynamodb = boto3.resource('dynamodb')
8table = dynamodb.Table(os.environ['TABLE_NAME'])
9
10# Connection pool — reused across invocations in the same environment
11db_conn = psycopg2.connect(os.environ['DATABASE_URL'])
12
13def handler(event, context):
14    # Handler runs on every invocation
15    # db_conn and table are already initialized
16    order_id = event['orderId']
17    result = table.get_item(Key={'orderId': order_id})
18    return result.get('Item')

Cold start latency varies by runtime and package size:

• Python, Node.js, Ruby: 100–500ms
• Java, C# (.NET): 500ms–2s (JVM/CLR initialization) — Java functions can use Lambda SnapStart (Java 11, 17, and 21 Corretto runtimes) to dramatically reduce cold starts (typically to under 300ms) by snapshotting the initialized state
• Container images: 1–10s depending on image size and layer caching

Reduce cold starts by:

• Keeping deployment packages small (Lambda layers for shared deps, tree-shaking for Node.js)
• Using Provisioned Concurrency to pre-initialize environments
• Choosing runtimes with fast initialization (Python, Node.js) for latency-sensitive functions

Memory and CPU

Lambda allocates CPU proportionally to memory. More memory = more CPU. There is no separate CPU setting.

1# Create function with 1024 MB memory (gets proportionally more CPU than 128 MB)
2aws lambda create-function \
3  --function-name payments-processor \
4  --runtime python3.12 \
5  --role arn:aws:iam::012345678901:role/LambdaExecutionRole \
6  --handler handler.handler \
7  --zip-file fileb://function.zip \
8  --memory-size 1024 \
9  --timeout 30 \
10  --environment Variables='{TABLE_NAME=payments-prod,DATABASE_URL=postgresql://...}'

For CPU-bound functions, increasing memory reduces execution time — sometimes enough to lower total cost despite the higher per-GB-second rate. The AWS Lambda Power Tuning tool automates finding the optimal memory setting by running the function at different memory sizes and comparing cost × performance.

Concurrency

Lambda scales by adding more execution environments. Each execution environment handles one invocation at a time. Concurrency = number of in-flight invocations.

1# Set reserved concurrency (limits maximum concurrent invocations for this function)
2aws lambda put-function-concurrency \
3  --function-name payments-processor \
4  --reserved-concurrent-executions 100
5
6# Set provisioned concurrency (pre-warms environments to eliminate cold starts)
7aws lambda put-provisioned-concurrency-config \
8  --function-name payments-processor \
9  --qualifier prod    # Must be a published version or alias
10  --provisioned-concurrent-executions 10

Reserved concurrency: caps the function's maximum concurrency. Useful for protecting downstream resources (databases, APIs) from Lambda scaling faster than they can handle.

Provisioned concurrency: keeps N execution environments initialized at all times, eliminating cold starts for those N concurrent invocations. Costs money whether or not the environments are used — approximately $0.000064646 per GB-second, which works out to ~$0.23/hour for a 1024 MB function in us-east-1. Use for latency-sensitive functions where cold starts are unacceptable.

The default account concurrency limit is 1,000 for all regions (the maximum number of concurrent Lambda executions across all functions; requestable increase via AWS Support). Separate from this is the burst scaling rate — how fast Lambda can add new execution environments: 3,000 new environments/minute in us-east-1, us-west-2, and eu-west-1; 500/minute in most other regions. Hitting the burst limit causes a brief throttle while Lambda scales; hitting the concurrency limit causes sustained throttling until executions complete. Reserve a concurrency budget for critical functions — if a lower-priority function scales unboundedly, it can exhaust the account limit and throttle critical functions.

Event Source Mappings

SQS

Lambda polls SQS and invokes the function with batches of messages. On success, Lambda deletes the messages from the queue. On failure, messages return to the queue and eventually go to the dead-letter queue.

1# Create SQS event source mapping
2aws lambda create-event-source-mapping \
3  --function-name payments-processor \
4  --event-source-arn arn:aws:sqs:us-east-1:012345678901:payments-queue \
5  --batch-size 10 \
6  --maximum-batching-window-in-seconds 5 \
7  --function-response-types ReportBatchItemFailures

ReportBatchItemFailures lets the function return partial batch success — return only the failed message IDs, and Lambda retries only those messages rather than the entire batch:

1def handler(event, context):
2    failed_items = []
3
4    for record in event['Records']:
5        try:
6            process_message(record['body'])
7        except Exception as e:
8            # Add failed message ID to list
9            failed_items.append({'itemIdentifier': record['messageId']})
10
11    # Return partial failure — only failed messages will be retried
12    return {'batchItemFailures': failed_items}

Without ReportBatchItemFailures, any exception causes the entire batch to be retried — successfully processed messages in the batch are re-processed, which requires idempotency in the handler.

SQS visibility timeout should be set to at least 6 times the function timeout (AWS's minimum recommendation). If your function timeout is 30 seconds, set the visibility timeout to at minimum 180 seconds — this gives Lambda sufficient time to process and allows for retries before the message becomes visible again to other consumers.

DynamoDB Streams

Lambda processes DynamoDB Streams for change data capture — reacting to inserts, updates, and deletes in a DynamoDB table:

1# Create DynamoDB Streams event source mapping
2aws lambda create-event-source-mapping \
3  --function-name order-change-processor \
4  --event-source-arn arn:aws:dynamodb:us-east-1:012345678901:table/orders/stream/2026-01-01T00:00:00.000 \
5  --starting-position TRIM_HORIZON \
6  --batch-size 100 \
7  --bisect-batch-on-function-error \
8  --destination-config '{"OnFailure":{"Destination":"arn:aws:sqs:us-east-1:012345678901:order-dlq"}}'
9  # --bisect-batch-on-function-error: on failure, split the batch in half to isolate the bad record

1def handler(event, context):
2    for record in event['Records']:
3        event_name = record['eventName']    # INSERT, MODIFY, REMOVE
4
5        if event_name == 'INSERT':
6            new_item = record['dynamodb']['NewImage']
7            order_id = new_item['orderId']['S']
8            # Process new order
9
10        elif event_name == 'MODIFY':
11            old_image = record['dynamodb']['OldImage']
12            new_image = record['dynamodb']['NewImage']
13            # React to order status change
14
15        elif event_name == 'REMOVE':
16            # OldImage is present only when stream view type is OLD_IMAGE or NEW_AND_OLD_IMAGES
17            old_item = record['dynamodb'].get('OldImage', {})
18            # React to deletion

DynamoDB Streams deliver each record at least once. Design handlers to be idempotent — if the same event is delivered twice, processing it twice should produce the same result.

Lambda Layers

Layers are ZIP archives attached to a Lambda function that extend the execution environment. They're mounted at /opt in the execution environment. Use layers for:

• Shared libraries used across multiple functions
• Large dependencies (numpy, pandas, ML model weights) kept separate from function code
• Custom runtimes

1# Create a layer with shared Python dependencies
2pip install -r requirements.txt -t python/lib/python3.12/site-packages/
3zip -r layer.zip python/
4
5aws lambda publish-layer-version \
6  --layer-name payments-dependencies \
7  --compatible-runtimes python3.12 \
8  --zip-file fileb://layer.zip
9
10# Attach layer to a function
11aws lambda update-function-configuration \
12  --function-name payments-processor \
13  --layers arn:aws:lambda:us-east-1:012345678901:layer:payments-dependencies:3

ZIP-based functions can use up to 5 layers. The combined unzipped size of a function's code + all layers must be under 250 MB. Container image functions do not use Lambda Layers — dependencies are packaged into the container image directly.

AWS-provided layers: AWS publishes layers for common use cases — the Lambda Insights extension layer (adds detailed CloudWatch metrics), the Parameters and Secrets Lambda extension (provides a local cache for SSM Parameter Store and Secrets Manager, reducing API calls and latency), and the AWS X-Ray SDK.

1# Attach the Parameters and Secrets extension layer
2aws lambda update-function-configuration \
3  --function-name payments-processor \
4  --layers arn:aws:lambda:us-east-1:177933569100:layer:AWS-Parameters-and-Secrets-Lambda-Extension:11
5
6# The extension provides a local HTTP server — access secrets without API calls
7# GET http://localhost:2773/secretsmanager/get?secretId=prod/payments/database

Container Image Functions

Lambda supports container images up to 10 GB (vs 250 MB for ZIP deployments). This is useful for ML inference, data processing pipelines, and functions with large dependencies.

1FROM public.ecr.aws/lambda/python:3.12
2
3# Copy requirements first for layer caching
4COPY requirements.txt .
5# Install to the standard Python path — the Lambda base image is already configured correctly.
6# Avoid --target $LAMBDA_TASK_ROOT with official base images; it can cause import conflicts.
7RUN pip install -r requirements.txt
8
9COPY handler.py ${LAMBDA_TASK_ROOT}
10
11CMD ["handler.handler"]

1# Build and push to ECR
2aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 012345678901.dkr.ecr.us-east-1.amazonaws.com
3
4docker build -t payments-processor .
5docker tag payments-processor:latest 012345678901.dkr.ecr.us-east-1.amazonaws.com/payments-processor:latest
6docker push 012345678901.dkr.ecr.us-east-1.amazonaws.com/payments-processor:latest
7
8# Create Lambda function from container image
9aws lambda create-function \
10  --function-name payments-processor \
11  --package-type Image \
12  --code ImageUri=012345678901.dkr.ecr.us-east-1.amazonaws.com/payments-processor:latest \
13  --role arn:aws:iam::012345678901:role/LambdaExecutionRole \
14  --memory-size 2048 \
15  --timeout 60

Container images have longer cold start times than ZIP deployments because Lambda must pull and initialize the container image. Lambda caches images across execution environments in the same region — after initial warm-up, subsequent cold starts in the same region use the cached layers.

Function URLs and API Gateway

Lambda functions can be invoked via HTTPS with two approaches:

Lambda Function URLs (direct): a dedicated HTTPS endpoint for a single function. No API Gateway required. Supports IAM auth or no auth. Best for simple webhooks, internal APIs, or functions triggered by a single endpoint.

1# Create a function URL with IAM auth
2aws lambda create-function-url-config \
3  --function-name payments-processor \
4  --auth-type AWS_IAM \
5  --cors '{
6    "AllowOrigins": ["https://app.codingprotocols.com"],
7    "AllowMethods": ["POST"],
8    "AllowHeaders": ["content-type"]
9  }'

API Gateway (full API): routing, request/response transformation, rate limiting, usage plans, API keys, request validation. Use when you need a full REST or HTTP API with multiple routes, stages, and access controls.

1# Create HTTP API (API Gateway v2 — lower cost, lower latency than REST API)
2aws apigatewayv2 create-api \
3  --name payments-api \
4  --protocol-type HTTP \
5  --target arn:aws:lambda:us-east-1:012345678901:function:payments-processor
6
7# The integration automatically creates a Lambda permission — manually add if needed
8aws lambda add-permission \
9  --function-name payments-processor \
10  --statement-id apigateway-invoke \
11  --action lambda:InvokeFunction \
12  --principal apigateway.amazonaws.com \
13  --source-arn "arn:aws:execute-api:us-east-1:012345678901:abc123/*"

Lambda@Edge and CloudFront Functions

Lambda@Edge runs Lambda functions at CloudFront edge locations — closer to users, before requests reach the origin. Use cases: A/B testing, request rewriting, auth token validation, HTTP security headers.

1// Lambda@Edge function — runs at CloudFront viewer-request stage
2// Must be deployed in us-east-1 regardless of distribution region
3exports.handler = async (event) => {
4    const request = event.Records[0].cf.request;
5    const headers = request.headers;
6
7    // Redirect www to non-www
8    const host = headers.host[0].value;
9    if (host.startsWith('www.')) {
10        return {
11            status: '301',
12            statusDescription: 'Moved Permanently',
13            headers: {
14                location: [{ key: 'Location', value: `https://${host.slice(4)}${request.uri}` }],
15            },
16        };
17    }
18
19    return request;
20};

Lambda@Edge constraints vs standard Lambda:

• Must be deployed in us-east-1 (replicated to edge automatically)
• No environment variables (use SSM Parameter Store with caching at init time)
• No VPC access
• Smaller size limits: 1 MB compressed / 10 MB uncompressed for viewer-request/response functions; 50 MB compressed / 250 MB uncompressed for origin-request/response functions
• No container images (Layers are supported at all stages)

CloudFront Functions (not Lambda@Edge): for very simple logic (URL rewrites, header manipulation), CloudFront Functions run in a JavaScript runtime at the edge with sub-millisecond execution. Cheaper than Lambda@Edge — CloudFront Functions cost $0.0000001/invocation with no duration charge, while Lambda@Edge costs $0.0000006/invocation plus $0.00000625/GB-second of duration. The total cost difference is larger than the per-invocation comparison alone suggests. CloudFront Functions are limited to 10ms execution time and no network calls.

IAM for Lambda

Lambda uses two IAM roles:

Execution role: what the Lambda function can do — permissions to call DynamoDB, S3, SQS, Secrets Manager, etc. Attached to the function.

Resource-based policy: what can invoke the Lambda function. API Gateway, S3, SNS, and other services need a lambda:InvokeFunction permission in the function's resource policy.

1# Execution role — what the function can call
2aws iam create-role \
3  --role-name LambdaPaymentsRole \
4  --assume-role-policy-document '{
5    "Version": "2012-10-17",
6    "Statement": [{
7      "Effect": "Allow",
8      "Principal": {"Service": "lambda.amazonaws.com"},
9      "Action": "sts:AssumeRole"
10    }]
11  }'
12
13# Attach basic execution policy (CloudWatch Logs)
14aws iam attach-role-policy \
15  --role-name LambdaPaymentsRole \
16  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
17
18# Add DynamoDB access
19aws iam put-role-policy \
20  --role-name LambdaPaymentsRole \
21  --policy-name DynamoDBAccess \
22  --policy-document '{
23    "Version": "2012-10-17",
24    "Statement": [{
25      "Effect": "Allow",
26      "Action": ["dynamodb:GetItem", "dynamodb:PutItem", "dynamodb:UpdateItem"],
27      "Resource": "arn:aws:dynamodb:us-east-1:012345678901:table/payments-prod"
28    }]
29  }'

Serverless Architectural Patterns

Fan-Out via SNS

One event triggers multiple Lambda functions independently:

API Gateway → Lambda (publish) → SNS Topic → Lambda A (send email)
                                           → Lambda B (update analytics)
                                           → Lambda C (trigger fulfillment)

1# Lambda publishes to SNS; each subscription triggers a separate function
2aws sns create-topic --name order-events
3
4aws sns subscribe \
5  --topic-arn arn:aws:sns:us-east-1:012345678901:order-events \
6  --protocol lambda \
7  --notification-endpoint arn:aws:lambda:us-east-1:012345678901:function:send-confirmation-email

Queue-Based Load Leveling via SQS

SQS decouples producers from Lambda, absorbing traffic spikes:

API (high-volume writes) → SQS → Lambda (processes at its own pace)

Lambda scales up to handle the queue backlog but is constrained by reserved-concurrent-executions and the downstream services it calls. The queue buffers events during traffic peaks — no requests are dropped, they just wait in the queue.

EventBridge for Event-Driven Coordination

EventBridge routes events from many AWS services (and your application) to multiple targets based on event patterns:

1# Route all EC2 state changes to a Lambda for auditing
2aws events put-rule \
3  --name ec2-state-changes \
4  --event-pattern '{
5    "source": ["aws.ec2"],
6    "detail-type": ["EC2 Instance State-change Notification"]
7  }' \
8  --state ENABLED
9
10aws events put-targets \
11  --rule ec2-state-changes \
12  --targets 'Id=audit-function,Arn=arn:aws:lambda:us-east-1:012345678901:function:ec2-audit'

For scheduled invocations (cron-style), use EventBridge Scheduler:

1# Invoke a Lambda every day at 2 AM UTC
2aws scheduler create-schedule \
3  --name daily-cleanup \
4  --schedule-expression "cron(0 2 * * ? *)" \
5  --flexible-time-window '{"Mode": "OFF"}' \
6  --target '{
7    "Arn": "arn:aws:lambda:us-east-1:012345678901:function:daily-cleanup",
8    "RoleArn": "arn:aws:iam::012345678901:role/SchedulerRole",
9    "Input": "{\"action\": \"cleanup\"}"
10  }'

When Lambda Works and When It Doesn't

Lambda works well for:

• Event-driven processing: S3 uploads → thumbnail generation, DynamoDB Streams → search index updates, SQS messages → order processing
• Scheduled tasks: daily reports, nightly cleanups, periodic polling
• Webhooks: receiving events from third-party services, processing GitHub webhooks, Stripe events
• Infrequent or bursty workloads: batch jobs that run once a day, reporting functions that run when triggered
• API backends with simple business logic: CRUD operations backed by DynamoDB or RDS Proxy

Lambda doesn't work well for:

• Long-running processes: Lambda maximum timeout is 15 minutes. Use ECS Fargate tasks or Step Functions for longer workflows
• Sustained high-throughput: at high sustained request rates, Lambda cost exceeds EC2 or EKS. The break-even point depends on function duration and memory — generally > ~100ms average duration at high RPS tips toward containers
• Stateful workloads: Lambda execution environments are ephemeral and can be replaced at any time. Any state must live in DynamoDB, ElastiCache, or S3
• Low-latency APIs with cold start sensitivity: if p99 latency matters and cold starts happen regularly, Provisioned Concurrency adds cost, or containers may be a better fit
• WebSocket connections: Lambda can handle WebSocket connections via API Gateway, but each message is a separate Lambda invocation — expensive and architecturally awkward for real-time bidirectional protocols

Frequently Asked Questions

How do I debug Lambda cold starts?

CloudWatch metrics report init duration separately from function duration in the REPORT log line:

REPORT RequestId: abc123 Duration: 45.23 ms  Billed Duration: 46 ms  Memory Size: 1024 MB  Max Memory Used: 87 MB  Init Duration: 312.45 ms

Init Duration only appears on cold starts. Use CloudWatch Logs Insights to find functions with high cold start rates:

fields @timestamp, @message
| filter @message like /Init Duration/
| parse @message "Init Duration: * ms" as init_duration
| stats avg(init_duration) as avg_init, count() as cold_starts by bin(1h)
| sort avg_init desc

What's the difference between Lambda and Step Functions?

Lambda executes a single function. Step Functions orchestrates multiple Lambda functions (and other services) into a workflow — sequencing, branching, parallel execution, retries, and error handling across steps.

Use Step Functions when:

• Your workflow has multiple sequential steps (order processing: validate → charge → fulfill → notify)
• You need to handle failures and retries at the workflow level
• A step takes longer than 15 minutes (Step Functions can wait for a callback for up to a year)
• You need to visualize workflow execution for debugging

How do I handle Lambda function versioning for zero-downtime deploys?

Lambda supports versions (immutable snapshots) and aliases (pointers to versions). Use aliases to route traffic between versions:

1# Publish current code as version 5
2aws lambda publish-version --function-name payments-processor
3
4# Update alias to point to version 5
5aws lambda update-alias \
6  --function-name payments-processor \
7  --name prod \
8  --function-version 5
9
10# Canary: route 10% of traffic to version 5, 90% stays on the alias's primary version
11aws lambda update-alias \
12  --function-name payments-processor \
13  --name prod \
14  --routing-config '{"AdditionalVersionWeights":{"5":0.1}}'

API Gateway or EventBridge targets the alias ARN (payments-processor:prod). Switching versions is atomic — update the alias, and all new invocations immediately use the new version.

For SQS, SNS, and EventBridge as the event sources that trigger Lambda, see AWS Messaging: SQS, SNS, and EventBridge. For Step Functions orchestrating Lambda into multi-step workflows, see AWS Step Functions: Orchestrating Distributed Workflows. For ECS Fargate as the container alternative when Lambda doesn't fit, see AWS ECS vs EKS: Choosing the Right Container Orchestrator on AWS.

Optimizing Lambda cold start latency for a latency-sensitive API, designing a fan-out event processing architecture on SQS and SNS, or deciding whether to run a workload on Lambda or ECS? Talk to us at Coding Protocols — we help platform teams choose the right serverless patterns for their workload characteristics.