AWS Reference Architecture for a High-Traffic SaaS Application

The architecture at a glance

"High-traffic" means different things to different products. A B2B dashboard with 5,000 tenants and a consumer app with a million daily sessions put load on different parts of the stack. Still, most SaaS products we build on AWS share one shape. There is a stateless request tier that can scale out, a relational core that has to stay correct, and an asynchronous tier that absorbs spikes and slow work. The goal is to keep the stateless tier truly stateless and to protect the database from connection storms. Anything that doesn't need an answer inside the request goes on a queue.

This design suits a product owner who wants a SaaS application to grow from thousands to millions of requests a day without re-platforming. It is not the cheapest way to run an early MVP. For a pre-product-market-fit product we usually cut this down to a single region with fewer services and one Aurora instance with a reader, then add parts as real traffic shows where the load is.

Edge: CloudFront, WAF and the load balancer

Every public hostname goes through CloudFront, including API hostnames that barely cache anything. This gives us one place for TLS policy and HTTP/2 and HTTP/3. Clients also connect to an edge location near them, and the WAF is attached before traffic reaches the VPC. For static assets and user uploads, CloudFront reads from S3 using origin access control, so the bucket is never public.

• AWS WAF rules: start with the AWS managed core rule set, known-bad-inputs and IP reputation lists. Add rate-based rules scoped by path, so login and password-reset endpoints get much tighter limits than read APIs. Run new rules in COUNT mode for a week before switching them to BLOCK.
• Cache policy: give hashed static assets long TTLs. Send API paths through with caching disabled and forward only the headers the origin really needs. Forwarding every header quietly destroys the cache hit ratio.
• Origin protection: the ALB only accepts traffic from the CloudFront managed prefix list plus a secret origin header, which blocks anyone who finds the ALB DNS name and tries to skip the WAF.
• ALB: path- and host-based routing to target groups. Health checks call a cheap /healthz that does not touch the database. Otherwise a slow database makes every task fail its health check at the same moment and the service takes itself offline.

Compute: ECS Fargate vs EKS vs Lambda

This choice causes more debate than any other in the stack. Our default for product teams without a dedicated platform team is ECS on Fargate. There are no nodes to patch, it plugs straight into ALB, IAM and CloudWatch, and the operational surface is small. We pick EKS when the organisation already runs Kubernetes or needs its ecosystem. We use Lambda for spiky, event-driven or glue workloads rather than as the main request path of a busy API.

Data: Aurora PostgreSQL, replicas and RDS Proxy

We choose Aurora PostgreSQL for most SaaS cores. Storage is replicated six ways across three AZs and grows automatically. Failover to a replica is usually fast, and readers share the same storage volume, so replica lag is typically low. Whether you need Aurora Serverless v2 or provisioned instances depends on the load shape. Serverless v2 suits variable load. Provisioned instances with reserved pricing are usually cheaper at steady, high utilisation.

Read replicas: use them on purpose

Send reads to the reader endpoint only when the code path can tolerate slightly stale data. Lists, search, analytics and dashboards usually can. "Read your own write" flows cannot. A user who saves a record and then gets redirected to a page read from a lagging replica will report a bug that you can't reproduce. We make this explicit in the data-access layer with separate writer and reader connection pools, rather than a proxy that guesses by parsing SQL.

RDS Proxy

PostgreSQL connections are expensive, and Fargate scale-out or Lambda concurrency can open thousands of them in seconds. RDS Proxy pools and multiplexes connections, keeps connections open during failover, and can authenticate with IAM so no database passwords sit in task definitions. Watch out for connection pinning. Session-level state such as SET statements, temporary tables, advisory locks or some prepared-statement patterns pins a client to one backend connection and removes the multiplexing benefit. Check the DatabaseConnectionsCurrentlySessionPinned metric after every ORM upgrade.

Tenant isolation decisions (shared schema with row-level security, schema-per-tenant or database-per-tenant) belong at this layer too. We cover them in detail in our guide to multi-tenant data isolation in PostgreSQL.

Caching with ElastiCache

ElastiCache (Valkey or Redis OSS engine) takes care of four jobs in a typical build: session and token storage, cache-aside for expensive read models, distributed rate limiting, and short-lived locks or idempotency keys. Rules we hold to:

• Tenant-prefix every key (t:{tenantId}:...) so one tenant's data can never be served to another and a single tenant's cache can be flushed.
• Every entry gets a TTL. Treat the cache as disposable. If the product breaks when the cache is flushed, it is being used as a database.
• Protect against stampedes with request coalescing or a short lock around recomputation of hot keys, plus jittered TTLs.
• Use cluster mode with replicas across AZs for anything the request path depends on, and alarm on evictions and memory fragmentation, not only CPU.

Async work: SQS, EventBridge and S3

Anything slow, retryable or fan-out goes off the request path: email, PDF generation, exports, third-party webhooks, search indexing, billing sync. We use SQS for work queues, where one consumer group must complete each job. EventBridge handles domain events such as "invoice.paid", where several independent consumers react and new consumers get added later without changing the publisher.

// Worker contract we enforce for every queue consumer
handle(message):
  key = message.idempotencyKey            // set by publisher
  if store.alreadyProcessed(key): ack; return
  doWork(message)                         // must be safe to retry
  store.markProcessed(key, ttl = 7 days)
  ack
// SQS: visibility timeout > p99 processing time
// redrive policy: maxReceiveCount = 5 -> DLQ, alarm on DLQ depth > 0

SQS standard queues deliver at least once, so every consumer must be idempotent. Use FIFO queues only when ordering per entity really matters, and pick a message group ID such as the tenant or aggregate ID so throughput still scales. For events that must match a database write, use the transactional outbox pattern. Write the event to an outbox table in the same transaction, then relay it to EventBridge. This avoids the "saved but never published" failure.

S3 holds uploads, exports and backups. Clients upload directly with pre-signed URLs, so large files never pass through the API tasks. Lifecycle rules move old objects to cheaper storage classes, and S3 events trigger processing such as virus scanning or thumbnailing through Lambda or SQS.

Multi-AZ and autoscaling signals

Run everything in at least two AZs, and preferably three. That means ALB subnets, a minimum of one ECS task per AZ, Aurora readers in separate AZs and ElastiCache replicas. NAT gateways are per-AZ as well. A single shared NAT is a cost saving that turns into a regional outage when its AZ has problems.

CPU is a weak scaling signal for most SaaS APIs, because they wait on I/O. Better signals:

Scaling compute without scaling the database just moves the bottleneck. Always cap the maximum task count at a level the database, through RDS Proxy, can actually serve. Load-test the whole path, not just the stateless tier.

Observability with CloudWatch and OpenTelemetry

We instrument services with OpenTelemetry SDKs and run the AWS Distro for OpenTelemetry collector as a sidecar or central service. Traces go to X-Ray or CloudWatch Application Signals, or to a vendor backend if the client already has one. Because the instrumentation is vendor-neutral, changing the backend later means changing collector configuration, not application code.

• Logs: structured JSON with trace_id, tenant_id and request_id on every line. Set retention per log group, because the default "never expire" gets expensive.
• Metrics: RED metrics (rate, errors, duration) per endpoint, queue age and DLQ depth, database connections, pinning and replica lag, and cache hit ratio.
• SLOs and alerts: alert on symptoms users feel, such as error budget burn rate and p95 latency on key journeys. Keep resource thresholds on dashboards, not pagers.
• Per-tenant views: in multi-tenant SaaS, "the system is fine" while one large tenant is timing out is a common blind spot. Tag telemetry by tenant tier.

Security baseline

We apply this baseline before the first production deploy, not after a questionnaire from an enterprise prospect:

• Account structure: AWS Organizations with separate accounts for production, staging, shared services and log archive. Service control policies deny disabling CloudTrail, leaving approved regions or creating IAM users with long-lived keys.
• IAM least privilege: one task role per service, scoped to specific resources (this queue, this bucket prefix, this secret). Humans sign in through IAM Identity Center with short-lived sessions. CI deploys through OIDC federation, not stored access keys.
• VPC layout: public subnets hold only the ALB and NAT. Application and data tiers sit in private subnets. Security groups reference other security groups rather than CIDR ranges. VPC endpoints for S3, ECR, Secrets Manager and CloudWatch keep that traffic off the internet and lower NAT data charges.
• Encryption: KMS customer-managed keys for Aurora, S3, SQS and secrets, with key policies that separate who administers keys from who uses them. TLS everywhere, including to the database.
• Secrets: Secrets Manager with rotation for database credentials, injected into ECS tasks as secrets, never as plain environment variables in the task definition.
• Detection: CloudTrail (organisation trail), GuardDuty, Security Hub with a foundational benchmark, AWS Config rules, and ECR image scanning in CI.

For regulated workloads, our security and compliance practices sit on top of this baseline.

Infrastructure as code: Terraform or CDK

Nothing in production is created by clicking in the console. Both tools work well, and we choose based on the team:

• Terraform / OpenTofu: our default when the organisation runs multiple clouds or SaaS providers (DNS, monitoring, identity), or when infrastructure is owned by a separate ops function. Remote state goes in S3 with locking, one state per environment and component, and plan output is posted on every pull request.
• AWS CDK: a good fit when the application team owns the infrastructure and writes TypeScript or C#. Its higher-level constructs cut boilerplate, and synthesised CloudFormation gives drift detection and rollbacks.

# Terraform: ECS service scaling on requests per target (excerpt)
resource "aws_appautoscaling_policy" "api_rps" {
  name               = "api-requests-per-target"
  policy_type        = "TargetTrackingScaling"
  resource_id        = aws_appautoscaling_target.api.resource_id
  scalable_dimension = "ecs:service:DesiredCount"
  service_namespace  = "ecs"

  target_tracking_scaling_policy_configuration {
    predefined_metric_specification {
      predefined_metric_type = "ALBRequestCountPerTarget"
      resource_label         = "${aws_lb.main.arn_suffix}/${aws_lb_target_group.api.arn_suffix}"
    }
    target_value       = var.requests_per_task   # from load testing, not a guess
    scale_in_cooldown  = 300
    scale_out_cooldown = 60
  }
}

Pipelines that apply this code are described in our CI/CD and automated testing pipeline. Our DevOps services team usually owns the modules and guard rails.

Cost controls

AWS bills for high-traffic SaaS rarely blow up because of compute. They grow through data transfer, NAT gateway processing, log ingestion, idle environments and oversized databases. Controls we set up from day one:

• Tagging policy (env, service, owner, tenant-tier) enforced in IaC, with cost allocation tags turned on and AWS Budgets alerts per account.
• Commitments after the pattern is stable: Compute Savings Plans cover Fargate and Lambda, and reserved instances suit steady Aurora and ElastiCache nodes. Don't commit in the first months while sizing is still changing.
• Graviton (ARM) task and database instances where dependencies support them. They usually cost less for similar work, but benchmark your own workload.
• Fargate Spot for interruption-tolerant workers. Scale non-production to zero outside working hours.
• VPC endpoints to cut NAT processing charges, CloudFront to reduce origin egress, and log sampling for high-volume debug output.
• Unit economics: track cost per tenant or per thousand requests, not only the total bill. Growth that raises cost per tenant is an architecture problem.

Disaster recovery: RPO and RTO tiers

Multi-AZ covers the loss of a data centre. It does not cover a bad migration that deletes data, a compromised account or a regional incident. Set DR tiers with the business based on what an hour of downtime or lost data actually costs, and then pay for exactly that tier.

Most B2B SaaS products we work on land on tier 1 or tier 2. Whatever tier you choose, it only counts if you rehearse it. Run a restore into an isolated account on a schedule and time it. Point-in-time recovery also protects against the most common "disaster", which is a bad deploy or migration.

Moving an existing platform onto this shape is a project of its own. Our cloud migration services usually move it in stages behind CloudFront, and our cloud consulting team can review a current design against it.

Pre-launch checklist

• WAF in block mode with rate rules on auth endpoints; ALB accepts only CloudFront traffic.
• Health checks independent of the database; a minimum of one task per AZ.
• RDS Proxy in front of Aurora; pinning metric reviewed; writer and reader pools separated in code.
• Every queue has a DLQ, an alarm and idempotent consumers; outbox used for DB-coupled events.
• Autoscaling tuned from a load test of the full path, with maximums that fit database capacity.
• Traces, structured logs with tenant IDs, SLO burn-rate alerts.
• Least-privilege task roles, OIDC deploys, KMS keys, rotated secrets, GuardDuty on.
• All infrastructure in Terraform or CDK with plan review; budgets and tagging enforced.
• DR tier agreed and restore rehearsed with a measured result.

For a fuller version to work through with your team, use our SaaS architecture checklist. For deployment strategy on mobile clients that call this backend, see Flutter vs React Native architecture.

Frequently asked questions

Should a new SaaS product start on ECS Fargate or EKS?

For most product teams without dedicated platform engineers, ECS on Fargate is the safer default: no nodes or control plane to operate and tight integration with ALB, IAM and CloudWatch. EKS makes sense when you already run Kubernetes, have many services, or need its ecosystem. Containers move from ECS to EKS without an application rewrite.

Do we need RDS Proxy with Aurora PostgreSQL?

If your compute scales out quickly, as Fargate services and Lambda functions do, RDS Proxy is usually worth it. It pools connections, protects the database from connection storms and holds connections open during failover. Monitor session pinning, because some ORM and session patterns reduce its benefit.

Can Lambda handle the main API for a high-traffic SaaS?

It can, but under sustained high concurrency it is often more expensive and harder to reason about than containers, and it is a weak fit for long-lived connections. We typically use Lambda for event-driven work, webhooks, scheduled jobs and low-traffic APIs, and containers for the main request path.

What RPO and RTO should a B2B SaaS target?

It depends on what downtime and data loss cost your customers and contracts. Many B2B products choose backup-and-restore or pilot-light tiers, with RPO from seconds to hours and RTO from under an hour to several hours. Agree the tier with the business, then rehearse restores so the numbers are measured rather than assumed.

Terraform or AWS CDK for SaaS infrastructure?

Both work. Terraform or OpenTofu suits teams managing several providers or a separate ops function. CDK suits application teams who own their infrastructure and prefer TypeScript or C#. What matters more is that all production infrastructure is in code, reviewed and applied from CI.