Multi-Region Kubernetes on Azure: Global Failover Guide

Deploy Kubernetes across Azure regions with traffic management, data consistency patterns, and disaster recovery strategies. Practical YAML and CLI examples.

sohail
By Sohail Ahmad
Posted
0

Running Kubernetes in a single region works well until you need to serve users globally or survive a regional outage. A data center issue, an unexpected traffic surge in one geography, or the need to comply with data residency regulations can push you beyond what a single region can handle. Multi-region deployments solve this, but they introduce real complexity: routing traffic intelligently, keeping data consistent across regions, and recovering from failures quickly.

I’ll walk through the practical decisions and patterns you need to run production Kubernetes across multiple Azure regions, with concrete examples you can adapt to your infrastructure.

Why Multi-Region Matters

A single-region AKS cluster gives you simplicity. Everything is fast, local, and predictable. But it’s also a single point of failure. If the region goes down, your application goes down. If you need to serve users in Asia and Europe with low latency, a single US-based region won’t cut it.

Multi-region deployments address three core needs:

  • Resilience: Survive regional outages and automatically failover to healthy regions.
  • Performance: Serve users from a region close to them, reducing latency.
  • Compliance: Keep data within specific geographic boundaries as regulations require.

The tradeoff is operational overhead. You now manage multiple clusters, coordinate deployments, synchronize state, and handle edge cases like split-brain scenarios or stale cache. This complexity is worth it if your business depends on global availability.

Routing Traffic Across Regions

The first decision is how to route incoming traffic to the right region. Azure offers two main options: Traffic Manager and Application Gateway.

Azure Traffic Manager

Traffic Manager operates at the DNS level. When a client queries your domain, Traffic Manager returns the IP address of an endpoint (usually a public IP or FQDN) based on a routing policy. It’s simple, global, and works well for region selection.

Common routing policies:

  • Priority: Always route to the primary region; failover to secondary only if primary is unhealthy.
  • Weighted: Distribute traffic across regions by percentage (useful for gradual rollouts or A/B testing).
  • Geographic: Route based on the client’s geographic location.
  • Performance: Route to the region with lowest latency.

Here’s an example setup with Terraform:

resource "azurerm_traffic_manager_profile" "global" {
  name                   = "myapp-tm"
  resource_group_name    = azurerm_resource_group.rg.name
  traffic_routing_method = "Priority"
  dns_config {
    relative_name = "myapp"
    ttl           = 60
  }
  monitor_config {
    protocol                    = "HTTPS"
    port                        = 443
    path                        = "/health"
    interval_in_seconds         = 30
    tolerated_number_of_failures = 3
  }
}

resource "azurerm_traffic_manager_azure_endpoint" "us" {
  name               = "us-east-endpoint"
  profile_name       = azurerm_traffic_manager_profile.global.name
  resource_group_name = azurerm_resource_group.rg.name
  target             = azurerm_public_ip.us_ingress.fqdn
  priority           = 1
}

resource "azurerm_traffic_manager_azure_endpoint" "eu" {
  name               = "eu-west-endpoint"
  profile_name       = azurerm_traffic_manager_profile.global.name
  resource_group_name = azurerm_resource_group.rg.name
  target             = azurerm_public_ip.eu_ingress.fqdn
  priority           = 2
}

Traffic Manager monitors each endpoint via health checks. If the primary region fails to respond to HTTPS requests on the /health path, Traffic Manager automatically fails over to the secondary. As documented in Azure’s multi-region deployment guide, this automatic failover happens without requiring application changes or manual intervention.

Application Gateway with Multi-Region Setup

Application Gateway operates at Layer 7 (application layer) and can route based on hostnames, URL paths, or custom rules. For multi-region, you typically place one Application Gateway per region and use Traffic Manager in front of them.

This gives you the best of both: Application Gateway handles intelligent request routing within a region (e.g., routing API requests to one backend pool, UI requests to another), and Traffic Manager handles cross-region failover.

In each region, configure Application Gateway to route to your AKS ingress controller:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: api-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    appgw.ingress.kubernetes.io/backend-path-prefix: "/"
spec:
  rules:
  - host: api.myapp.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 80

Data Consistency Across Regions

Traffic routing is straightforward. Keeping your data consistent across regions is where the real decisions come in.

You have three main patterns, each with different tradeoffs:

Eventual Consistency with Asynchronous Replication

Write to the primary region, replicate asynchronously to secondaries. Data is consistent “eventually,” meaning there’s a window where secondaries lag behind the primary.

This pattern is simple and scales well. You can failover to a secondary, but you might lose recent writes if the primary crashes before replication completes.

Example: PostgreSQL with streaming replication to a standby in another region.

-- On primary (US-East)
CREATE PUBLICATION all_tables FOR ALL TABLES;

-- On secondary (EU-West)
CREATE SUBSCRIPTION all_tables CONNECTION 'dbname=mydb host=us-primary user=replicator' 
PUBLICATION all_tables;

In your application, all writes go to the primary. Reads can come from either primary or secondary. If the primary fails, promote the secondary and update your application config to point there.

Read Replicas for Geographic Proximity

Place read-only replicas in each region and route read traffic to the nearest replica. Writes still go to the primary region, but reads are fast and local.

This reduces latency for reads without sacrificing consistency. The tradeoff is cost: you’re running multiple database instances.

Azure Database for PostgreSQL and MySQL both support read replicas across regions:

az postgres server replica create \
  --name mydb-eu-replica \
  --source-server mydb \
  --resource-group myapp-rg \
  --location westeurope

In your application, use a DNS alias or connection string that points to the nearest replica for reads:

const readPool = new Pool({
  host: process.env.DB_READ_HOST, // e.g., mydb-eu-replica.postgres.database.azure.com
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
  database: process.env.DB_NAME,
});

const writePool = new Pool({
  host: process.env.DB_PRIMARY_HOST, // e.g., mydb.postgres.database.azure.com
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
  database: process.env.DB_NAME,
});

// Read from nearest replica
const result = await readPool.query('SELECT * FROM users WHERE id = $1', [userId]);

// Write to primary
await writePool.query('UPDATE users SET name = $1 WHERE id = $2', [newName, userId]);

Geo-Replication for Strong Consistency

Some databases support synchronous replication, where writes are acknowledged only after they’re persisted in multiple regions. This gives you strong consistency but adds latency to every write.

Azure Cosmos DB is built for this. It offers multiple consistency levels (strong, bounded staleness, session, eventual) and automatic failover across regions:

az cosmosdb create \
  --name myapp-cosmos \
  --resource-group myapp-rg \
  --locations regionName=eastus failoverPriority=0 \
  --locations regionName=westeurope failoverPriority=1 \
  --consistency-policy default-consistency-level Strong

When you write to Cosmos DB with Strong consistency, the write is synchronously replicated to all configured regions before the write is acknowledged. If the primary region fails, Cosmos DB automatically promotes the next region in the failover priority list.

The cost is higher latency for writes, since you’re waiting for multiple regions to acknowledge. Use this pattern when consistency is critical (financial transactions, inventory) and latency is acceptable.

Backup and Disaster Recovery

Even with multi-region replication, you need backups. Replication protects against regional failures, but not against data corruption, ransomware, or accidental deletion.

Backing Up Kubernetes State with Velero

Velero is an open-source tool for backing up and restoring Kubernetes resources and persistent volumes. Install it in each cluster:

helm repo add vmware-tanzu https://vmware-tanzu.github.io/helm-charts
helm install velero vmware-tanzu/velero \
  --namespace velero \
  --create-namespace \
  --set configuration.backupStorageLocation.bucket=myapp-backups \
  --set configuration.backupStorageLocation.provider=azure \
  --set configuration.schedules.daily.schedule="0 2 * * *" \
  --set configuration.schedules.daily.template.ttl="720h"

This creates daily backups at 2 AM UTC, retained for 30 days. Velero stores backups in Azure Blob Storage, which you can access from any region.

To restore from a backup in a different region:

velero restore create --from-backup daily-20240115 --namespace velero

Azure Site Recovery for Infrastructure

Azure Site Recovery handles recovery of VMs and entire infrastructure stacks. It’s useful if you’re running stateful workloads outside Kubernetes or need to replicate entire regions.

Set up replication from your primary region to a secondary:

az site-recovery fabric create \
  --resource-group myapp-rg \
  --vault-name myapp-vault \
  --name primary-fabric \
  --type AzureIaas

az site-recovery protection-container create \
  --resource-group myapp-rg \
  --vault-name myapp-vault \
  --fabric-name primary-fabric \
  --name primary-container

Site Recovery continuously replicates your VMs to the secondary region. In case of outage, you can failover with a single command. The RTO (recovery time objective) is typically minutes, and RPO (recovery point objective) is seconds to minutes depending on your configuration.

Real-World Considerations

Latency and Cost

Multi-region deployments introduce latency between regions. If you write to US-East and read from EU-West, you’re crossing the Atlantic. Asynchronous replication means that read might be stale by milliseconds to seconds.

Data transfer between regions costs money. Outbound traffic from Azure is charged per GB. A 100 GB daily replication between regions can cost hundreds of dollars monthly. Consider your data volume and replication frequency carefully.

Use Azure’s pricing calculator and factor in compute, storage, and egress costs for each region. Sometimes a single large region with CDN for static content is cheaper and simpler than true multi-region.

Compliance and Data Residency

Many regulations (GDPR, CCPA, etc.) require data to stay within specific geographic boundaries. If you’re serving EU customers, their data must stay in EU regions.

Design your architecture to respect these boundaries. Use separate databases for each region, not a single global database. In Kubernetes, use namespaces or separate clusters per region to enforce data locality.

Testing Failover

Failover scenarios are easy to get wrong. Test them regularly in a staging environment that mirrors production.

Create a failover runbook: which services fail over first, what are the dependencies, how do you validate the secondary region is healthy, how do you fail back to primary. Automate as much as possible. Manual steps during an outage are error-prone.

Run a quarterly failover drill. Pick a time, actually fail over to the secondary region, verify everything works, then fail back. You’ll discover issues before they hit production.

Observability Across Regions

Single-region monitoring is challenging. Multi-region monitoring requires visibility into:

  • Which region is handling traffic at any moment.
  • Latency between regions and to end users.
  • Data replication lag (how far behind is the secondary).
  • Failed deployments or unhealthy pods in any region.

Use a centralized monitoring stack. Azure Monitor works well for this; send logs and metrics from all regions to a single Log Analytics workspace:

apiVersion: v1
kind: ConfigMap
metadata:
  name: ama-metrics-prometheus-config
  namespace: kube-system
data:
  prometheus-config: |
    global:
      scrape_interval: 30s
    scrape_configs:
    - job_name: 'kubernetes-pods'
      kubernetes_sd_configs:
      - role: pod
      relabel_configs:
      - source_labels: [__meta_kubernetes_namespace]
        action: replace
        target_label: namespace
      - source_labels: [__meta_kubernetes_pod_name]
        action: replace
        target_label: pod

Then query across regions to find anomalies. Alert on replication lag, failover events, and regional outages.

Putting It Together: A Multi-Region Architecture

Here’s a practical architecture you can build on:

Two AKS clusters: one in US-East (primary), one in EU-West (secondary). Traffic Manager routes incoming requests to the primary. If primary is unhealthy, Traffic Manager fails over to secondary.

PostgreSQL in US-East with streaming replication to EU-West. Reads go to the nearest replica. Writes always go to primary.

Velero runs in both clusters, backing up to a shared Azure Blob Storage account.

Azure Monitor collects logs and metrics from both clusters. Alerts fire if replication lag exceeds 5 minutes or Traffic Manager detects a failed health check.

A runbook documents the failover procedure: promote the EU-West replica to primary, update Traffic Manager to point there, and validate that applications are healthy.

This setup gives you resilience (survive a region failure), performance (serve users from nearby), and confidence that you can recover quickly from disaster.

Start Small, Scale Deliberately

Don’t jump straight to five regions across continents. Start with two regions, get the patterns working, understand the operational overhead, then expand.

Multi-region Kubernetes is powerful, but it’s not free. You trade simplicity for resilience and performance. Make sure the tradeoff is worth it for your use case. If you’re a startup with a single product in one market, a single well-provisioned region with solid backups might be enough. If you’re a global SaaS with millions of users, multi-region is mandatory.

The patterns I’ve outlined here, Traffic Manager, read replicas, Velero, and centralized monitoring, will serve you well as you scale from single-region to global infrastructure.

What’s the difference between Traffic Manager and Application Gateway for multi-region routing?

Traffic Manager operates at the DNS level and routes based on region health, geographic location, or latency. It’s simple and global but coarse-grained. Application Gateway operates at Layer 7 (HTTP/HTTPS) and can route based on URL paths, hostnames, or custom rules. For multi-region, use Traffic Manager to choose the region, then Application Gateway within each region to handle intelligent request routing.

How do I handle data consistency across regions without losing writes?

Use read replicas for your primary use case: writes go to the primary region, reads come from nearby replicas. This balances consistency with performance. For critical data that cannot tolerate any loss, use synchronous replication (like Cosmos DB with Strong consistency) or accept the latency cost. For less critical data, asynchronous replication with eventual consistency is simpler and cheaper.

What happens to my Kubernetes state if a region fails?

Kubernetes state (deployments, services, configmaps) is stored in etcd within the cluster. If the region fails, that cluster goes down. Velero backs up this state to Azure Blob Storage, which is accessible from any region. You can restore the entire cluster state in a different region, but it takes time (minutes to hours depending on cluster size). For critical workloads, run redundant clusters in multiple regions and fail over via traffic routing, not by restoring from backup.

How often should I test failover to the secondary region?

Test at least quarterly. Set up a staging environment that mirrors your production multi-region setup, then practice failing over and failing back. Document the steps, identify issues, and refine your runbook. The goal is to make failover so routine that when you actually need it, you’re confident it will work.

How much does multi-region Kubernetes on Azure cost compared to single-region?

Roughly double to triple, depending on configuration. You’re running two clusters (double compute), two databases (double storage), replication between regions (egress charges), and backups. Use Azure’s pricing calculator for your specific workload. Consider whether the resilience and performance gains justify the cost for your business.

sohail

Sohail Ahmad

Leave a Reply

Your email address will not be published. Required fields are marked *