DNS & Load Balancing

DNS architecture, record management, routing policies, and multi-layer load balancing strategies on AWS and GCP — from Route 53 health checks to GCP NEGs and Kubernetes Gateway API.

DNS Fundamentals

DNS Resolution Flow

# Full recursive resolution for api.example.com

1. Client queries local stub resolver (OS)
   → checks /etc/hosts, then local DNS cache

2. Local stub resolver queries Recursive Resolver (e.g. 8.8.8.8 or VPC DNS at .2)

3. Recursive resolver (if not cached):
   → queries Root nameservers (a.root-servers.net … m.root-servers.net, 13 anycast clusters)
   → Root returns referral: "ask .com TLD servers"

4. → queries .com TLD nameservers (managed by Verisign)
   → TLD returns: "NS records for example.com are at ns-xxx.awsdns-xx.com"

5. → queries authoritative NS for example.com (Route 53 / Cloud DNS)
   → Returns: api.example.com A 203.0.113.45 TTL 300

6. Recursive resolver caches result (for TTL duration), returns to client

# Negative caching: NXDOMAIN responses are also cached for the SOA MINIMUM TTL
# Verify with:
dig api.example.com +trace
dig @8.8.8.8 api.example.com +stats   # shows query time and when answer was cached

DNS Record Types

Record	Purpose	Example
A	IPv4 address mapping	api.example.com. 300 IN A 203.0.113.45
AAAA	IPv6 address mapping	api.example.com. 300 IN AAAA 2001:db8::1
CNAME	Alias to another hostname (cannot coexist with other records at same name)	www.example.com. CNAME example.com.
MX	Mail exchange with priority	example.com. MX 10 mail.example.com.
TXT	Text (SPF, DKIM, domain verification)	example.com. TXT "v=spf1 include:_spf.google.com ~all"
NS	Authoritative nameserver delegation	example.com. NS ns-1234.awsdns-45.com.
SOA	Zone authority, serial, refresh, retry, expire, minimum TTL	One per zone, defines zone properties
SRV	Service location (host + port + priority)	_https._tcp.example.com. SRV 10 5 443 api.example.com.
PTR	Reverse DNS lookup (IP → hostname)	45.113.0.203.in-addr.arpa. PTR api.example.com.
CAA	Certificate Authority Authorization	example.com. CAA 0 issue "letsencrypt.org"

# Common DNS debugging commands
dig api.example.com A                    # A record lookup
dig api.example.com AAAA                 # IPv6
dig api.example.com MX                   # mail records
dig api.example.com TXT                  # TXT records (SPF, DKIM)
dig api.example.com ANY                  # all record types (may be blocked)
dig -x 203.0.113.45                      # reverse PTR lookup
dig api.example.com +short               # IP only output
dig @ns-1234.awsdns-45.com api.example.com  # query specific nameserver
dig api.example.com +dnssec              # check DNSSEC signatures

TTL Strategy

AWS Route 53

Hosted Zones

# Public Hosted Zone: resolves from the internet
# Private Hosted Zone: resolves only within associated VPCs

# Create private hosted zone (Terraform)
resource "aws_route53_zone" "private" {
  name = "internal.example.com"
  vpc {
    vpc_id = aws_vpc.prod.id
  }
  # Can associate multiple VPCs (cross-account with aws_route53_vpc_association_authorization)
}

# Associate private zone with additional VPCs
resource "aws_route53_zone_association" "staging" {
  zone_id = aws_route53_zone.private.zone_id
  vpc_id  = aws_vpc.staging.id
}

# Split-horizon DNS: same domain name, different records for internal vs external
# public zone:  api.example.com → 203.0.113.45 (public ALB)
# private zone: api.example.com → 10.0.1.10 (internal service)

Routing Policies

Policy	Use Case	How it works
Simple	Single resource, no health checks	Returns one or multiple values (random selection for multiple)
Weighted	Canary deployments, A/B testing	Distribute traffic by percentage weight (e.g. 90/10)
Latency-based	Multi-region active-active	Routes to region with lowest measured latency for client
Failover	Active-passive DR	Primary record served; if health check fails, switch to secondary
Geolocation	Data residency, content localization	Routes based on client continent/country/state
Geoproximity	Fine-grained geographic routing	Routes based on client-to-resource geographic distance with bias
Multivalue Answer	Simple load distribution with health	Returns up to 8 healthy IPs; client picks randomly
IP-based	Route specific ISPs/prefixes	Routes based on client IP CIDR ranges you define

# Route 53 — Failover routing with health check (Terraform)
resource "aws_route53_health_check" "primary" {
  fqdn              = "api-us-east-1.example.com"
  port              = 443
  type              = "HTTPS"
  resource_path     = "/health"
  failure_threshold = 3
  request_interval  = 30   # seconds
  tags = { Name = "primary-health-check" }
}

resource "aws_route53_record" "api_primary" {
  zone_id        = aws_route53_zone.public.zone_id
  name           = "api.example.com"
  type           = "A"
  set_identifier = "primary"
  failover_routing_policy { type = "PRIMARY" }
  health_check_id = aws_route53_health_check.primary.id
  alias {
    name                   = aws_lb.primary.dns_name
    zone_id                = aws_lb.primary.zone_id
    evaluate_target_health = true
  }
}

resource "aws_route53_record" "api_secondary" {
  zone_id        = aws_route53_zone.public.zone_id
  name           = "api.example.com"
  type           = "A"
  set_identifier = "secondary"
  failover_routing_policy { type = "SECONDARY" }
  alias {
    name                   = aws_lb.secondary.dns_name   # us-west-2 ALB
    zone_id                = aws_lb.secondary.zone_id
    evaluate_target_health = true
  }
}

# Weighted routing for canary deployment (10% to new version)
resource "aws_route53_record" "api_v2_canary" {
  zone_id        = aws_route53_zone.public.zone_id
  name           = "api.example.com"
  type           = "A"
  set_identifier = "canary-v2"
  weighted_routing_policy { weight = 10 }
  alias {
    name    = aws_lb.canary.dns_name
    zone_id = aws_lb.canary.zone_id
    evaluate_target_health = true
  }
}

GCP Cloud DNS

# Create public managed zone
gcloud dns managed-zones create example-com \
  --dns-name="example.com." \
  --description="Production public zone" \
  --visibility=public \
  --dnssec-state=on

# Create private managed zone
gcloud dns managed-zones create internal-example \
  --dns-name="internal.example.com." \
  --visibility=private \
  --networks=prod-vpc

# Add records
gcloud dns record-sets create api.example.com. \
  --zone=example-com \
  --type=A \
  --ttl=300 \
  --rrdatas=203.0.113.45

gcloud dns record-sets create api.example.com. \
  --zone=example-com \
  --type=AAAA \
  --ttl=300 \
  --rrdatas=2001:db8::1

# DNS Server Policy — override resolver for VPC (e.g. forward to on-prem DNS)
gcloud dns policies create on-prem-forward \
  --networks=prod-vpc \
  --alternative-name-servers=192.168.1.53,192.168.1.54 \
  --description="Forward to on-prem DNS"

# DNS Response Policy (local override for specific names — split-horizon)
gcloud dns response-policies create corp-overrides \
  --networks=prod-vpc \
  --description="Internal overrides"

gcloud dns response-policies rules create api-override \
  --response-policy=corp-overrides \
  --dns-name="api.example.com." \
  --local-data=name="api.example.com.",type="A",ttl=60,rrdatas="10.0.1.10"

Load Balancing Fundamentals

Load Balancing Algorithms

Algorithm	How it works	Best for
Round Robin	Requests sent to backends sequentially in rotation	Homogeneous backends, stateless services
Weighted Round Robin	Round robin with weight factors (higher weight = more requests)	Canary deployments, heterogeneous instance sizes
Least Connections	New request sent to backend with fewest active connections	Long-lived connections (WebSocket, gRPC streaming)
IP Hash	Client IP hashed to consistently pick same backend	Stateful apps needing session affinity (sticky sessions)
Random with Two Choices	Pick 2 backends randomly, send to one with fewer connections	Low-overhead approximation of least connections at scale
Least Response Time	Combines least connections + fastest response time	Heterogeneous backends with variable latency

Layer 4 vs Layer 7 Load Balancing

Feature	Layer 4 (Transport)	Layer 7 (Application)
Operates on	TCP/UDP connections	HTTP/HTTPS requests
Routing decisions	Source/dest IP + port	URL path, hostname, headers, cookies
TLS termination	Pass-through (backend terminates) or terminate	Always terminates TLS, then re-encrypts or HTTP to backend
Performance	Ultra-low latency, high throughput	Slightly higher latency (HTTP parsing overhead)
AWS equivalent	NLB	ALB
GCP equivalent	Network LB / TCP Proxy	Global HTTP(S) LB
Source IP	Preserved (with proxy protocol or direct)	Via X-Forwarded-For header

AWS Load Balancers

Application Load Balancer (ALB)

# ALB Terraform — multi-rule listener

resource "aws_lb" "prod" {
  name               = "prod-alb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.alb.id]
  subnets            = aws_subnet.public[*].id
  enable_deletion_protection = true

  access_logs {
    bucket  = aws_s3_bucket.alb_logs.bucket
    prefix  = "prod-alb"
    enabled = true
  }
}

resource "aws_lb_listener" "https" {
  load_balancer_arn = aws_lb.prod.arn
  port              = 443
  protocol          = "HTTPS"
  ssl_policy        = "ELBSecurityPolicy-TLS13-1-2-2021-06"
  certificate_arn   = aws_acm_certificate.wildcard.arn

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.api_v1.arn
  }
}

# HTTP → HTTPS redirect
resource "aws_lb_listener" "http" {
  load_balancer_arn = aws_lb.prod.arn
  port              = 80
  protocol          = "HTTP"
  default_action {
    type = "redirect"
    redirect {
      port        = "443"
      protocol    = "HTTPS"
      status_code = "HTTP_301"
    }
  }
}

# Listener rules — path-based routing
resource "aws_lb_listener_rule" "api_v2" {
  listener_arn = aws_lb_listener.https.arn
  priority     = 100

  condition {
    path_pattern { values = ["/api/v2/*"] }
  }
  action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.api_v2.arn
  }
}

resource "aws_lb_listener_rule" "admin" {
  listener_arn = aws_lb_listener.https.arn
  priority     = 200

  condition {
    host_header { values = ["admin.example.com"] }
  }
  condition {
    source_ip { values = ["10.100.0.0/16"] }   # restrict to corporate network
  }
  action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.admin.arn
  }
}

resource "aws_lb_listener_rule" "maintenance" {
  listener_arn = aws_lb_listener.https.arn
  priority     = 999
  condition {
    http_header {
      http_header_name = "X-Maintenance-Bypass"
      values           = ["secret-bypass-token"]
    }
  }
  action { type = "forward"; target_group_arn = aws_lb_target_group.api_v1.arn }
}

# Target group with health check
resource "aws_lb_target_group" "api_v1" {
  name        = "api-v1"
  port        = 8080
  protocol    = "HTTP"
  vpc_id      = aws_vpc.prod.id
  target_type = "ip"   # for ECS Fargate / direct IP

  health_check {
    path                = "/health"
    protocol            = "HTTP"
    matcher             = "200"
    interval            = 30
    timeout             = 5
    healthy_threshold   = 2
    unhealthy_threshold = 3
  }

  stickiness {
    type            = "lb_cookie"
    cookie_duration = 86400
    enabled         = false   # enable only if stateful
  }
}

Network Load Balancer (NLB)

# NLB key characteristics:
# - Static Elastic IPs per AZ (whitelistable)
# - Preserves source IP (unlike ALB which uses X-Forwarded-For)
# - TLS termination at NLB, or TCP pass-through
# - Ultra-low latency (~100 microseconds vs ALB ~1ms)
# - Supports UDP (for DNS, syslog, game servers, QUIC)
# - PrivateLink endpoint service requires NLB

resource "aws_lb" "nlb" {
  name               = "prod-nlb"
  internal           = false
  load_balancer_type = "network"
  enable_cross_zone_load_balancing = true

  subnet_mapping {
    subnet_id     = aws_subnet.public[0].id
    allocation_id = aws_eip.nlb[0].id   # static Elastic IP per AZ
  }
  subnet_mapping {
    subnet_id     = aws_subnet.public[1].id
    allocation_id = aws_eip.nlb[1].id
  }
}

resource "aws_lb_listener" "nlb_tls" {
  load_balancer_arn = aws_lb.nlb.arn
  port              = 443
  protocol          = "TLS"
  ssl_policy        = "ELBSecurityPolicy-TLS13-1-2-2021-06"
  certificate_arn   = aws_acm_certificate.wildcard.arn

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.nlb_tcp.arn
  }
}

resource "aws_lb_listener" "nlb_udp" {
  load_balancer_arn = aws_lb.nlb.arn
  port              = 53
  protocol          = "UDP"
  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.dns.arn
  }
}

Gateway Load Balancer (GWLB)

GCP Load Balancers

Global HTTP(S) Load Balancer

# GCP Global HTTPS LB: anycast IP, Google's edge PoPs worldwide
# Terminates SSL at PoP (closest to user), then proxies to backend over Google backbone

resource "google_compute_global_address" "lb_ip" {
  name = "prod-lb-ip"
}

resource "google_compute_managed_ssl_certificate" "default" {
  name = "prod-cert"
  managed {
    domains = ["api.example.com", "www.example.com"]
  }
}

resource "google_compute_backend_service" "api" {
  name                  = "api-backend"
  protocol              = "HTTP"
  port_name             = "http"
  load_balancing_scheme = "EXTERNAL_MANAGED"
  timeout_sec           = 30

  backend {
    group           = google_compute_region_instance_group_manager.api.instance_group
    balancing_mode  = "UTILIZATION"
    max_utilization = 0.8
  }

  health_checks = [google_compute_health_check.api.id]

  # Cloud CDN
  enable_cdn = true
  cdn_policy {
    cache_mode = "CACHE_ALL_STATIC"
    default_ttl = 3600
  }

  # Cloud Armor security policy
  security_policy = google_compute_security_policy.waf.id
}

# URL Map — path-based routing
resource "google_compute_url_map" "prod" {
  name            = "prod-url-map"
  default_service = google_compute_backend_service.api.id

  host_rule {
    hosts        = ["api.example.com"]
    path_matcher = "api-paths"
  }

  path_matcher {
    name            = "api-paths"
    default_service = google_compute_backend_service.api.id

    path_rule {
      paths   = ["/v2/*"]
      service = google_compute_backend_service.api_v2.id
    }
    path_rule {
      paths   = ["/static/*"]
      service = google_compute_backend_service.cdn.id
    }
  }
}

resource "google_compute_target_https_proxy" "prod" {
  name             = "prod-https-proxy"
  url_map          = google_compute_url_map.prod.id
  ssl_certificates = [google_compute_managed_ssl_certificate.default.id]
  ssl_policy       = google_compute_ssl_policy.modern.id
}

resource "google_compute_global_forwarding_rule" "https" {
  name                  = "prod-https-rule"
  target                = google_compute_target_https_proxy.prod.id
  port_range            = "443"
  ip_address            = google_compute_global_address.lb_ip.address
  load_balancing_scheme = "EXTERNAL_MANAGED"
}

Network Endpoint Groups (NEGs)

NEG Type	Use Case	Backend
Zonal (GCE)	VM instances or containers in a zone	GCE VMs, GKE pods (directly)
Internet NEG	External endpoint (FQDN or IP)	Third-party APIs, on-prem, partner services
Serverless NEG	Cloud Run, App Engine, Cloud Functions	Serverless workloads behind GCP LB
Private Service Connect NEG	Consume PSC services via LB	Published services in other VPCs

# Serverless NEG for Cloud Run (expose via Global HTTPS LB)
resource "google_compute_region_network_endpoint_group" "cloudrun" {
  name                  = "cloudrun-neg"
  network_endpoint_type = "SERVERLESS"
  region                = "us-central1"
  cloud_run {
    service = google_cloud_run_service.api.name
  }
}

resource "google_compute_backend_service" "cloudrun" {
  name                  = "cloudrun-backend"
  load_balancing_scheme = "EXTERNAL_MANAGED"
  protocol              = "HTTPS"
  backend {
    group = google_compute_region_network_endpoint_group.cloudrun.id
  }
}

# Internet NEG — route to external endpoint through GCP LB (for Cloud Armor protection)
resource "google_compute_global_network_endpoint_group" "external_api" {
  name                  = "partner-api-neg"
  network_endpoint_type = "INTERNET_FQDN_PORT"
  default_port          = 443
}

resource "google_compute_global_network_endpoint" "partner" {
  global_network_endpoint_group = google_compute_global_network_endpoint_group.external_api.name
  fqdn                          = "api.partner.com"
  port                          = 443
}

Kubernetes Ingress vs Gateway API

Feature	Ingress	Gateway API
Maturity	Stable, widely adopted	GA since Kubernetes 1.28
Role separation	Single object (admin + developer mixed)	GatewayClass (infra), Gateway (ops), HTTPRoute (dev)
Protocol support	HTTP/HTTPS only	HTTP, TCP, UDP, gRPC, TLS
Cross-namespace routing	Limited	First-class support via ReferenceGrant
Traffic splitting	Controller-specific annotations	Native via weight in HTTPRoute

# NGINX Ingress — path and host routing
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: prod-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  ingressClassName: nginx
  tls:
  - hosts: [api.example.com]
    secretName: api-tls-cert
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /api/v1
        pathType: Prefix
        backend:
          service: { name: api-v1, port: { number: 80 } }
      - path: /api/v2
        pathType: Prefix
        backend:
          service: { name: api-v2, port: { number: 80 } }

---
# Gateway API — HTTPRoute with traffic splitting
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: api-canary
spec:
  parentRefs:
  - name: prod-gateway
    namespace: infra
  hostnames: [api.example.com]
  rules:
  - matches:
    - path: { type: PathPrefix, value: /api }
    backendRefs:
    - name: api-v1
      port: 80
      weight: 90
    - name: api-v2
      port: 80
      weight: 10   # 10% canary traffic

---
# cert-manager ClusterIssuer (Let's Encrypt)
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef: { name: letsencrypt-prod }
    solvers:
    - http01:
        ingress: { class: nginx }

Health Checks

# AWS ALB health check (Terraform)
health_check {
  enabled             = true
  path                = "/health"
  protocol            = "HTTP"
  matcher             = "200-299"
  port                = "traffic-port"
  interval            = 30        # seconds between checks
  timeout             = 5         # fail if no response in 5s
  healthy_threshold   = 2         # 2 consecutive successes = healthy
  unhealthy_threshold = 3         # 3 consecutive failures = unhealthy
}
# Grace period: set deregistration_delay.timeout_seconds = 30 (drain existing connections)

# GCP health check
resource "google_compute_health_check" "api" {
  name                = "api-health-check"
  check_interval_sec  = 10
  timeout_sec         = 5
  healthy_threshold   = 2
  unhealthy_threshold = 3

  https_health_check {
    port         = 8443
    request_path = "/healthz"
  }
}

# gRPC health check (for gRPC services on GCP)
resource "google_compute_health_check" "grpc" {
  name = "grpc-health"
  grpc_health_check {
    port         = 50051
    grpc_service_name = "my.service.Health"
  }
}

# Kubernetes Probes
livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 10   # wait before first check (app startup time)
  periodSeconds: 10
  failureThreshold: 3        # restart pod after 3 failures

readinessProbe:
  httpGet:
    path: /ready
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 5
  failureThreshold: 3        # remove from LB endpoints after 3 failures