Load Balancer Best Practices: Improve Scalability and Reliability

Troubleshooting Load Balancer Performance: Common Issues and Fixes

Date: February 7, 2026

This article walks through the most common load balancer performance problems, how to diagnose them, and practical fixes you can apply quickly. It assumes a typical cloud or on-premises environment using popular load balancer types (L4 Network Load Balancer, L7 Application Load Balancer/ALB, and software proxies like HAProxy or NGINX).

1. High latency to backend services

• Symptoms: Increased request response times; CPU on backends low while requests queue; tail latencies spike.
• Causes: Slow backend processing, connection saturation, inefficient health checks, or TLS termination overhead.
• Diagnostics:
  1. Measure end-to-end latency and break down client→LB, LB→backend, backend processing using tracing or logs.
  2. Check backend connection metrics (active connections, wait queues) and TCP retransmits.
  3. Inspect TLS handshake times if TLS is terminated at the LB.
• Fixes:
  • Offload TLS to dedicated terminators or enable session reuse (TLS session tickets/OCSP stapling).
  • Tune backend worker/thread pools and database queries.
  • Increase backend instances or scale horizontally.
  • Adjust keepalive and connection reuse settings on LB and backends.

2. Uneven traffic distribution (hot backends)

• Symptoms: Some backend instances show far higher load than peers.
• Causes: Sticky sessions misconfiguration, inaccurate health checks, inconsistent hashing/keying, or poor session affinity settings.
• Diagnostics:
  1. Inspect LB distribution algorithm (round-robin, least-connections, hash-based).
  2. Verify session cookie settings and any application-level affinity logic.
  3. Confirm backends report healthy status consistently.
• Fixes:
  • Use least-connections or weighted round-robin for dynamic load.
  • Disable sticky sessions unless required; if used, set appropriate cookie TTLs.
  • Ensure consistent hashing keys (e.g., same header) across requests.
  • Replace failing instances; make health checks more granular (application-level).

3. Connection limits / resource exhaustion

• Symptoms: New connections dropped or refused, ⁄₅₀₃ errors from LB, or backend logs showing EMFILE/socket exhaustion.
• Causes: File descriptor limits, ephemeral port exhaustion, insufficient LB or backend capacity.
• Diagnostics:
  1. Monitor connection counts, file descriptor usage (ulimit), ephemeral port usage.
  2. Check LB worker threads/process counts and error logs.
• Fixes:
  • Raise OS limits (ulimit -n), tune tcp_tw_reuse/time_wait settings, increase ephemeral port range.
  • Enable connection pooling and keepalives to reduce churn.
  • Scale LB processes or instances horizontally.

4. Health check flapping and false negatives

• Symptoms: Healthy backends marked unhealthy intermittently; traffic diverted unexpectedly.
• Causes: Overly strict health checks, transient application spikes, network jitter, or mismatched health endpoint behavior.
• Diagnostics:
  1. Correlate health check failures with application logs and network metrics.
  2. Verify health endpoint response time and payload stability.
• Fixes:
  • Make health checks tolerant: increase timeout, lower frequency, require consecutive failures.
  • Use deeper, application-level checks (e.g., DB connectivity) instead of simple TCP.
  • Ensure health endpoint is lightweight and deterministic.

5. TLS/SSL performance problems

• Symptoms: High CPU on LB, slow handshakes, increased client latency.
• Causes: CPU-bound crypto operations, misconfigured ciphers, lack of session reuse, or missing hardware acceleration.
• Diagnostics:
  1. Measure CPU utilization on LB during TLS peaks.
  2. Inspect SSL handshake counts and per-handshake times.
• Fixes:
  • Enable TLS session resumption (tickets or IDs) and OCSP stapling.
  • Prefer modern, efficient cipher suites (ECDHE with AES-GCM or ChaCha20-Poly1305).
  • Offload crypto to hardware (HSMs) or dedicated TLS proxies.
  • Terminate TLS at edge and use internal plain or mTLS connections where acceptable.

6. Misrouted or dropped requests (routing rules mistakes)

• Symptoms: Requests fail or go to wrong backend pool, 404s for valid paths.
• Causes: Incorrect host/path rules, regex bugs, header mismatches, or precedence errors in rule sets.
• Diagnostics:
  1. Review LB routing rules and precedence ordering.
  2. Reproduce failing requests and capture request headers and matched rule.
• Fixes:
  • Simplify and test routing rules; add logging to show matched rules.
  • Normalize incoming headers (case, trailing slashes) before matching.
  • Use explicit rule ordering and unit tests for complex regex-based routes.

7. Health and autoscaling not aligned

• Symptoms: Autoscaler triggers too slowly or scales based on LB metrics that don’t reflect real load.
• Causes: Using the wrong metrics (e.g., CPU instead of request latency), health probes masking real load.
• Diagnostics:
  1. Compare autoscaler triggers with actual traffic patterns and request latencies.
  2. Check LB metrics used by autoscaler.
• Fixes:
  • Use request-based metrics (RPS, latency, queue length) to drive autoscaling.
  • Add scale-down grace periods to avoid rapid churn.
  • Ensure healthchecks don’t falsely indicate readiness before app is fully ready.

8. Logging, observability, and tracing gaps

• Symptoms: Hard to root-cause issues; lack of correlation between LB logs and backend traces.
• Causes: Missing request IDs, inconsistent log formats, or sampling gaps.
• Diagnostics:
  1. Verify request IDs are injected at the edge and propagated downstream.
  2. Check that LB and backend logs include timestamps, latency, status codes.
• Fixes:
  • Inject and propagate unique request IDs (X-Request-ID).
  • Export structured logs and align timestamp formats and timezones.
  • Enable distributed tracing and correlate LB spans with backend spans.

9. DDoS and abusive traffic impact

• Symptoms: Legitimate traffic degraded during spikes; LB overwhelmed by malformed or high-rate requests.
• Causes: Lack of rate limiting, insufficient edge protections.
• Diagnostics:
  1. Inspect request patterns, geo-distribution, and request rates.
  2. Check WAF/edge protections and LB dropped packet counts.
• Fixes:
  • Apply rate limiting, WAF rules, and geo-blocking where appropriate.
  • Use CDN or DDoS protection services in front of LB.
  • Configure SYN cookies and TCP rate limiting at the network edge.

10. Configuration drift and deployment errors

• Symptoms: Unexpected behavior after config changes; inconsistent environments.
• Causes: Manual edits, no versioning, or missing automated testing.
• Diagnostics:
  1. Compare current config to version-controlled baseline.
  2. Review recent change history and deployment logs.
• Fixes:
  • Store LB configs in IaC (Terraform, CloudFormation) and use CI to validate.
  • Add staged rollouts and automated tests for routing and health checks.
  • Use feature flags and canary deployments for rule changes.

Quick troubleshooting checklist

1. Check LB and backend health metrics (latency, error rates, connection counts).
2. Trace a slow/failing request end-to-end with request IDs.
3. Verify health check configuration and backend readiness.
4. Inspect TLS handshake metrics and tune session resumption.
5. Confirm routing rules and header normalization.
6. Scale or increase capacity if connection limits are reached.
7. Improve observability: inject request IDs, enable traces and structured logs.

When to escalate

• Persistent 5xx across many backends despite healthy probes.
• Rapid connection or FD exhaustion not fixed by tuning.
• Outages correlated with LB software/firmware bugs—contact vendor support.

If you want, I can produce a tailored troubleshooting runbook for your specific load balancer type (AWS ALB/NLB, GCP, HAProxy, NGINX) with exact commands, metrics to watch, and sample config snippets.