DevOps & Platform

Nginx Load Balancing: Round Robin, Least Conn, and IP Hash

Intermediate45 min to complete14 min readJune 1, 2026Updated August 19, 2026

Quick answer

Configure Nginx as a production reverse proxy and load balancer — upstream blocks, load balancing algorithms, passive health checks, keepalive connections to upstream, and a complete TLS-terminating config.

Installing Nginx
Nginx as a Reverse Proxy
The Upstream Block
Load Balancing Algorithms
Server Parameters

intermediate · 45 min

Before you begin

Linux intermediate — comfortable editing config files and using systemd
Basic networking — TCP ports, DNS, HTTP/HTTPS
A running Linux server with Nginx installed

Nginx

Load Balancing

Reverse Proxy

Infrastructure

DevOps

Nginx Load Balancing: Round Robin, Least Conn, and IP Hash

Nginx is one of the most widely deployed reverse proxies and load balancers in production. Whether you're in front of a three-node API cluster, a Kubernetes Ingress controller, or a CDN origin, the underlying mechanism is the same: an upstream block defines a pool of backend servers, and proxy_pass forwards requests to it.

This tutorial covers the load balancing algorithms, server parameters, health checking, keepalive optimisation, and a complete production config.

Installing Nginx

bash

1# Ubuntu / Debian
2apt update && apt install nginx -y
3systemctl enable --now nginx
4
5# Verify
6nginx -v
7curl http://localhost

Config files live in /etc/nginx/. The main file is /etc/nginx/nginx.conf. Site configs go in /etc/nginx/sites-enabled/ (or /etc/nginx/conf.d/ on RHEL-based distros).

Nginx as a Reverse Proxy

Before adding load balancing, understand the basic reverse proxy pattern:

nginx

1# /etc/nginx/sites-enabled/api
2server {
3    listen 80;
4    server_name api.example.com;
5
6    location / {
7        proxy_pass http://localhost:3000;
8        proxy_set_header Host              $host;
9        proxy_set_header X-Real-IP         $remote_addr;
10        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
11        proxy_set_header X-Forwarded-Proto $scheme;
12    }
13}

The four proxy_set_header directives are almost always needed:

Header	Purpose
`Host`	Passes the original hostname to the backend
`X-Real-IP`	Client's actual IP address
`X-Forwarded-For`	Chain of IPs for multi-hop proxies
`X-Forwarded-Proto`	Whether the original request was HTTP or HTTPS

Without these, your backend sees the Nginx worker's IP as the client and may generate incorrect redirect URLs.

The Upstream Block

Load balancing uses a named upstream block placed outside the server block:

nginx

1upstream api_servers {
2    server 10.0.0.1:3000;
3    server 10.0.0.2:3000;
4    server 10.0.0.3:3000;
5}
6
7server {
8    listen 80;
9    server_name api.example.com;
10
11    location / {
12        proxy_pass http://api_servers;    # Reference the upstream by name
13    }
14}

Load Balancing Algorithms

Round Robin (default)

Requests are distributed sequentially. No directive needed — this is Nginx's default.

nginx

upstream api_servers {
    server 10.0.0.1:3000;
    server 10.0.0.2:3000;
    server 10.0.0.3:3000;
}

Request 1 → server 1, Request 2 → server 2, Request 3 → server 3, Request 4 → server 1...

Best for: uniform request sizes and similar server specs.

Weighted Round Robin

Servers with a higher weight receive proportionally more requests:

nginx

upstream api_servers {
    server 10.0.0.1:3000 weight=3;    # Receives 3 out of every 5 requests
    server 10.0.0.2:3000 weight=2;    # Receives 2 out of every 5 requests
}

Best for: backend servers with different CPU/memory capacities.

Least Connections (`least_conn`)

Each new request goes to the server with the fewest active connections:

nginx

upstream api_servers {
    least_conn;
    server 10.0.0.1:3000;
    server 10.0.0.2:3000;
    server 10.0.0.3:3000;
}

Best for: requests with variable response times (database queries, file uploads, external API calls). Round robin can pile up slow requests on one server; least_conn distributes the actual load more evenly.

This is the best default for most APIs.

IP Hash (`ip_hash`)

Routes each client to the same backend server based on a hash of their IP address:

nginx

upstream api_servers {
    ip_hash;
    server 10.0.0.1:3000;
    server 10.0.0.2:3000;
}

The same client IP always hits the same server — sticky sessions without cookies.

Caveats:

Load distributes unevenly if many clients are behind the same NAT IP
Adding or removing a server rehashes all client mappings, disrupting sessions
Use only when your application has genuine session affinity requirements that can't be solved with a shared session store (Redis)

Server Parameters

nginx

1upstream api_servers {
2    server 10.0.0.1:3000;
3    server 10.0.0.2:3000 weight=2;          # Higher weight — gets twice as many requests
4
5    server 10.0.0.3:3000 backup;            # Only used when all primary servers are down
6    server 10.0.0.4:3000 down;             # Permanently marked unavailable
7                                            # (keep in ip_hash upstreams to preserve mappings)
8
9    server 10.0.0.5:3000 max_fails=3 fail_timeout=30s;
10    # max_fails=3 — mark server as failed after 3 consecutive failures
11    # fail_timeout=30s — keep it marked failed for 30 seconds, then retry
12}

Defaults: max_fails=1, fail_timeout=10s.

Passive Health Checks

Nginx open source uses passive health checks: it monitors real traffic and marks a server as unavailable when it fails max_fails times within fail_timeout seconds. After the timeout expires, Nginx sends a single probe request to check if the server recovered.

nginx

1upstream api_servers {
2    server 10.0.0.1:3000 max_fails=3 fail_timeout=30s;
3    server 10.0.0.2:3000 max_fails=3 fail_timeout=30s;
4}
5
6server {
7    location / {
8        proxy_pass http://api_servers;
9        proxy_connect_timeout    5s;
10        proxy_read_timeout      60s;
11
12        # Retry the next upstream server on these conditions
13        proxy_next_upstream error timeout http_502 http_503 http_504;
14        proxy_next_upstream_tries 2;    # Try at most 2 servers before returning an error
15    }
16}

proxy_next_upstream — transparent failover: if the first server fails, Nginx retries the request on the next available server. Safe for GET requests; be cautious with POST (non-idempotent) requests.

Active health checks — probing /health on a schedule — require Nginx Plus. For open source Nginx, use an external tool (HAProxy, Consul, or a load balancer health check) or the third-party nginx_upstream_check_module.

Keepalive Connections to Upstream

By default, Nginx opens a new TCP connection for each upstream request. This is expensive under load (TCP handshake + TLS handshake for HTTPS upstreams).

Enable connection reuse with keepalive:

nginx

1upstream api_servers {
2    least_conn;
3    server 10.0.0.1:3000;
4    server 10.0.0.2:3000;
5    keepalive 32;        # Keep up to 32 idle connections cached per worker process
6}
7
8server {
9    location / {
10        proxy_pass http://api_servers;
11        proxy_http_version 1.1;          # Required — HTTP/1.0 doesn't support keepalive
12        proxy_set_header Connection "";  # Clear the Connection header (removes "close")
13    }
14}

The combination of proxy_http_version 1.1 and proxy_set_header Connection "" is required. Without them, Nginx defaults to HTTP/1.0 upstream requests which include Connection: close, preventing keepalive.

A Complete Production Config

nginx

1# /etc/nginx/sites-enabled/api
2upstream api {
3    least_conn;
4    server 10.0.0.1:3000 max_fails=3 fail_timeout=30s;
5    server 10.0.0.2:3000 max_fails=3 fail_timeout=30s;
6    server 10.0.0.3:3000 max_fails=3 fail_timeout=30s;
7    keepalive 32;
8}
9
10# Redirect HTTP → HTTPS
11server {
12    listen 80;
13    server_name api.example.com;
14    return 301 https://$host$request_uri;
15}
16
17server {
18    listen 443 ssl;
19    server_name api.example.com;
20
21    ssl_certificate     /etc/letsencrypt/live/api.example.com/fullchain.pem;
22    ssl_certificate_key /etc/letsencrypt/live/api.example.com/privkey.pem;
23    ssl_protocols       TLSv1.2 TLSv1.3;
24    ssl_prefer_server_ciphers off;    # Let clients use their preferred cipher order
25
26    # Logging with upstream server IP for debugging
27    log_format upstream_log '$remote_addr → $upstream_addr [$time_local] '
28                             '"$request" $status upstream_rt=$upstream_response_time';
29    access_log /var/log/nginx/api_access.log upstream_log;
30
31    location / {
32        proxy_pass http://api;
33
34        # Keepalive
35        proxy_http_version 1.1;
36        proxy_set_header Connection "";
37
38        # Forwarding headers
39        proxy_set_header Host              $host;
40        proxy_set_header X-Real-IP         $remote_addr;
41        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
42        proxy_set_header X-Forwarded-Proto $scheme;
43
44        # Timeouts
45        proxy_connect_timeout  5s;
46        proxy_read_timeout    60s;
47        proxy_send_timeout    60s;
48
49        # Failover
50        proxy_next_upstream error timeout http_502 http_503 http_504;
51        proxy_next_upstream_tries 2;
52    }
53
54    # Internal health check endpoint (no upstream)
55    location /nginx-health {
56        access_log off;
57        return 200 "OK\n";
58        add_header Content-Type text/plain;
59    }
60}

Testing and Applying Changes

bash

1# Always test before applying
2nginx -t
3
4# Reload gracefully (no dropped connections)
5systemctl reload nginx
6
7# Verify which upstream server handled each request
8# (using the upstream_log format above)
9tail -f /var/log/nginx/api_access.log
10
11# Test load balancing manually
12for i in {1..9}; do
13    curl -s http://api.example.com/whoami
14done
15
16# Check active connections (requires stub_status module)
17# Add to a server block: location /nginx_status { stub_status; }
18curl http://localhost/nginx_status

Algorithm Comparison

Algorithm	Best for	Avoid when
Round Robin	Uniform request size, similar server specs	Requests have variable processing time
Least Conn	Variable request duration (APIs, file ops)	Almost never — this is the best default
Weighted RR	Mixed server capacities	Weights are frequently adjusted
IP Hash	Session-dependent apps, no shared session store	Adding/removing servers disrupts sessions

Frequently Asked Questions

Which load balancing algorithm should I use?

Round robin unless you have a reason otherwise. Least connections helps when request durations vary a lot, so a slow request does not keep queueing behind itself. IP hash gives session affinity at the cost of uneven distribution — reach for it only when the application genuinely cannot handle a request landing anywhere.

What is the difference between passive and active health checks?

Passive checks observe real traffic and mark a backend down after failures, which is free but only notices when a user request already failed. Active checks probe independently, so a broken backend is removed before traffic reaches it — that is an NGINX Plus feature in the official build.

How do I take a backend out of rotation without dropping connections?

Mark it as draining rather than removing it, so existing connections finish while new ones go elsewhere. Removing it outright cuts in-flight requests. If your build lacks that directive, mark the server down and reload — nginx keeps old workers alive until their in-flight requests finish, so nothing is cut. Setting weight=0 is not an option: nginx rejects it as an invalid parameter and refuses to load the config.

Why do my backends see NGINX's IP instead of the client's?

Because the proxy terminates the connection. Forward the original address in a header and configure the backend to trust it — but only trust that header from your proxy, since a client can otherwise set it to anything and defeat any IP-based logic you have.

What's Next

Fix: Nginx 'Too Many Open Files' — the most common production Nginx error and how to fix it at all three levels
Kubernetes Networking & Ingress — how ingress-nginx wraps these concepts to serve Kubernetes workloads

We built Podscape to simplify Kubernetes workflows like this — logs, events, and cluster state in one interface, without switching tools.