What Is a Load Balancer? How Traffic Routing and Failover Work

What a load balancer actually does: Layer 4 vs Layer 7, round robin vs least connections, health checks, sticky sessions, failover, and real 2026 pricing.

Original diagram explaining load balancer routing, traffic layers and availability dependencies

At some point every growing site hits the same wall: one server is not enough. The fix is several servers – and then you need something to decide which one answers each request.

A load balancer sits between your users and your servers and decides, for every incoming request, which server should handle it. When one server gets busy, it spreads the work. When one server dies, it stops sending it traffic. That is the whole job, done millions of times a day.

Without one, your website lives on a single machine. Every request, every visitor, every background job competes for the same CPU and memory. That machine has a ceiling, and it is also a single point of failure: one bad deploy, one failed disk, one traffic spike, and the site is down for everyone at once.

The concept is older than the cloud. Hardware appliances did this job in data centers for years; today’s cloud balancers are the same idea rented by the hour, plus an API. The algorithms and failure modes barely changed, which is why the fundamentals below are worth learning once and reusing for a decade.

If you are already comparing managed edge platforms rather than learning the basics, our Azure Front Door vs CloudFront breakdown is the better starting point. Everyone else, read on.

This guide explains how load balancers actually work: the two types that matter, the algorithms they use to pick a server, how health checks keep dead servers out of rotation, and what it all costs. It assumes a normal web application, not a network engineering background.

How a load balancer fits into an application

Clients connect to the load balancer’s public address. It is the only address the outside world ever needs. The balancer accepts the request, chooses a backend server from its pool, forwards the request, and returns the response. The client never learns which server answered, and the servers never talk to clients directly.

Original diagram tracing an HTTP request through a load balancer to a healthy backend

That indirection buys three things, and none of them requires clever hardware. Capacity: ten servers can serve what one cannot. Reliability: a failed server is bypassed while it is fixed. And flexibility: you can update servers one at a time without users noticing, which is how most zero-downtime deploys work. A site that can survive one failed server is also most of the way to surviving a failed availability zone.

Notice what is not in the diagram: a database. The balancer spreads application servers, but the database stays a shared resource below them, and it becomes the next ceiling once the app tier scales. Connection pooling and read replicas are the usual next chapter.

If you are still choosing the stack underneath, our guide on choosing a tech stack covers the earlier decisions. If the design work itself needs outside help, this is also the point where IT architecture consultants tend to earn their fee.

Layer 4 vs Layer 7: what the balancer can see

Every load balancer makes its choice with some information. How much information depends on which layer it works at, and that choice shapes everything it can and cannot do.

Original diagram comparing Layer 4 load balancers that read IPs and ports with Layer 7 balancers that read URLs, headers and cookies

A Layer 4 balancer works at the transport level. It sees IP addresses, ports, and whether the traffic is TCP or UDP, then forwards the whole connection. It is extremely fast and handles enormous connection counts, but it cannot route /api to one pool and /static to another because it never reads the URL.

A Layer 7 balancer works at the application level. It reads the HTTP request itself – path, headers, cookies, method – before choosing a backend. That enables path-based routing, per-host rules, and header-based canary releases. The tradeoff is cost: it terminates TLS and parses every request, which takes real CPU.

A useful mental model: Layer 4 is a mail sorting office that reads only the postal code, while Layer 7 opens the envelope and reads the letter. Both are legitimate designs; they answer different questions.

Every major cloud sells both kinds. AWS splits them into separate products – the Layer 7 Application Load Balancer and the Layer 4 Network Load Balancer – while edge platforms like the ones in our Front Door vs CloudFront comparison fold Layer 7 balancing into a bigger bundle.

In practice, most web applications use Layer 7 because the routing flexibility is worth the compute. Layer 4 shows up for raw throughput, for non-HTTP protocols like databases and game servers, and as the front door to a fleet of Layer 7 balancers.

Load-balancing algorithms, with a worked example

Once a balancer can receive traffic, it needs a rule for choosing a server. That rule is the load-balancing algorithm, and it runs on every single request. The simplest is round robin: first request to server A, second to B, third to C, fourth back to A. It needs no state and works well when requests cost roughly the same and servers are identical.

Original diagram of round robin load balancing sending requests one, two, three and four to servers A, B, C and back to A

Round robin breaks down when requests are not equal. If request nine triggers a ten-second report while request ten is a static file, one server quietly accumulates heavy work. Weighted round robin lets you give a bigger server more turns, but it still cannot see how busy each server actually is. Every self-hosted option in our HAProxy alternatives guide supports all of these algorithms out of the box.

Least connections fixes that by counting. The balancer tracks how many active connections each server holds and sends the next request to the emptiest one. Long-lived requests naturally pile up on servers that have capacity for them.

Original diagram of least connections balancing sending a new request to the server holding four connections instead of twelve or seven

Two more are worth knowing. IP hash pins a client to a server based on its address, which keeps a session on one machine without cookies. Consistent hashing spreads cache keys or users across servers in a way that survives servers being added and removed, which is why caches and large-scale systems prefer it.

A worked example shows why the choice matters. Three servers, nine requests, and one slow report query: under round robin each server gets three requests, but the report lands on one of them together with two normal requests. Under least connections, the report’s server looks busy after thirty seconds, so the next arrivals go elsewhere. Same hardware, visibly different p95 latency.

None of these is “best.” A small web app does fine on round robin or least connections. Pick fancier algorithms only when you have measured the problem they solve, because an algorithm that needs more state is also an algorithm that can fail in more ways. The self-hosted tools in our HAProxy alternatives guide all support these, so there is no need to shop on algorithm support alone.

Health checks: answering a port is not enough

Balancing only helps if the balancer knows which servers are healthy. It checks by probing each server on a schedule – commonly every few seconds – and a server that fails enough probes in a row leaves the rotation until it passes again. Good health data is also how teams watching multi-cloud infrastructure decide whether a whole region, not just a server, should leave the rotation.

Original diagram of load balancer health checks keeping two servers in rotation and taking a timed-out server out

There are two probe styles. Active checks ask the server a question on a timer. Passive checks watch real traffic and mark a server sick when its responses turn into errors. Most managed balancers use active checks; service meshes and some proxies add passive ones on top.

The common mistake is a shallow check. A server can accept TCP connections while its application is deadlocked, its database is unreachable, or its disk is full. A useful health endpoint checks the things the app actually needs – database connectivity, cache reachability, disk headroom – and returns failure fast when one is broken.

Tune the timing with care. Check too slowly and users hit a dead server for a minute before it is removed. Check too aggressively and a brief GC pause marks a healthy server sick, shrinking your pool at the worst moment. Typical starting points are a five-second interval with two or three failures before removal.

One more subtlety: fail open or fail closed. If every backend fails its checks, some balancers stop all traffic, while others keep routing on the theory that a sick server beats an error page. Decide which behavior you want before you need it, and monitor the check results themselves – a pool silently running at half size is how small incidents become outages.

Good health data is also how teams watching multi-cloud infrastructure decide whether a whole region, not just a server, should leave the rotation.

Sticky sessions and shared state

Sticky sessions sound like a feature. Treat them as a warning sign instead: they exist to protect applications that keep per-user state on one machine.

Some applications remember who you are on the server that served you: a shopping cart in local memory, a logged-in session on disk. Send that user’s next request to a different server and the cart is gone. Sticky sessions solve this by pinning each user to one backend, usually with a cookie.

Original diagram of stateless app servers behind a load balancer all reading the same shared session store

Session affinity has one more hidden cost: it makes deployments fragile, because draining a server means waiting for every pinned session to expire. Teams that rely on it end up scheduling deploys at 3 AM out of fear, not necessity.

Stickiness works, but it undermines the point of balancing. Load stops spreading evenly, and when the pinned server fails, the user’s state dies with it anyway. The stronger pattern is to make servers stateless: put sessions in a shared store like Redis or the database, so any server can answer any request.

Statelessness pays off beyond balancing. Deploys get simpler because any server can be drained and replaced. Autoscaling works because new servers are instantly useful. And failures stop being user-visible events that require apology emails.

If you are designing that data layer, the questions overlap with data governance: who owns session data, how long it lives, and where it may be stored. Teams that would rather hand the whole design to specialists often start from our cloud consulting shortlist.

Load balancer, reverse proxy and CDN are related, not identical

These three overlap enough that vendor marketing blurs them. A reverse proxy is any server that forwards requests to backends – it may balance, but it may also just route to one app. A load balancer is a reverse proxy whose job is spreading load and surviving failures. A CDN caches content on servers near users and answers many requests before they ever reach you.

Original diagram showing users reaching a CDN edge first, then a load balancer acting as a reverse proxy in front of app servers

API gateways sit next to this family too, but their job is policy – authentication, rate limits, versioning – rather than spreading load. Some products do both, which is convenient until you need to reason about which layer rate-limited a request.

Real architectures stack them. The CDN absorbs static traffic at the edge, the load balancer routes dynamic requests, the app servers do the work. Edge platforms now blur the line further – our Azure Front Door vs CloudFront comparison covers two services that are CDN, load balancer, and WAF in one box.

The balancer itself must not be a single point of failure

There is one more trap in this layer: make sure the failover address moves too. Two healthy balancers are useless if DNS still points at the dead one’s IP. Managed services hand you a stable hostname for exactly this reason.

Move every user request through one box and that box becomes your most fragile component. Production balancers therefore run in pairs: an active one handles traffic, a standby watches it with a heartbeat, and the standby takes over the shared address if the active one stops answering.

Original diagram of an active and standby load balancer pair exchanging heartbeats in front of the same app pool

Failover is only part of the answer. Both balancers should live in different failure domains – different racks, different availability zones – so the event that kills one cannot kill both. Managed services do this behind the scenes; self-hosted pairs make it your problem.

Cloud load balancers hide this from you – AWS, Azure, and Cloudflare run their balancers as managed, redundant services. Self-hosted NGINX or the tools in our HAProxy alternatives roundup do not: if you install it on one VM, you have built a highly available app pool behind a very available-looking single point of failure.

What load balancing costs

Managed cloud balancers charge two ways: a per-hour base and a usage meter. On AWS, an Application Load Balancer is $0.0225 per hour – about $16.50 a month before usage – plus $0.008 per LCU-hour, where LCUs measure connections, data, and rule evaluations. A Network Load Balancer is $0.0225 per hour plus $0.006 per NCLU-hour. Those are list prices from the AWS pricing page; small sites typically land in the $20 to $50 per month range, but measure your own traffic before trusting any estimate.

Worked example on AWS list prices: a small site holding 100 active connections, moving 1 GB per hour, and evaluating a handful of rules uses roughly one LCU, so the month works out near $22 for the ALB plus about $6 of LCU time. Real bills vary with traffic shape, which is why the LCU dimensions in the AWS docs are worth ten minutes before you commit.

Azure’s Standard Load Balancer charges per configured rule plus data processed, and Cloudflare sells load balancing as an add-on to its plans. Exact numbers change, so check the current pricing pages before budgeting.

The self-hosted route costs compute instead of invoices. NGINX, HAProxy, and Traefik are free to run on VMs you already pay for, which is why cost-sensitive teams compare self-hosted balancers before accepting managed prices. The hidden line item is your time: config, failover pairs, TLS renewal, and 3 AM pages.

Original diagram of AWS Application Load Balancer list-price math: 0.0225 dollars per hour plus 0.008 per LCU-hour, about 22 to 28 dollars a month for a small site

Do not forget the meter behind the meter. Outbound data transfer from the load balancer to the internet is billed at the provider’s normal egress rates, and at scale that line dwarfs the balancer itself. Budget the pair together.

If the application is not built yet, cost planning belongs in the build conversation with your SaaS development partner, and hosting choice matters too – shared hosts like the ones in our FastComet vs SiteGround review do not offer real balancers at all, while enterprise platforms like those in WordPress VIP vs AEM bake them in.

A staging checklist before you put a balancer in front of users

Run this checklist in staging, before the launch that forced you to add a balancer.

Test the boring path first. Send normal traffic through the balancer and confirm logs, headers, and client IPs survive – most apps need the X-Forwarded-For header configured or every request looks like it came from the balancer.

Check the edges of normal too: oversized headers, slow clients, WebSocket upgrades, and HTTP/2 can all behave differently once a balancer terminates and re-originates connections.

Then break things on purpose. Kill a backend mid-test and watch it leave rotation. Fill a health check’s dependencies and confirm it reports sick. Drop the active balancer and time the failover. A balancer you have never seen fail is a hypothesis, not infrastructure.

Load test while you still can afford surprises. A balancer adds a ceiling of its own – connections per second, bandwidth, rule evaluations – and finding that ceiling during a launch is a story you do not want to tell. Managed services publish their quotas; self-hosted boxes have kernel and file-descriptor limits that need raising.

Finally, write down the rollback. Who owns the balancer config, how do you route around it, and where do DNS records point if it must be replaced? If your team is small, browse our Apps directory for vetted partners who have done this before.

How we put this guide together

The mechanics here come from AWS, Azure, NGINX, and HAProxy documentation, and the pricing figures are list prices from official pricing pages, checked the day this guide was updated. Diagrams are original. Where costs depend on your traffic we say so instead of inventing a benchmark – the same standard as how we rank companies.

Frequently asked questions

Does a load balancer make a website faster?

Indirectly. It does not speed up any single request, but it stops slow requests from piling onto one server while others idle. It also lets you serve more users at once and removes the downtime that a single failed server would cause.

Is a load balancer the same as a reverse proxy?

No. A reverse proxy forwards requests to one or more backends for any reason – routing, TLS, caching. A load balancer is a reverse proxy whose specific job is distributing traffic across a pool and routing around failures. Most balancers are reverse proxies; not every reverse proxy balances.

Can I use a load balancer with only one server?

You can, and managed cloud balancers will happily bill you for it. You gain TLS termination and a stable public address, but no resilience or capacity. With one server, the money is usually better spent on a bigger machine until you can run two.

What happens when a backend server fails?

Health checks notice within seconds and the balancer stops sending that server traffic. Users see brief errors at worst while the checks catch up. When the server passes checks again, it rejoins the pool automatically.

Do I need sticky sessions?

Only if your app stores user state on individual servers and cannot be changed quickly. Prefer a shared session store so any server can serve any user; use sticky sessions as a bridge while you get there, not as the destination.

Layer 4 or Layer 7 for a small application?

Layer 7, almost always. The routing flexibility, TLS handling, and HTTP-aware health checks are worth the modest extra compute at small scale. Layer 4 earns its place at very high connection rates or for non-HTTP protocols like databases.

Can a load balancer replace backups?

No. It protects against server failure, not data loss. A balancer happily routes requests to a pool of servers all reading a corrupted database. Backups and replication are a separate layer of the design.


Leave a Reply

Your email address will not be published. Required fields are marked *