What are load balancers and why use them? A system design perspective.

Improve application scalability, performance and availability with load balancers. Illustrations included!

Load balancing is a system design concept that refers to distributing a set of tasks evenly across multiple servers.

Why use load balancers?

Imagine a server with a public ip address hosting a website.
The server receives requests from a single user as shown below:

Now imagine that the number of users and requests increase 100 times.
This is going to overload the server causing crashes, slow response times and inevitably a bad user experience.

To fix this, we can add an additional server. Some of the requests can be directed to this server to reduce load on the first server. Adding additional servers in this way to handle increased traffic is known as horizontal scaling or scaling out.

But the problem now becomes that the user doesn't know whether to direct traffic at server 1 with ip 1.1.2.0 or server 2 with ip 12.1.1.0. Furthermore, there is nothing preventing users from directing traffic to server 1. In this case, it would have been a waste to add an additional server and server 1 would still be overloaded.

A load balancer would be the ideal solution in this case.

All traffic would be directed at the load balancer with a public ip address. The ip addresses of the servers would be made private. This way, they are only accessible through the load balancer. The load balancer would direct incoming traffic to the servers depending on what load balancing algorithm is configured.

Next we look at some routing algorithms used by load balancers.

Load balancing algorithms

Round robin:

Client requests are directed to each server in turn. So with two servers the requests would be directed as follows: server1, server2, server1, server2...

A problem with round robin is that there is no consideration for the load at a particular server. If the servers have varied processing times it can be a problem. For instance, if server1 takes a long time to process requests than server2, then it can still result in server1 being over-burdened. Since, requests are distributed evenly.

Least outstanding requests:

To fix the issue above, the least outstanding requests algorithm may be used. LOR is context-aware and acts based on the amount of load on each server.

This algorithm chooses the server which has the least number of outstanding requests. An outstanding request is a request that is still pending. So, for example if server1 has 5 outstanding requests (5 clients still waiting to receive the response) and server2 has 2, then the next request will be routed to server2.

Load balancers also provide additional benefits which are important to mention. We'll look at these next.

Performance, availability, scalability ...
Benefits of using load balancers

There are many reasons to use load balancers.

Load balancers increase availability of your application. If a particular server goes down, the load balancer will simply route the request to one of the other servers.

They also provide scalability. You can easily add more servers if load increases and the load balancer will automatically start routing requests to the new server. This also works the other way. If load decreases, you can remove some servers without reconfiguring the load balancer.

Finally, they provide enhanced performance by distributing load across servers as shown previously. In addition, they also provide security as the server ip addresses can be made private so that they are not directly accessible over the internet.

When not to use load balancers?

Load balancers incur an additional cost. So in cases, where you have a single server or have low traffic there is no compelling reason to use a load balancer.

I hope this post was useful. This was just an introduction to load balancers. Major cloud providers have load balancers in their list of services. There exist also several types of load balancers. AWS offers application load balancers, network load balancers and gateway load balancers. We will look at them in a future post!

Have any questions? Leave a comment below 😇

I hope you liked this post.
Please consider subscribing below to never miss content in the future.