Interview Preparation | Building Blocks - Load Balancers

December 20th, 2023

Introduction

This summary will serve to further cement my learnings taken when reviewing the Load Balancers module in the Grokking the Modern System Design Interview course, and I hope will provide some learnings to you as well.

Introduction to Load Balancing

Millions of requests in a data center necessitate load balancing to distribute the load among servers. The load balancer (LB) divides client requests among available servers to prevent overload or crashes.

Importance of Load Balancers

The load balancing layer, following the firewall, is crucial. Load balancers are essential for scalability, availability, and performance improvement. They will seamlessly upscale/downscale, hide faults, and optimize resource utilization.

Capabilities of Load Balancers

  1. Scalability: Easily increase application capacity by adding servers.
  2. Availability: Maintain system availability despite server faults or failures.
  3. Performance: Optimize response time by directing requests to less-loaded servers.

Load Balancer Functionality

Load balancers sit between clients and servers, managing traffic flow. They are strategically placed in a three-tier architecture between web, application, and database servers.

Load Balancer Placement

  1. Between end users and web servers/application gateway.
  2. Between web servers and application servers.
  3. Between application servers and database servers.

Services Offered by Load Balancers

Load balancers provide crucial services beyond load distribution, enhancing system functionality.

Key Services

  1. Health Checking: Monitors server health and reliability using the heartbeat protocol.
  2. TLS Termination: Handles TLS termination with clients, reducing server burden.
  3. Predictive Analytics: Predicts traffic patterns through analytics.
  4. Reduced Human Intervention: Automates tasks, minimizing system administration efforts.
  5. Service Discovery: Forwards client requests to appropriate servers by consulting the service registry.
  6. Security: Mitigates attacks, enhancing security at different OSI model layers.

Global Server Load Balancing (GSLB)

  1. GSLB Functionality: Distributes traffic load globally across multiple geographical regions.
  2. Decision Factors: Involves considerations like users’ locations, server numbers, and data center health.
  3. Automatic Zonal Failover: GSLB offers automatic failover and can be deployed on-premises or as Load Balancing as a Service (LBaaS).

Local Load Balancing

  1. Focus: Optimizes efficiency and resource utilization within a data center.

Global Server Load Balancing (GSLB) Techniques

GSLB intelligently forwards globally arriving traffic, considering factors such as geographic locations, data center health, and server numbers.

Load Balancing in DNS

DNS can perform GSLB by responding with multiple IP addresses. It uses round-robin for load balancing, but with limitations.

Round-Robin in DNS

  1. Mechanism: Reorders IP addresses in response to DNS queries.
  2. Limitations: Uneven load distribution among end-servers, potential service availability issues due to crashes.
  3. Widespread Use: Despite limitations, round-robin is widely used by DNS service providers, emphasizing short TTL for effective load balancing.

Local Load Balancers - Need and Challenges

While DNS is vital, it has limitations, prompting the need for local load balancers within a data center.

DNS Limitations

  1. Packet Size: DNS’s small packet size may not include all server IP addresses.
  2. Client Behavior Control: Limited control over client behavior, and potential selection of busy data center IP addresses.
  3. Geolocation Challenges: Challenges in determining the closest server address.
  4. Recovery Slowdown: Slow recovery in case of failures due to DNS caching mechanisms.

Algorithms of Load Balancers

Load balancers utilize various algorithms to distribute client requests. Key algorithms include:

Key Load Balancing Algorithms

  1. Round-robin Scheduling: Sequential forwarding of requests to servers.
  2. Weighted Round-robin: Assigns weights to nodes for proportional load distribution.
  3. Least Connections: Assigns newer requests to servers with fewer existing connections.
  4. Least Response Time: Directs requests to servers with the shortest response time.
  5. IP Hash and URL Hash: Hashes IP or URL for specialized application service assignments.

Static versus Dynamic Algorithms

  1. Static Algorithms: Based on existing server knowledge, suitable for single routers or commodity machines.
  2. Dynamic Algorithms: Consider real-time server states, leading to improved forwarding decisions.

Stateful versus Stateless LBs

  1. Stateful Load Balancing: Maintains session information for clients and servers.
  2. Stateless Load Balancing: Faster and lightweight, uses consistent hashing, but may lack resilience in dynamic environments.

Types of Load Balancers

Load balancing can occur at different OSI layers, offering varying functionalities:

Layer 4 and Layer 7 Load Balancers

  1. Layer 4 Load Balancers: Operate at the transport layer, ensuring the same communication is forwarded to the same backend server.
  2. Layer 7 Load Balancers: Application-aware, making decisions based on application-specific data, offering smart inspection capabilities.

Load Balancer Deployment

Load balancing is a multi-tiered process, involving different layers for optimal performance:

Tiered Load Balancer Deployment

  1. Tier-0 and Tier-1 LBs: DNS and ECMP routers divide incoming traffic.

  2. Tier-2 LBs: Utilize layer 4 load balancing, ensuring packets are forwarded to the same tier-3 LBs.

  3. Tier-3 LBs: Layer 7 LBs directly interact with backend servers, providing health monitoring and reducing end-server burden.

    To summarize, tier 1 balances the load among the load balancers themselves. Tier 2 enables a smooth transition from Tier 1 to Tier 3 in case of failures, whereas Tier 3 does the actual load balancing between back-end servers. Each tier performs other tasks to reduce the burden on end-servers.

Load Balancer Implementations

  1. Hardware Load Balancers: Stand-alone devices with performance benefits but higher costs and potential availability issues.
  2. Software Load Balancers: Flexible, programmable, and cost-effective, implemented on commodity hardware.
  3. Cloud Load Balancers: Introduced with Cloud LBaaS, providing global traffic management between zones with advantages in ease of use, cost-effectiveness, and flexibility.