Cloud Companies Have Been Doing Load Balancing All Wrong—Here’s a Smarter Way

:::info Authors:

(1) Tianyi Cui, University of Washington ([email protected]);

(2) Chenxingyu Zhao, University of Washington ([email protected]);

(3) Wei Zhang, Microsoft ([email protected]);

(4) Kaiyuan Zhang, University of Washington ([email protected]).

:::

:::tip Editor's note: This is Part 5 of 6 of a study detailing attempts to optimize layer-7 load balancing. Read the rest below.

:::

• Abstract and 1 Introduction

• 2 Background and 2.1 Programmable Multi-core SmartNICs

• 2.2 Load balancers

• 3 SmartNIC-based Load Balancers and 3.1 Offloading to SmartNICs: Challenges and Opportunities

• 3.2 Laconic Overview

• 3.3 Lightweight Network Stack

• 3.4 Lightweight synchronization for shared data

• 3.5 Acceleration with Hardware Engine

• 4 Evaluation and 4.1 Experiment Setup

• 4.2 End-to-end Throughput

• 4.3 End-to-end Latency

• 4.4 Evaluating the Benefits of Key Techniques

• 4.5 Real-world Workload

• 5 Related Work

• 6 Conclusion and References

• A Appendix

Load Balancers. There are multiple works investigating high-performance load balancers. Katran [16] is an L4 load balancer implemented in eBPF. SilkRoad [38] and Hula [27] leverage programmable switches to build a high-performance L4 load balancer. Maglev [14] proposed by Google takes advantage of consistent hashing. Beamer [42] provides consistency without maintaining state. DRILL [19, 20] performs micro load balancing. Tiara [45] proposes a new hardware architecture for L4 load balancing. All of these works are based on connection-less load balancing and are unable to satisfy the need for connection-oriented L7 load balancing.

\ Network stack acceleration. TAS [28] provides a host TCP networking acceleration service. FlexTOE [44] and IOTCP [30] offload the host TCP stack onto the SmartNICs. Snap [36] is a high-performance networking stack proposed by Google. AccelTCP [39] offloads the control path of TCP onto the SmartNIC. Kim, D., Lee, S., and Park, K [29] explore offloading the TLS control plane onto the SmartNIC. Zilberman [47] offloads packet processing with XDP. Netchannel [11] disaggregates the network stack. They only offload a part of the control path of the networking protocols onto the SmartNIC and do not touch L7 protocols.

\ Measurements for SmartNIC. Multiple works provide valuable measurement for SmartNICs [26, 34]. Girondi et al. measured the performance of different high-speed concurrent hashtables [21]. We consulted these numbers during the design process.

\

:::info This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license.

:::

\