Kubernetes supports load balancing for inbound traffic. But what about Kubernetes services inside the cluster?
When in-cluster services communicate, a load balancer called kube-proxy forwards requests to service pods at random. You can use Istio to add more complex load balancing methods, enabled by Envoy.
Envoy supports multiple load balancing methods, including random, round-robin, and least request.
Let’s see how to use Istio to add least request load balancing for a service called payments, which processes all transactions for a web frontend. The payments service is backed by three pods.
In this least request algorithm, the client-side Envoy will first choose two instances at random. Then, it will forward the request to the instance with the fewest number of active requests, to help ensure even load balancing across all instances.
To enable this functionality, we create an Istio DestinationRule with a LEAST_CONN load balancer:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: payments-load-balancer
spec:
host: payments.prod.svc.cluster.local
trafficPolicy:
loadBalancer:
simple: LEAST_CONN
Check out the Istio docs to see how to add multiple load balancing methods for a single host.