gRPC Envoy Nghttp2 and Load Balancing

- grpc go

I’ve been using gRPC at work and in several personal projects for months and happy with it, but when it comes to load balancing gRPC does not come with batteries included.

For a long time the only document was the Load Balancing draft in the gRPC repo, the clients should implement a Picker interface to know about the servers, so the pooling and controling the load were handled by the clients.
HTTP/2 was new and most of the reverse proxies implementations were not capable of load balancing gRPC HTTP2 frames, the only solution was to use a TCP load balancer, generating errors, improper and weird behaviours for the clients.

At least two projects are now supporting gRPC load balancing easily.

Here are some notes to simply load balance two gRPC Helloworld server! running on ports 50050 & 50051.

frontend=*,8000;no-tls
backend=localhost,50050;;no-tls;proto=h2;fall=2;rise=2
backend=localhost,50051;;no-tls;proto=h2;fall=2;rise=2
workers=8
    "clusters": [
      {
        "name": "local_service",
        "connect_timeout_ms": 250,
        "type": "static",
        "lb_type": "least_request",
        "features": "http2",
        "hosts": [
          {
            "url": "tcp://127.0.0.1:50050"
          },
          {
            "url": "tcp://127.0.0.1:50051"
          }
        ]
      }
    ]

You can then tweak the greeter_client to loop for requests, so you can simulate a client doing multiple requests while killing/restarting your servers.

for {  
    r, err := c.SayHello(context.Background(), &pb.HelloRequest{Name: name}) 
    if err != nil { 
        log.Printf("could not greet: %v", err)
    }
    log.Printf("Greeting: %s", r.Message)
}

And modify the greeter_server to show on which port/server you get your response:

// SayHello implements helloworld.GreeterServer
func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
    return &pb.HelloReply{Message: "Hello " + in.Name + port}, nil 
}

Those tests aren’t enabling any TLS so use grpc.WithInsecure().

Note that Envoy is also capable of bridging your HTTP/1.1 queries to gRPC, which is a killer feature (I haven’t tested it yet) , you would normally do it by code with gRPC-gateway.

Envoy is really new and I’m still digging into but already proves itself to be a complete load balancing proxy solution with or without gRPC in your stack.