Skip to content
Snippets Groups Projects
Commit b4227370 authored by Mark D. Roth's avatar Mark D. Roth
Browse files

Code review changes and other improvements.

parent ce0105fa
No related branches found
No related tags found
No related merge requests found
doc/images/load-balancing.png

27.1 KiB | W: | H:

doc/images/load-balancing.png

27.4 KiB | W: | H:

doc/images/load-balancing.png
doc/images/load-balancing.png
doc/images/load-balancing.png
doc/images/load-balancing.png
  • 2-up
  • Swipe
  • Onion skin
This diff is collapsed.
Load Balancing in gRPC
=======================
======================
# Objective
# Scope
To design a load balancing API between a gRPC client and a Load Balancer to
instruct the client how to send load to multiple backend servers.
This document explains the design for load balancing within gRPC.
# Background
## Per-Call Load Balancing
It is worth noting that load-balancing within gRPC happens on a per-call
basis, not a per-connection basis. In other words, even if all requests
come from a single client, we still want them to be load-balanced across
all servers.
## Approaches to Load Balancing
Prior to any gRPC specifics, we explore some usual ways to approach load
balancing.
## Proxy Model
### Proxy Model
Using a proxy provides a solid trustable client that can report load to the load
balancing system. Proxies typically require more resources to operate since they
......@@ -21,7 +29,7 @@ latency to the RPCs.
The proxy model was deemed inefficient when considering request heavy services
like storage.
## Balancing-aware Client
### Balancing-aware Client
This thicker client places more of the load balancing logic in the client. For
example, the client could contain many load balancing policies (Round Robin,
......@@ -41,7 +49,7 @@ It would also significantly complicate the client's code: the new design hides
the load balancing complexity of multiple layers and presents it as a simple
list of servers to the client.
## External Load Balancing Service
### External Load Balancing Service
The client load balancing code is kept simple and portable, implementing
well-known algorithms (e.g., Round Robin) for server selection.
......@@ -104,9 +112,7 @@ works:
a load balancer address, and a [service config](service_config.md)
that indicates which client-side load-balancing policy to use (e.g.,
`round_robin` or `grpclb`).
2. The client instantiates the load balancing policy, which is then
responsible for deciding which requests will be sent to which
addresses.
2. The client instantiates the load balancing policy.
- Note: If all addresses returned by the resolver are balancer
addresses, then the client will use the `grpclb` policy, regardless
of what load-balancing policy was requested by the service config.
......@@ -114,19 +120,28 @@ works:
by the service config. If no load-balancing policy is requested
by the service config, then the client will default to a policy
that picks the first available server address.
3. In the case of the `grpclb` policy, it opens a stream to one of the
balancer addresses returned by the resolver. It asks the balancer for
the server addresses to use for the server name originally requested by
the client (i.e., the same one originally passed to the name resolver).
- Note: Currently, the `grpclb` policy ignores any non-balancer
addresses returned by the resolver. However, in the future, it may
be changed to use these addresses as a fallback in case no balancers
can be contacted.
4. The gRPC servers to which the load balancer is directing the client
may report load to the load balancers, if that information is needed
by the load balancer's configuration.
5. The load balancer returns a server list to the gRPC client. If the
server list is empty, the call will block until a non-empty one is
received.
6. The gRPC client will send RPCs to the gRPC servers contained in
the server list from the Load Balancer.
3. The load balancing policy creates a subchannel to each server address.
- For all policies *except* `grpclb`, this means one subchannel for each
address returned by the resolver. Note that these policies
ignore any balancer addresses returned by the resolver.
- In the case of the `grpclb` policy, the workflow is as follows:
a. The policy opens a stream to one of the balancer addresses returned
by the resolver. It asks the balancer for the server addresses to
use for the server name originally requested by the client (i.e.,
the same one originally passed to the name resolver).
- Note: The `grpclb` policy currently ignores any non-balancer
addresses returned by the resolver. However, in the future, it
may be changed to use these addresses as a fallback in case no
balancers can be contacted.
b. The gRPC servers to which the load balancer is directing the client
may report load to the load balancers, if that information is needed
by the load balancer's configuration.
c. The load balancer returns a server list to the gRPC client's `grpclb`
policy. The `grpclb` policy will then create a subchannel to each of
server in the list.
4. For each RPC sent, the load balancing policy decides which
subchannel (i.e., which server) the RPC should be sent to.
- In the case of the `grpclb` policy, the client will send requests
to the servers in the order in which they were returned by the load
balancer. If the server list is empty, the call will block until a
non-empty one is received.
......@@ -20,12 +20,18 @@ uses the syntax:
scheme://authority/endpoint_name
```
Here, `scheme` indicates the name-system to be used. Example schemes to
be supported include:
Here, `scheme` indicates the name-system to be used. Currently, we
support the following schemes:
* `dns`
- `dns`
* `etcd`
- `ipv4` (IPv4 address)
- `ipv6` (IPv6 address)
- `unix` (path to unix domain socket -- unix systems only)
In the future, additional schemes such as `etcd` could be added.
The `authority` indicates some scheme-specific bootstrap information, e.g.,
for DNS, the authority may include the IP[:port] of the DNS server to
......@@ -38,7 +44,7 @@ syntax of the endpoint name is dictated by the scheme in use.
### Resolver Plugins
The gRPC client library will switch on the scheme to pick the right
The gRPC client library will use the specified scheme to pick the right
resolver plugin and pass it the fully qualified name string.
Resolvers should be able to contact the authority and get a resolution
......@@ -46,7 +52,7 @@ that they return back to the gRPC client library. The returned contents
include:
- A list of resolved addresses, each of which has three attributes:
- The address itself, in IP:port form.
- The address itself, including both IP address and port.
- A boolean indicating whether the address is a backend address (i.e.,
the address to use to contact the server directly) or a balancer
address (for cases where [external load balancing](load-balancing.md)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment