Implement PickFirst load balancer by nathanielford · Pull Request #2570 · hyperium/tonic

nathanielford · 2026-03-25T15:30:03Z

Motivation

Full implementation of the pick first load balancer, including 'Happy eyeballs' features.

Solution

Load balancing implementation to pick the first available endpoint to connect to, maintaining stickiness across endpoint updates if configured. Handles accepting new LB configuration and subchannel reconstruction.

Prototype is at https://github.com/nathanielford/grpc-rust-testbed/tree/main/pick_first_lib

Notes

Ended up including all happy eyeball features because it wasn't clear where best to slice the line. Considering this a full implementation, and it should be reviewed as such.
This does use tokio::spawn and tokio::time, which may need to be replaced to make things runtime agnostic. Please comment in the PR whether this is the case.

arjan-bal

Leaving initial comments while I review the remaining changes.

arjan-bal · 2026-03-30T16:55:20Z

+}
+
+#[cfg(test)]
+mod test {


There are some tricky edge cases found by Go's e2e-style tests that we discovered when implementing a new dual-stack pick_first LB in Go. I added unit tests for them to ensure the balancer behaved as expected. These tests are mainly in two files:

pickfirst_ext_test.go: These are e2e-style tests that create a gRPC channel to a fake server.

pickfirst_test.go: These use a mock channel, similar to the tests in Rust.

It may not be possible to write e2e-style tests in Rust right now since we don't have the same test utilities as Go. However, we can still test the same scenarios using a mock channel.

I would recommend seeing if Gemini can convert these Go tests into tests for Rust, skipping the Happy Eyeballs ones. This would act as a conformance test. I did something similar to generate tests for credentials in the following way:

Clone gRPC Go.

Point the Gemini CLI to the test files for gRPC Go and gRPC Rust.

Ask Gemini to create similar test cases for gRPC Rust.

If it gives decent results without much effort, that's great; otherwise, we can improve test coverage later.

I think I was able to do this in the tests, particularly the last two tests in pick_first.rs. LMK if you think something is misaligned with these!

…t and stickiness

…ss, and updated sync testing framework

…A61 endpoint handling This commit implements the PickFirst load balancer policy for Tonic gRPC, focusing on: - Efficient subchannel management with backoff preservation. - "Stickiness" support: continuing to use an existing Ready subchannel if it remains in resolver updates. - Compliance with gRFC A61: endpoints are now shuffled before being flattened into an address list, ensuring multiple addresses for a single endpoint (e.g., IPv4/IPv6) stay together. - Clean state reset: subchannels and selected state are now cleared when receiving an empty address list. - Alignment with the updated synchronous testing framework in master. Includes comprehensive test coverage for basic connection, failover, stickiness, exhaustion, deterministic endpoint shuffling, de-duplication, and empty updates.

…active failover This change enhances the PickFirst load balancing policy to better support gRFC A61 (Happy Eyeballs) and improve connection establishment latency. Key changes: - Implement IPv6/IPv4 address interleaving in `compile_address` to ensure subsequent connection attempts alternate between protocol families. - Introduce a `subchannel_states` cache in `PickFirstPolicy` to track the connectivity status of managed subchannels. - Refactor connection logic to use a `frontier_index` and proactively skip subchannels known to be in `TransientFailure` (e.g., during backoff). - Update `advance_frontier` to safely maintain the index within the bounds of the address list, ensuring the policy remains reactive to recovery. - Add deterministic unit tests for shuffling and interleaving logic.

…balls

arjan-bal · 2026-05-04T18:09:30Z

+            let error = self
+                .last_connection_error
+                .clone()
+                .unwrap_or_else(|| "all addresses in transient failure".to_string());


I think if we update self.last_connection_error with an error message in the following two events, it is guaranteed to be present:

When a subchannel starts in TRANSIENT_FAILURE.

When a subchannel enters TRANSIENT_FAILURE.

Then we can call .expect() and avoid using a custom error message generated by the LB.

arjan-bal · 2026-05-04T18:33:21Z

+        // Update the cache for all updates.
+        self.subchannel_states
+            .insert(subchannel.address(), state.clone());


Before inserting, we should first ensure that the address is part of the current list of subchannels. If not, the function should return early. This can happen when a subchannel update has been queued by the channel while a resolver updated arrives, removing the address.

arjan-bal · 2026-05-04T18:40:16Z

+        }
+
+        // Steady State Retries: Automatically connect when a subchannel goes IDLE.
+        if self.steady_state.is_some() && state.connectivity_state == ConnectivityState::Idle {


nit: It would be good to group the handling of steady_state and the first pass into an if/else block, and handle common transitions (e.g., READY or READY->IDLE) before this block. This would allow readers to understand the first pass and steady-state behavior separately.

arjan-bal · 2026-05-04T18:41:44Z

+        // Steady State Failures: Count failures across all subchannels.
+        if let Some(ref mut ss) = self.steady_state {
+            if state.connectivity_state == ConnectivityState::TransientFailure {
+                if ss.record_failure(self.subchannels.len()) {


nit: As a minor simplification, we can pass the channel controller to record_failure and let it handle requesting re-resolution itself.

I moved all the steady-state stuff into the steady state handler. I think that it makes it a bit cleaner, to keep all the state tracking there. LMK if you think something is leaky though!

arjan-bal · 2026-05-04T18:48:57Z

+                    if let Some(next_sc) = self.advance_frontier(false) {
+                        let next_sc = next_sc.clone(); // Clone to avoid borrow issues
+                        self.trigger_subchannel_connection(next_sc, channel_controller);
+                    } else {


We need to cancel the timer in the else case also.It can probably happen before the if/else block.

Comparing the references of timer_handle with the Go implementation, we also need to abort the timer when the LB policy is dropped and when a new resolver update arrives.

This should all be handled - though do please audit all the possible cases!

…mediate READY activation

…preserve stickiness

…austion

…ctations

…stLB

…ollision unit test

…stLB

…nually written hash impl

…into refactor/AddressHashing

…stLB # Conflicts: # grpc/Cargo.toml

…nformance tests

nathanielford · 2026-05-07T19:47:05Z

Note that this now includes #2631, which will merge first so as to keep a clearer record. However, I needed the fix to get the tests ported from Go to work.

nathanielford requested a review from dfawley March 25, 2026 15:30

nathanielford self-assigned this Mar 25, 2026

nathanielford requested review from arjan-bal and removed request for dfawley March 25, 2026 16:17

nathanielford force-pushed the implement/PickFirstLB branch 3 times, most recently from a442272 to 73397ff Compare March 25, 2026 17:29

nathanielford assigned arjan-bal and unassigned nathanielford and arjan-bal Mar 25, 2026

dfawley assigned arjan-bal Mar 27, 2026

arjan-bal reviewed Mar 30, 2026

View reviewed changes

arjan-bal assigned nathanielford Mar 30, 2026

arjan-bal reviewed Mar 30, 2026

View reviewed changes

arjan-bal removed their assignment Mar 30, 2026

nathanielford force-pushed the implement/PickFirstLB branch from 73397ff to bdf7fc3 Compare April 6, 2026 19:32

nathanielford force-pushed the implement/PickFirstLB branch from 7ea10ad to e1bcaf4 Compare April 27, 2026 20:55

nathanielford requested a review from arjan-bal April 27, 2026 21:38

nathanielford marked this pull request as ready for review April 27, 2026 21:39

arjan-bal assigned arjan-bal and unassigned nathanielford Apr 28, 2026

arjan-bal added the A-grpc-next label Apr 28, 2026

nathanielford added 6 commits April 29, 2026 15:17

Implement PickFirst load balancer with optimized subchannel managemen…

9a3be62

…t and stickiness

WIP: implementation of PickFirstLB

aa39042

Implement PickFirst LB with optimized subchannel management, stickine…

db800a7

…ss, and updated sync testing framework

grpc: implement Racing Selection and Timer Cancellation for Happy Eye…

98b1b41

…balls

arjan-bal reviewed May 4, 2026

View reviewed changes

arjan-bal assigned nathanielford and unassigned nathanielford and arjan-bal May 4, 2026

Partial feedback integration

9cbb1a4

nathanielford force-pushed the implement/PickFirstLB branch from 880fb6f to 9cbb1a4 Compare May 4, 2026 23:19

nathanielford added 11 commits May 6, 2026 17:45

lb/pick_first: restrict visibility of builder and config

8180473

lb/pick_first: refactor subchannel updates for mode separation and im…

2954173

…mediate READY activation

lb/pick_first: early return on untracked subchannel updates

168e7a8

lb/pick_first: encapsulate timer cancellation into a helper function

5467799

lb/pick_first: scan for READY subchannels during resolver updates to …

0a77a0f

…preserve stickiness

lb/pick_first: reconnect to IDLE subchannels upon connection pass exh…

c17dbaf

…austion

lb/pick_first: harden pass exhaustion error tracking with strict expe…

9449f22

…ctations

name_resolution: include structural attributes in Address PartialEq

12ab353

name_resolution: include structural attributes in Address PartialEq

1018fda

Merge remote-tracking branch 'hyperium/master' into implement/PickFir…

6698786

…stLB

Merge branch 'master' into refactor/AddressHashing

b0aa655

nathanielford force-pushed the implement/PickFirstLB branch from b02264a to 6698786 Compare May 6, 2026 23:54

nathanielford added 9 commits May 6, 2026 23:56

name_resolution: derive structural equality for Address and add map c…

acd6b2e

…ollision unit test

Merge branch 'master' into refactor/AddressHashing

4d47842

Merge remote-tracking branch 'hyperium/master' into implement/PickFir…

3a32745

…stLB

Update test to remove coverage of derived traits, add coverage for ma…

cc467f5

…nually written hash impl

Merge remote-tracking branch 'nathanielford/refactor/AddressHashing' …

0fa6603

…into refactor/AddressHashing

Merge branch 'master' into refactor/AddressHashing

3f3800f

Merge remote-tracking branch 'hyperium/master' into implement/PickFir…

5ecf935

…stLB # Conflicts: # grpc/Cargo.toml

Merge branch 'refactor/AddressHashing' into implement/PickFirstLB

eb7c378

lb/pick_first: refactor test boilerplate, harden comments, and add co…

c869cb6

…nformance tests

Conversation

nathanielford commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Solution

Notes

Uh oh!

arjan-bal left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nathanielford commented May 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nathanielford commented Mar 25, 2026 •

edited

Loading

arjan-bal left a comment •

edited

Loading