PR: Federated Relay Pool Implementation 🚀#1076
Conversation
There was a problem hiding this comment.
Pull request overview
Implements a federated relay pool for croc, adding a pool API server for relay registration/heartbeat/discovery, client-side relay discovery + scoring, and new CLI relay roles to support community-hosted relays.
Changes:
- Add a Fiber-based pool API server with relay storage, TTL cleanup, registration, heartbeats, probing, and discovery endpoints.
- Add client-side pool relay discovery + latency/uptime-based selection, integrated into send/receive flows.
- Extend CLI/relay to support
main|community|privateroles, automatic registration/heartbeats, and post-transfer relay recruitment messaging.
Reviewed changes
Copilot reviewed 17 out of 18 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| src/utils/utils.go | Improves public IP detection, adds public IP validation + TCP latency measurement helpers. |
| src/tcp/tcp.go | Adds relay self-registration loop startup and introduces RunWithOptions. |
| src/tcp/options.go | Adds WithPublicRegistration TCP server option. |
| src/tcp/registration.go | Implements pool registration + heartbeat client for relays. |
| src/pool/storage_impl.go | Adds in-memory relay store with TTL, warmup, uptime tracking, and TCP probing. |
| src/pool/server.go | Adds Fiber pool API server bootstrap and graceful shutdown logic. |
| src/pool/handler.go | Adds pool API endpoints: /register, /heartbeat, /relays, /health. |
| src/pool/client.go | Adds Fiber-based pool client for fetching ready relays. |
| src/pool/client_test.go | Tests pool client relay filtering and uptime preservation. |
| src/models/constants.go | Introduces DEFAULT_POOL constant. |
| src/croc/relay_selection.go | Adds relay sampling + scoring logic and pool discovery helper. |
| src/croc/relay_selection_test.go | Tests relay address parsing behavior (IPv4/IPv6/hostnames). |
| src/croc/croc.go | Integrates pool discovery into send/receive and adds receiver recruitment message. |
| src/cli/cli.go | Adds --pool and relay role modes, starts integrated pool server for main, and enables relay registration. |
| go.mod | Adds Fiber v3 dependency and bumps Go version. |
| go.sum | Updates dependency checksums. |
| README.md | Documents federated pool usage and updated relay command docs. |
| Dockerfile | Updates Go builder image version. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
|
How does it work? |
|
Check PROPOSAL.md |
|
Thank @pablodz , no rush, Please ping me with the results of your tests once finished! |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 17 changed files in this pull request and generated 8 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 17 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 17 changed files in this pull request and generated 8 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
|
@schollz, the proposal for a federated pool is on PROPOSAL.md that will we removed before merge if pass |
|
Some thoughts, even before considering merging. I think I need buy-in from some community members on this. It is a big change. As the sole maintainer its even more complexity for me to maintain. That is all fine as long as this feature gets used. So I would ask that we have dedicated people providing nodes to use this before implementing it. I don't want to be in a situation where I am the only one providing nodes...its basically then back to where we started...except that I'm paying more (more servers) and I'm supporting more (problems with users creating issues related to the federatino). |
|
Sure! Even if I donated 10 TB of unused server bandwidth, it’d still just be the two of us collaborating. Maybe building a SaaS around this is the way to go? I'm picturing a managed 'Croc SaaS' for companies that move files bigger than 1 GB or burn through 1 TB+ of bandwidth. As a quick way to spread the word, the CLI could just print a short message or ad to stdout when people handle large files. You could also add enterprise-grade compliance so companies have the option to self-host or just use the managed service. A lot of companies would rather just use a ready-to-go SaaS so they don't have to deal with deployment overhead, and it'd still be really cost-effective for large enterprises. It's a model that works well—just look at projects like FastAPI. It’s fully open-source, but they also offer managed cloud services and enterprise solutions right out of the box |
PROPOSAL.md