Skip to content

UCP/RMA: Add GET and PUT rendezvous protocols#11482

Open
tvegas1 wants to merge 9 commits into
openucx:masterfrom
tvegas1:rma_rndv
Open

UCP/RMA: Add GET and PUT rendezvous protocols#11482
tvegas1 wants to merge 9 commits into
openucx:masterfrom
tvegas1:rma_rndv

Conversation

@tvegas1
Copy link
Copy Markdown
Contributor

@tvegas1 tvegas1 commented May 22, 2026

What?

Add rendezvous-based protocols for PUT and GET:

  • put/rndv as an RMA_RTS wrapper over regular RNDV receive
  • get/rndv as a push-only RNDV wrapper using spontaneous RTR_REQ.

Why?

Reuse existing rendezvous flows for RMA transfers to support cases where native RMA cannot directly access either the source or destination.

How?

PUT using put/rndv:

  • RNDV_SEND creation on origin, RMA_RTS tx, RNDV_RECV creation on target

GET uses a rndv push flow only, using get/rndv:

  • RNDV_RECV creation on origin, RTR_REQ tx, RNDV_SEND creation on target, data push and relevant AMs.

Common:

  • RNDV fragmentation, mtype staging, pipeline flow, rkey packing, ATP/ATS handling, request cleanup
    Config for instance:
  • UCX_MAX_RNDV_RAILS applies through the reused RNDV data path, UCX_RNDV_SCHEME is functional for put/rndv.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant