C++ binding thread-safety and ownership

[chuyue86]

I'm implementing an RPC system using iceoryx2 C++, and very satisfied with performance.
My main confusion comes from the “Thread Safety“ section

1. Meaning of “thread-safe”
   What exactly does “ports are thread-safe” mean in the C++ wrapper? Does it guarantee that calls such as Publisher::loan/send and Subscriber::receive can be invoked concurrently from multiple threads? Or does the caller still need external synchronization?

2. Moving / cross-thread usage of Sample / SampleMut
   The comment in sample.hpp states “DO NOT MOVE THE SAMPLE INTO ANOTHER THREAD!”. Can I read sample in another thread？

3. Root cause of issue Memory corruption issue using C++ binding #624

• What was the final conclusion for issue Memory corruption issue using C++ binding #624? Was it determined to be user misuse (e.g., moving Sample across threads) or a library bug (e.g., a race in management structures under heavy concurrency)?

4. Recommended patterns for high-concurrency / high-throughput (C++)

• For C++ users (who do not have Rust’s Send/Sync type-level checks), what patterns do maintainers recommend in high-concurrency scenarios? Examples: per-thread independent publishers, never transferring Sample across threads, copying payload in the receiving thread and forwarding the copy to workers, etc. Are there official docs or example snippets we should follow?

[elfenpiff]

Hey @chuyue86, first of all, I think we need to improve the documentation on our side, so thank you for raising your questions!

What exactly does “ports are thread-safe” mean in the C++ wrapper?

It means that every method in the

• Publisher, Subscriber, (pubsub messaging pattern)
• Listener, Notifier (event messaging pattern)
• Server, Client (request response messaging pattern)
• Reader, Writer (blackboard messaging pattern)

can be called from multiple threads concurrently. All language bindings use the {ipc|local}_threadsafe::Service Rust service type (see: https://github.com/eclipse-iceoryx/iceoryx2/tree/main/examples/rust/service_types) under the hood since languages like C, C++ or Python do not provide the same guarantees as Rust does. And since we are used to safety-critical systems, we choose this approach since it is less bug-prone.

Or does the caller still need external synchronization?

No

Moving / cross-thread usage of Sample / SampleMut
The comment in sample.hpp states “DO NOT MOVE THE SAMPLE INTO ANOTHER THREAD!”. Can I read sample in another thread？

This is out of date. You can move the Samples (PubSub), Requests, PendingResponse and ActiveRequests (ReqRes) safely in other threads and read them. But those are not threadsafe, meaning you cannot acquire in a publisher a SampleMut and modify it from multiple threads, or read and write in on the publisher side from multiple threads.

What was the final conclusion for issue #624? Was it determined to be user misuse (e.g., moving Sample across threads) or a library bug (e.g., a race in management structures under heavy concurrency)?

The conclusion was: API bug caused a user misuse. We follow the paradigm that the API shall be easy to use and hard to misuse. Since iceoryx2 is often used in concurrent environments, it can easily happen that a port is accessed from multiple threads or that the payload moves between threads. Therefore, we refactored the language bindings so that they are all based on the threadsafe service variants.

But it comes at a cost. Just run the benchmark cargo run --bin benchmark-publish-subscribe --release -- --bench-all and you see the effect of the additional mutex in the threadsafe service variants. If you are absolutely sure you never accidentally use the ports from multiple threads or move the sample between threads, you can also use the non-threadsafe service variant and reduce the latency even further by using here the {ipc|local}::Service, see: https://github.com/eclipse-iceoryx/iceoryx2/blob/main/iceoryx2-ffi/c/src/api/service.rs#L51

For C++ users (who do not have Rust’s Send/Sync type-level checks), what patterns do maintainers recommend in high-concurrency scenarios? Examples: per-thread independent publishers, never transferring Sample across threads, copying payload in the receiving thread and forwarding the copy to workers, etc. Are there official docs or example snippets we should follow?

No not yet. We will try to add some docs to give you some guidance here. So thanks for the hint! Now some hints just for you ;)

Examples: per-thread independent publishers

If you want to have a nice debugging experience in a larger project, I would recommend one node per thread, every thread opens the same service and creates its thread local ports. With the iox2 tooling you can then list the active nodes/threads and also see what the ports in the individual threads are doing.

never transferring Sample across threads,

By default, you can transfer samples across threads to read them, but you should not transfer a SampleMut on the publisher side into multiple threads and read/write at the same time. But reading concurrently on the subscriber side is totally fine.

copying payload in the receiving thread and forwarding the copy to workers,

This is not necessary - you can share the Sample sample with multiple workers/threads.

Are there official docs or example snippets we should follow?

Take a look at the iceoryx2 book: https://ekxide.github.io/iceoryx2-book/main/ but it does not yet cover all of your questions.

Now, a question from my side: are you able to share with us some details for what you are using iceoryx2 for? We are always happy to get to know the projects that use iceoryx2.

[elfenpiff]

@chuyue86 I created an issue: #1400 and with the next release v0.9 the docs should be improved.

[chuyue86]

Thank you very much for your response，it has cleared up a problem that I've been struggling with lately. Perhaps it really is time for me to start learning Rust :-)

My team is planning to integrate iceoryx2 into our cluster's IPC system. This system is primarily designed for large-scale video decoding and analysis, consisting of various decoupled services.

We previously relied on gRPC for communication, but as the system scales, we are now looking into testing iceoryx2 as our underlying IPC layer, as performance is our top priority. I am particularly excited about several items on your roadmap, such as GPU shared memory, memfd, and the zenoh-tunnel integration.

We are currently in the early experimental stage. Since our team's primary languages are C/C++, Go, and Python, navigating the iceoryx2 source code can be quite challenging. Therefore, your explanation was incredibly helpful. Thanks again!

[elBoberido]

@chuyue86 glad we could help. Btw. we also do training on iceoryx2 if you need your team to speed up quickly. You can find the contact details in the main readme.

Also, as a fun fact, we are currently preparing a demo between iceoryx2 and gRPC to transport a lot of image data between processes and the difference between the two technologies is quite impressive.