Allow out-of-order submission of messages in LocalMessageBroker.

[ehpor]

Note

This PR is still work in progress.

Summary

This PR fixes out-of-order message handling in the local message broker by refactoring the message availability tracking from separate first/last atomics to a unified bitmap-based approach.

The Problem

Previously, the message broker tracked message availability using two separate counters: first_frame_id (oldest available) and last_frame_id (newest available + 1). This worked for sequential publishing but couldn't handle out-of-order messages. For example, if message 5 arrived before message 4, there was no way to track that 5 was available while message 4 was still missing.

The Solution: Bitmap-Based Availability Tracking

Instead of tracking just a range [first, last), we now track which specific messages in the circular buffer are available using a bitmap packed into a single atomic variable.

How it works:

The availability field is a 64-bit atomic containing two pieces of information:

• Upper 48 bits: The last message ID (the next ID that will be assigned). These 48 bits still allow us to publish messages for ~9 years at 1 million messages/sec.
• Lower 16 bits: A bitmap indicating which of the previous 16 messages are actually available.

Bitmap interpretation:

• Bit 0 = message last-1 (the newest message)
• Bit 1 = message last-2
• Bit 2 = message last-3
• ...and so on

Example:
If last = 10 and the bitmap is 0b0000000000010101 (binary):

• Bit 0 set → message 9 is available
• Bit 2 set → message 7 is available
• Bit 4 set → message 5 is available
• Messages 6 and 8 are not yet available (out of order!)

This allows the broker to correctly handle out-of-order message publishing while maintaining lock-free concurrent access between publishers and subscribers.

Note: packing everything in a single 64bit atomic allows us to atomically update both at the same time. We can perform additional operations using CAS loops (done in TryReserve(), TryReserveNext() and TryMakeAvailable().

Key Changes

1. Refactored TopicHeader availability tracking Converted from separate first_frame_id/last_frame_id atomics to a single availability atomic containing packed last_id and 16-bit bitmap.

2. Added C++17 bit operations Implemented countl_zero and countr_zero functions (backported from C++20) in Util.h/Util.inl to efficiently count leading/trailing zeros in the bitmap.

3. Moved message ID to MessageHeader This is required since each nested topic has a different message ID and we need to track all.

4. API improvements:

   • Changed GetOldestMessageId()/GetNewestMessageId() to GetBegin()/GetEnd() (half-open range semantics: [begin, end))
   • Added TryReserve(), TryReserveNext(), TryMakeAvailable() methods for atomic reservation and publishing
   • Requiring subscriptions to access messages (removed duplicate access path that bypassed subscription tracking)