rfcs: proposal for an asynchronous verbose mode

[avmanerikar]

Link to rendered document: [link]
Link to PoC implementation: [link]

Addresses MFDNN-13603 and MFDNN-12088.

[avmanerikar]

The PoC implementation for asynchronous verbose profiling fails in the following scenario:

The asynchronous callbacks responsible for calculating and printing profiling information rely on stream_profiler_t to access event execution times. When these callbacks are delayed beyond the point where the stream is destroyed, accessing the profiler becomes impossible and can potentially trigger a segmentation fault during timing computation.

The following example reproduces this scenario for OpenCL GPU runtimes:

ONEDNN_VERBOSE_USE_SYNC=0 ONEDNN_VERBOSE=1 ./tests/benchdnn/benchdnn -v5 --engine=gpu --matmul --mode=R --repeats-per-prb=80000 128x128:128x128

The fix requires a non-blocking mechanism to retain the lifecycle of the profiler and its registered events until the callbacks complete verbose logging.

Candidate Solutions:

• Pass a snapshot of profiler events in the callback payload - The caveat is that this approach involves mutex-locked operations.
• Defer stream destruction until callbacks are executed - While this approach is blocking, the blocking occurs during stream destruction rather than during primitive execution.