UCX Memory Pool Trace

This tool can be used to monitor memory pool usage and behavior within the UCX library. In particular, it provides visibility into the memory pool used to allocate bounce buffers at runtime when reliable connected (RC) queue pairs are employed. The library uses GOTCHA to intercept the following UCX functions:

• uct_iface_mpool_init
• ucs_mpool_cleanup
• ucs_mpool_grow

Provided Accessor API

• void *MPOOL_TRACER_get_next_bounce_buffer(size_t *)
• void MPOOL_TRACER_prefetch_bounce_buffer(const size_t, const int)
• void MPOOL_TRACER_builtin_prefetch_bounce_buffer(const size_t, const int)
• void *bounce_buffer_seek(ucs_mpool_t *mp, int index,size_t *buffer_size)

Example usage with OpenMPI

$ mpirun -x LD_PRELOAD=libUCXMpoolTracer.so ./app