Importance of vendored dependencies

[frenchwr]

Topic

Importance of vendored dependencies

Context

I hope to create a Debian package (.deb) of oneCCL for inclusion in the Ubuntu archive, likely for our 26.10 release.

Question

Does oneCCL support replacing the vendored dependencies for oneCCL in the deps directory with system dependencies available from a distro's package manager (e.g. apt in Ubuntu)? Does oneCCL require very specific versions of each of these dependencies?

[dvrogozh]

I looked into the deps and oneCCL cmake. Here is my breakdown on the dependencies and from where they can be taken:

Dependency	Comments
hwloc	apt-get install libhwloc-dev
itt	Not packed by Ubuntu, but it's open source project: https://github.com/intel/ittapi
level_zero	apt-get install libze-dev
mpi	Is that a closed source dependency?
ofi	apt-get install libfabric-dev
pmix	apt-get install libpmix-dev
umf	apt-get install libumf-dev (that's a new package - Ubuntu 26.04 is needed)
drm	Not in deps folder, but oneCCL actually depends on it and has manual detection of this library

I attempted to create the following PR to use all system installed dependencies which I could:

• Add build path to use system dependencies with USE_SYSTEM_DEPS #200

PR excludes MPI and ITT paths from the build. PR did not yet handle level_zero as well because I stepped into the issue that some code paths are not protected by ENABLE_MPI. As a result such build fails with:

/opt/oneccl/src/atl/util/pm/pmi_resizable_rt/pmi_resizable/kvs/internal_kvs.cpp: In member function 'virtual kvs_status_t internal_kvs::kvs_main_server_address_reserve(char*)':
/opt/oneccl/src/atl/util/pm/pmi_resizable_rt/pmi_resizable/kvs/internal_kvs.cpp:405:23: error: 'MPI_COMM_WORLD' was not declared in this scope
  405 |         MPI_Comm_rank(MPI_COMM_WORLD, &rank);
      |                       ^~~~~~~~~~~~~~
/opt/oneccl/src/atl/util/pm/pmi_resizable_rt/pmi_resizable/kvs/internal_kvs.cpp:405:9: error: 'MPI_Comm_rank' was not declared in this scope
  405 |         MPI_Comm_rank(MPI_COMM_WORLD, &rank);
      |         ^~~~~~~~~~~~~

I need either resolution for the issue above or actual MPI library we can use for the open source build. I have reported the issue in:

• Build fails with -DENABLE_MPI=OFF #201

CC: @blazej-smorawski

[dvrogozh]

In the respect to ITT dependency, it's essential ingredient to support performance profiling. As I noted before it's open source library, but it's not being packaged to Debian and Ubuntu. Most projects just prefer to build it from sources as ITT trends to be a static library and maintainers push back to make it shared (see intel/ittapi#150). Fedora still packages it making it shared (I've no idea if they have any issues with it after that). Either way, I believe oneCCL needs to at least enable building ITT from sources, so I have filed:

• Support building ITT dependency from sources #203

@frenchwr : please, consider if you want to package ITT or leave it as is. I know Debian/Ubuntu already has few projects packaged who uses ITT building it from sources. So I suggest to avoid packaging it directly and just continue the practice.