Skip to content

Improve trackio experiment tracking with monitoring and artifacts support - #409

Open
dameikle wants to merge 1 commit into
eole-nlp:mainfrom
dameikle:trackio-improvements
Open

Improve trackio experiment tracking with monitoring and artifacts support#409
dameikle wants to merge 1 commit into
eole-nlp:mainfrom
dameikle:trackio-improvements

Conversation

@dameikle

@dameikle dameikle commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

This pull request significantly enhances EOLE's integration with Trackio for experiment reporting, focusing on improved configuration management, artifact logging, and robust testing. The changes introduce new configuration options, ensure reproducibility by uploading both original and effective config files as artifacts, and add comprehensive unit and integration tests for Trackio support.

Trackio integration improvements:

  • Added new Trackio configuration options to LoggingConfig (trackio_group, trackio_auto_log_cpu, trackio_auto_log_gpu, trackio_system_log_interval, trackio_log_config_artifact) to enable finer control over experiment grouping, system metrics logging, and artifact uploads.
  • Enhanced documentation to detail all Trackio options, system metric logging behavior, and artifact upload/versioning logic. [1] [2]

Config artifact handling and reproducibility:

  • Implemented logic to upload both the original and the effective (fully validated and default-filled) config files as Trackio artifacts, with sanitized names for artifact safety; artifact upload failures do not disrupt metric logging.
  • Ensured that CLI-only runs (without a config file) still upload the effective config artifact for reproducibility.

Testing and reliability:

  • Added comprehensive unit tests for Trackio artifact upload logic, artifact naming, and error handling in test_report_manager.py. [1] [2]
  • Introduced a real integration test and subprocess script to verify Trackio artifact and metric logging end-to-end, gated by an environment variable and optional dependencies. [1] [2]

These changes ensure robust, reproducible experiment reporting with Trackio, with clear documentation and strong test coverage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant