CorrelationCenter

Compute the correlations between the outputs of the TrainingCenter.

Feel free to start a discussion to ask anything here

To setup the virtual environment:

pip install -e .

I Residual

It fetches the data from TrainingCenter, adds to it the predictions and the feature importances from the dumps and compute the residuals. The outcoming data is stored at data/main_category/category.feather.

To launch this step:

./residual/shell_script/run_residual.sh

II Log hazard ratio

It uses the residuals from age prediction computed in the last step as a predictor for survival along with the age, the sex and the ethnicities. It outputs the results on a google sheet.

To launch this step:

./log_hazard_ratio/shell_script/run_log_hazard_ratio.sh

III Feature importances

It stores the feature importances, that were gathered in the first step, at data/feature_importances/main_category/category.feather.

To launch this step:

./feature_importances/run_feature_importances.sh

IV Correlation

It computes the correlations between the residuals and between the feature importances from the data that has been stored in the first step. The outcoming correlations are stored at data/correlation/residual/ and data/correlation/feature_importances/.

To launch this step:

./correlation/run_residual_correlation.sh
./correlation/run_feature_importances_correlation.sh

Structure to have before launching the jobs

┣ 📦TrainingCenter
┃  ┗ 📂data
┃     ┣ 📂examination
┃     ┃ ┗ 📜[category].feather
┃     ┣ 📂laboratory
┃     ┃ ┗ 📜[category].feather
┃     ┗ 📂questionnaire
┃       ┗ 📜[category].feather
┣ 📦CorrelationCenter
   ┗ 📂[...]

Structure of the data folder when the jobs are finished

 📂data
 ┣ 📂correlation
 ┃ ┣ 📂feature_importances
 ┃ ┃ ┣ 📜pearson_[...].feather
 ┃ ┃ ┣ 📜spearman_[...].feather
 ┃ ┗ 📂residual
 ┃   ┣ 📜number_participants_[...].feather
 ┃   ┣ 📜pearson_[...].feather
 ┃   ┗ 📜spearman_[...].feather
 ┣ 📂examination
 ┃ ┗ 📜[category].feather
 ┣ 📂feature_importances
 ┃ ┣ 📂examination
 ┃ ┃ ┗ 📜[category].feather
 ┃ ┣ 📂laboratory
 ┃ ┃ ┗ 📜[category].feather
 ┃ ┗ 📂questionnaire
 ┃ ┃ ┗ 📜[category].feather
 ┣ 📂laboratory
 ┃ ┗ 📜[category].feather
 ┗ 📂questionnaire
   ┗ 📜[category].feather