Summary
Currently, a submission that fails to be compiled or doesn't have the required files is catched by the PreFlight Step, which already aborts the pipeline execution. Since these checks are fundamental for the submission to be successfully graded.
However, these errors are logged in the ERROR level, causing observability tools to believe these failures are application failures. Logs in ERROR level should actually be related to a application procedure that was expected to work but didn't (like saving grading results, fetching a template, etc). These PreFlight errors are indeed expected, this Step is known for returning it.
Steps to reproduce
Send a submission that doesn't have all required files or doesn't compile and check the logs.
Observed behavior
PreFlight errors are logged in ERROR level.
Expected behavior
These errors should be logged as INFO or WARNING, since they don't reflect a problem in the application.
Environment (optional)
No response
References (optional)
No response
Summary
Currently, a submission that fails to be compiled or doesn't have the required files is catched by the PreFlight Step, which already aborts the pipeline execution. Since these checks are fundamental for the submission to be successfully graded.
However, these errors are logged in the ERROR level, causing observability tools to believe these failures are application failures. Logs in ERROR level should actually be related to a application procedure that was expected to work but didn't (like saving grading results, fetching a template, etc). These PreFlight errors are indeed expected, this Step is known for returning it.
Steps to reproduce
Send a submission that doesn't have all required files or doesn't compile and check the logs.
Observed behavior
PreFlight errors are logged in ERROR level.
Expected behavior
These errors should be logged as INFO or WARNING, since they don't reflect a problem in the application.
Environment (optional)
No response
References (optional)
No response