Data Science Template: implement expect_metric test function

[ArthurCRodrigues]

Parent Issue

Part of #338 — Feature: Data Science Template (data_science)

---

Overview

Implement the expect_metric test function that executes the student's program, extracts a numeric metric from stdout using a regex pattern, and validates it against a threshold with a configurable comparison operator. This is the primary test for evaluating model performance metrics (accuracy, precision, recall, RMSE, etc.).

---

Interface

class ExpectMetricTest(BaseExecutionTest):
    @property
    def name(self):
        return "expect_metric"

---

Parameters

Parameter	Type	Required	Description
program_command	string/dict	✓	Command to execute. Supports multi-language dict and "CMD" auto-resolution.
metric_pattern	string	✓	Regex pattern with a named group (?P<value>...) to extract the metric value from stdout.
condition	string	✓	Comparison operator: >=, <=, ==, >, <
threshold	float	✓	The threshold value to compare against.
tolerance	float	✗	Tolerance for == comparisons (default: 0.001). Ignored for inequality operators.
inputs	list[string]	✗	Stdin inputs if the program requires them.

---

Scoring Logic

Binary scoring:

• 100 if the extracted metric satisfies the condition against the threshold.
• 0 if the condition is not met, or if the metric cannot be extracted.

Condition evaluation:

• >=: extracted_value >= threshold
• <=: extracted_value <= threshold
• >: extracted_value > threshold
• <: extracted_value < threshold
• ==: abs(extracted_value - threshold) <= tolerance

---

Execution Flow

1. Run program_command in the sandbox (via run_sandbox_execution from BaseExecutionTest).
2. Check for base errors (timeout, compilation error, runtime error) → score 0.
3. Apply metric_pattern regex to the full stdout string.
4. If no match found → score 0 with "metric not found in output" message.
5. Extract the value named group from the first match.
6. Attempt float() conversion on the extracted string.
7. If conversion fails → score 0 with "extracted value is not numeric" message.
8. Evaluate condition against threshold.
9. Return score 100 (pass) or 0 (fail) with a descriptive report.

---

Error Handling

Scenario	Result
Program times out	Score 0, report timeout
Program crashes	Score 0, report runtime error
Regex has no match in stdout	Score 0, report "metric not found in output" with the pattern used
Regex missing (?P<value>...) group	Score 0, report "invalid metric_pattern: must contain named group 'value'"
Extracted value is not numeric	Score 0, report "extracted value 'X' is not a valid number"
Invalid condition operator	Score 0, report "invalid condition: must be >=, <=, ==, >, or <"
Condition not met	Score 0, report "metric {value} does not satisfy {condition} {threshold}"

---

Example Usage

Basic accuracy threshold

{
  "name": "expect_metric",
  "parameters": {
    "program_command": "python3 train.py",
    "metric_pattern": "accuracy:\\s*(?P<value>[\\d.]+)",
    "condition": ">=",
    "threshold": 0.85
  },
  "weight": 40
}

RMSE upper bound

{
  "name": "expect_metric",
  "parameters": {
    "program_command": "python3 evaluate.py",
    "metric_pattern": "RMSE:\\s*(?P<value>[\\d.]+)",
    "condition": "<=",
    "threshold": 2.5
  },
  "weight": 30
}

Exact value with tolerance

{
  "name": "expect_metric",
  "parameters": {
    "program_command": "python3 stats.py",
    "metric_pattern": "mean:\\s*(?P<value>[\\d.]+)",
    "condition": "==",
    "threshold": 42.7,
    "tolerance": 0.05
  },
  "weight": 20
}

---

Implementation Notes

• Inherit from BaseExecutionTest to reuse sandbox execution and error checking.
• Validate metric_pattern compiles as valid regex before execution. If invalid, fail early with a clear message.
• Validate that the regex contains a named group called value — check via re.compile(pattern).groupindex.
• Validate condition is one of the 5 allowed operators during execute().
• Use re.search() (not re.match()) so the pattern can match anywhere in stdout.
• If stdout contains multiple matches, use the first match.
• All user-facing messages must go through the t() translation function.

---

Translation Keys Needed

• data_science.expect_metric.description
• data_science.expect_metric.report.success
• data_science.expect_metric.report.metric_not_found
• data_science.expect_metric.report.invalid_pattern
• data_science.expect_metric.report.not_numeric
• data_science.expect_metric.report.invalid_condition
• data_science.expect_metric.report.condition_not_met

---

Acceptance Criteria

• Test function registered in DataScienceTemplate.tests dict with key "expect_metric"
• Passes when extracted metric satisfies the condition
• Fails with informative message when regex doesn't match stdout
• Fails when extracted value is not a valid float
• All 5 condition operators work correctly (>=, <=, ==, >, <)
• Tolerance is applied correctly for == comparisons
• Invalid regex pattern is caught and reported gracefully
• Missing (?P<value>...) group is detected and reported
• Unit tests cover all success/failure/edge-case scenarios
• Translation keys added for pt-BR and en