Data Science Template: implement expect_stdout_value test function

[ArthurCRodrigues]

Parent Issue

Part of #338 — Feature: Data Science Template (data_science)

---

Overview

Implement the expect_stdout_value test function that executes the student's program, extracts a specific value from stdout using a regex pattern, and validates it against an expected value with tolerance. This is a lightweight test for quick validation of printed statistics, counts, or derived values without requiring file artifacts.

---

Interface

class ExpectStdoutValueTest(BaseExecutionTest):
    @property
    def name(self):
        return "expect_stdout_value"

---

Parameters

Parameter	Type	Required	Description
program_command	string/dict	✓	Command to execute. Supports multi-language dict and "CMD" auto-resolution.
extraction_pattern	string	✓	Regex with named group (?P<value>...) to extract the target value from stdout.
expected_value	float or string	✓	The expected value. If numeric (int/float), tolerance-based comparison is used. If string, exact match after strip.
tolerance	float	✗	Numeric tolerance (default: 0.001). Ignored when expected_value is a string.
inputs	list[string]	✗	Stdin inputs if the program requires interactive input.

---

Scoring Logic

Binary scoring:

• 100 if extracted value matches expected_value (within tolerance for numerics).
• 0 if value doesn't match, cannot be extracted, or program fails.

Value comparison rules:

• If expected_value is numeric (int or float): extract value, convert to float, compare with abs(actual - expected) <= tolerance.
• If expected_value is a string: extract value, strip whitespace, compare with ==.

---

Execution Flow

1. Run program_command in the sandbox via run_sandbox_execution.
2. Check for base errors (timeout, compilation, runtime) → score 0.
3. Validate extraction_pattern compiles and contains (?P<value>...) group.
4. Apply regex to stdout using re.search().
5. If no match → score 0 with "value not found in output" message.
6. Extract the value named group.
7. Determine comparison type based on expected_value type:
   • If expected_value is int or float: convert extracted string to float and compare with tolerance.
   • If expected_value is str: strip both sides and compare with ==.
8. Return score 100 (match) or 0 (mismatch) with descriptive report.

---

Difference from expect_metric

Aspect	expect_metric	expect_stdout_value
Purpose	Threshold validation (>= 0.85)	Exact value matching (== 34.7)
Condition	Configurable operator (>=, <=, ==, >, <)	Always equality (with tolerance)
Value type	Always numeric	Numeric or string
Use case	"Is accuracy good enough?"	"Is the computed mean correct?"

expect_metric answers "does this metric meet a threshold?" while expect_stdout_value answers "did the student compute the correct value?"

---

Error Handling

Scenario	Result
Program times out	Score 0, report timeout
Program crashes	Score 0, report runtime error
Regex doesn't match stdout	Score 0, report "value not found in output" with the pattern
Regex missing (?P<value>...) group	Score 0, report "invalid extraction_pattern"
Extracted value not convertible to float (when expected is numeric)	Score 0, report "extracted value 'X' is not numeric"
Value doesn't match	Score 0, report expected vs actual

---

Example Usage

Numeric value with tolerance

{
  "name": "expect_stdout_value",
  "parameters": {
    "program_command": "python3 stats.py",
    "extraction_pattern": "mean_age:\\s*(?P<value>[\\d.]+)",
    "expected_value": 34.7,
    "tolerance": 0.1
  },
  "weight": 20
}

Integer count (exact)

{
  "name": "expect_stdout_value",
  "parameters": {
    "program_command": "python3 analysis.py stats",
    "extraction_pattern": "total_rows:\\s*(?P<value>\\d+)",
    "expected_value": 1500,
    "tolerance": 0
  },
  "weight": 25
}

String value

{
  "name": "expect_stdout_value",
  "parameters": {
    "program_command": "python3 classify.py",
    "extraction_pattern": "best_model:\\s*(?P<value>\\w+)",
    "expected_value": "RandomForest"
  },
  "weight": 15
}

---

Implementation Notes

• Inherit from BaseExecutionTest to reuse sandbox execution and error checking.
• Type detection for expected_value: check if it's int or float → numeric path; otherwise → string path.
• For the numeric path, convert extracted string with float(). Catch ValueError for non-numeric extractions.
• Validate extraction_pattern at the start of execute(): compile regex and check "value" in pattern.groupindex.
• Use re.search() (not re.match()) to allow matching anywhere in stdout.
• If multiple matches exist, use the first match.
• All user-facing messages must go through the t() translation function.

---

Translation Keys Needed

• data_science.expect_stdout_value.description
• data_science.expect_stdout_value.report.success
• data_science.expect_stdout_value.report.value_not_found
• data_science.expect_stdout_value.report.invalid_pattern
• data_science.expect_stdout_value.report.not_numeric
• data_science.expect_stdout_value.report.mismatch

---

Acceptance Criteria

• Test function registered in DataScienceTemplate.tests dict with key "expect_stdout_value"
• Numeric comparison with tolerance works correctly
• String comparison with strip works correctly
• Fails with informative message when regex doesn't match
• Fails when extracted value cannot be converted to float (numeric mode)
• Invalid pattern (no named group) is detected and reported
• Type detection correctly distinguishes numeric vs string expected_value
• Unit tests cover numeric match, numeric mismatch, string match, string mismatch, regex failures
• Translation keys added for pt-BR and en