Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -100,13 +100,17 @@ start-autograder:


# Sandbox Management
sandbox-build-all: sandbox-build-python sandbox-build-java sandbox-build-node sandbox-build-cpp
sandbox-build-all: sandbox-build-python sandbox-build-pyds sandbox-build-java sandbox-build-node sandbox-build-cpp
@echo "✅ All sandbox images built successfully!"

sandbox-build-python:
@echo "Building Python sandbox image..."
docker build -t sandbox-py:latest -f sandbox_manager/images/Dockerfile.python sandbox_manager/images/

sandbox-build-pyds:
@echo "Building Python Data Science sandbox image..."
docker build -t sandbox-pyds:latest -f sandbox_manager/images/Dockerfile.python-ds sandbox_manager/images/

sandbox-build-java:
@echo "Building Java sandbox image..."
docker build -t sandbox-java:latest -f sandbox_manager/images/Dockerfile.java sandbox_manager/images/
Expand Down
16 changes: 15 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,21 @@ Validates HTML, CSS, and JavaScript files.
| `check_css_property` | Validate CSS rules | `selector`, `property`, `expected_value` |

And much more! Check the [WebDev Template Documentation](docs/templates/web_dev.md) for the full list of tests.
#### 4. Custom Templates

#### 4. Data Science Template
Validates data science assignments including ML model metrics, JSON/CSV outputs, and generated artifacts. Requires Python DS sandbox.

| Test Name | Description | Key Parameters |
|-----------|-------------|----------------|
| `expect_stdout_value` | Extract a value from stdout and compare to an expected value | `extraction_pattern`, `expected_value`, `tolerance` |
| `expect_metric` | Extract metric from stdout and validate vs threshold | `metric_pattern`, `condition`, `threshold` |
| `expect_csv_output` | Validate generated CSV file contents | `artifact_path`, `expected_columns`, `expected_shape`, `expected_values` |
| `expect_json_output` | Validate generated JSON structure and keys | `artifact_path`, `required_keys`, `expected_values` |
| `expect_model_artifact` | Verify a model file was produced | `artifact_path`, `min_size_bytes` |
Comment on lines +228 to +234

Check the [Data Science Template Documentation](docs/template-library/data_science.md) for full details.

#### 5. Custom Templates
Upload your own test functions for specialized grading contexts:

```python
Expand Down
3 changes: 2 additions & 1 deletion autograder/services/command_resolver.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ class CommandResolver:
# Default command patterns for each language
DEFAULT_COMMANDS = {
Language.PYTHON: "python3 {filename}",
Language.PYTHON_DS: "python3 {filename}",
Language.JAVA: "java {classname}",
Language.NODE: "node {filename}",
Language.CPP: "./{executable}",
Expand Down Expand Up @@ -105,7 +106,7 @@ def _auto_resolve_command(
This is used when program_command is "CMD" placeholder.
"""
command = None
if language == Language.PYTHON:
if language in (Language.PYTHON, Language.PYTHON_DS):
command = f"python3 {fallback_filename}" if fallback_filename else "python3 main.py"
elif language == Language.JAVA:
# Java requires class name without .java extension
Expand Down
2 changes: 2 additions & 0 deletions autograder/services/template_library_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from autograder.template_library.api_testing import ApiTestingTemplate
from autograder.template_library.input_output import InputOutputTemplate
from autograder.template_library.static_analysis import StaticAnalysisTemplate
from autograder.template_library.data_science import DataScienceTemplate

logger = logging.getLogger(__name__)

Expand All @@ -37,6 +38,7 @@ class TemplateLibraryService:
"api": ApiTestingTemplate,
"input_output": InputOutputTemplate,
"static_analysis": StaticAnalysisTemplate,
"data_science": DataScienceTemplate,
}

def __new__(cls):
Expand Down
3 changes: 3 additions & 0 deletions autograder/template_library/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,12 @@
from autograder.template_library.web_dev.template import WebDevTemplate
from autograder.template_library.api_testing import ApiTestingTemplate
from autograder.template_library.input_output import InputOutputTemplate
from autograder.template_library.data_science import DataScienceTemplate

__all__ = [
"WebDevTemplate",
"ApiTestingTemplate",
"InputOutputTemplate",
"DataScienceTemplate",
]

4 changes: 2 additions & 2 deletions autograder/template_library/api_testing.py
Original file line number Diff line number Diff line change
Expand Up @@ -109,9 +109,9 @@ def execute(self, files, sandbox: SandboxContainer, endpoint: str = "", expected
data = response.json()
if data.get(expected_key) == expected_value:
score = 100
report = t("api_testing.check_response_json.report.success", locale=locale, endpoint=endpoint, key=expected_key, value=expected_value)
report = t("api_testing.check_response_json.report.success", locale=locale, endpoint=endpoint, field_key=expected_key, value=expected_value)
else:
report = t("api_testing.check_response_json.report.failure", locale=locale, endpoint=endpoint, key=expected_key, expected=expected_value, actual=data.get(expected_key))
report = t("api_testing.check_response_json.report.failure", locale=locale, endpoint=endpoint, field_key=expected_key, expected=expected_value, actual=data.get(expected_key))
except json.JSONDecodeError:
report = t("api_testing.check_response_json.report.invalid_json", locale=locale, endpoint=endpoint)

Expand Down
Loading
Loading