Skip to content

[M5] Build AI 3D evaluation harness for regression samples #34

Description

@weiuou

Goal

Create an offline evaluation process for comparing Meshy, TripoSR, Hunyuan3D, TRELLIS, and procedural fallback against the existing regression sample manifest.

Scope

  • Add a documented evaluation harness that reads docs/qa/regression-sample-suite.json positive samples.
  • Record provider success, mesh parseability, vertex/face counts, paperability repair, decimation, unfolding, export success, page count, part count, and fallback status.
  • Add a result template for manual ratings: subject similarity, cutability, and whether the sample should enter manual paper QA.
  • Do not require actual sample binaries in the repo.

Acceptance Criteria

  • Evaluation can be run in dry-run/mock mode in CI or local tests.
  • Output format is stable enough to compare providers across runs.
  • Documentation explains how to run Meshy API and local model candidates.

Metadata

Metadata

Assignees

No one assigned

    Labels

    aiAI model and provider integrationalgorithmImage, geometry, and generation workqaTesting and acceptance workresearchResearch, evaluation, and model comparison

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions