Evidence-first industrial visual anomaly detection and pixel localization. ForgeSight-AD compares PatchCore, DINOv2 nearest-neighbour features, PaDiM, and a fixed Train-only fusion rule on all 15 MVTec AD categories.
在线作品集 · 中文一页案例 · 中文简历与面试材料 · 完整结果说明
The practical benchmark fits category-specific one-class models from 3,629 Train-normal images and evaluates all 1,725 official Test images and masks. Values below are category macro means.
| Method | Image AUROC | Pixel AUROC | Pixel AP | AUPRO@0.30 |
|---|---|---|---|---|
| PatchCore, WRN50-2 | 0.9664 | 0.9638 | 0.5676 | 0.8911 |
| DINOv2-B/14 nearest neighbour | 0.9612 | 0.9443 | 0.5228 | 0.8433 |
| Tail-gated PatchCore + DINOv2 | 0.9728 | 0.9705 | 0.4852 | 0.9020 |
| PaDiM, ResNet-18 | 0.9550 | 0.9747 | 0.5479 | 0.9237 |
The result is deliberately not summarized as “one model wins”:
- the fixed fusion rule gives the strongest image AUROC and improves PatchCore AUPRO by 0.01084 on average, with positive deltas on 9/15 categories;
- PaDiM gives the strongest localization result and beats PatchCore AUPRO on 10/15 categories;
- fusion Pixel AP falls below PatchCore, so the evidence supports a particular overlap/image-ranking trade-off, not universal dominance.
No package, model, dataset, or network download is needed:
git clone https://github.com/nikashen/ForgeSight-AD.git
cd ForgeSight-AD
python -m http.server 8015 --directory docsOpen http://127.0.0.1:8015/. The static page contains only aggregate benchmark facts, architecture, trade-offs, and public-release boundaries. It does not pretend to run model inference.
The repository wrappers provide the same review mode:
.\run_project.ps1 -Task reviewOn macOS/Linux, run ./run_project.sh review.
flowchart LR
A[Train-normal images] --> B[Aspect-preserving 280px transform]
B --> C1[WRN50-2 PatchCore]
B --> C2[DINOv2 NN]
B --> C3[ResNet-18 PaDiM]
C1 --> D[Held-out normal calibration]
C2 --> D
D --> E[Fixed tail-gated fusion]
C1 --> F[Image score + pixel map]
C3 --> F
E --> F
F --> G[Metrics, reports, local FastAPI Lab]
Engineering highlights:
- Leakage-safe calibration: fixed seed-17 Train-normal fold for q95 thresholds and branch calibration; Test scores, labels, and masks do not tune the threshold or fusion constants.
- Bounded memory: PatchCore and DINOv2 deterministically pre-cap at 20,000 patch candidates before a 1% greedy coreset; PaDiM uses streaming covariance statistics and chunked precision construction.
- Pixel-correct evaluation: aspect-preserving resize and reflection padding to a 280 × 280 canvas; padding is excluded from practical pixel metrics.
- Failure evidence: the earlier lower-CDF fusion regressed on 15/15 categories and remains documented; the fixed tail gate was designed before Test scoring.
- Interactive prototype: validated bundle loading, upload/camera input checks, lazy category activation, sequential inference, heatmap/overlay output, and explicit production boundaries.
The public JSON reports are path-sanitized projections of two private clean-run reports. Metric arrays, category counts, aggregate values, runtime records, configuration, hashes, and source commit identifiers are preserved. Only the machine-local dataset root and split-manifest path were changed.
| Public report | Original private SHA-256 | Source commit |
|---|---|---|
| Three-method benchmark | e1fae8e5…e246c7d | 911322b4…a976e0 |
| PaDiM benchmark | 2a348d40…709879 | 977b2b24…a7ab6d |
The exact original-to-public hash mapping and complete transformation list are recorded in the publication manifest. The private source commits are provenance identifiers; they are intentionally not reachable from this sanitized single-root history. Therefore this release does not claim that the reports were freshly reproduced from the public commit.
See public release boundary, experiment protocol, metrics, and data provenance.
The complete local setup adds real category-specific PatchCore inference to the benchmark dashboard. It is separate from the zero-install public review because it requires MVTec AD, pretrained weights, and locally generated model bundles.
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install --upgrade pip
.\.venv\Scripts\python.exe -m pip install -e ".[serve]"
# Obtain MVTec AD and weights under their upstream terms, then fit 15 bundles.
.\.venv\Scripts\forgesight.exe live-export --device cuda
# Start the local inference and benchmark interface.
.\.venv\Scripts\forgesight.exe demo --host 127.0.0.1 --port 8015macOS/Linux uses the same commands with .venv/bin/python and .venv/bin/forgesight. Open http://127.0.0.1:8015/demo.
The recorded RTX 3050 Ti diagnostic covers 15 local bundles and 150 warm calls:
- model prediction latency: 58.24 / 110.60 ms p50/p95;
- complete serialized service latency: 340.20 / 399.15 ms p50/p95;
- estimated sequential throughput: 2.94 FPS;
- peak CUDA reserved memory: 358 MiB.
These measurements exclude HTTP transport, browser rendering, camera capture, concurrency, factory I/O, and production SLA. They are a local diagnostic, not a real-time production claim. See the machine-readable report.
The measured Windows reference environment is Python 3.10, PyTorch 2.6.0+cu118, torchvision 0.21.0+cu118, NumPy 1.26.4, SciPy 1.15.3, scikit-learn 1.5.1, and Pillow 11.3.0.
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install --upgrade pip
.\.venv\Scripts\python.exe -m pip install -e ".[dev,serve]"
.\.venv\Scripts\python.exe -m pytest -qCI downloads neither MVTec data nor pretrained weights. Real GPU benchmarks, formal Development selection, Frozen Final evaluation, and deployment measurements remain separate evidence-producing workflows.
After preparing the upstream dataset and weights, the runner checkpoints after every category and resumes only when all settings and input hashes match.
.\.venv\Scripts\python.exe -B -m forgesight.resume_runner benchmark --device cuda --output artifacts\resume\practical-benchmark\reproduction.json
.\.venv\Scripts\python.exe -B -m forgesight.resume_runner benchmark --methods padim_resnet18 --device cuda --output artifacts\resume\practical-benchmark\padim-reproduction.jsonOn the measured laptop GPU, the three-method run took 1003.8 seconds and stayed below 854 MiB CUDA reserved memory. The PaDiM run took 466.1 seconds and stayed below 142 MiB.
- This is a fixed-seed practical benchmark, not a multi-seed significance study and not a completed Formal Frozen Final.
- Metrics are measured on MVTec AD, not on a target factory camera, lighting, product revision, or defect-cost distribution.
- The practical pixel metrics use the valid resized 280 × 280 region, not original-resolution masks.
- The local service is serialized and category-specific. It has not validated batch processing, high concurrency, 30 FPS, ONNX/TensorRT, edge deployment, authentication, observability, or production SLA.
- MVTec AD images/masks, downloaded weights, memory banks, model bundles, logs, and predictions are not included in this public repository.
Repository-authored code is MIT licensed. Dataset, pretrained-weight, and Python package terms remain independent; see third-party notices.