Canine Intelligence

Canine Intelligence is a local-first computer-vision application that locates up to five dogs, predicts one of 133 breed labels, and estimates four visible expression patterns. It supports image upload, drag and drop, camera capture, and adaptive live-camera polling.

Expression output describes visual patterns only. It is not a veterinary, emotional, medical, or behavioral assessment.

Ringkasan Bahasa Indonesia: aplikasi ini mendeteksi hingga lima anjing dalam satu foto, menampilkan tiga kandidat ras dari 133 label, dan mengestimasi empat pola ekspresi visual. Aplikasi dapat berjalan lokal, tidak sengaja menyimpan foto yang diunggah, serta menandai hasil yang tidak pasti. Bobot model tidak disertakan di Git dan harus disediakan secara terpisah.

Contents

• What is included
• Inference architecture
• Requirements
• Local setup
• Running the application
• Health endpoints
• Prediction API
• Configuration
• Testing and quality checks
• Evaluation and calibration
• Docker and deployment
• Project structure
• Known limitations
• Troubleshooting
• Privacy and security
• Contributing
• License

What is included

• YOLOv8 dog detection with a lower-confidence retry for difficult portraits.
• Explicitly marked full-image fallback when detector recall is insufficient and the breed classifier is highly decisive.
• Batched breed and expression inference for multi-dog images.
• Top-three breed candidates, Top-1/Top-2 margin, and separate detector/breed/expression uncertainty.
• Interactive bounding boxes and result cards with per-dog crop thumbnails.
• Responsive offline UI with English/Indonesian switching, keyboard navigation, focus-trapped modal, search, and reduced-motion support.
• Strict decoded-image validation, pixel/byte limits, request IDs, security headers, and in-process inference rate limiting.
• Separate liveness and readiness endpoints, model warm-up, structured timing data, and a production Waitress entrypoint.
• Unit/contract tests, GitHub Actions, Dependabot, Docker assets, detector recall evaluation, and confidence calibration tooling.

Inference architecture

JPG / PNG / WEBP
        │
        ▼
Validate bytes, decoded format, EXIF orientation, and pixel count
        │
        ▼
YOLOv8n dog detector (COCO class 16)
        │
        ├── primary detection ≥ 0.35
        ├── low-confidence retry ≥ 0.10 (marked uncertain)
        └── optional full-image candidate
                  │ accepted only by a high breed score + margin
                  ▼
       Batch all dog crops once per classifier
        ├── MobileNetV2 → 133 breed labels + Top 3 + margin
        └── EfficientNetB3 → 4 visual expression scores
                  │
                  ▼
       Versioned JSON + uncertainty + per-stage timings

The full-image fallback is deliberately labeled detection_uncertain. It improves portfolio usability but is not a substitute for a separately validated dog/non-dog or fine-tuned detection model.

Requirements

• Python 3.11 (recommended and used by CI).
• Git and approximately 8 GB of free disk space for the Python environment and model artifacts.
• A 64-bit operating system. Windows, Linux, and macOS are suitable for local development.
• The three model artifacts listed below. They are intentionally excluded from Git because of their size.
• A modern browser with camera permission for live capture. Camera access outside localhost normally requires HTTPS.

CPU inference is supported. A compatible GPU can reduce latency, but GPU setup depends on the TensorFlow, PyTorch, driver, and operating-system combination and is outside the repository's default installation.

Local setup

Windows PowerShell

git clone https://github.com/LaboNapitupulu/Dog_Classifier.git
cd Dog_Classifier
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Linux or macOS

git clone https://github.com/LaboNapitupulu/Dog_Classifier.git
cd Dog_Classifier
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

For notebook-based training and evaluation, install the optional packages after the runtime dependencies:

python -m pip install -r requirements-training.txt

Model artifacts

Place the ignored artifacts at:

Dog_Classifier/
├── yolov8n.pt
└── models/
    ├── best_dog_breed_model.h5
    └── best_dog_emotion_model.h5

Expected architectures, input sizes, preprocessing functions, thresholds, label hashes, and artifact SHA-256 values are tracked in models/model_metadata.json. Only use weights obtained from a trusted source. The application will refuse artifacts whose model output shape does not match the persisted label order.

This repository currently does not publish model-weight download URLs. To reproduce the exact application, obtain the matching artifacts from the repository owner. To create new artifacts, follow the notebooks and evaluation guide, then update the metadata and hashes as a new model release. Do not commit large weights directly to Git.

Validate the tracked labels, metadata, and local artifact hashes:

python scripts/validate_label_order.py
python scripts/validate_model_metadata.py --verify-artifacts

Running the application

Development server:

python app.py

Production-style local server:

python scripts/run_production.py

Open http://127.0.0.1:5000.

The development server is intended for local debugging only. Use the Waitress entrypoint or the Docker image for production-style serving. On the first readiness request, model loading and warm-up may take tens of seconds on a CPU-only machine.

Health endpoints

Endpoint	Meaning
GET /health/live	Process is alive; does not load models.
GET /health/ready	Loads, validates, and optionally warms every model; returns 503 until ready.
GET /health	Backward-compatible alias for readiness.

Prediction API

POST /predict expects multipart field file with JPG, PNG, or WEBP content.

Example with curl:

curl -X POST http://127.0.0.1:5000/predict \
  -H "X-Request-ID: local-demo-001" \
  -F "file=@dog.jpg"

PowerShell users should call curl.exe explicitly because older PowerShell versions alias curl to Invoke-WebRequest:

curl.exe -X POST http://127.0.0.1:5000/predict `
  -H "X-Request-ID: local-demo-001" `
  -F "file=@dog.jpg"

Successful API v2 responses contain:

• results[] with stable per-dog IDs and bounding boxes.
• detector source, detector confidence, and detection_uncertain.
• breed score, Top-1/Top-2 margin, Top 3 candidates, and breed_uncertain.
• expression scores and emotion_uncertain.
• fallback_used, calibration_status, and stage-level timings_ms.

Abbreviated successful response:

{
  "api_version": 2,
  "is_dog": true,
  "fallback_used": false,
  "calibration_status": "uncalibrated",
  "original_width": 1600,
  "original_height": 1067,
  "results": [
    {
      "id": 1,
      "box": [214, 96, 1320, 1010],
      "detection_source": "primary",
      "detection_confidence": 0.91,
      "detection_uncertain": false,
      "breed": "Golden Retriever",
      "breed_confidence": 0.74,
      "breed_margin": 0.52,
      "breed_uncertain": false,
      "top_breeds": [
        {"breed": "Golden Retriever", "confidence": 0.74},
        {"breed": "Labrador Retriever", "confidence": 0.22}
      ],
      "emotion": "happy",
      "emotion_confidence": 0.68,
      "emotion_uncertain": false,
      "probabilities": {
        "angry": 0.04,
        "happy": 0.68,
        "relaxed": 0.23,
        "sad": 0.05
      }
    }
  ],
  "timings_ms": {
    "detector": 142.4,
    "breed": 81.7,
    "expression": 95.2,
    "total": 320.1
  }
}

Scores above are illustrative, not an accuracy claim. Floating-point values and latency vary by image and hardware.

API status codes

Status	Code	Meaning
200	—	Prediction completed or a health endpoint is ready.
400	missing_file, empty_filename, unsupported_extension, invalid_image	The upload is missing or cannot be safely decoded.
413	file_too_large	Request exceeds the 16 MB upload limit.
422	no_dog_detected	No dog or acceptable fallback candidate was found.
429	rate_limited	Per-process request window is full; inspect Retry-After.
500	model_load_failed, prediction_failed	Unexpected model or inference failure; inspect server logs and X-Request-ID.
503	models_unavailable	One or more required artifacts are absent or invalid.

All responses include X-Request-ID. Supplying the same request header makes it easier to correlate client failures with server logs.

Configuration

Use .env.example as a reference and export the required values in the shell or process manager. The application does not automatically load .env files.

Variable	Default	Purpose
HOST	127.0.0.1	Waitress bind address. Use 0.0.0.0 only behind an appropriate firewall/proxy.
PORT	5000	Waitress listening port.
LOG_LEVEL	INFO	Python log level for the production entrypoint.
WEB_THREADS	4	Waitress request threads. Model execution remains protected by locks.
WEB_CHANNEL_TIMEOUT	120	Waitress idle channel timeout in seconds.
DOG_CLASSIFIER_DEBUG	false	Enable Flask debug mode only when running app.py; never enable publicly.
DOG_CLASSIFIER_WARMUP	true	Compile model inference graphs before serving traffic.
DOG_CLASSIFIER_FULL_IMAGE_FALLBACK	true	Allow explicitly uncertain full-image fallback.
DOG_CLASSIFIER_RATE_LIMIT	true	Enable the local sliding-window limiter.
DOG_CLASSIFIER_RATE_LIMIT_REQUESTS	120	Requests allowed per window/IP/process.
DOG_CLASSIFIER_RATE_LIMIT_WINDOW	60	Rate-limit window in seconds.

For multi-instance public deployment, enforce rate limits at the gateway or a shared store; the built-in limiter is intentionally process-local.

Example for one PowerShell session:

$env:HOST = "127.0.0.1"
$env:PORT = "5000"
$env:LOG_LEVEL = "INFO"
python scripts/run_production.py

Fixed safety limits

The current API accepts JPG, PNG, and WEBP files up to 16 MB and rejects decoded images above 25 million pixels. Up to five detected dogs are returned per image. These limits are intentionally code-controlled rather than client-controlled.

Testing and quality checks

The lightweight test environment avoids downloading TensorFlow, PyTorch, and model weights in CI by replacing inference modules with deterministic fakes. This validates the Flask contract, upload safety, error handling, uncertainty flags, and critical frontend behavior. Real-model smoke tests remain a separate local responsibility.

Install test dependencies and run the same checks used by GitHub Actions:

python -m pip install -r requirements-test.txt
ruff check app.py src scripts tests
python -m compileall -q app.py src scripts tests
python scripts/validate_model_metadata.py
python -m unittest discover -s tests -v
node --check static/js/script.js

With the local weights present, also verify their hashes and label compatibility:

python scripts/validate_model_metadata.py --verify-artifacts
python scripts/validate_label_order.py

CI runs on pushes to main, pull requests, and manual workflow dispatch. Routine Dependabot version-update PRs are disabled to keep the repository's branch list focused; dependency upgrades are reviewed and tested manually. Repository security updates can still be managed through GitHub's Dependabot security settings.

Evaluation and calibration

The repository does not claim production-grade accuracy. Existing model weights are portfolio artifacts and the checked-in thresholds are explicitly marked uncalibrated.

The training notebooks now include a low-learning-rate fine-tuning stage with frozen Batch Normalization. Breed training uses train/validation/test directories. Expression training requires an independent data/emotion_test/<class> set before final reporting.

Detector recall smoke test on a known-dog directory:

python scripts/evaluate_detector_recall.py data/breeds/test --sample 100 --seed 42

Fit temperature scaling and suggest an abstention threshold from an NPZ containing probabilities [N,C] and integer labels [N]:

python scripts/calibrate_confidence.py evaluation/breed_predictions.npz --target-accuracy 0.80 --output evaluation/breed_calibration.json

See Model evaluation guide before publishing accuracy or confidence claims.

Docker and deployment

Model artifacts are excluded from the build context. Supply them from a trusted artifact store or mount them at runtime.

docker build -t canine-intelligence .
docker run --rm -p 5000:5000 `
  -v "${PWD}\models:/app/models:ro" `
  -v "${PWD}\yolov8n.pt:/app/yolov8n.pt:ro" `
  canine-intelligence

See Deployment guide for readiness, resources, logging, reverse proxies, and model artifact handling.

Project structure

Dog_Classifier/
├── app.py                         # Flask API, safety controls, readiness
├── src/
│   ├── inference.py               # Detection, batching, fallback, warm-up
│   └── breeds.py                  # Persisted label order
├── templates/ + static/           # Offline bilingual accessible UI
├── models/model_metadata.json     # Labels, preprocessing, hashes, thresholds
├── notebooks/                     # Reproducible training/fine-tuning flows
├── scripts/                       # Validation, calibration, evaluation, serving
├── tests/                         # API and frontend contracts
├── .github/                       # CI and dependency updates
└── Dockerfile

Known limitations

• Breed and expression scores are currently uncalibrated. A score of 0.80 must not be interpreted as an 80% real-world probability.
• The 133 labels are closed-set classes. Mixed breeds, unsupported breeds, unusual grooming, puppies, and domain-shifted images may produce misleading candidates.
• Expression labels describe visible image patterns only and cannot establish emotion, pain, temperament, intent, health, or welfare.
• YOLOv8n uses the generic COCO dog class rather than a detector fine-tuned for this application's image domain.
• Full-image fallback can improve recall for close portraits but is intentionally marked uncertain and may accept a confident non-dog classification error.
• TensorFlow and PyTorch are loaded in the same process, which increases cold-start time, RAM use, and container size.
• The built-in rate limiter is process-local and is not sufficient by itself for horizontally scaled public deployments.
• Uploaded images are processed synchronously; the current server does not provide a job queue or streaming API.

Do not use this application for veterinary, safety-critical, legal, breeding, insurance, or animal-behavior decisions.

Troubleshooting

Readiness returns 503

Call GET /health/ready and inspect missing_models. Confirm the three artifact paths, then run:

python scripts/validate_model_metadata.py --verify-artifacts

Replace any file whose hash or output dimensions do not match the tracked metadata, then restart the process. A previous model-load failure is cached for process safety.

First request is slow

Model import, loading, and graph warm-up are expensive on CPU. Keep DOG_CLASSIFIER_WARMUP=true, wait for /health/ready before sending traffic, and use the production entrypoint. Additional Waitress threads do not create extra model copies, but additional processes do.

Camera does not open

Grant browser permission and close other applications using the camera. Browser camera APIs generally require localhost or an HTTPS origin. File upload remains available when camera access is unavailable.

Upload is rejected

Confirm the extension is .jpg, .jpeg, .png, or .webp, the encoded file is no larger than 16 MB, and the decoded dimensions stay below 25 million pixels. Renaming another file type to an image extension will not bypass content validation.

No dog is detected

Use an evenly lit image that includes the face and body, avoids heavy occlusion, and leaves some space around the dog. A 422 response is an abstention, not proof that the image contains no dog.

Dependency installation fails

Use a fresh Python 3.11 virtual environment and update pip first. TensorFlow and PyTorch compatibility varies by Python version and GPU setup; reproduce the pinned CPU-capable environment before adding accelerator-specific packages.

Privacy and security

Uploaded images are decoded in memory and are not intentionally persisted. Avoid logging images, raw multipart data, or personal metadata. Public deployments should terminate TLS at a reverse proxy, add shared rate limiting, monitor resource use, and keep model artifacts read-only.

See SECURITY.md for reporting and deployment boundaries.

Contributing

1. Create a short-lived branch from the latest main.
2. Keep model weights, datasets, uploaded images, secrets, and generated evaluation outputs out of Git.
3. Add or update tests for every API or frontend-contract change.
4. Run the complete quality-check sequence above.
5. Open a focused pull request describing behavior changes, model/data provenance, and validation results.

The repository intentionally keeps main as its only long-lived branch. Model changes should include updated metadata, hashes, evaluation protocol, and a clear statement of limitations; accuracy claims require an independent labeled test set.

License

Released under the MIT License. Model artifacts and datasets may have their own licenses; verify those terms before redistribution or commercial use.