SharkTrackKit

SharkTrackKit is a Swift package for macOS that brings computer vision–based detection of elasmobranchs (sharks and rays) in underwater images and videos to native Swift applications.

Built as a lightweight wrapper of the open-source SharkTrack project, it provides a modern async/await API for running detections, tracking progress, collecting generated screenshots, and accessing typed detection metadata from Swift and SwiftUI. Learn more about SharkTrack here: https://github.com/filippovarini/sharktrack

Features

• Detect sharks and rays in underwater images and videos using SharkTrack
• Native async/await Swift API
• Streaming progress events for SwiftUI and AppKit
• Typed detection results and metadata
• Automatic runtime discovery and validation
• Configurable processing presets
• Support for bundled production runtimes

Requirements

• Swift 6.1 or later
• macOS 13 or later
• Xcode with Swift Package Manager support
• uv for preparing the local development runtime

The setup script installs or reuses uv for preparing the local runtime. It does not require sudo, install global Python packages, or modify the system Python installation.

Installation

Add SharkTrackKit using Swift Package Manager:

dependencies: [
    .package(url: "https://github.com/dorypiacek/SharkTrackKit.git", branch: "main")
]

Then add the library product to your target:

.product(name: "SharkTrackKit", package: "SharkTrackKit")

Runtime Setup For Development

SharkTrackKit is intentionally lightweight: the Swift package does not bundle the large SharkTrack/Python runtime.

SwiftPM package sources are usually stored in build-system-managed caches, such as Xcode's DerivedData SourcePackages/checkouts directory or command-line SwiftPM's .build/checkouts directory. Do not treat those cache locations as the runtime install location.

Install the development runtime once:

scripts/install_runtime.sh

By default, the installer writes the runtime to:

~/Library/Application Support/SharkTrackKit/runtimes/<runtime-version>/

The initial setup can take several minutes because it installs Python, native dependencies, and a pinned SharkTrack checkout. The setup does not use sudo, install global Python packages, or modify system Python.

Swift runtime discovery uses the same canonical location. A host app can use the default processor without knowing where SwiftPM cached the package:

let processor = SharkTrackProcessor()

Sandboxed Apps

The installed development runtime is intended for local development and CI. A sandboxed macOS app cannot freely read and execute this external Python virtual environment unless it has explicit access.

For day-to-day development, run the host app's Debug configuration without App Sandbox, or inject a runtime path the app is allowed to access. For sandboxed release builds, bundle a frozen, signed runtime inside the app and initialize SharkTrackKit with SharkTrackRuntime.appBundleExecutable().

Do not rely on the external development runtime as the end-user distribution model for a sandboxed app.

For tests and CI, set SHARKTRACKKIT_HOME to override the base install directory:

export SHARKTRACKKIT_HOME=/absolute/path/to/test/SharkTrackKitHome

Custom runtime directories are still supported explicitly:

let runtime = SharkTrackRuntime(location: .directory(runtimeRootURL))
let processor = SharkTrackProcessor(runtime: runtime)

You can validate setup explicitly before starting a long processing task:

let processor = SharkTrackProcessor()
try await processor.prepareRuntime()

If setup is missing, incomplete, incompatible, or broken, SharkTrackKit throws a typed SharkTrackError with the exact expected path and setup command.

Quick Start

import SharkTrackKit

let processor = SharkTrackProcessor()

let result = try await processor.process(
    input: inputMovieURL,
    output: outputDirectoryURL,
    mode: .automatic
)

print(result.screenshots)

For MaxN review workflows, use the recall-oriented preset:

let result = try await processor.process(
    input: inputMovieURL,
    output: outputDirectoryURL,
    mode: .accurate,
    options: .maxNReview
)

let maxNValues = result.screenshots.compactMap(\.maxN)
let detections = result.detections

Progress Events

Use events(input:output:mode:options:) to receive progress updates suitable for SwiftUI, AppKit, logging, or background processing.

for try await event in processor.events(input: inputURL, output: outputURL, mode: .accurate, options: .maxNReview) {
    switch event {
    case .preparingRuntime:
        status = "Preparing runtime"
    case .installingDependencies:
        status = "Installing dependencies"
    case .processingFile(let current, let total, let name):
        progress = Double(current) / Double(max(total, 1))
        status = "Processing \(name)"
    case .screenshotProcessed(let screenshot):
        screenshots.append(screenshot.url)
    case .message(let message):
        logger.info("\(message)")
    case .completed(let result):
        screenshots = result.screenshots.map(\.url)
    }
}

Public API

• SharkTrackProcessor: High-level API for one-shot and streaming processing.
• SharkTrackRuntime: Runtime discovery and validation for development, bundled, directory, and executable layouts.
• SharkTrackProcessingOptions: Configurable confidence thresholds, IoU, image size, tracker settings, frame rate, and MaxN review presets.
• SharkTrackMode: .automatic, .fast, and .accurate.
• SharkTrackEvent: Progress, runtime, dependency, processing, screenshot, message, and completion events.
• SharkTrackResult: Processed output, screenshots, metadata, duration, and statistics.
• SharkTrackScreenshot: Screenshot URL with timestamp, confidence, track ID, frame number, source video, and MaxN.
• SharkTrackDetection: Typed review candidate with screenshot URL, source media, track ID, review frame, MaxN, timestamp, confidence, and status.
• SharkTrackError: Typed errors for runtime discovery, execution, invalid input/output, and cancellation.

Runtime Packaging

SharkTrackKit supports both development and frozen runtime layouts. Both layouts must include a .runtime-version file matching SharkTrackRuntime.requiredRuntimeVersion.

Development layout created by scripts/install_runtime.sh:

~/Library/Application Support/SharkTrackKit/runtimes/<runtime-version>/
  .runtime-version
  .venv/
  bin/
    sharktrack_runner.py
  SharkTrack/

Recommended production app resource layout:

Example.app/Contents/Resources/SharkTrackRuntime/
  .runtime-version
  sharktrack-runner/
    sharktrack-runner
    _internal/

The frozen runner includes Python, SharkTrack, model files, and all required native dependencies. Bundle it with the host application, then sign and notarize it as part of the application's normal release process.

Development code can use the default runtime after setup:

let processor = SharkTrackProcessor()

For development, tests, or custom distribution pipelines, inject another runtime:

let runtime = SharkTrackRuntime(location: .directory(runtimeRootURL))
let processor = SharkTrackProcessor(runtime: runtime)

or:

let runtime = SharkTrackRuntime(location: .executable(runnerURL))
let processor = SharkTrackProcessor(runtime: runtime)

App releases that bundle the frozen runtime should use:

let runtime = try SharkTrackRuntime.appBundleExecutable()
let processor = SharkTrackProcessor(runtime: runtime)

Building the Runner

To build the frozen runner, provide a SharkTrack source checkout and Python environment:

SHARKTRACK_SOURCE=/path/to/sharktrack \
SHARKTRACK_VENV=/path/to/sharktrack/.venv \
BuildSupport/build_runner.sh

SHARKTRACK_VENV is optional when the virtual environment lives at $SHARKTRACK_SOURCE/.venv.

The output is copied into:

.runtime/SharkTrackRuntime/sharktrack-runner

See Documentation/BuildingRuntime.md and Documentation/RuntimeContract.md for additional runtime details.

Testing

Run:

swift test

Unit tests use lightweight mock runtimes, allowing CI to verify process execution, progress parsing, result parsing, screenshot discovery, and MaxN metadata handling without running the full ML runtime.

License

The Swift package source is licensed under the MIT License. Bundled runtimes, model files, and third-party Python dependencies remain subject to their respective licenses.