OverlapIndex (OI)

This package provides an implementation of the Overlap Index (OI), a cluster-validity measure designed to quantify the degree of overlap between data classes or clusters. The OI can be updated online with ARTMAP-based backends, or computed in batch with offline clustering backends, making it useful for streaming, continual learning, large-scale representation analysis, and embedding-space diagnostics.

The implementation supports multiple swappable clustering backends:

• Fuzzy ARTMAP and Hypersphere ARTMAP for incremental / online updates.
• KMeans and MiniBatchKMeans for offline centroid-based analysis.
• BallCover for offline greedy landmark-ball covers, useful when the goal is to preserve class-support geometry for downstream shape or topology analysis.

---

Installation

To install OverlapIndex, simply use pip:

pip install overlapindex

That installs the default batch-oriented dependencies. To enable the incremental ART backends as well, install the optional ART extra:

pip install "overlapindex[art]"

The core package and optional art extra support Python 3.9 through 3.14.

Or to install directly from the most recent source:

pip install git+https://github.com/NiklasMelton/OverlapIndex.git@develop

The complete user guide and source-generated API reference are published on Read the Docs.

To validate the documentation locally:

python -m pip install -r docs/requirements.txt
python -m pip install -e .
sphinx-build --fail-on-warning --keep-going -b html docs docs/_build/html

---

Overview

The Overlap Index is bounded in the interval [0, 1] and has the following interpretation:

• OI = 1.0
  Indicates perfect class separation (no overlap).

• 0.0 < OI < 1.0 Indicates partial overlap or separation at the fitted prototype resolution.

• OI = 0.0 Indicates complete overlap between classes.

The index is computed incrementally by tracking shared cluster activations between pairs of classes and aggregating class-wise overlap into a global measure.

---

Key Properties

• Incremental and Offline Modes
  ARTMAP backends support streaming updates via add_sample and mini-batch updates via add_batch. Offline backends such as KMeans, MiniBatchKMeans, and BallCover support batch computation through add_batch.

• Label-Aware
  Can be applied both to labeled raw data and to intermediate representations (e.g., neural network activations).

• Geometry-Agnostic
  Works well on arbitrary geometric structures of data. No geometric constraints are assumed.

---

Visual Behavior

The examples below sweep two synthetic populations from fully interleaved to well separated. Gaussian clouds and vertical bars vary the distance between their centers; concentric rings vary the difference between their radii. Each response curve is the mean across repeated deterministic draws, and the shaded band shows one standard deviation.

Regenerate the discrete figure from the repository root with:

poetry run python examples/visualize_discrete_overlap_sweeps.py

The script writes img/discrete_overlap_sweeps.png by default and accepts --output PATH for a different destination.

---

Typical Use Cases

The Overlap Index can be used in several settings:

• Unsupervised clustering evaluation
  As an iCVI, OI provides insight into the quality of a clustering partition as it evolves over time.

• Class separability analysis
  Measures the degree of overlap in labeled datasets without requiring a classifier.

• Representation monitoring in deep learning
  Tracks how class separation changes across layers or training epochs.

• Backbone evaluation for transfer learning
  Compares feature extractors, where higher OI values indicate better class separation in the backbone embeddings.

---

Implementation Notes

• ART-based clustering is performed using artlib’s FuzzyARTMAP or HypersphereARTMAP.
• artlib is an optional dependency and is only required when using the "Fuzzy" or "Hypersphere" backends.
• Offline centroid backends fit one clustering model per class and concatenate the resulting class-owned prototypes into global cluster ids.
• The BallCover backend fits one greedy ball cover per class and treats ball centers as class-owned prototypes.
• KMeans and MiniBatchKMeans accept SciPy sparse feature matrices and normalize sparse formats to CSR internally. BallCover and ART backends require dense feature arrays.
• Sparse feature matrices remain sparse during fitting, slicing, prediction, multi-label expansion, and overlap scoring. KMeans centroids and the bounded prototype-distance score blocks are still dense.
• Normalize input features before fitting. Examples in this repository use MinMaxScaler for convenience.
• ART backends complement-code inputs internally and therefore require features in the [0, 1] interval.
• Offline backends (KMeans, MiniBatchKMeans, and BallCover) consume normalized features directly and do not apply complement coding.
• Overlap is estimated by monitoring shared best-matching units (BMUs) or top prototype activations between class pairs.
• The global OI is computed as the macro mean of per-class minimum pairwise overlap scores, so each observed class contributes equally to index.
• A support-weighted companion score is available through weighted_index for workflows that need the score to reflect observed class frequencies.
• Global aggregation can exclude one or more label ids through exclude_classes without removing those labels from fitting, singleton scores, or pairwise scores.
• Offline backends support multi-label targets supplied as per-sample label collections or as dense/sparse binary indicator matrices. Rectangular Python lists retain collection semantics; convert them to NumPy or SciPy to request indicator-matrix semantics explicitly.
• Multi-label directional pairs are evaluated only where the source label is present and the competitor is absent. Pairs without such rows are exposed in unevaluable_pairs_; source labels with no evaluable selected pairs are exposed in unevaluable_labels_, assigned NaN, and omitted from global summaries. Fitting raises if no non-excluded label remains evaluable.
• Labels that own fewer than two prototypes are exposed in under_prototyped_labels_ and emit a warning because top-two scoring is degenerate in that case. Their scores are still computed.

---

Basic Usage

from sklearn.preprocessing import MinMaxScaler
from overlapindex import OverlapIndex

# Normalize features before fitting.
X = MinMaxScaler().fit_transform(X)

# MiniBatchKMeans is the default backend and is recommended for most offline use cases.
oi = OverlapIndex(
    kmeans_k=10,
    kmeans_kwargs={"random_state": 0},
)

# sklearn-style API
oi.fit(X, y)
score = oi.index

Sparse feature matrices can be passed directly to either centroid backend:

from scipy import sparse

X_csr = sparse.csr_matrix(X)
oi.fit(X_csr, y)

The fitted value is available through oi.index. For users who prefer update methods that return the current score directly, add_batch(X, y) is also supported.

Multi-Label Targets

import numpy as np

# Sequence-of-collections form
y_multilabel = [{"cat"}, {"cat", "pet"}, {"dog", "pet"}]
oi.fit(X, y_multilabel)

# Binary indicator form: positive column indices become integer labels
Y_indicator = np.asarray([[1, 0, 0], [1, 1, 0], [0, 1, 1]])
oi.fit(X, Y_indicator)

multilabel_pair_mode="all" evaluates all directional label pairs. multilabel_pair_mode="top_m" with a positive integer top_m restricts each source label to its nearest prototype-owning competitors. KMeans, MiniBatchKMeans, and BallCover support both modes.

Excluding Classes From Global Aggregation

exclude_classes lets you keep a label fully involved in overlap evaluation while omitting it from the two global summary scores:

oi = OverlapIndex(exclude_classes=0)
oi = OverlapIndex(exclude_classes=[0, "unlabeled"])

This is useful for segmentation workflows where only foreground objects are labeled but background-only samples should still contribute to pairwise overlap counts. A common pattern is to create one background class containing those samples, then pass that class id to exclude_classes. The background class will still appear in singleton_index, pairwise_index, and prototype ownership; only index and weighted_index omit it from aggregation.

Online ARTMAP Usage

from overlapindex import OverlapIndex

# For ARTMAP backends, batches should already be scaled into [0, 1].

oi = OverlapIndex(
    model_type="Hypersphere",
    rho=0.9,
    match_tracking="MT+",
)

for X_batch, y_batch in stream:
    oi.partial_fit(X_batch, y_batch)
    score = oi.index

For single-sample streams, ARTMAP backends also support add_sample(x, y), which updates the model and returns the current score directly. Labeled mini-batches can also be passed to add_batch(X, y).

API Styles

OverlapIndex supports both sklearn-style methods and direct score-returning update methods:

Method	Returns	Typical use
fit(X, y)	self	Full offline fitting on a labeled dataset.
partial_fit(X, y)	self	Incremental batch updates for ARTMAP backends; offline backends refit on the provided batch.
score() / score(X, y)	float	Read the current index, or refit on labeled data and return the new score.
predict(X)	np.ndarray	Return the highest-scoring global prototype id for each sample.
fit_predict(X, y)	np.ndarray	Fit and return per-sample prototype ids.
add_batch(X, y)	float	Batch update when the current OI score is needed immediately.
add_sample(x, y)	float	Single-sample online update for ARTMAP backends.

After fit or partial_fit, read the current score from oi.index or call score().

For model_type="KMeans", model_type="MiniBatchKMeans", and model_type="BallCover", partial_fit(X, y) is a convenience wrapper around recomputing the index on the provided labeled batch. Only the ARTMAP backends perform true incremental updates across calls.

Full fit(X, y) and score(X, y) calls always construct a fresh backend. fit_offline(..., reset_state=False) is accepted only for explicit ARTMAP continuation; offline backends reject it because their prototype ids cannot be safely accumulated across independent refits.

Count, neighborhood, projection, permutation, threshold, and chunk-size parameters use strict validation: booleans, strings, and fractional values are not coerced to integers. Radii and temperatures must be finite and positive, and class-specific dictionaries must provide a valid entry for every observed label.

If a batch is empty or contains only one unique class, OverlapIndex emits a RuntimeWarning and leaves the score at its default value of 1.0.

Clustering Backends

OverlapIndex uses model_type="MiniBatchKMeans" by default and supports several backend families through the model_type parameter:

model_type	Update mode	Description
"Fuzzy"	Online / batch	Incremental Fuzzy ARTMAP backend. Requires the optional art extra.
"Hypersphere"	Online / batch	Incremental Hypersphere ARTMAP backend. Requires the optional art extra.
"KMeans"	Offline batch only	Fits one scikit-learn KMeans model per class.
"MiniBatchKMeans"	Offline batch only	Default backend. Fits one scikit-learn MiniBatchKMeans model per class; recommended for larger datasets.
"BallCover"	Offline batch only	Fits one greedy landmark-ball cover per class. Useful when preserving class-support geometry is important.

Offline backends should be used with fit or add_batch. They do not support add_sample because their prototypes are fit from a complete labeled batch.

Top-two overlap scoring requires at least two owned prototypes per class for a well-resolved estimate. If fitting yields fewer than two, the estimator warns and continues; inspect under_prototyped_labels_ before reporting the score.

KMeans backend

from overlapindex import OverlapIndex

OI = OverlapIndex(
    model_type="KMeans",
    kmeans_k=10,
    kmeans_kwargs={"random_state": 0},
)

OI.fit(X, y)
score = OI.index

MiniBatchKMeans backend

from overlapindex import OverlapIndex

OI = OverlapIndex(
    model_type="MiniBatchKMeans",
    kmeans_k=10,
    kmeans_kwargs={
        "random_state": 0,
        "batch_size": 8192,
        "n_init": 1,
    },
)

OI.fit(X, y)
score = OI.index

BallCover backend

from overlapindex import OverlapIndex

OI = OverlapIndex(
    model_type="BallCover",
    ballcover_k="auto",
    ballcover_radius=0.25,
    ballcover_kwargs={
        "metric": "auto",
        "cover_fraction": 1.0,
    },
)

OI.fit(X, y)
score = OI.index

The BallCover backend supports one automatic cover parameter at a time:

• ballcover_k="auto" with a fixed ballcover_radius greedily adds balls until the requested cover fraction is reached.
• ballcover_k=<int> with ballcover_radius="auto" selects a fixed number of landmarks and infers the radius needed to cover the requested fraction of samples.

metric="auto" uses Euclidean distance in lower-dimensional spaces and cosine geometry for high-dimensional inputs such as embedding vectors. Users can override this with metric="euclidean" or metric="cosine".

Iris Dataset Example

from sklearn.datasets import load_iris
import numpy as np
from overlapindex import OverlapIndex

# Load dataset
iris = load_iris()

# Feature matrix (shape: [150, 4])
X = iris.data.astype(np.float64)

# Target vector (shape: [150,])
y = iris.target.astype(np.int64)

# Normalize the data (required)
x_max = X.max(axis=0)
x_min = X.min(axis=0)
X = (X - x_min) / (x_max - x_min)

# Instantiate the OI object with a reproducible centroid fit
OI = OverlapIndex(kmeans_kwargs={"random_state": 0})

# Calculate the Overlap Index
OI.fit(X, y)
print(OI.index)

The exact fitted score depends on backend settings and library versions. Set a random seed, as above, whenever a result must be repeatable.

Additional runnable examples are available in the examples/ directory.

---

Continuous Targets

ContinuousOverlapIndex is a regression-capable companion estimator for continuous targets. It preserves the OI interpretation by measuring whether feature-space prototype overlap occurs between incompatible empirical target distributions:

• COI = 1.0 indicates no observed harmful continuous-target overlap.
• 0.0 < COI < 1.0 indicates partial continuous-target separation.
• COI = 0.0 indicates complete or permutation-equivalent overlap.

The reported COI is always bounded to [0, 1]. A loss_ratio_ above 1.0 identifies worse-than-null target disagreement while the reported score remains at the 0.0 lower endpoint.

The main visual example uses three genuinely continuous regression problems: a smooth latent signal with increasing observation fidelity, a folded latent trajectory that is progressively unfolded in feature space, and a continuous covariate that is gradually recovered. The example uses eight target cells so the estimator evaluates fine-grained target structure rather than reducing each problem to a high-versus-low split. It uses strict nearest-competitor adjacency so each ideal target-ordered endpoint approaches the 1.0 upper anchor.

Regenerate the continuous figure with:

poetry run python examples/visualize_continuous_overlap_sweeps.py

An additional gallery demonstrates a heteroscedastic target field becoming progressively clean and a multivariate oscillator target observed with increasing fidelity:

poetry run python examples/visualize_continuous_overlap_additional_sweeps.py

These examples use six refit permutations per score to keep the complete sweeps practical to reproduce. Increase n_null_permutations in the scripts when adapting them for final quantitative reporting.

Version 1 is offline-first and supports model_type="MiniBatchKMeans", model_type="KMeans", and model_type="BallCover". ARTMAP online support is intentionally deferred for continuous targets.

from sklearn.preprocessing import MinMaxScaler
from overlapindex import ContinuousOverlapIndex

X = MinMaxScaler().fit_transform(X)

coi = ContinuousOverlapIndex(
    model_type="MiniBatchKMeans",
    kmeans_k=8,
    kmeans_kwargs={"random_state": 0},
    adjacency_mode="soft_topk",
    top_k=5,
    feature_temperature=1.0,
    null_mode="auto",
    n_target_cells="auto",
    n_null_permutations=20,
    random_state=0,
)

coi.fit(X, y_regression)
score = coi.index

ContinuousOverlapIndex.random_state seeds target-cell construction, projection directions, permutation sampling, and the selected feature backend. An explicit random_state inside kmeans_kwargs or ballcover_kwargs takes precedence over the top-level value.

The KMeans-backed continuous paths also accept SciPy sparse feature matrices. The same CSR matrix is reused by prototype construction, adjacency scoring, and permutation-null refits without materializing a dense copy of the complete feature matrix. Continuous targets remain dense numeric arrays.

For univariate regression targets, target_cover="auto" uses quantile target cells and target_distance="auto" uses 1D Wasserstein distance. For multivariate regression targets, target_cover="auto" uses KMeans target cells and target_distance="auto" uses sliced Wasserstein distance.

By default, COI builds prototype adjacency with adjacency_mode="soft_topk". Each sample spreads one unit of competitor mass across up to top_k nearby non-own prototypes using a temperature-scaled softmax over backend feature scores. Lower feature_temperature values make the weighting approach the legacy hard_top1 behavior, while adjacency_mode="hard_top1" remains available for strict single-competitor scoring and backwards comparison.

COI stores empirical target measures per feature prototype instead of reducing targets to means or variances. By default, null_mode="auto" uses the current refit-permutation null on smaller workloads and automatically switches to a faster fixed-structure permutation null when n_samples * n_null_permutations >= 100_000. The refit null rebuilds target cells and feature prototypes for each target shuffle so that random target assignments calibrate near 0.0. The fixed-structure null keeps the fitted prototype geometry and shuffles target values across that structure, which is substantially faster on large datasets but should be treated as an approximate calibration mode. Use null_mode="refit_permutation" for final reporting when you want the strongest null semantics. As with discrete OI, use enough prototypes per target cell for overlap structure to be observable; one prototype per cell is usually too coarse for separation diagnostics.

Key diagnostics after fitting include:

• actual_loss_, null_loss_, and loss_ratio_
• null_mode_, null_loss_samples_, and auto_null_work_
• macro_index_ and weighted_index
• prototype_index_, prototype_loss_, and prototype_target_values_

The continuous calibration changed in the 0.1.3 alpha series. At the loss-ratio level, the new pre-bound calibration equals 2 * legacy_score - 1, after which values are bounded to [0, 1]. Do not compare historical COI values directly with scores from this calibration.

---

Parameters

• rho (float)
  Vigilance parameter controlling cluster granularity for ARTMAP backends.

• r_hat (float, Hypersphere ARTMAP only)
  Maximum cluster radius for the Hypersphere backend.

• model_type ("Fuzzy" | "Hypersphere" | "KMeans" | "MiniBatchKMeans" | "BallCover")
  Clustering backend used to create class-owned prototypes. Defaults to "MiniBatchKMeans".

• match_tracking (str)
  Match-tracking strategy used during ARTMAP learning.

• kmeans_k (int or dict)
  Number of clusters per class for KMeans and MiniBatchKMeans backends.

• kmeans_kwargs (dict, optional)
  Keyword arguments forwarded to the selected scikit-learn KMeans backend.

• ballcover_k (int, dict, or "auto")
  Number of balls per class, class-specific ball counts, or "auto" for greedy fixed-radius covering.

• ballcover_radius (float, dict, or "auto")
  Ball radius, class-specific radii, or "auto" when using a fixed number of balls.

• ballcover_kwargs (dict, optional)
  Additional BallCover options such as metric, cover_fraction, chunk_size, max_balls, and random_state.

• exclude_classes (None, scalar label, or iterable of labels)
  Label ids to omit from the global index and weighted_index aggregation while leaving all fitting and per-class overlap outputs intact.

• offline_chunk_size (positive int or None) Maximum row block used for vectorized offline prototype scoring.

• multilabel_pair_mode ("all" or "top_m") Directional competitor selection strategy for multi-label offline scoring.

• top_m (positive int, required for "top_m") Maximum number of prototype-nearest competitors selected per source label.

---

The default parameters are intended for offline batch use with MiniBatchKMeans. For online or continual-learning workflows, explicitly choose model_type="Fuzzy" or model_type="Hypersphere". For very large ART-based runs, smaller rho values (0.5-0.7) may improve run-time performance.

---

Output

• index
  Global macro Overlap Index across all observed classes that are not listed in exclude_classes. This is the default class-balanced score and is usually preferred for imbalance-sensitive separation analysis.

• weighted_index
  Support-weighted Overlap Index across observed classes that are not listed in exclude_classes. This weights each included class's singleton_index value by its positive sample count, which can be useful when reporting should reflect observed class frequencies.

• singleton_index[y]
  Minimum pairwise overlap score for class y.

• pairwise_index[(y, b)]
  Pairwise overlap score between classes y and b.

• under_prototyped_labels_ Labels owning fewer than two prototypes after fitting.

• unevaluable_pairs_ / unevaluable_labels_ Multi-label diagnostics for directional comparisons without sufficient rows.

---

Intended Audience

This package is intended for researchers and practitioners working on:

• incremental and continual learning,
• clustering validation,
• representation learning,
• transfer learning

License

The source code is licensed under the GNU Affero General Public License v3.0 or later (AGPLv3-or-later). Commercial licenses are available; please contact the maintainer through GitHub.