Instance Space Analysis for Python
Understand algorithm performance across the instance space of a problem
Instance Space Analysis (ISA) relates the structural features of problem instances to the performance of a portfolio of algorithms. This Python package projects instances into a 2D or 3D space, fits algorithm selectors, and constructs algorithm footprints. A footprint describes a region where an algorithm is expected to perform well under a chosen performance threshold.
Use ISA to compare algorithm strengths, examine feature and performance relationships, inspect benchmark diversity, and recommend algorithms for new instances. Gaps in the observed space can help identify where additional instances would improve coverage. The conclusions depend on the features, performance measure, and collected instances. Footprints do not guarantee good performance for every new instance within their boundaries.
PILOT supplies the projected space to CLOISTER and PYTHIA. TRACE uses the trained labels and coordinates.
Get Started
Getting Started
Install the package and build a complete instance space from a runnable example.
Metadata Format
Prepare features, algorithm outcomes and instance identifiers.
Build and Explore
Inspect stages, save a checkpoint and evaluate new instances.
Options Reference
Every configuration field, its default, accepted JSON aliases and effect.
Classes and Reference
Start with InstanceSpace to build and explore an analysis. Use Model to inspect fitted results, save an archive, or export files.
Each stage page provides syntax, a description, parameters, results, and examples:
| Stage | Purpose |
|---|---|
| PREPROCESSING | Select metadata columns and filter missing inputs. |
| PRELIM | Define good performance and fit data transformations. |
| SIFTED | Select informative features. |
| PILOT | Project features into a 2D or 3D instance space. |
| CLOISTER | Estimate empirical bounds. |
| PYTHIA | Train algorithm selectors and score observed outcomes. |
| TRACE | Construct footprints and evaluate membership. |
Classifiers and Tuning explains the learners and search strategies. Outputs and Plotting describes file formats, geometry, and figures. Custom Stages explains pipeline extensions.
The API Reference supplies exact Python signatures and result fields. MATLAB Compatibility describes shared contracts and current differences. Use Troubleshooting for input, configuration, and workflow errors.
Installation
Use Python 3.12 in a virtual environment:
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install instancespace
On Windows, activate the environment with .\.venv\Scripts\Activate.ps1 in PowerShell. The distribution and import names are both instancespace. Python workflows do not require MATLAB, GNU Octave, or MATLAB toolboxes.
Reproducibility
Keep the input metadata, complete options, package version, and dependency versions with each analysis. Choose performance direction, threshold mode, epsilon, projection dimensions, and footprint method explicitly. Stage seeds of None inherit general.seed, whose Python default is 0.
A fixed seed does not guarantee identical optimizer paths or floating-point results across platforms or implementations. Use independent evaluation data to assess the complete analysis. PYTHIA cross-validation summaries remain conditional on the fitted preprocessing, feature selection, projection, and tuning.
AI-Assisted Analysis
The repository includes a Python ISA skill for AI assistants. It covers data checks, option choices, stage interpretation, and evaluation limits. Its implementation guidance applies to this Python package.
Methodology
- Smith-Miles, K. and Muñoz, M. A. (2023). Instance Space Analysis for Algorithm Testing: Methodology and Software Tools. ACM Computing Surveys, 55(12), Article 255. DOI.
- Simpson, C., Muñoz, M. A., Kandanaarachchi, S. and Campello, R. J. G. B. (2025). ISA3: A 3-dimensional expansion of Instance Space Analysis. Machine Learning, 114, 240. DOI.
For the Python software, cite:
License and Support
The package uses the PolyForm Noncommercial License 1.0.0. Report defects and request features through GitHub Issues. See Contributing for development checks and Security for private security reporting.