InstanceSpace Python v0.4.1

instancespace.instance_space

Perform instance space analysis on given dataset and configuration.

Construct an instance space from data and configuration files located in a specified directory. The instance space is represented as a Model object, which encapsulates the analytical results and metadata of the instance space analysis.

class ExploreStage(enum.Enum):

One step of the explore()-time inference pipeline.

These lightweight identifiers describe the lazy orchestration order. Scientific inference is owned by the corresponding predictive stage; metadata reconciliation and conditional evaluation remain coordinated by InstanceSpace.

PRELIM = <ExploreStage.PRELIM: 'prelim'>
SIFTED = <ExploreStage.SIFTED: 'sifted'>
PILOT = <ExploreStage.PILOT: 'pilot'>
PYTHIA = <ExploreStage.PYTHIA: 'pythia'>
TRACE = <ExploreStage.TRACE: 'trace'>
EVALUATION = <ExploreStage.EVALUATION: 'evaluation'>
class AnnotatedExploreOutput(typing.NamedTuple):

The yielded output of running one explore-time step.

Mirrors stage_runner.AnnotatedStageOutput's (stage, output) shape.

AnnotatedExploreOutput(stage: ExploreStage, output: Any)

Create new instance of AnnotatedExploreOutput(stage, output)

stage: ExploreStage

Alias for field number 0

output: Any

Alias for field number 1

class InstanceSpace:

Build an instance-space pipeline and apply its fitted model to new data.

Construct with validated Metadata and InstanceSpaceOptions objects. build() returns a fitted Model; explore() returns predictions, selections and optional evaluation metrics without refitting. Staged runs expose intermediate outputs. save() and load() preserve a complete checkpoint, including the runner needed for exploration.

Examples

With metadata and options prepared as in the Getting Started guide:

>>> space = InstanceSpace(metadata, options)
>>> model = space.build()

Create an output directory before calling explicit export methods. Only load checkpoints from trusted producers. Call close() when the reusable execution pool is no longer needed.

InstanceSpace( metadata: instancespace.Metadata, options: instancespace.InstanceSpaceOptions, stages: list[type[instancespace.stages.stage.Stage[typing.Any, typing.Any]]] = [<class 'instancespace.stages.PreprocessingStage'>, <class 'instancespace.stages.PrelimStage'>, <class 'instancespace.stages.SiftedStage'>, <class 'instancespace.stages.PilotStage'>, <class 'instancespace.stages.PythiaStage'>, <class 'instancespace.stages.CloisterStage'>, <class 'instancespace.stages.TraceStage'>], additional_initial_inputs_type: type[typing.NamedTuple] | None = None, progress_reporter: instancespace.ProgressReporter | None = None)

Initialise the InstanceSpace.

Parameters
  • metadata (Metadata): The feature and algorithm performance data for the instance space.
  • options (InstanceSpaceOptions): The options for the instance space.
  • stages (list[StageClass], optional): The list of stages to run.
  • additional_initial_inputs_type (type[NamedTuple] | None, optional): More initial inputs for plugins.
  • progress_reporter (ProgressReporter | None, optional): The reporter that records the progress of build() and run_stage(). For example, a SLURM job can use HttpProgressReporter to send a callback to an orchestrator after each stage. The default reporter does nothing, so the behavior does not change if you omit this argument.
metadata: instancespace.Metadata

Get metadata.

options: instancespace.InstanceSpaceOptions

Get options.

model: instancespace.Model

Get model.

Raises
  • StageRunningError: If the InstanceSpace is not built.
Returns
  • Model: The model that build() made.
def save(self, path: pathlib.Path | str, secret_key: bytes | None = None) -> None:

Checkpoint this InstanceSpace to path via signed joblib.

Captures the entire pipeline state - not just a finished Model - so a partially-built InstanceSpace (e.g. one stage run per SLURM job, with the next stage triggered by a separate, later job) can be reconstructed by load() and continue from exactly where it left off via run_stage(). Signing follows the same scheme as Model.save()/Model.load(): see that method's docstring for the exact secret_key semantics and the production invariant (always sign, never accept a user-supplied path server-side).

@classmethod
def load( cls: type[InstanceSpace], path: pathlib.Path | str, secret_key: bytes | None = None) -> InstanceSpace:

Restore an InstanceSpace checkpoint previously written by save().

Same four verification cases as Model.load() (matched/mismatched/ missing/unexpected signature) - see that method's docstring.

def close(self) -> None:

Release resources held across staged calls (currently: the TRACE pool).

Safe to call even if nothing has been built yet. A subsequent staged call recreates the pool lazily when parallel execution is enabled.

def plot_sources( self, ax: matplotlib.axes._axes.Axes | None = None) -> matplotlib.axes._axes.Axes:

Scatter training instances in the 2D instance space, coloured by source.

See instancespace.plotting.plot_sources.

def plot_portfolio( self, ax: matplotlib.axes._axes.Axes | None = None) -> matplotlib.axes._axes.Axes:

Scatter instances coloured by their best-performing algorithm.

See instancespace.plotting.plot_portfolio.

def plot_good( self, algo: str | int, ax: matplotlib.axes._axes.Axes | None = None) -> matplotlib.axes._axes.Axes:

Scatter instances coloured by PYTHIA's good/bad prediction for one algorithm.

See instancespace.plotting.plot_good.

def plot_footprint( self, algo: str | int, kind: str = 'good', ax: matplotlib.axes._axes.Axes | None = None) -> matplotlib.axes._axes.Axes:

Draw one algorithm's trained footprint polygon(s) over training instances.

See instancespace.plotting.plot_footprint.

def build(self) -> instancespace.Model:

Build the instance space, in one call, start to finish.

Options will be broken down to sub fields to be passed to stages. You can override inputs to stages. Progress is reported via the configured progress_reporter (a no-op by default - see __init__), the same as run_stage().

Returns
  • Model: The output of all stages.
def run_iter( self) -> Generator[instancespace.stage_runner.AnnotatedStageOutput, None, None]:

Run all stages, yielding between so the data can be examined.

Yields
  • AnnotatedStageOutput: The output of each stage, with the name of the stage that ran. The name is necessary because the stages in one schedule can run in any order.
def run_stage( self, stage: type[instancespace.stages.stage.Stage[~IN, ~OUT]], **arguments: Any) -> ~OUT:

Run a single stage.

All inputs to the stage must either be present from previously ran stages, or be given as arguments to this function. Arguments to this function have priority over outputs from previous stages.

Progress is reported via the configured progress_reporter (a no-op by default - see __init__). This is the entry point a SLURM job that runs one stage per invocation should call: load a checkpoint with InstanceSpace.load() (or construct fresh for the first stage), call run_stage() for the stage this invocation was told to run, then save() the result so a later, separately-triggered job can resume. self.model becomes available once the last stage in the schedule has been run this way, exactly as after build().

Parameters
  • stage (StageClass): The stage to run.
  • **arguments (Any): More inputs for the stage. If an input is missing, the stage uses the output of a previous stage.
Returns
  • OUT: The output of the stage.
def run_until_stage( self, stage: type[instancespace.stages.stage.Stage[typing.Any, typing.Any]], **arguments: Any) -> dict[str, typing.Any]:

Run all stages until the specified stage, as well as the specified stage.

Parameters
  • stage (StageClass): A stage in the last wave to execute.
  • **arguments (Any): Input overrides for this run. After the stage runs, all downstream stages can use these overrides.
Returns
  • dict[str, Any]: The raw output dict of all the stages that ran.
explore_results: list[instancespace.ExploreResult]

Get list of explore results from previous explore() calls.

Returns
  • list[ExploreResult]: List of explore results, in order of execution.
def explore( self, test_metadata: instancespace.Metadata, *, dataset_id: str | None = None) -> instancespace.ExploreResult:

Apply trained instance space model to new test data.

This method applies the transformations and models learned during build() to new test instances. It performs:

  1. Feature preprocessing (PRELIM normalization)
  2. Feature selection (SIFTED)
  3. Dimensionality reduction to the trained 2D/3D PILOT projection
  4. Algorithm performance prediction (PYTHIA SVMs)
  5. Footprint membership analysis (TRACE)
Parameters
  • test_metadata (Metadata): New instances that have the same feature columns as the training data. The method matches the feature columns by name, not by position. Thus, the columns can be in any order. This behavior is intentional. It is not a strict order check like MATLAB's featureOrderMismatch.
  • dataset_id (str | None, optional): The identifier for this test data set. If you do not supply an ID, the method makes an ID from a timestamp.
Returns
  • ExploreResult: The projected coordinates, the algorithm predictions, and the footprint membership for the test instances.
Raises
  • RuntimeError: If build() has not been called before explore().
  • ValueError: If the test_metadata features are not the same as the training features. Also if the trained and explored projection dimensions are different.
def explore_stage_iter( self, test_metadata: instancespace.Metadata) -> Generator[AnnotatedExploreOutput, None, None]:

Run explore() one stage at a time, yielding each stage's output.

Same computation and trained model as explore(), but instead of returning a single ExploreResult it yields an AnnotatedExploreOutput (an (ExploreStage, output) pair - the explore-time counterpart of run_iter()'s AnnotatedStageOutput) after each inference stage, so the intermediate result of every stage can be inspected or plotted before the next one runs. In order: ExploreStage.PRELIM/SIFTED/PILOT yield the transformed feature or coordinate array, PYTHIA yields (y_hat, pr0_hat, selection0), TRACE yields (in_good, in_best), and - only when test_metadata carries algorithm performance columns - EVALUATION yields the ground-truth-vs-prediction evaluation fields as an _EvaluationResult. EVALUATION is omitted entirely (not yielded at all) when the test set has no ground truth, matching explore()'s own conditional field population.

When the test set introduces algorithms absent from training (full MATLAB alignment), PYTHIA's and TRACE's yielded arrays are widened to include a column per new algorithm (False/0 placeholders - no trained classifier or footprint exists for them), matching MATLAB's own evaluateTestSet reconciliation happening before PYTHIA/TRACE run, not after.

Parameters
  • test_metadata (Metadata): New instances that have the same feature columns as the training data.
Yields
  • AnnotatedExploreOutput: The stage that just ran and its output.
Raises
  • RuntimeError: If build() has not been called before explore().
  • ValueError: If the test_metadata features are not the same as the training features. TraceStage.predict() checks for a TRACE/projection dimension mismatch. This check occurs only when the lazy iterator gets to TRACE, after the iterator yields PYTHIA. This order is intentional.
def instance_space_from_files( metadata_filepath: pathlib.Path, options_filepath: pathlib.Path) -> InstanceSpace | None:

Construct an instance space object from 2 files.

Parameters
  • metadata_filepath (Path): Path to the metadata csv file.
  • options_filepath (Path): Path to the options json file.
Returns
  • InstanceSpace | None: A new instance space object instantiated with metadata and options from the specified files, or None if the initialization fails.
def instance_space_from_directory( directory: pathlib.Path) -> InstanceSpace | None:

Construct an instance space object from a directory.

Parameters
  • directory (Path): Path to a directory that contains metadata.csv and options.json.
Returns
  • InstanceSpace | None: A new instance space object instantiated with metadata and options from the specified directory, or None if the initialization fails.