InstanceSpace Python v0.4.1

instancespace.stages.pilot

PILOT: Obtaining a two- or three-dimensional projection.

Projecting Instances with Linearly Observable Trends (PILOT) is a dimensionality reduction algorithm which aims to facilitate the identification of relationships between instances and algorithms by unveiling linear trends in the data, increasing from one edge of the space to the opposite.

class PilotInput(typing.NamedTuple):

Inputs for the Pilot stage.

Attributes
  • x (NDArray[np.double]): The feature matrix (instances x features) to process.
  • y (NDArray[np.double]): The data points for the selected feature.
  • feat_labels (list[str]): List feature names.
  • pilot_options (PilotOptions): The options for the PILOT stage.
  • parallel_options (ParallelOptions): The parallel options, specifying whether to run in parallel and the number of cores - used to parallelise the numerical solver's ntries restart loop.
  • general_options (GeneralOptions): General options (e.g. the RNG seed), not specific to any one stage.
  • y_bin (NDArray[np.bool_]): Binary matrix indicating good algorithm performance per instance, used only when pilot_options.adjust_rotation is set.
PilotInput( x: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], y: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], feat_labels: list[str], pilot_options: instancespace.data.options.PilotOptions, parallel_options: instancespace.data.options.ParallelOptions, general_options: instancespace.data.options.GeneralOptions, y_bin: numpy.ndarray[typing.Any, numpy.dtype[numpy.bool_]])

Create new instance of PilotInput(x, y, feat_labels, pilot_options, parallel_options, general_options, y_bin)

x: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]

Alias for field number 0

y: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]

Alias for field number 1

feat_labels: list[str]

Alias for field number 2

pilot_options: instancespace.data.options.PilotOptions

Alias for field number 3

parallel_options: instancespace.data.options.ParallelOptions

Alias for field number 4

general_options: instancespace.data.options.GeneralOptions

Alias for field number 5

y_bin: numpy.ndarray[typing.Any, numpy.dtype[numpy.bool_]]

Alias for field number 6

class PilotPredictInput(typing.NamedTuple):

Inputs for applying a fitted PILOT projection.

PilotPredictInput(x: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]])

Create new instance of PilotPredictInput(x,)

x: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]

Alias for field number 0

class PilotOutput(typing.NamedTuple):

Outputs for the Pilot stage.

Attributes
  • X0 (NDArray[np.double] | None): The start points of the numerical solver, one column for each trial. None if PILOT does not use the numerical solver.
  • alpha (NDArray[np.double] | None): The solution vector of each numerical trial, one column for each trial. None if PILOT does not use the numerical solver.
  • eoptim (NDArray[np.double] | None): The final value of the error function for each numerical trial. None if PILOT does not use the numerical solver.
  • perf (NDArray[np.double] | None): For each numerical trial, the Pearson correlation between the distances in feature space and the distances in the projected space. None if PILOT does not use the numerical solver.
  • a (NDArray[np.double]): The projection matrix (dims x features). PLS subtracts pilot_x_mean before projection.
  • z (NDArray[np.double]): The projected instances (instances x dims).
  • c (NDArray[np.double]): The matrix that estimates the algorithm performance from z (algorithms x dims).
  • b (NDArray[np.double]): The matrix that estimates the features from z (features x dims).
  • error (NDArray[np.double]): The sum of the squared errors between the input data and its estimate.
  • r2 (NDArray[np.double]): The coefficient of determination (R^2) of the estimate, one value for each feature and algorithm column.
  • pilot_summary (pd.DataFrame): The projection matrix a, rounded to four decimals, with feature labels.
  • pilot_x_mean (NDArray[np.double] | None): Fitted feature mean for PLS; None for an uncentred standard projection.
  • viewpoint (PilotViewpointResult | None): The optimized viewpoints for the algorithm groups of a 3D projection. None for a 2D projection.
PilotOutput( X0: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]] | None, alpha: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]] | None, eoptim: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]] | None, perf: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]] | None, a: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], z: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], c: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], b: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], error: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], r2: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], pilot_summary: pandas.core.frame.DataFrame, viewpoint: instancespace.stages.pilot_viewpoint.PilotViewpointResult | None = None, pilot_x_mean: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]] | None = None)

Create new instance of PilotOutput(X0, alpha, eoptim, perf, a, z, c, b, error, r2, pilot_summary, viewpoint, pilot_x_mean)

X0: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]] | None

Alias for field number 0

alpha: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]] | None

Alias for field number 1

eoptim: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]] | None

Alias for field number 2

perf: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]] | None

Alias for field number 3

a: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]

Alias for field number 4

z: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]

Alias for field number 5

c: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]

Alias for field number 6

b: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]

Alias for field number 7

error: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]

Alias for field number 8

r2: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]

Alias for field number 9

pilot_summary: pandas.core.frame.DataFrame

Alias for field number 10

viewpoint: instancespace.stages.pilot_viewpoint.PilotViewpointResult | None

Alias for field number 11

pilot_x_mean: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]] | None

Alias for field number 12

class PilotStage(instancespace.stages.stage.Stage[instancespace.stages.pilot.PilotInput, instancespace.stages.pilot.PilotOutput], instancespace.stages.stage.PredictiveStage[instancespace.stages.pilot.PilotPredictInput, instancespace.data.model.PilotOut, numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]]):

Class for PILOT stage.

PilotStage( x: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], y: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], feat_labels: list[str])

Initialize the Pilot stage.

The Initialize functon is used to create a Pilot class.

Parameters
  • x (NDArray[np.double]): The feature matrix (instances x features) to process.
  • y (NDArray[np.double]): The data points for the selected feature.
  • feat_labels (list[str]): The feature names.
Returns
  • None
x
y
feat_labels
@staticmethod
def predict( inputs: PilotPredictInput, fitted: instancespace.data.model.PilotOut) -> numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]:

Project with the fitted PLS mean; older models retain linear inference.

@staticmethod
def pilot( x: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], y: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], feat_labels: list[str], options: instancespace.data.options.PilotOptions, general_options: instancespace.data.options.GeneralOptions, y_bin: numpy.ndarray[typing.Any, numpy.dtype[numpy.bool_]] | None = None, _do_output: bool = True, parallel_options: instancespace.data.options.ParallelOptions | None = None) -> PilotOutput:

Run the PILOT dimensionality reduction algorithm.

Parameters
  • x (NDArray[double]): The feature matrix (instances x features) to process.
  • y (NDArray[double]): The data points for the selected feature.
  • feat_labels (list[str]): List feature names.
  • options (PilotOptions): The options enabled for the Pilot Class.
  • general_options (GeneralOptions): General options for all stages, for example the RNG seed.
  • y_bin (NDArray[np.bool_] | None): Binary matrix (instances x algorithms) indicating good algorithm performance. Required only when options.adjust_rotation is set.
  • parallel_options (ParallelOptions | None): Whether (and how much) to parallelise the numerical solver's ntries restart loop. None (the default for direct callers that don't pass one) runs the restarts sequentially, same as flag=False.
Returns
  • PilotOutput: Outputs from the Pilot stage.
@staticmethod
def adjust_rotation( z: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], bad_instances: numpy.ndarray[typing.Any, numpy.dtype[numpy.bool_]], theta: float = 135.0) -> tuple[numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]]:

Rotate the 2D projection so poorly-solved instances face theta.

Ported from PyISpace's pilot.adjust_rotation() (gitlab.com/ita-ml/pyispace). Rotation preserves all pairwise distances in z, so it only changes the space's visual orientation

  • error, R2, and footprint areas are unaffected. The centroid of the instances flagged by bad_instances is rotated to sit at theta degrees (135 = upper-left quadrant, matching PyISpace's default), so that similar datasets come out consistently oriented across runs.
Parameters
  • z (NDArray[np.double]): The 2D projection (instances x 2) to rotate.
  • bad_instances (NDArray[np.bool_]): Boolean mask selecting the instances whose centroid should be placed at theta.
  • theta (float): Target angle, in degrees, for the bad-instance centroid.
Returns
  • NDArray[np.double]: The rotated projection, same shape as z.
  • NDArray[np.double]: The 2x2 rotation matrix applied.
@staticmethod
def analytic_solve( x: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], x_bar: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], n: int, m: int, cost_weight: float = 1.0, _do_output: bool = True, dims: int = 2) -> tuple[numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]]:

Solve the projection problem analytically.

Parameters
  • x (NDArray[np.double]): The feature matrix (instances x features) to process.
  • x_bar (NDArray[np.double]): Combined matrix of X and Y.
  • n (int): Number of original features.
  • m (int): Total number of features including appended Y.
  • cost_weight (float): Scalar performance-reconstruction weight (MATLAB's costWeight). Scales the performance block relative to the feature block before the eigendecomposition; 1.0 weights both blocks equally.
  • dims (int): Output projection dimensionality, either 2 or 3.
Returns
  • NDArray[np.double]: Matrix A.
  • NDArray[np.double]: Matrix B.
  • NDArray[np.double]: Matrix C.
  • NDArray[np.double]: Matrix Z.
  • NDArray[np.double]: The mean squared error between x_bar and its low-dimensional approximation.
  • NDArray[np.double]: The coefficient of determination between x_bar and its low-dimensional approximation.
@staticmethod
def pls_solve( x: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], y: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], x_bar: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], m: int, dims: int = 2, _do_output: bool = True) -> tuple[numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]]:

Solve the projection problem via Partial Least Squares.

MATLAB's method='pls' alternative to the analytic/numeric eigen-decomposition solvers above - a genuinely different algorithm, not a variant of either. dims matches MATLAB's own plsregress(X, Y, d) call.

Implements the SIMPLS routine used by MATLAB R2026a's plsregress. Both inputs are mean-centred but not variance-scaled, matching the default MATLAB call in PILOT.m.

Parameters
  • x (NDArray[np.double]): The feature matrix (instances x features) to process.
  • y (NDArray[np.double]): The performance matrix (instances x algorithms).
  • x_bar (NDArray[np.double]): Combined matrix of X and Y.
  • m (int): Total number of features including appended Y.
  • dims (int): Output projection dimensionality, either 2 or 3.
Returns
  • NDArray[np.double]: Matrix A (dims x n), the projection matrix: Z = (X - mean(X)) @ A.T.
  • NDArray[np.double]: Matrix Z (instances x dims), the projected coordinates.
  • NDArray[np.double]: Matrix C (dims x q), the performance-block reconstruction matrix.
  • NDArray[np.double]: Matrix B (n x dims), the feature-block reconstruction matrix.
  • NDArray[np.double]: The sum of squared reconstruction error between x_bar and its low-dimensional approximation.
  • NDArray[np.double]: The per-column coefficient of determination between x_bar and its low-dimensional approximation.
@staticmethod
def numerical_solve( x: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], hd: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], x0: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], x_bar: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], n: int, m: int, alpha: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], eoptim: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], perf: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], opts: instancespace.data.options.PilotOptions, general_options: instancespace.data.options.GeneralOptions, _do_output: bool = True, parallel_options: instancespace.data.options.ParallelOptions | None = None) -> tuple[int, numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]]]:

Solve the projection problem numerically.

Parameters
  • x (NDArray[np.double]): The feature matrix (instances x features) to process.
  • hd (NDArray[np.double]): Condensed pairwise distances between the instances in feature space. The quality of each trial is the Pearson correlation between these distances and the distances in the projected space.
  • x0 (NDArray[np.double]): Initial guess for the solution.
  • x_bar (NDArray[np.double]): Combined matrix of X and Y.
  • n (int): Number of original features.
  • m (int): Total number of features including appended Y.
  • alpha (NDArray[np.double]): Flattened parameter vector containing both A (dimsn size) and B (mdims size) matrices.
  • eoptim (NDArray[np.double]): Optimized error function.
  • perf (NDArray[np.double]): Optimized performance matrix.
  • opts (PilotOptions): Configuration options for PILOT.
  • general_options (GeneralOptions): General options for all stages, for example verbosity.
  • parallel_options (ParallelOptions | None): Whether (and how much) to parallelise the ntries restarts across OS processes (matching MATLAB's parfor). None or flag=False runs the restarts sequentially. Ignored - falls back to sequential - when already running inside another process pool's worker (see multiprocessing.parent_process() check below): SIFTED's GA fitness function calls into pilot() from inside its own ProcessPoolExecutor workers, and opening a second, nested pool there would reintroduce MATLAB's nested-parfor-inside-GA bug. In practice this never triggers today, since SIFTED always calls pilot() with analytic=True (bypassing this numerical branch entirely) - this check guards against that invariant ever changing silently.
Returns
  • int: The index of the trial that has the highest distance correlation.
  • NDArray[np.double]: Flattened parameter vector containing both A (dimsn size) and B (mdims size) matrices.
  • NDArray[np.double]: Optimized error function.
  • NDArray[np.double]: Optimized performance matrix.
@staticmethod
def error_function( alpha: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], x_bar: numpy.ndarray[typing.Any, numpy.dtype[numpy.float64]], n: int, m: int, cost_weight: float = 1.0, d: int = 2) -> float:

Error function used for numerical optimization in the PILOT algorithm.

Parameters
  • alpha (NDArray[np.float64]): Flattened parameter vector containing both A (dn size) and B (md size) matrices.
  • x_bar (NDArray[np.float64]): Combined matrix of X and Y.
  • n (int): Number of original features.
  • m (int): Total number of features including appended Y.
  • cost_weight (float): Scalar weight applied to the performance columns' squared error (MATLAB's pilotErrorFcn costWeight). 1.0 weights every column equally.
  • d (int): Output projection dimensionality, either 2 or 3.
Returns
  • float: The mean squared error between x_bar and its low-dimensional approximation.