instancespace.stages.prelim
Performing preliminary data processing.
The main focus is on the prelim function, which prepares the input data for further
analysis and modeling.
The prelim function takes feature and performance data matrices along with a set of
processing options, and performs various preprocessing tasks such as normalization,
outlier detection and removal, and binary performance classification. These tasks are
guided by the options specified in the InstanceSpaceOptions object.
Ground-truth binary-performance fields derived from raw algorithm performance.
Attributes
- y (NDArray[np.double]):
Relative (or absolute) performance matrix -
y_rawtransformed by the samemax_perf/abs_perfbranching used to derivey_bin. - y_bin (NDArray[np.bool_]): Binary matrix indicating instances with good algorithm performance.
- y_best (NDArray[np.double]):
Best observed performance value for each instance across all algorithms
(zero values substituted with
eps, matching MATLAB). - p (NDArray[np.int_]): 1-based index of the best-performing algorithm per instance, ties broken with MATLAB-compatible seeded random selection.
- num_good_algos (NDArray[np.double]): Number of algorithms with good performance, per instance.
- beta (NDArray[np.bool_]):
Whether each instance clears
beta_threshold's fraction of good algorithms.
Create new instance of BinaryPerformance(y, y_bin, y_best, p, num_good_algos, beta)
Compute the binary measure of algorithm performance from raw Y.
Ports MATLAB PRELIM.m's binary-performance section: an algorithm is
"good" for an instance if its performance is within epsilon of the best
(relative) or better than epsilon outright (absolute), per
perf_opts.max_perf/abs_perf. Ties for the best algorithm use MATLAB's
seeded twister stream and randi selection rule.
Shared by PrelimStage._prelim() (training, ground truth for the
training set) and InstanceSpace.explore()'s evaluation path (ground
truth for a test set using the trained PerformanceOptions) - one
implementation, not a second one written to match it by hand.
Clip each feature column of x to [lo_bound, hi_bound].
Equivalent to (and verified bit-for-bit identical, including NaN
handling, against) the mask-multiply arithmetic this replaces -
np.clip simply expresses the same operation without hand-writing it
twice. Shared by PrelimStage._bound() (training, clipping to bounds
just fit from this same data) and InstanceSpace._explore_prelim()
(test data, clipping to the trained model's stored bounds) - one
implementation, not a second one written to match it by hand.
Apply a Box-Cox transform at a known lambda, then z-score at known mu/sigma.
Verified bit-for-bit identical to scipy.stats.zscore(stats.boxcox(x,
lambda_), ddof=1) when mu/sigma are themselves x's own
box-cox'd mean/std (ddof=1) - i.e. this is a drop-in replacement for
recomputing the z-score from scratch, not a new formula. Shared by
PrelimStage._normalise() (training, applying the lambda/mu/sigma it
just fit from this same column) and InstanceSpace._explore_prelim()
(test data, applying the trained model's stored lambda/mu/sigma) - one
implementation, not a second one written to match it by hand. x must
already be positive (the min-shift-by-1 step happens before this call).
Inputs for the Prelim stage.
Attributes
- x (NDArray[np.double]): Feature matrix where each row represents an instance, and each column represents a feature.
- y (NDArray[np.double]): Performance matrix of algorithms, with rows as instances and columns as algorithms.
- x_raw (NDArray[np.double]): Unprocessed feature matrix, containing raw values of each instance-feature pair.
- y_raw (NDArray[np.double]): Unprocessed performance matrix, containing raw performance values for each instance-algorithm pair.
- s (pd.Series | None): Optional series for additional selection during processing, if available.
- inst_labels (pd.Series): Labels for each instance in the dataset, used for identification.
- prelim_options (PrelimOptions): Configuration options specific to the Prelim stage.
- selvars_options (SelvarsOptions): Options for selecting variables within the Prelim stage, affecting criteria and file indices.
- general_options (GeneralOptions): General options (e.g. the RNG seed), not specific to any one stage.
- algo_labels (list[str]): Algorithm names aligned with performance columns; propagated after pruning.
Create new instance of PrelimInput(x, y, x_raw, y_raw, s, inst_labels, prelim_options, selvars_options, general_options, algo_labels)
Inputs for applying fitted PRELIM feature transformations.
Outputs for the Prelim stage.
Attributes
- med_val (NDArray[np.double]): Median values of each feature across instances in the processed data.
- iq_range (NDArray[np.double]): Interquartile range of each feature, representing the spread of data between the 25th and 75th percentiles.
- hi_bound (NDArray[np.double]): Upper bound values for each feature based on specified statistical measures.
- lo_bound (NDArray[np.double]): Lower bound values for each feature based on specified statistical measures.
- min_x (NDArray[np.double]): Minimum values for each feature in the raw feature matrix.
- lambda_x (NDArray[np.double]): Box-Cox transformation parameters for each feature, if applicable.
- mu_x (NDArray[np.double]): Mean values of each feature across instances in the processed data.
- sigma_x (NDArray[np.double]): Standard deviation of each feature across instances in the processed data.
- min_y (float): Minimum value observed in the raw performance data.
- lambda_y (NDArray[np.double]): Box-Cox transformation parameters for the performance matrix, if applicable.
- sigma_y (NDArray[np.double]): Standard deviation of performance values across instances.
- mu_y (NDArray[np.double]): Mean values of performance across instances for each algorithm.
- x (NDArray[np.double]): Processed feature matrix, where each row represents an instance and each column represents a feature.
- y (NDArray[np.double]): Processed performance matrix, containing performance values for each instance-algorithm pair.
- x_raw (NDArray[np.double]): Original, unprocessed feature matrix containing raw values of each instance-feature pair.
- y_raw (NDArray[np.double]): Original, unprocessed performance matrix containing raw values for each instance-algorithm pair.
- y_bin (NDArray[np.bool_]): Binary matrix indicating instances with good algorithm performance (True if performance is good).
- y_best (NDArray[np.double]): Best observed performance value for each instance across all algorithms.
- p (NDArray[np.int_]): Index of the best-performing algorithm for each instance (1-based, matching MATLAB).
- num_good_algos (NDArray[np.double]): Number of good algorithms per instance.
- beta (NDArray[np.bool_]): Per-instance beta-easy indicator: good count strictly exceeds the configured fraction of retained algorithms.
- instlabels (pd.Series | None): Labels for each instance in the dataset, if provided.
- data_dense (DataDense | None): Dense data representation, if available, containing compressed or alternative feature representations.
- s (pd.Series | None): Optional source labels aligned with the retained instance rows.
Create new instance of PrelimOutput(med_val, iq_range, hi_bound, lo_bound, min_x, lambda_x, mu_x, sigma_x, min_y, lambda_y, sigma_y, mu_y, x, y, x_raw, y_raw, y_bin, y_best, p, num_good_algos, beta, instlabels, data_dense, s, algo_labels)
See file docstring.
See file docstring.
Apply fitted bounds and normalisation without re-fitting PRELIM.
Perform preliminary processing on the input data 'x' and 'y'.
Parameters
- x (NDArray[np.double]): The feature matrix (instances x features) to process.
- y (NDArray[np.double]): The performance matrix (instances x algorithms) to process.
- x_raw (NDArray[np.double]): The raw feature matrix.
- y_raw (NDArray[np.double]): The raw performance matrix.
- s (pd.Series | None): The instance source labels, or None.
- inst_labels (pd.Series): The instance labels.
- prelim_opts (PrelimOptions): The options for the preliminary processing.
- selvars_opts (SelvarsOptions): The options for the instance and feature selection.
- general_opts (GeneralOptions): The general options for all stages.