instancespace.stages.sifted
SIFTED Stage: Feature Selection and Optimisation.
This module implements the SIFTED stage, which focuses on feature selection, clustering, and optimization to enhance data analysis outcomes. The SIFTED algorithm performs several key steps:
- Conduct feature correlation analysis to identify the most relevant features based on their correlation with performance metrics.
- Cluster features to group similar ones, further reducing dimensionality and improving selection efficiency.
- Select feature combinations with a genetic algorithm, projecting candidates through PILOT in the configured dimensionality and maximizing the worst per-algorithm KNN cross-validation accuracy. A per-search cache avoids recomputing candidate subsets.
- Filter instances using a filtering method if the corresponding flag is raised
The SIFTED stage is encapsulated in the SiftedStage class, which manages
the entire feature selection process. It includes utility methods for
calculating correlations, evaluating clusters, and selecting optimal feature
combinations based on performance metrics.
Dependencies:
- numpy
- pandas
- pygad
- scipy
- scikit-learn
Classes
SiftedStage : The primary class that implements the SIFTED stage, providing methods for feature selection, clustering, and optimization through various algorithms.
Functions
sifted(x, y, y_bin, x_raw, y_raw, beta, num_good_algos, y_best, p, inst_labels, s, feat_labels, opts, opts_selvars, data_dense, parallel_options): The main function to execute the SIFTED stage for processing feature selection and performance optimization using provided datasets and configuration options.
Raised when there is not enough feature to continue feature selection.
Inputs for the Sifted stage.
Attributes
- x (NDArray[np.double]): Feature matrix to be processed (instances x features).
- y (NDArray[np.double]): Algorithm performance matrix (instances x algorithms).
- y_bin (NDArray[np.bool_]): Binary matrix indicating good algorithm performance per instance and algorithm.
- x_raw (NDArray[np.double]): Unprocessed feature matrix (instances x features).
- y_raw (NDArray[np.double]): Unprocessed performance matrix (instances x algorithms).
- beta (NDArray[np.bool_]): Per-instance beta-easy labels (instances x 1).
- num_good_algos (NDArray[np.double]): Number of good algorithms for each instance.
- y_best (NDArray[np.double]): Best observed algorithm performance per instance.
- p (NDArray[np.int_]): One-based best-algorithm index for each instance (instances x 1).
- inst_labels (pd.Series): Labels identifying instances in the dataset.
- feat_labels (list[str]): List of labels for each feature.
- s (pd.Series, optional): Selection series used during the processing stage, if any.
- sifted_options (SiftedOptions): Configuration options for the Sifted stage.
- selvars_options (SelvarsOptions): Configuration options for variable selection.
- data_dense (DataDense, optional): Dense data representation, if applicable.
- parallel_options (ParallelOptions): Configuration options for parallel processing.
- general_options (GeneralOptions): General options (e.g. the RNG seed), not specific to any one stage.
Create new instance of SiftedInput(x, y, y_bin, x_raw, y_raw, beta, num_good_algos, y_best, p, inst_labels, feat_labels, s, sifted_options, selvars_options, data_dense, parallel_options, general_options, pilot_options)
Inputs for applying fitted SIFTED feature selection.
Outputs from the Sifted stage.
Attributes
- x (NDArray[np.double]): Processed feature matrix (instances x features).
- y (NDArray[np.double]): Processed algorithm performance matrix (instances x algorithms).
- y_bin (NDArray[np.bool_]): Binary matrix indicating good algorithm performance for processed instances.
- x_raw (NDArray[np.double]): Unprocessed feature matrix (instances x features).
- y_raw (NDArray[np.double]): Unprocessed performance matrix (instances x algorithms).
- beta (NDArray[np.bool_]): Per-instance beta-easy labels.
- num_good_algos (NDArray[np.double]): Number of good algorithms per retained instance.
- y_best (NDArray[np.double]): Best observed algorithm performance per instance after processing.
- p (NDArray[np.int_]): One-based best-algorithm index for each instance.
- inst_labels (pd.Series): Labels for each instance in the dataset.
- s (pd.Series, optional): Optional source labels aligned with the retained instances.
- feat_labels (list[str]): List of feature labels.
- selvars (NDArray[np.intc]): Array of indices for selected features after the sifted stage.
- rho (NDArray[np.double], optional): Array of coefficients or weights for features after the sifted stage.
- pval (NDArray[np.double], optional): Performance metric values for features after the sifted stage.
- silhouette_scores (list[float], optional): List of silhouette scores for the sifted features.
- clust (NDArray[np.bool_], optional): Boolean array indicating selected features (True if selected, False otherwise).
Create new instance of SiftedOutput(x, y, y_bin, x_raw, y_raw, beta, num_good_algos, y_best, p, inst_labels, s, feat_labels, selvars, rho, pval, silhouette_scores, clust)
Stage in the processing pipeline for feature selection and instance filtering.
Attributes
- MIN_FEAT_REQUIRED (int): Minimum number of features required for selection (default is 3).
- KFOLDS (int): Number of folds for cross-validation (default is 5).
- parallel_options (ParallelOptions): Options for parallel processing.
Methods
__init__(x: NDArray[np.double], y: NDArray[np.double], y_bin: NDArray[np.bool_], x_raw: NDArray[np.double], y_raw: NDArray[np.double], beta: NDArray[np.bool_], num_good_algos: NDArray[np.double], y_best: NDArray[np.double], p: NDArray[np.int_], inst_labels: pd.Series, s: pd.Series | None, feat_labels: list[str], opts: SiftedOptions, parallel_options: ParallelOptions) -> None: Initializes the SiftedStage with provided parameters.
_inputs() -> type[SiftedInput]: Returns the input type for the SIFTED stage.
_outputs() -> type[SiftedOutput]: Returns the output type for the SIFTED stage.
_compute_correlation(x: NDArray[np.double], y: NDArray[np.double]) -> tuple[NDArray[np.double], NDArray[np.double]]: Calculate the Pearson correlation coefficients between features and performance metrics.
evaluate_cluster(x_aux: NDArray[np.double], rng: np.random.Generator) -> tuple[list[float], NDArray[np.intc]]: Evaluates clusters based on silhouette scores for different cluster configurations.
_find_best_combination(x_aux: NDArray[np.double], clust: NDArray[np.bool_], selvars: NDArray[np.intc], rng: np.random.Generator) -> tuple[NDArray[np.double], NDArray[np.intc]]: Finds the best combination of features using a genetic algorithm.
cost_fcn(instance: pygad.GA, solutions: NDArray[np.intc], _solution_idx: int) -> float: Fitness function to evaluate the quality of a solution in the genetic algorithm
select_features_by_clustering(x_aux: NDArray[np.double], rng: np.random.Generator) -> tuple[NDArray[np.bool_], NDArray[np.intc]]: Selects features based on clustering results.
select_features_by_performance() -> tuple[NDArray[np.double], NDArray[np.double] , NDArray[np.double], NDArray[np.intc]]: Selects features based on correlation with performance metrics.
_sifted(opts_selvars: SelvarsOptions, data_dense: DataDense | None) -> SiftedOutput: Executes feature selection and optional instance filtering based on provided options.
sifted(x: NDArray[np.double], y: NDArray[np.double], y_bin: NDArray[np.bool_], x_raw: NDArray[np.double], y_raw: NDArray[np.double], beta: NDArray[np.bool_], num_good_algos: NDArray[np.double], y_best: NDArray[np.double], p: NDArray[np.int_], inst_labels: pd.Series, s: pd.Series | None, feat_labels: list[str], opts: SiftedOptions, opts_selvars: SelvarsOptions, data_dense: DataDense | None, parallel_options: ParallelOptions) -> SiftedOutput: Executes the core functionality of the Sifted stage for processing feature selection.
_run(inputs: SiftedInput) -> SiftedOutput: Executes the SIFTED stage of the pipeline using the provided inputs.
Define the input variables for the stage.
Parameters
- x (NDArray[np.double]): Feature matrix to be processed (instances x features).
- y (NDArray[np.double]): Algorithm performance matrix (instances x algorithms).
- y_bin (NDArray[np.bool_]): Binary matrix indicating good algorithm performance.
- feat_labels (list[str]): List of feature labels.
- x_raw (NDArray[np.double]): Raw feature matrix.
- y_raw (NDArray[np.double]): Raw performance matrix.
- beta (NDArray[np.bool_]): Per-instance beta-easy labels.
- num_good_algos (NDArray[np.double]): Number of good algorithms for each instance.
- y_best (NDArray[np.double]): Best algorithm performance for each instance.
- p (NDArray[np.int_]): One-based best-algorithm index for each instance.
- inst_labels (pd.Series): Instance labels for the dataset.
- s (pd.Series | None): The instance source labels, or None.
- opts (SiftedOptions): The configuration options for the SIFTED stage.
- parallel_options (ParallelOptions): An instance of ParallelOptions containing parallel processing parameters.
- general_opts (GeneralOptions): General options (e.g. the RNG seed), not specific to any one stage.
Apply fitted feature indices without re-running selection.
Public method to access the private _sifted method.
Core Sifted stage to process feature selection and performance.
Parameters
- x (NDArray[np.double]): Feature matrix (instances x features).
- y (NDArray[np.double]): Performance matrix of algorithms (instances x algorithms).
- y_bin (NDArray[np.bool_]): Binary matrix indicating instances with good algorithm performance.
- x_raw (NDArray[np.double]): Unprocessed feature matrix.
- y_raw (NDArray[np.double]): Unprocessed performance matrix.
- beta (NDArray[np.bool_]): Per-instance beta-easy labels.
- num_good_algos (NDArray[np.double]): Number of good algorithms per instance.
- y_best (NDArray[np.double]): Best algorithm performance for each instance.
- p (NDArray[np.int_]): Array of correlation p-values for features.
- inst_labels (pd.Series): Labels for each instance in the dataset.
- s (pd.Series | None): Series for optional selection during processing.
- feat_labels (list[str]): List of feature labels.
- opts (SiftedOptions): Configuration options for the Sifted stage.
- opts_selvars (SelvarsOptions): Options for selecting variables within the stage.
- data_dense (DataDense | None): Dense data representation, if available.
- parallel_options (ParallelOptions): Options for parallel processing.
- general_options (GeneralOptions): General options (e.g. the RNG seed), not specific to any one stage.
Returns
- SiftedOutput: Output of the Sifted stage, including selected feature and algorithm indices,correlation coefficients, performance metrics, silhouette scores, and feature selection flags.
Notes
This static method initialises a SiftedStage instance and calls its _sifted
method, which uses the provided options and data to execute the Sifted stage
pipeline and generate results.
Select features based on correlation with performance.
Returns
- x_aux (NDArray[np.double]): Filtered feature matrix after selection.
- rho (NDArray[np.double]): Pearson correlation coefficients between features and performance.
- pval (NDArray[np.double]): p-values for the correlation coefficients.
- selvars (NDArray[np.intc]): Indices of the selected features.
Select features based on clustering.
Parameters
- x_aux (NDArray[np.double]): Auxiliary feature matrix used for clustering.
- rng (np.random.Generator): Random number generator for reproducibility.
Returns
- clust (NDArray[np.bool_]): Boolean matrix where each column represents a cluster.
- cluster_labels (NDArray[np.intc]): Labels for the clusters.
Fitness function to evaluate the quality of solution in GA.
Parameters
- instance (pygad.GA): The instance of the genetic algorithm.
- solutions (NDArray[np.intc]): The array of integer values representing the solution to be evaluated.
- solution_idx (int): The index of the solution being evaluated.
Returns
- float: The fitness score of the solution: the worst (lowest) per-
algorithm k-NN classification accuracy.
pygad.GAalways maximizes whatever this returns, so maximizing the minimum per-algorithm accuracy directly is equivalent to MATLAB'sfitcknn/kfoldLoss-basedcostfcn(core/SIFTED.m), whichga()minimizes asmax_i(loss_i)--max_i(1 - acc_i) = min_i(acc_i) - 1, the same optimum up to an additive constant without changing the optimal feature set.
Evaluate cluster based on silhouette scores.
Parameters
- x_aux (NDArray[np.double]): Auxiliary feature matrix used for clustering.
- rng (np.random.Generator): Random number generator for reproducibility.
Returns
- silhouette_scores (list[float]): Silhouette scores for different cluster configurations.
- cluster_labels (NDArray[np.intc]): Labels for the selected cluster configuration.