instancespace.stages.pythia
Train and evaluate algorithm selectors from 2D or 3D instance coordinates.
PYTHIA standardizes coordinates, fits one binary learner per retained algorithm, and combines predictions with a saved training selection policy. The registry supports SVM, KNN, tree, kernel Naive Bayes, linear and ensemble learners. Hyperparameters use Sobol search, Bayesian search or supplied values. Evaluation applies fitted classifiers without refitting and scores only observed outcomes.
Inputs for the Pythia stage.
Attributes
- z (NDArray[np.double]): The feature matrix.
- y (NDArray[np.double]): The performance metrics.
- y_bin (NDArray[np.bool_]): The binary labels.
- y_best (NDArray[np.double]): The best performance metrics.
- algo_labels (list[str]): The algorithm labels.
- pythia_options (PythiaOptions): The options for the Pythia stage.
- parallel_options (ParallelOptions): The parallel options, specifiy whether run in parallel and number of cores.
- general_options (GeneralOptions): General options (e.g. the RNG seed), not specific to any one stage.
Create new instance of PythiaInput(z, y_raw, y_bin, y_best, algo_labels, pythia_options, parallel_options, general_options)
Outputs from the Pythia stage.
Attributes
- mu (NDArray[np.double] | list[float]): The mean values of the raw feature coordinates.
- sigma (NDArray[np.double] | list[float]): The standard deviations used to normalize raw feature coordinates.
- w (NDArray[np.double]): The weight matrix used for cost-sensitive classification.
- cp (list[StratifiedKFold | None]):
The per-algorithm Stratified K-Fold cross-validators. A degenerate or
skipped algorithm has
None, matching MATLAB's empty cell entry. - svm (list[ClassifierMixin | None]):
The trained classifiers, one per algorithm -
SVCinstances unlessPythiaOptions.classifierselected a different registered type. The field is still namedsvmfor backward compatibility; it holds whateverPythiaOptions.classifierchose to train,'svm'by default. A skipped algorithm hasNone, matching MATLAB's empty classifier cell. - cvcmat (NDArray[np.double]): Confusion matrix for each algorithm
- y_sub (NDArray[np.bool_]): The binary predicted labels for each algorithm.
- y_hat (NDArray[np.bool_]): The final predicted labels for each algorithm.
- pr0_sub (NDArray[np.double]):
The predicted cross-validated probabilities of class 0 ("bad"),
matching MATLAB's
Pr0subconvention. - pr0_hat (NDArray[np.double]):
The predicted probabilities of class 0 ("bad") on the full data,
matching MATLAB's
Pr0hatconvention. - box_consnt (list[float]): First hyperparameter in reporting units; BoxConstraint for SVM.
- k_scale (list[float]):
Second hyperparameter in reporting units, or NaN for one-parameter
learners. For SVM this is KernelScale, with estimator coefficient
gamma = 1 / KernelScale**2. - accuracy (list[float]): Held-out accuracy for each algorithm classifier.
- precision (list[float]): Held-out precision for each algorithm classifier.
- recall (list[float]): Recall scores for each algorithm
- selection0 (NDArray[np.int_]): The selected algorithm indices for each instance.
- selection1 (NDArray[np.int_]): The backup selected algorithm indices for each instance.
- pythia_summary (pd.DataFrame): A summary table for performance statistics of all algorithms.
Create new instance of PythiaOutput(mu, sigma, w, cp, svm, cvcmat, y_sub, y_hat, pr0_sub, pr0_hat, box_consnt, k_scale, accuracy, precision, recall, selection0, selection1, pythia_summary)
Inputs for applying fitted PYTHIA classifiers to projected instances.
Predictions and precision-weighted selections for unseen instances.
Inputs for evaluating fitted-classifier predictions against test truth.
MATLAB-compatible per-algorithm confusion counts and rates.
Create new instance of PythiaEvaluateOutput(accuracy, precision, recall, cvcmat)
Pythia stage for automated algorithm selection.
The PythiaStage class is the main class for the Pythia stage. It
contains the main function pythia that runs the Pythia stage.
Methods
_inputs() -> type[PythiaInput] Return the input type for the Pythia stage.
_outputs() -> type[PythiaOutput] Return the output type for the Pythia stage.
_run(inputs: PythiaInput) -> PythiaOutput Run the Pythia stage.
pythia(z: NDArray[np.double], y: NDArray[np.double], y_bin: NDArray[np.bool_], y_best: NDArray[np.double], algo_labels: list[str], opts: PythiaOptions, parallel_options: ParallelOptions) -> PythiaOutput Main method that perform automated algorithm selection.
_fit_classifier(z: NDArray[np.double], y_bin: NDArray[np.bool_], w: NDArray[np.double], skf: StratifiedKFold, classifier_name: str, is_poly_kernel: bool, param_space: dict[str, list[float]] | None, use_weights: bool, parallel_options: ParallelOptions, general_options: GeneralOptions, n_tuning_iter: int) -> _ClassifierResult Train the classifier selected by PythiaOptions.classifier.
_display_overall_perf(precision: list[float], accuracy: list[float]) -> None Output overall performance metrics.
_compute_znorm(z: NDArray[np.double]) -> tuple[NDArray[np.double], NDArray[np.double], NDArray[np.double]] Compute normalized feature matrix.
_check_precalcparams(params: NDArray[np.double] | None, nalgos: int) -> NDArray[np.double] | None Check pre-calculated hyper-parameters.
_determine_selections(nalgos: int, precision: list[float], y_hat: NDArray[np.bool_], y_bin: NDArray[np.bool_]) -> tuple[NDArray[np.int_], NDArray[np.int_]] Determine the selections based on the precision metrics.
_generate_summary(nalgos: int, algo_labels: list[str], y: NDArray[np.double], y_hat: NDArray[np.bool_], y_bin: NDArray[np.bool_], y_best: NDArray[np.double], selection0: NDArray[np.int_], selection1: NDArray[np.int_], accuracy: list[float], precision: list[float], recall: list[float], box_consnt: list[float], k_scale: list[float]) -> pd.DataFrames Generate a summary of the results.
Define the input for the Pythia stage.
Parameters
- z (NDArray[np.double]): The feature matrix.
- y_raw (NDArray[np.double]): The performance metrics.
- y_bin (NDArray[np.bool_]): The binary labels.
- y_best (NDArray[np.double]): The best performance metrics.
- algo_labels (list[str]): The algorithm labels.
Apply persisted PYTHIA state without fitting or mutating classifiers.
This is MATLAB's PYTHIAevalMode prediction core: normalize new
coordinates with the parameters stored by the training run, apply each
fitted classifier, pad test-only algorithms with empty predictions, and
make the same precision-weighted recommendation as the build path.
Evaluate predictions with MATLAB's explicit confusion-count formulas.
MATLAB stores confusion counts in column-major order: [TN, FN, FP, TP].
Only observed instance/algorithm pairs contribute. inputs.observed
has shape (n_instances, n_trained_algorithms) and must come from raw
outcome availability, not binary labels. Accuracy uses the observed
count for each algorithm. Missing classifiers, wholly unobserved
algorithms and test-only algorithms retain NaN counts and rates.
Run the Pythia stage.
Parameters
- z (NDArray[np.double]): The feature matrix.
- y (NDArray[np.double]): The performance metrics.
- y_bin (NDArray[np.bool_]): The binary labels.
- y_best (NDArray[np.double]): The best performance metrics.
- algo_labels (list[str]): The algorithm labels.
- opts (PythiaOptions): The options for the Pythia stage.
- parallel_options (ParallelOptions): The parallel options, specifiy whether run in parallel and number of cores.
- general_options (GeneralOptions): General options (e.g. the RNG seed), not specific to any one stage.
Returns
- PythiaOutput: The output of the Pythia stage.