Conditional PED-ANOVA: Hyperparameter Importance in Hierarchical & Dynamic Search Spaces

Kaito Baba   Yoshihiko Ozaki   Shuhei Watanabe

We propose conditional PED-ANOVA (condPED-ANOVA), a principled framework for estimating hyperparameter importance (HPI) in conditional search spaces, where the presence or domain of a hyperparameter can depend on other hyperparameters. Although the original PED-ANOVA provides a fast and efficient way to estimate HPI within the top-performing regions of the search space, it assumes a fixed, unconditional search space and therefore cannot properly handle conditional hyperparameters. To address this, we introduce a conditional HPI for top-performing regions and derive a closed-form estimator that accurately reflects conditional activation and domain changes. Experiments show that naive adaptations of existing HPI estimators yield misleading or uninterpretable importances in conditional settings, whereas condPED-ANOVA consistently provides meaningful importances that reflect the underlying conditional structure.

🚀 Quick Start 🚀

✨️ cond_ped_anova is fully compatible with Optuna’s built-in hyperparameter importance API.

1. Installation

uv add git+https://github.com/kAIto47802/condPED-ANOVA.git
# If you're using pip:
# pip install git+https://github.com/kAIto47802/condPED-ANOVA.git

2. Usage

import optuna
from cond_ped_anova import CondPedAnovaImportanceEvaluator


def objective(trial: optuna.trial.Trial) -> float:
    c = trial.suggest_float("c", 0.0, 1.0)
    if c < 0.5:
        x = trial.suggest_float("x", -5.0, -2.0)
        return x
    else:
        y = trial.suggest_float("y", 2.0, 5.0)
        return y


sampler = optuna.samplers.RandomSampler(seed=42)
study = optuna.create_study(direction="minimize", sampler=sampler)
study.optimize(objective, n_trials=1000)

evaluator = CondPedAnovaImportanceEvaluator()
importance = optuna.importance.get_param_importances(study, evaluator=evaluator)

📊 Reproducing the Results 📊

🔁 All experiments in our paper are fully reproducible.

1. Clone this Repository & Install Dependencies

git clone https://github.com/kAIto47802/condPED-ANOVA.git

uv sync --python 3.13 --extra experiments
# If you're using pip:
# pip install --upgrade pip  # enable PEP 660 support
# pip install -e '.[experiments]'

2. Run Experiments

Running the commands below reproduces the corresponding experiments and regenerates the exact figures:

• condPED-ANOVA on synthetic problems (Figures 1, 7, and 11):

  ./run_cond_ped_anova.sh

• Baseline methods on synthetic problems (Figures 2, 9, and 10):

  ./run_baselines.sh

• Ablation study (Figures 3 and 4):

  ./run_ablations.sh

• condPED-ANOVA on real-world benchmarks (Figures 5 and 14 and Tables 1 and 2):

  # prepare YAHPO Gym data
  git clone https://github.com/slds-lmu/yahpo_data.git
  
  ./run_yahpo_gym.sh

• Runtime comparison (Figure 6):

  ./run_runtime_comparison.sh

• condPED-ANOVA Results with Different $N$ (Figure 8):

  ./run_cond_ped_anova_different_n.sh

• Additional Experiments on Synthetic Problems (Figures 12 and 13):

  ./run_cond_ped_anova_additional.sh

3. Check the Results

The generated figures will be saved in the figures/ directory. The raw experiment outputs (pickled results) and the values used in the tables are saved under results/.

Note

If a LaTeX environment is not available, the figures will be rendered without LaTeX, which may slightly change font rendering and layout.

🔖 Citation 🔖

If you find condPED-ANOVA useful in your research, please consider citing the following paper:

@article{baba2026condpedanova,
  title={Conditional {PED-ANOVA}: Hyperparameter Importance in Hierarchical \& Dynamic Search Spaces},
  author={Baba, Kaito and Ozaki, Yoshihiko and Watanabe, Shuhei},
  journal={arXiv preprint arXiv:2601.20800},
  year={2026},
}