Supervised data suffers severe selection bias when labels are expensive. We formulate a MDP over posterior beliefs on model performance and solve it with pathwise policy gradients computed through an auto-differentiable pipeline. The paper is available here.
Key Features:
- Adaptive Labeling - MDPs with combinatorial action space
- Uncertainty Quantification - Gaussian Processes, Deep Learning based UQ methodologies (Ensembles, Ensemble+, ENNs)
- Policy parametrization through K-subset sampling
- Policy gradients through Autodiff - Smoothed differentiable pipeline
Project_Name/
│
├── Main/ # Source code for the project
│ ├── gp_experiments
│ │ ├── gp_pipeline_regression
│ │ │ ├── run_pipeline_long_horizon.py
│ │ │ ├── run_pipeline_pg_long_horizon.py
│ │ │ ├── run_pipeline_active_learning_long_horizon.py
│ │ │ └── ....
│ │ │
│ │ └── gp_pipeline_regression_real_data
│ │ ├── run_pipeline_long_horizon.py
│ │ ├── run_pipeline_pg_long_horizon.py
│ │ ├── run_pipeline_active_learning_long_horizon.py
│ │ └── ....
│ └── ensemble_plus_experiments
│ ├── ensemble_plus_pipeline_regression
│ │ ├── run_enn_pipeline_1a.py
│ │ ├── run_pipeline_pg_ensemble_plus_long_horizon.py
│ │ └── ....
│ └── ensemble_plus_pipeline_regression_active_learning
│ ├── run_enn_pipeline_1a_active_learning.py
│ └── ....
│
│
├── src/ # Source code for ongoing research (under development)
│ ├── autodiff # Autodiff (Smoothed-Differentiable) pipeline development - different UQ methodologies, integration with baselines
│ │ ├── gp
│ │ ├── ensemble_plus
│ │ ├── enn
│ │ └── deprecated # Deprecated code
│ ├── baselines # REINFORCE based policy gradient pipeline development
│ └── notebooks # Notebooks for unit tests, testing individual components of the pipeline
│
├── requirements.txt # List of dependencies
└── README.md # Project documentation
-
Clone the repository:
git clone https://github.com/namkoong-lab/adaptive-labeling.git cd adaptive-labeling
-
Install dependencies:
We use Python 3.10.13 for our experiments.
pip install -r requirements.txt
To ensure a consistent and isolated environment, you can create a virtual environment using venv
or conda
.
python -m venv env
source env/bin/activate # On Windows use `env\Scripts\activate`
pip install -r requirements.txt
conda create -n project_env python=3.10
conda activate project_env
pip install -r requirements.txt
-
We currently use weights and biases (Link) to track our project and our code is integrated with wandb. See (Link) for setting up an account.
-
Accordingly one might need to edit files for including their own ``ENTITY'' name on wandb.
For example - In line 288 of "Main/gp_experiments/gp_pipeline_regression/run_pipeline_long_horizon.py" - put your own entity name
-
After setting up the environment, one can run various pipelines (AUTODIFF, REINFORCE, ACTIVE LEARNING) of the project using following command line (similar commnad line for other pipelines) :
python Main/gp_experiments/gp_pipeline_regression/run_pipeline_long_horizon.py --config_file_path Main/gp_experiments/gp_pipeline_regression/config_sweep_0.json --project_name gp_adaptive_sampling_final_run
@inproceedings{
mittal2024adaptive,
title={Adaptive Labeling for Efficient Out-of-distribution Model Evaluation},
author={Daksh Mittal and Yuanzhe Ma and Shalmali Joshi and Hongseok Namkoong},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=uuQQwrjMzb}
}