Advisor is the hyper parameters tuning system for black box optimization.
It is the open-source implementation of Google Vizier with these features.
- Easy to use with API, SDK, WEB and CLI
- Support abstractions of Study and Trial
- Included search and early stop algorithms
- Recommend parameters with trained model
- Same programming interfaces as Google Vizier
- Command-line tool just like Microsoft NNI.
- Grid Search
- Random Search
- Bayesian Optimization
- TPE(Hyperopt)
- Random Search(Hyperopt)
- Simulate Anneal(Hyperopt)
- Quasi Random(Chocolate)
- Grid Search(Chocolate)
- Random Search(Chocolate)
- Bayes(Chocolate)
- CMAES(Chocolate)
- MOCMAES(Chocolate)
- SMAC Algorithm
- Bayesian Optimization(Skopt)
- Early Stop First Trial Algorithm
- Early Stop Descending Algorithm
- Performance Curve Stop Algorithm
It is easy to setup advisor service in local machine.
pip install advisor
advisor_admin server start
Then go to http://127.0.0.1:8000
in the browser and submit tuning jobs.
git clone --depth 1 https://github.com/tobegit3hub/advisor.git && cd ./advisor/
advisor run -f ./advisor_client/examples/python_function/config.json
advisor study describe -s demo
Run server with official package.
advisor_admin server start
Or run with official docker image.
docker run -d -p 8000:8000 tobegit3hub/advisor
Or run with docker-compose.
wget https://raw.githubusercontent.com/tobegit3hub/advisor/master/docker-compose.yml
docker-compose up -d
Or run in Kubernetes cluster.
wget https://raw.githubusercontent.com/tobegit3hub/advisor/master/kubernetes_advisor.yaml
kubectl create -f ./kubernetes_advisor.yaml
Or run from scratch with source code.
git clone --depth 1 https://github.com/tobegit3hub/advisor.git && cd ./advisor/
pip install -r ./requirements.txt
./manage.py migrate
./manage.py runserver 0.0.0.0:8000
Install with pip
or use docker container.
pip install advisor
docker run -it --net=host tobegit3hub/advisor bash
Use the command-line tool.
export ADVISOR_ENDPOINT="http://127.0.0.1:8000"
advisor study list
advisor study describe -s "demo"
advisor trial list --study_name "demo"
Use admin tool to start/stop server.
advisor_admin server start
advisor_admin server stop
Use the Python SDK.
client = AdvisorClient()
# Create the study
study_configuration = {
"goal": "MAXIMIZE",
"params": [
{
"parameterName": "hidden1",
"type": "INTEGER",
"minValue": 40,
"maxValue": 400,
"scalingType": "LINEAR"
}
]
}
study = client.create_study("demo", study_configuration)
# Get suggested trials
trials = client.get_suggestions(study, 3)
# Complete the trial
trial = trials[0]
trial_metrics = 1.0
client.complete_trial(trial, trial_metrics)
Please checkout examples for more usage.
Study configuration describe the search space of parameters. It supports four types and here is the example.
{
"goal": "MAXIMIZE",
"randomInitTrials": 1,
"maxTrials": 5,
"maxParallelTrials": 1,
"params": [
{
"parameterName": "hidden1",
"type": "INTEGER",
"minValue": 1,
"maxValue": 10,
"scalingType": "LINEAR"
},
{
"parameterName": "learning_rate",
"type": "DOUBLE",
"minValue": 0.01,
"maxValue": 0.5,
"scalingType": "LINEAR"
},
{
"parameterName": "hidden2",
"type": "DISCRETE",
"feasiblePoints": "8, 16, 32, 64",
"scalingType": "LINEAR"
},
{
"parameterName": "optimizer",
"type": "CATEGORICAL",
"feasiblePoints": "sgd, adagrad, adam, ftrl",
"scalingType": "LINEAR"
},
{
"parameterName": "batch_normalization",
"type": "CATEGORICAL",
"feasiblePoints": "true, false",
"scalingType": "LINEAR"
}
]
}
Here is the configuration file in JSON format for advisor run
.
{
"name": "demo",
"algorithm": "BayesianOptimization",
"trialNumber": 10,
"concurrency": 1,
"path": "./advisor_client/examples/python_function/",
"command": "./min_function.py",
"search_space": {
"goal": "MINIMIZE",
"randomInitTrials": 3,
"params": [
{
"parameterName": "x",
"type": "DOUBLE",
"minValue": -10.0,
"maxValue": 10.0,
"scalingType": "LINEAR"
}
]
}
}
Or use the equivalent configuration file in YAML format.
name: "demo"
algorithm: "BayesianOptimization"
trialNumber: 10
path: "./advisor_client/examples/python_function/"
command: "./min_function.py"
search_space:
goal: "MINIMIZE"
randomInitTrials: 3
params:
- parameterName: "x"
type: "DOUBLE"
minValue: -10.0
maxValue: 10.0
List all the studies and create/delete the studies easily.
List the detail of study and all the related trials.
List all the trials and create/delete the trials easily.
List the detail of trial and all the related metrics.
You can edit the source code and test without re-deploying the server and client.
git clone [email protected]:tobegit3hub/advisor.git
cd ./advisor/advisor_client/
python ./setup.py develop
export PYTHONPATH="/Library/Python/2.7/site-packages/:$PYTHONPATH"