Version 2.0.0b1 (#892)

- Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support multi-fidelity, multi-objective and multi-threading by nature now.
- Expected behaviour for ask-and-tell interface ensured (also for Successive Halving).
- Continuing a run is now fully supported.
- Added more examples.
- Updated documentation based on new implementation.
- Added benchmark to compare different versions.

Co-authored-by: Sarah Krebs <[email protected]>
Co-authored-by: Carolin Benjamins <[email protected]>
Co-authored-by: dengdifan <[email protected]>
Co-authored-by: helegraf <[email protected]>
Co-authored-by: timruhkopf <[email protected]>
Co-authored-by: Alexander Tornede <[email protected]>
Co-authored-by: Eric Kalosa-Kenyon <[email protected]>

Author: 8 people

.flake8

@@ -6,5 +6,7 @@ extend-exclude =
     .venv
     build
 extend-ignore =
-    E203  # No whitespace before ':' in [x : y]
-    E731  # No lambdas — too strict
+    # No whitespace before ':' in [x : y]
+    E203
+    # No lambdas — too strict
+    E731

.github/workflows/pre-commit.yml

@@ -25,15 +25,16 @@ jobs:
       with:
         submodules: recursive
 
-    - name: Setup Python 3.9
+    - name: Setup Python 3.10
       uses: actions/setup-python@v4
       with:
-        python-version: 3.9
+        python-version: "3.10"
 
     - name: Install pre-commit
       run: |
         pip install pre-commit
         pre-commit install
+
     - name: Run pre-commit
       run: |
         pre-commit run --all-files

.gitignore

@@ -143,4 +143,8 @@ dmypy.json
 docker
 
 # Others
-src
+src
+
+# Pycharm
+.idea
+.vscode

.pre-commit-config.yaml

@@ -2,7 +2,6 @@
 # Click the repo and update to latest tags.
 # If things break on update, raise an issue
 
-
 repos:
   - repo: https://github.com/pycqa/isort
     rev: 5.10.1
@@ -35,13 +34,12 @@ repos:
       - id: mypy
         name: mypy smac
         files: smac
-        exclude: "scripts|tests|smac/utils/dependencies.py|smac/__init__.py"
+        exclude: "scripts|tests"
 
-  - repo: https://gitlab.com/pycqa/flake8
-    rev: 4.0.1
+  - repo: https://github.com/PyCQA/flake8
+    rev: 5.0.4
     hooks:
       - id: flake8
         name: flake8 smac
         files: smac
         exclude: "scripts|tests"
-

changelog.md renamed to CHANGELOG.md

@@ -1,3 +1,14 @@
+# 2.0.0b1
+
+- Completely reimplemented the intensifiers (including Successive Halving and Hyperband): All intensifiers support 
+multi-fidelity, multi-objective and multi-threading by nature now.
+- Expected behaviour for ask-and-tell interface ensured (also for Successive Halving).
+- Continuing a run is now fully supported.
+- Added more examples.
+- Updated documentation based on new implementation.
+- Added benchmark to compare different versions.
+
+
 # 2.0.0a2
 
 ## Bugfixes

CONTRIBUTING.md

@@ -40,18 +40,18 @@ If you found a bug, please provide us the following information:
 ## Work on New Features
 
 To work on new features, create a fork of the original repository and implement the feature there. When you are happy with the final result you can create a pull request which we will review.
-A good tutorial on how to do this is in the Github Guide: [Fork a repo](https://help.github.com/articles/fork-a-repo/).
+A good tutorial on how to do this is in the GitHub Guide: [Fork a repo](https://help.github.com/articles/fork-a-repo/).
 
 ## <a name="pull-requests"></a> Pull Requests
 
-The target branch for your pull request has to be the development branch. If you have not worked with pull requests, please take a look at [how to Contribute to an Open Source Project on GitHub](https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github) or read more about it in the official github documentation <https://help.github.com/articles/about-pull-requests/>
+The target branch for your pull request has to be the development branch. If you have not worked with pull requests, please take a look at [how to Contribute to an Open Source Project on GitHub](https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github) or read more about it in the official GitHub documentation <https://help.github.com/articles/about-pull-requests/>
 
 If you know how to fix a bug or implement your own feature, follow this small guide:
 
 - Check the issue tracker if someone has already reported the same idea or found the same bug.
 - Create a pull request, after you have implemented your changes in your fork. Using a separate branch for the fix is required.
 - Pull request must include tests.
-- We are using the Google Style Python Docstrings. Take a look at this [example](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html). 
+- We are using the Numpy Style Python Docstrings. Take a look at this [example](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html#example-numpy). 
 - The code should follow the PEP8 coding convention.
 
 We try to react as fast as possible to your pull request.

Makefile

@@ -133,7 +133,7 @@ clean-data:
 # Will echo the commands to actually publish to be run to publish to actual PyPi
 # This is done to prevent accidental publishing but provide the same conveniences
 publish: clean build
-	read -p "Did you update the version number?"
+	read -p "Did you update the version number? Did you add the old version to docs/conf.py?"
 
 	$(PIP) install twine
 	$(PYTHON) -m twine upload --repository testpypi ${DIST}/*

README.md

@@ -1,4 +1,4 @@
-# Sequential Model Algorithm Configuration (SMAC)
+# SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization
 
 
 [![Tests](https://github.com/automl/SMAC3/actions/workflows/pytest.yml/badge.svg?branch=main)](https://github.com/automl/SMAC3/actions/workflows/pytest.yml)
@@ -15,25 +15,28 @@ aggressive racing mechanism to efficiently decide which of two configurations pe
 SMAC3 is written in Python3 and continuously tested with Python 3.8, 3.9, and 3.10. Its Random
 Forest is written in C++. In further texts, SMAC is representatively mentioned for SMAC3.
 
-> [Documention](https://automl.github.io/SMAC3)
+> [Documentation](https://automl.github.io/SMAC3)
 
 > [Roadmap](https://github.com/orgs/automl/projects/5/views/2)
 
 
 ## Important: Changes in v2.0
 
 With the next big major release of SMAC, we drastically boosted the user experience by improving the APIs and how the 
-pipelining is done (see [changelog](CHANGELOG.md)). However, cleaning the code base, we removed the command-line 
-interface (calling a target function from a script is still supported), runtime optimization, and pSMAC. Also,
- python 3.7 is not supported anymore. If you depend on those functionalities, we kindly ask you to keep using v1.4.
+pipelining is done (see [changelog](CHANGELOG.md)). All facades/intensifiers support multi-objective, multi-fidelity, 
+and multi-threading natively now! That includes having an ask-and-tell interface and continuing a run
+wherever you left off. pSMAC is removed because when specifying the number of workers, SMAC automatically uses 
+multi-threading for evaluating trials. When cleaning the code base, however, we removed the command-line 
+interface (calling a target function from a script is still supported), and runtime optimization. Also,
+python 3.7 is not supported anymore. If you depend on those functionalities, please keep using v1.4.
 
 We are excited to introduce the new major release and look forward to developing new features on the new code base. 
 We hope you enjoy this new user experience as much as we do. 🚀
 
 
 ## Installation
 
-This instruction is for the installation on a Linux system, for Windows and Mac and further information see the [documention](https://automl.github.io/SMAC3/main/installation.html).
+This instruction is for the installation on a Linux system, for Windows and Mac and further information see the [documentation](https://automl.github.io/SMAC3/main/1_installation.html).
 
 Create a new environment with python 3.10 and make sure swig is installed either on your system or
 inside the environment. We demonstrate the installation via anaconda in the following:
@@ -91,7 +94,7 @@ smac = HyperparameterOptimizationFacade(scenario, train)
 incumbent = smac.optimize()
 ```
 
-More examples can be found in the [documention](https://automl.github.io/SMAC3/main/examples/).
+More examples can be found in the [documentation](https://automl.github.io/SMAC3/main/examples/).
 
 
 ## License
@@ -114,8 +117,8 @@ SMAC3 is developed by the [AutoML Groups of the Universities of Hannover and
 Freiburg](http://www.automl.org/).
 
 If you have found a bug, please report to [issues](https://github.com/automl/SMAC3/issues). Moreover, we are 
-appreciating any kind of help. Find our guidlines for contributing to this package 
-[here](https://github.com/automl/SMAC3/blob/master/.github/CONTRIBUTING.md).
+appreciating any kind of help. Find our guidelines for contributing to this package 
+[here](CONTRIBUTING.md).
 
 If you use SMAC in one of your research projects, please cite our 
 [JMLR paper](https://jmlr.org/papers/v23/21-0888.html):

benchmark/.gitignore

@@ -0,0 +1 @@
+!src

benchmark/README.md

@@ -0,0 +1,22 @@
+# SMAC Benchmark (Beta)
+
+Calculates multiple tasks for specified versions and compares performance.
+Each version is derived from pip and installed in an empty environment (``SMACBenchmark``).
+
+
+## Results
+
+![trajectory_trials](report/trajectory_walltime.png "Trajectory (Walltime)")
+
+
+## Getting Started
+
+- Make sure you have anaconda installed.
+- Execute ``bash ./run.sh`` inside the benchmark directory. You can specify the versions to run there.
+- Have your current version of SMAC installed in the environment ``SMAC``. After the old versions have been finished, the script benchmarks the current version.
+- Alternatively, just execute ``python src/benchmark.py`` with a SMAC environment of your   choice.
+
+
+## Note
+
+- Versions before 2.0 might not support the new sklearn (>1.2) anymore