Optimization Benchmarking

Release 0.8.9: Beta

17 Feb 2017

Today, version 0.8.9, the “Patched Beta” release of our optimizationBenchmarking.org software suite, was published. Compared to the previous release, this new version comes much closer to our goal to reduce the workload of researchers and practitioners during the development of algorithms to allow them to spend more time on thinking.

Besides all the functionality offered by the previous releases, it introduces a new process for obtaining high-level conclusions about problem hardness and algorithm behaviors. This process takes the raw data from experiments together with meta-information about algorithm setups and problem instances as input. It applies a sequence of machine learning technologies, namely curve fitting, clustering, and classification, to find which features make a problem instance hard and which algorithm setup parameters cause which algorithm behavior. We just submitted an article about this new process for review.

Like version 0.8.7, our new software release comes in three flavors:

  1. evaluator/GUI
  2. dockerized evaluator/GUI
  3. evaluator/CLI

The GUI is a stand-alone server providing a web-based interface for managing your experimental data, creating meta data, and running the evaluator.

The dockerized version of the GUI comes as a, well, Docker image. This image has all the software requirements pre-installed, including the Java 8 OpenJDK, TeX Live, R, the needed R packages, and ghostscript. Using the evaluator as Docker image requires more disk space, but apart from Docker itself, you don’t need to install anything. This seems to be quite a convenient scenario to me, given that you would otherwise need to install R and several of its packages, TeX Live, and such and such, to fully enjoy all features of our system.

The CLI, i.e., the command line interface, contains only the pure evaluator component and is to be executed manually. This may make sense when running it as component of another, non-Java process. If you would want to use the software inside your Java program, you can just the libraries directly. If you want to know how to do that, <a href=”mailto:tweise@ustc.edu.cn>contact me</a>.

Anyway, both the GUI and the CLI come as stand-alone jars and you can run them directly via java -jar evaluatorGui.jar (in case of the GUI) and java -jar evaluator.jar in case of the evaluator. In the dockerized GUI version, this is already automatically done for you.

[home] • [status] • [atom feed] • [rss feed]

Contact: Dr.  Thomas Weise, http://www.it-weise.de, tweise@ustc.edu.cn