Optimization Benchmarking

Downloads & Installation

For using the software, you may first download it. The software requires a set of other tools, which you then should install in order to get everything to work smoothly. Alternatively, if you are using Linux, MacOS, or Windows (on a 64 bit system), you can also use the dockerized version of the GUI. The latter is recommended, as it only requires you to install Docker and then you are ready to go!

Dockerized Version: Only Install Docker, Nothing Else!

The graphical user interface (GUI) of our optimizationBenchmarking.org evaluator has now been “dockerized”. Docker is an application that allows you to define, publish, and run containers. Containers are something like lightweight VMs, they live as normal processes on the same kernel as the OS under Linux and as small Virtual Box VMs under Windows and Mac OS. Docker can be installed following the guidelines below:

  • for Linux, you can run curl -fsSL https://get.docker.com/ | sh on your command line and everything is done automatically (if you have curl installed, which is normally the case),
  • for Windows
  • for Mac OS

After doing this, you can start our container by typing the following command into a normal terminal (Linux), the Docker Quickstart Terminal (Mac OS), or the Docker Toolbox Terminal (Windows):

docker run -t -d -p 9999:8080/tcp optimizationbenchmarking/evaluator-gui

The first time you run the program, this will download the software once (and only once). Once the container is started, you can access it with your browser at address

  • http://localhost:9999 under Linux or
  • http://<dockerIP>:9999 under Windows and Mac OS, where dockerIP is the IP address of your Docker container. This address is displayed when you run the container. You can also obtain it with the command docker-machine ip default.

The container contains a full installation of my system, including the Java 8 OpenJDK, TeX Live, R, the needed R packages, and ghostscript. No further setup is needed. It is thus about 600 MB in size.

Here and here you can find the command line options explained. This will allow you to use our system efficiently.

Downloads of Java Software Version

This only holds if you want to run the jars directly and do not use Docker. Otherwise, this step is irrelevant for you – if you use Docker, nothing else is required.

So if you want to use the jars directly, maybe because you already have R and Java and TeX Live installed and want to save disk space: You may enjoy the optimizationBenchmarking.org software Java jar releases in two flavors:

  1. evaluator/GUI (Version: 0.8.7)
  2. evaluator/CLI (Version: 0.8.7)

The GUI is a stand-alone server providing a web-based interface for managing your experimental data, creating meta data, and running the evaluator.

The CLI, i.e., the command line interface, is a only the pure evaluator component to be executed manually.

Both software come as stand-alone jars and you can run them via java -jar evaluatorGui.jar (in case of the GUI) and java -jar evaluator.jar in case of the evaluator.

Installation of Java Software Version

This only holds if you want to run the jars directly and do not use Docker. Otherwise, this step is irrelevant for you – if you use Docker, nothing else is required.

The runtime environment for the optimizationBenchmarking.org framework requires a set a Java JDK, a LaTeX installation, and R with a set of pre-installed packages.

I recommend:

  • Java 8 OpenJDK (Linux: install ca-certificates-java and openjdk-8-jdk)
  • TeX Live for Linux (install: texlive-base, texlive-fonts-recommended, texlive-luatex, or texlive-xetex) or MiKTeX for Windows: Make sure to install the LaTeX, XeLaTeX and LuaLaTeX components
  • ghostscript
  • R (Linux: install libicu-dev, pkg-config, r-base=3.3.*, and r-base-dev=3.3.*)

For R we automatically pre-install libraries which may be needed by our software, namely

Under Linux, you may copy the following text to the command line (it may ask you for your sudo password a few times for installing these packages, given that you have R already installed:

R_params="repos=c(\"http://cran.us.r-project.org\", \"http://cran.uk.r-project.org\",  \"http://mirrors.ustc.edu.cn/CRAN\", \"http://cran.wu.ac.at/\", \"http://cran.utstat.utoronto.ca/\", \"http://cran.rstudio.com/\", \"https://cran.uni-muenster.de/\", \"http://ftp.iitm.ac.in/cran/\" ), dependencies=TRUE, clean=TRUE" &&\
sudo Rscript -e "update.packages(ask=FALSE, ${R_params})" &&\
sudo Rscript -e "if(!(require(\"apcluster\"))) install.packages(\"apcluster\", ${R_params})" &&\
sudo Rscript -e "if(!(require(\"cluster\"))) install.packages(\"cluster\", ${R_params})" &&\
sudo Rscript -e "if(!(require(\"fpc\"))) install.packages(\"fpc\", ${R_params})" &&\
sudo Rscript -e "if(!(require(\"mclust\"))) install.packages(\"mclust\", ${R_params})" &&\
sudo Rscript -e "if(!(require(\"NbClust\"))) install.packages(\"NbClust\", ${R_params})" &&\
sudo Rscript -e "if(!(require(\"stats\"))) install.packages(\"stats\", ${R_params})" &&\
sudo Rscript -e "if(!(require(\"vegan\"))) install.packages(\"vegan\", ${R_params})"


There are currently two slide sets with documentation about our system:

[home] • [status] • [atom feed] • [rss feed]

Contact: Dr.  Thomas Weise, http://www.it-weise.de, tweise@ustc.edu.cn