Basic Example

If you are not familiar with the main concepts of Bayesian Optimization, a quick introduction is available here. In this tutorial, we will explain how to create a new experiment in which a simple function ( \(-{(5 * x - 2.5)}^2 + 5\)) is maximized.

Let’s say we want to create an experiment called “myExp”. The first thing to do is to create the folder exp/myExp under the limbo root. Then add two files:

  • the main.cpp file
  • a python file called wscript, which will be used by waf to register the executable for building

The file structure should look like this:

limbo
|-- exp
     |-- myExp
          +-- wscript
          +-- main.cpp
|-- src
...

Next, copy the following content to the wscript file:

from waflib.Configure import conf

 def options(opt):
     pass


 def build(bld):
     bld(features='cxx cxxprogram',
         source='main.cpp',
         includes='. ../../src',
         target='myExp',
         uselib='BOOST EIGEN TBB LIBCMAES NLOPT',
         use='limbo')

For this example, we will optimize a simple function: \(-{(5 * x - 2.5)}^2 + 5\), using all default values and settings. If you did not compile with libcmaes and/or nlopt, remove LIBCMAES and/or NLOPT from ‘uselib’.

To begin, the main file has to include the necessary files:

1
2
3
4
5
6
#include <iostream>

// you can also include <limbo/limbo.hpp> but it will slow down the compilation
#include <limbo/bayes_opt/boptimizer.hpp>

using namespace limbo;

We also need to declare the Parameter struct:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
struct Params {
    struct bayes_opt_boptimizer : public defaults::bayes_opt_boptimizer {
    };

// depending on which internal optimizer we use, we need to import different parameters
#ifdef USE_NLOPT
    struct opt_nloptnograd : public defaults::opt_nloptnograd {
    };
#elif defined(USE_LIBCMAES)
    struct opt_cmaes : public defaults::opt_cmaes {
    };
#else
    struct opt_gridsearch : public defaults::opt_gridsearch {
    };
#endif

    // enable / disable the writing of the result files
    struct bayes_opt_bobase : public defaults::bayes_opt_bobase {
        BO_PARAM(int, stats_enabled, true);
    };

    // no noise
    struct kernel : public defaults::kernel {
        BO_PARAM(double, noise, 1e-10);
    };

    struct kernel_maternfivehalves : public defaults::kernel_maternfivehalves {
    };

    // we use 10 random samples to initialize the algorithm
    struct init_randomsampling {
        BO_PARAM(int, samples, 10);
    };

    // we stop after 40 iterations
    struct stop_maxiterations {
        BO_PARAM(int, iterations, 40);
    };

    // we use the default parameters for acqui_ucb
    struct acqui_ucb : public defaults::acqui_ucb {
    };
};

Here we are stating that the samples are observed without noise (which makes sense, because we are going to evaluate the function), that we want to output the stats (by setting stats_enabled to true), that the model has to be initialized with 10 samples (that will be selected randomly), and that the optimizer should run for 40 iterations. The rest of the values are taken from the defaults. By default limbo optimizes in \([0,1]\), but you can optimize without bounds by setting BO_PARAM(bool, bounded, false) in bayes_opt_bobase parameters. If you do so, limbo outputs random numbers, wherever needed, sampled from a gaussian centered in zero with a standard deviation of \(10\), instead of uniform random numbers in \([0,1]\) (in the bounded case). Finally limbo always maximizes; this means that you have to update your objective function if you want to minimize.

Then, we have to define the evaluation function for the optimizer to call:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
struct Eval {
    // number of input dimension (x.size())
    BO_PARAM(size_t, dim_in, 1);
    // number of dimensions of the result (res.size())
    BO_PARAM(size_t, dim_out, 1);

    // the function to be optimized
    Eigen::VectorXd operator()(const Eigen::VectorXd& x) const
    {
        double y = -((5 * x(0) - 2.5) * (5 * x(0) - 2.5)) + 5;
        // we return a 1-dimensional vector
        return tools::make_vector(y);
    }
};

It is required that the evaluation struct has the static function members dim_in() and dim_out(), specifying the input and output dimensions. Also, it should have the operator() expecting a const Eigen::VectorXd& of size dim_in(), and return another one, of size dim_out().

With this, we can declare the main function:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
int main()
{
    // we use the default acquisition function / model / stat / etc.
    bayes_opt::BOptimizer<Params> boptimizer;
    // run the evaluation
    boptimizer.optimize(Eval());
    // the best sample found
    std::cout << "Best sample: " << boptimizer.best_sample()(0) << " - Best observation: " << boptimizer.best_observation()(0) << std::endl;
    return 0;
}

The full main.cpp can be found here

Finally, from the root of limbo, run a build command, with the additional switch --exp myExp:

./waf build --exp myExp

Then, an executable named myExp should be produced under the folder build/exp/myExp. When running this executable, you should see something similar to this:

0 new point: 0.502378 value: 4.99986 best:4.99986
1 new point: 0.503035 value: 4.99977 best:4.99986
2 new point: 0.502521 value: 4.99984 best:4.99986
3 new point: 0.502533 value: 4.99984 best:4.99986
4 new point: 0.502556 value: 4.99984 best:4.99986
5 new point: 0.502585 value: 4.99983 best:4.99986
6 new point: 0.502618 value: 4.99983 best:4.99986
7 new point: 0.502643 value: 4.99983 best:4.99986
8 new point: 0.502646 value: 4.99983 best:4.99986
9 new point: 0.502673 value: 4.99982 best:4.99986
10 new point: 0.502383 value: 4.99986 best:4.99986
11 new point: 0.502262 value: 4.99987 best:4.99987
12 new point: 0.502111 value: 4.99989 best:4.99989
13 new point: 0.501921 value: 4.99991 best:4.99991
14 new point: 0.501679 value: 4.99993 best:4.99993
15 new point: 0.501383 value: 4.99995 best:4.99995
16 new point: 0.501055 value: 4.99997 best:4.99997
17 new point: 0.500751 value: 4.99999 best:4.99999
18 new point: 0.500517 value: 4.99999 best:4.99999
19 new point: 0.500358 value: 5 best:5
20 new point: 0.500256 value: 5 best:5
21 new point: 0.500189 value: 5 best:5
22 new point: 0.500145 value: 5 best:5
23 new point: 0.500114 value: 5 best:5
24 new point: 0.500092 value: 5 best:5
25 new point: 0.500075 value: 5 best:5
26 new point: 0.500063 value: 5 best:5
27 new point: 0.500054 value: 5 best:5
28 new point: 0.500046 value: 5 best:5
29 new point: 0.500039 value: 5 best:5
30 new point: 0.500035 value: 5 best:5
31 new point: 0.50003 value: 5 best:5
32 new point: 0.500027 value: 5 best:5
33 new point: 0.500024 value: 5 best:5
34 new point: 0.500022 value: 5 best:5
35 new point: 0.50002 value: 5 best:5
36 new point: 0.500018 value: 5 best:5
37 new point: 0.500016 value: 5 best:5
38 new point: 0.500015 value: 5 best:5
39 new point: 0.500014 value: 5 best:5
Best sample: 0.500014 - Best observation: 5

These lines show the result of each sample evaluation of the \(40\) iterations (after the random initialization). In particular, we can see that algorithm progressively converges toward the maximum of the function (\(5\)) and that the maximum found is located at \(x = 0.500014\).

Running the executable also created a folder with a name composed of YOUCOMPUTERHOSTNAME-DATE-HOUR-PID. This folder should contain two files:

limbo
|-- YOUCOMPUTERHOSTNAME-DATE-HOUR-PID
   +-- samples.dat
   +-- aggregated_observations.dat

The file samples.dat contains the coordinates of the samples that have been evaluated during each iteration, while the file aggregated_observations.dat contains the corresponding observed values.

If you want to display the different observations in a graph, you can use the python script print_aggregated_observations.py (located in limbo_root/src/tutorials). For instance, from the root of limbo you can run

python src/tutorials/print_aggregated_observations.py YOUCOMPUTERHOSTNAME-DATE-HOUR-PID/aggregated_observations.dat