How to specify bounds¶

Constraints vs bounds¶

optimagic distinguishes between bounds and constraints. Bounds are lower and upper bounds for parameters. In the literature, they are sometimes called box constraints. Examples for general constraints are linear constraints, probability constraints, or nonlinear constraints. You can find out more about general constraints in the next section on How to specify constraints.

Example objective function¶

Let’s again look at the sphere function:

import numpy as np

import optimagic as om

def fun(x):
    return x @ x

res = om.minimize(fun=fun, params=np.arange(3), algorithm="scipy_lbfgsb")
res.params.round(5)

array([ 0., -0.,  0.])

Array params¶

For params that are a numpy.ndarray, one can specify the lower and/or upper-bounds as an array of the same length.

Lower bounds

res = om.minimize(
    fun=fun,
    params=np.arange(3),
    bounds=om.Bounds(lower=np.ones(3)),
    algorithm="scipy_lbfgsb",
)
res.params

array([1., 1., 1.])

Lower & upper-bounds

res = om.minimize(
    fun=fun,
    params=np.arange(3),
    algorithm="scipy_lbfgsb",
    bounds=om.Bounds(
        lower=np.array([-2, -np.inf, 1]),
        upper=np.array([-1, np.inf, np.inf]),
    ),
)
res.params

array([-1.00000000e+00, -3.57647465e-08,  1.00000000e+00])

Pytree params¶

Now let’s look at a case where params is a more general pytree. We also update the sphere function by adding an intercept. Since the criterion always decreases when decreasing the intercept, there is no unrestricted solution. Lets fix a lower bound only for the intercept.

params = {"x": np.arange(3), "intercept": 3}


def fun(params):
    return params["x"] @ params["x"] + params["intercept"]

res = om.minimize(
    fun=fun,
    params=params,
    algorithm="scipy_lbfgsb",
    bounds=om.Bounds(lower={"intercept": -2}),
)
res.params

{'x': array([ 0.00000000e+00, -6.27294559e-08, -1.07934306e-08]),
 'intercept': np.float64(-2.0)}

optimagic tries to match the user provided bounds with the structure of params. This allows you to specify bounds for subtrees of params. In case your subtree specification results in an unidentified matching, optimagic will tell you so with a InvalidBoundsError.

params data frame¶

It often makes sense to specify your parameters in a pandas.DataFrame, where you can utilize the multiindex for parameter naming. In this case, you can specify bounds as extra columns lower_bound and upper_bound.

Note The columns are called *_bound instead of *_bounds like the argument passed to minimize or maximize.

import pandas as pd

params = pd.DataFrame(
    {"value": [0, 1, 2, 3], "lower_bound": [0, 1, 1, -2]},
    index=pd.MultiIndex.from_tuples([("x", k) for k in range(3)] + [("intercept", 0)]),
)
params

		value	lower_bound
x	0	0	0
	1	1	1
	2	2	1
intercept	0	3	-2

def fun(params):
    x = params.loc["x"]["value"].to_numpy()
    intercept = params.loc["intercept"]["value"].iloc[0]
    value = x @ x + intercept
    return float(value)

res = om.minimize(
    fun,
    params=params,
    algorithm="scipy_lbfgsb",
)
res.params

		value	lower_bound
x	0	0.0	0
	1	1.0	1
	2	1.0	1
intercept	0	-2.0	-2

Filtering algorithms¶

It is further possible to filter algorithms based on whether they support bounds, if bounds are required to run, and if infinite bounds are supported. The AlgoInfo class provides all information about the chosen algorithm, which can be accessed with algo.algo_info… . Suppose we are looking for a optimizer that supports bounds and strictly require them for the algorithm to run properly.

To find all algorithms that support bounds and cannot run without bounds, we can simply do:

from optimagic.algorithms import AVAILABLE_ALGORITHMS

algos_with_bounds_support = [
    algo
    for name, algo in AVAILABLE_ALGORITHMS.items()
    if algo.algo_info.supports_bounds
]
my_selection = [
    algo for algo in algos_with_bounds_support if algo.algo_info.needs_bounds
]
my_selection[0:3]

[om.algos.bayes_opt, om.algos.nlopt_crs2_lm, om.algos.nlopt_direct]

Similarly, to find all algorithms that support infinite values in bounds , we can do:

my_selection2 = [
    algo
    for algo in algos_with_bounds_support
    if algo.algo_info.supports_infinite_bounds
]
my_selection2[0:3]

[om.algos.fides, om.algos.nag_dfols, om.algos.nag_pybobyqa]

In case you you forget to specify bounds for a optimizer that strictly requires them or pass infinite values in bounds to a optimizer which does not support them, optimagic will raise an IncompleteBoundsError.

Coming from scipy¶

If params is a flat numpy array, you can also provide bounds in any format that is supported by scipy.optimize.minimize.