.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples\exploration-vs-exploitation.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_exploration-vs-exploitation.py>`
        to download the full example code or to run this example in your browser via Binder

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_exploration-vs-exploitation.py:


===========================
Exploration vs exploitation
===========================

Sigurd Carlen, September 2019.
Reformatted by Holger Nahrstaedt 2020

.. currentmodule:: skopt


We can control how much the acqusition function favors exploration and
exploitation by tweaking the two parameters kappa and xi. Higher values
means more exploration and less exploitation and vice versa with low values.

kappa is only used if acq_func is set to "LCB". xi is used when acq_func is
"EI" or "PI". By default the acqusition function is set to "gp_hedge" which
chooses the best of these three. Therefore I recommend not using gp_hedge
when tweaking exploration/exploitation, but instead choosing "LCB",
"EI" or "PI".

The way to pass kappa and xi to the optimizer is to use the named argument
"acq_func_kwargs". This is a dict of extra arguments for the aqcuisition
function.

If you want opt.ask() to give a new acquisition value immediately after
tweaking kappa or xi call opt.update_next(). This ensures that the next
value is updated with the new acquisition parameters.

This example uses :class:`plots.plot_gaussian_process` which is available
since version 0.8.

.. GENERATED FROM PYTHON SOURCE LINES 33-42

.. code-block:: Python


    print(__doc__)

    import numpy as np

    np.random.seed(1234)
    from skopt import Optimizer
    from skopt.plots import plot_gaussian_process








.. GENERATED FROM PYTHON SOURCE LINES 43-49

Toy example
-----------
First we define our objective like in the ask-and-tell example notebook and
define a plotting function. We do however only use on initial random point.
All points after the first one is therefore chosen by the acquisition
function.

.. GENERATED FROM PYTHON SOURCE LINES 49-63

.. code-block:: Python


    noise_level = 0.1


    # Our 1D toy problem, this is the function we are trying to
    # minimize
    def objective(x, noise_level=noise_level):
        return np.sin(5 * x[0]) * (1 - np.tanh(x[0] ** 2)) + np.random.randn() * noise_level


    def objective_wo_noise(x):
        return objective(x, noise_level=0)









.. GENERATED FROM PYTHON SOURCE LINES 64-67

.. code-block:: Python



    opt = Optimizer([(-2.0, 2.0)], "GP", n_initial_points=3, acq_optimizer="sampling")







.. GENERATED FROM PYTHON SOURCE LINES 68-69

Plotting parameters

.. GENERATED FROM PYTHON SOURCE LINES 69-79

.. code-block:: Python


    plot_args = {
        "objective": objective_wo_noise,
        "noise_level": noise_level,
        "show_legend": True,
        "show_title": True,
        "show_next_point": False,
        "show_acq_func": True,
    }








.. GENERATED FROM PYTHON SOURCE LINES 80-81

We run a an optimization loop with standard settings

.. GENERATED FROM PYTHON SOURCE LINES 81-89

.. code-block:: Python


    for i in range(30):
        next_x = opt.ask()
        f_val = objective(next_x)
        opt.tell(next_x, f_val)
    # The same output could be created with opt.run(objective, n_iter=30)
    _ = plot_gaussian_process(opt.get_result(), **plot_args)




.. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_001.png
   :alt: x* = -0.2913, f(x*) = -1.0409
   :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 90-94

We see that some minima is found and "exploited"

Now lets try to set kappa and xi using'to other values and
pass it to the optimizer:

.. GENERATED FROM PYTHON SOURCE LINES 94-95

.. code-block:: Python

    acq_func_kwargs = {"xi": 10000, "kappa": 10000}







.. GENERATED FROM PYTHON SOURCE LINES 96-104

.. code-block:: Python


    opt = Optimizer(
        [(-2.0, 2.0)],
        "GP",
        n_initial_points=3,
        acq_optimizer="sampling",
        acq_func_kwargs=acq_func_kwargs,
    )







.. GENERATED FROM PYTHON SOURCE LINES 105-107

.. code-block:: Python

    opt.run(objective, n_iter=20)
    _ = plot_gaussian_process(opt.get_result(), **plot_args)



.. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_002.png
   :alt: x* = -0.3083, f(x*) = -0.7990
   :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_002.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 108-111

We see that the points are more random now.

This works both for kappa when using acq_func="LCB":

.. GENERATED FROM PYTHON SOURCE LINES 111-120

.. code-block:: Python


    opt = Optimizer(
        [(-2.0, 2.0)],
        "GP",
        n_initial_points=3,
        acq_func="LCB",
        acq_optimizer="sampling",
        acq_func_kwargs=acq_func_kwargs,
    )







.. GENERATED FROM PYTHON SOURCE LINES 121-123

.. code-block:: Python

    opt.run(objective, n_iter=20)
    _ = plot_gaussian_process(opt.get_result(), **plot_args)



.. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_003.png
   :alt: x* = -0.1829, f(x*) = -0.8271
   :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_003.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 124-125

And for xi when using acq_func="EI": or acq_func="PI":

.. GENERATED FROM PYTHON SOURCE LINES 125-134

.. code-block:: Python


    opt = Optimizer(
        [(-2.0, 2.0)],
        "GP",
        n_initial_points=3,
        acq_func="PI",
        acq_optimizer="sampling",
        acq_func_kwargs=acq_func_kwargs,
    )







.. GENERATED FROM PYTHON SOURCE LINES 135-137

.. code-block:: Python

    opt.run(objective, n_iter=20)
    _ = plot_gaussian_process(opt.get_result(), **plot_args)



.. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_004.png
   :alt: x* = -0.3877, f(x*) = -0.8487
   :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_004.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 138-139

Now lets try MES with 50 points:

.. GENERATED FROM PYTHON SOURCE LINES 140-141

.. code-block:: Python

    acq_func_kwargs = {"n_min_samples": 150}







.. GENERATED FROM PYTHON SOURCE LINES 142-151

.. code-block:: Python


    opt = Optimizer(
        [(-2.0, 2.0)],
        "GP",
        n_initial_points=3,
        acq_func="MES",
        acq_optimizer="sampling",
        acq_func_kwargs=acq_func_kwargs,
    )







.. GENERATED FROM PYTHON SOURCE LINES 152-154

.. code-block:: Python

    opt.run(objective, n_iter=20)
    _ = plot_gaussian_process(opt.get_result(), **plot_args)



.. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_005.png
   :alt: x* = -0.2441, f(x*) = -0.8469
   :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_005.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 155-156

We can also favor exploitaton:

.. GENERATED FROM PYTHON SOURCE LINES 156-157

.. code-block:: Python

    acq_func_kwargs = {"xi": 0.000001, "kappa": 0.001}







.. GENERATED FROM PYTHON SOURCE LINES 158-166

.. code-block:: Python

    opt = Optimizer(
        [(-2.0, 2.0)],
        "GP",
        n_initial_points=3,
        acq_func="LCB",
        acq_optimizer="sampling",
        acq_func_kwargs=acq_func_kwargs,
    )







.. GENERATED FROM PYTHON SOURCE LINES 167-169

.. code-block:: Python

    opt.run(objective, n_iter=20)
    _ = plot_gaussian_process(opt.get_result(), **plot_args)



.. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_006.png
   :alt: x* = 1.6319, f(x*) = -0.1482
   :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_006.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 170-178

.. code-block:: Python

    opt = Optimizer(
        [(-2.0, 2.0)],
        "GP",
        n_initial_points=3,
        acq_func="EI",
        acq_optimizer="sampling",
        acq_func_kwargs=acq_func_kwargs,
    )







.. GENERATED FROM PYTHON SOURCE LINES 179-181

.. code-block:: Python

    opt.run(objective, n_iter=20)
    _ = plot_gaussian_process(opt.get_result(), **plot_args)



.. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_007.png
   :alt: x* = -0.2705, f(x*) = -1.0266
   :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_007.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 182-190

.. code-block:: Python

    opt = Optimizer(
        [(-2.0, 2.0)],
        "GP",
        n_initial_points=3,
        acq_func="PI",
        acq_optimizer="sampling",
        acq_func_kwargs=acq_func_kwargs,
    )







.. GENERATED FROM PYTHON SOURCE LINES 191-194

.. code-block:: Python

    opt.run(objective, n_iter=20)
    _ = plot_gaussian_process(opt.get_result(), **plot_args)




.. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_008.png
   :alt: x* = -0.2961, f(x*) = -0.9580
   :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_008.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 195-197

Note that negative values does not work with the "PI"-acquisition function
but works with "EI":

.. GENERATED FROM PYTHON SOURCE LINES 197-198

.. code-block:: Python

    acq_func_kwargs = {"xi": -1000000000000}







.. GENERATED FROM PYTHON SOURCE LINES 199-208

.. code-block:: Python


    opt = Optimizer(
        [(-2.0, 2.0)],
        "GP",
        n_initial_points=3,
        acq_func="PI",
        acq_optimizer="sampling",
        acq_func_kwargs=acq_func_kwargs,
    )







.. GENERATED FROM PYTHON SOURCE LINES 209-211

.. code-block:: Python

    opt.run(objective, n_iter=20)
    _ = plot_gaussian_process(opt.get_result(), **plot_args)



.. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_009.png
   :alt: x* = -0.3491, f(x*) = -0.7981
   :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_009.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 212-220

.. code-block:: Python

    opt = Optimizer(
        [(-2.0, 2.0)],
        "GP",
        n_initial_points=3,
        acq_func="EI",
        acq_optimizer="sampling",
        acq_func_kwargs=acq_func_kwargs,
    )







.. GENERATED FROM PYTHON SOURCE LINES 221-223

.. code-block:: Python

    opt.run(objective, n_iter=20)
    _ = plot_gaussian_process(opt.get_result(), **plot_args)



.. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_010.png
   :alt: x* = -1.5268, f(x*) = -0.1786
   :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_010.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 224-230

Changing kappa and xi on the go
-------------------------------
If we want to change kappa or ki at any point during our optimization
process we just replace opt.acq_func_kwargs. Remember to call
`opt.update_next()` after the change, in order for next point to be
recalculated.

.. GENERATED FROM PYTHON SOURCE LINES 230-231

.. code-block:: Python

    acq_func_kwargs = {"kappa": 0}







.. GENERATED FROM PYTHON SOURCE LINES 232-240

.. code-block:: Python

    opt = Optimizer(
        [(-2.0, 2.0)],
        "GP",
        n_initial_points=3,
        acq_func="LCB",
        acq_optimizer="sampling",
        acq_func_kwargs=acq_func_kwargs,
    )







.. GENERATED FROM PYTHON SOURCE LINES 241-242

.. code-block:: Python

    opt.acq_func_kwargs




.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    {'kappa': 0}



.. GENERATED FROM PYTHON SOURCE LINES 243-245

.. code-block:: Python

    opt.run(objective, n_iter=20)
    _ = plot_gaussian_process(opt.get_result(), **plot_args)



.. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_011.png
   :alt: x* = -0.6144, f(x*) = -0.2081
   :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_011.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 246-247

.. code-block:: Python

    acq_func_kwargs = {"kappa": 100000}







.. GENERATED FROM PYTHON SOURCE LINES 248-250

.. code-block:: Python

    opt.acq_func_kwargs = acq_func_kwargs
    opt.update_next()







.. GENERATED FROM PYTHON SOURCE LINES 251-253

.. code-block:: Python

    opt.run(objective, n_iter=20)
    _ = plot_gaussian_process(opt.get_result(), **plot_args)



.. image-sg:: /auto_examples/images/sphx_glr_exploration-vs-exploitation_012.png
   :alt: x* = -0.1506, f(x*) = -0.6871
   :srcset: /auto_examples/images/sphx_glr_exploration-vs-exploitation_012.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 46.614 seconds)


.. _sphx_glr_download_auto_examples_exploration-vs-exploitation.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/holgern/scikit-optimize/master?urlpath=lab/tree/notebooks/auto_examples/exploration-vs-exploitation.ipynb
        :alt: Launch binder
        :width: 150 px

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: exploration-vs-exploitation.ipynb <exploration-vs-exploitation.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: exploration-vs-exploitation.py <exploration-vs-exploitation.py>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_