Bayesian Optimization motivation (4/4): recommended application domains, remarks

Antonio Sala, UPV

Difficulty: ** ,       Relevance: ,      Duration: 19:33

Summary:

After a quick review of prior material, this video discusses the application domains where Bayesian optimization IS recommended (model-free experimental optimization, configuration of long computations in neural networks, fluids, …), as well as some circumstances in which it is NOT recommended (simple functions, high-dimensional problems). If you did not watch videos [BOmot1EN], [BOmot2EN] y [BOmot3EN], I recommend doing it now to get a broader view.

The video concludes with a couple of remarks on, first, the fact that determinism is a limit case of statistics with uncertainty size tending to zero, so there are quite a few deterministic optimization problems that admit a ‘statistical interpretation’ (say, Least squares, assuming normally distributed random measurement noise); there are also semi-parametric Bayesian optimization options.

The second remark pinpoints the fact that gradient-based methods may get stuck on local minima; hence, some algorithms (directed random search) use ’rolling the dice’ steps to try to explore and get the global optimum; however, these issues do not change the fact that the underlying problem is deterministic and no uncertainty is assumed to be present in the objective function.

A summary and conclusions reviewing the ideas on this video and the preceding ones in the series that started with video [BOmot1EN] end the video and the 4-video set that motivated and briefly outlined the main features of Bayesian optimization.

*Link to my [ whole collection] of videos in English. Link to larger [ Colección completa] in Spanish.