Perceiving Systems
Empirical Inference
Probabilistic Numerics
Conference Paper
2012
Quasi-Newton Methods: A New Direction
website+code pdf
Four decades after their invention, quasi- Newton methods are still state of the art in unconstrained numerical optimization. Although not usually interpreted thus, these are learning algorithms that fit a local quadratic approximation to the objective function. We show that many, including the most popular, quasi-Newton methods can be interpreted as approximations of Bayesian linear regression under varying prior assumptions. This new notion elucidates some shortcomings of classical algorithms, and lights the way to a novel nonparametric quasi-Newton method, which is able to make more efficient use of available information at computational cost similar to its predecessors.
| Author(s): | Hennig, Philipp and Kiefel, Martin |
| Links: | |
| Book Title: | Proceedings of the 29th International Conference on Machine Learning |
| Pages: | 25--32 |
| Year: | 2012 |
| Month: | July |
| Series: | ICML '12 |
| Editors: | John Langford and Joelle Pineau |
| Publisher: | Omnipress |
| Project(s): | |
| BibTeX Type: | Conference Paper (inproceedings) |
| Address: | New York, NY, USA |
| Event Name: | ICML 2012 |
| Event Place: | Edinburgh, Scotland, GB |
| URL: | http://www.is.tuebingen.mpg.de/fileadmin/user_upload/files/publications/2012/Hennig_Kiefel_ICML2012.pdf |
| Electronic Archiving: | grant_archive |
BibTeX
@inproceedings{optimization,
title = {Quasi-Newton Methods: A New Direction},
booktitle = {Proceedings of the 29th International Conference on Machine Learning},
abstract = {Four decades after their invention, quasi- Newton methods are still state of the art in unconstrained numerical optimization. Although not usually interpreted thus, these are learning algorithms that fit a local quadratic approximation to the objective function. We show that many, including the most popular, quasi-Newton methods can be interpreted as approximations of Bayesian linear regression under varying prior assumptions. This new notion elucidates some shortcomings of classical algorithms, and lights the way to a novel nonparametric quasi-Newton method, which is able to make more efficient use of available information at computational cost similar to its predecessors.},
pages = {25--32},
series = {ICML '12},
editors = {John Langford and Joelle Pineau},
publisher = {Omnipress},
address = {New York, NY, USA},
month = jul,
year = {2012},
author = {Hennig, Philipp and Kiefel, Martin},
url = {http://www.is.tuebingen.mpg.de/fileadmin/user_upload/files/publications/2012/Hennig_Kiefel_ICML2012.pdf},
month_numeric = {7}
}