Institute Homepage
Institute Homepage DE Sign In

Back

Empirical Inference Technical Report 2008

Infinite Kernel Learning

PDF Web
no image
Empirical Inference
Sebastian Nowozin

In this paper we consider the problem of automatically learning the kernel from general kernel classes. Specifically we build upon the Multiple Kernel Learning (MKL) framework and in particular on the work of (Argyriou, Hauser, Micchelli, & Pontil, 2006). We will formulate a Semi-Infinite Program (SIP) to solve the problem and devise a new algorithm to solve it (Infinite Kernel Learning, IKL). The IKL algorithm is applicable to both the finite and infinite case and we find it to be faster and more stable than SimpleMKL (Rakotomamonjy, Bach, Canu, & Grandvalet, 2007) for cases of many kernels. In the second part we present the first large scale comparison of SVMs to MKL on a variety of benchmark datasets, also comparing IKL. The results show two things: a) for many datasets there is no benefit in linearly combining kernels with MKL/IKL instead of the SVM classifier, thus the flexibility of using more than one kernel seems to be of no use, b) on some datasets IKL yields impressive increases in accuracy over SVM/MKL due to the possibility of using a largely increased kernel set. In those cases, IKL remains practical, whereas both cross-validation or standard MKL is infeasible.

Author(s): Gehler, PV. and Nowozin, S.
Links:
Number (issue): 178
Year: 2008
Month: October
Day: 0
Bibtex Type: Technical Report (techreport)
Digital: 0
Electronic Archiving: grant_archive
Institution: Max-Planck Institute for Biological Cybernetics, Tübingen, Germany
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik

BibTex 

@techreport{5617,
  title = {Infinite Kernel Learning},
  abstract = {In this paper we consider the problem of automatically learning the kernel from general kernel
  classes. Specifically we build upon the Multiple Kernel Learning (MKL) framework and in particular on the work
  of (Argyriou, Hauser, Micchelli, & Pontil, 2006). We will formulate a Semi-Infinite Program (SIP) to solve the
  problem and devise a new algorithm to solve it (Infinite Kernel Learning, IKL). The IKL algorithm is applicable
  to both the finite and infinite case and we find it to be faster and more stable than SimpleMKL (Rakotomamonjy,
  Bach, Canu, & Grandvalet, 2007) for cases of many kernels. In the second part we present the first large scale
  comparison of SVMs to MKL on a variety of benchmark datasets, also comparing IKL. The results show two
  things: a) for many datasets there is no benefit in linearly combining kernels with MKL/IKL instead of the SVM
  classifier, thus the flexibility of using more than one kernel seems to be of no use, b) on some datasets IKL yields
  impressive increases in accuracy over SVM/MKL due to the possibility of using a largely increased kernel set. In
  those cases, IKL remains practical, whereas both cross-validation or standard MKL is infeasible.},
  number = {178},
  organization = {Max-Planck-Gesellschaft},
  institution = {Max-Planck Institute for Biological Cybernetics, Tübingen, Germany},
  school = {Biologische Kybernetik},
  month = oct,
  year = {2008},
  slug = {5617},
  author = {Gehler, PV. and Nowozin, S.},
  month_numeric = {10}
}