Header logo is

The Infinite Hidden Markov Model

2002

Conference Paper

ei


We show that it is possible to extend hidden Markov models to have a countably infinite number of hidden states. By using the theory of Dirichlet processes we can implicitly integrate out the infinitely many transition parameters, leaving only three hyperparameters which can be learned from data. These three hyperparameters define a hierarchical Dirichlet process capable of capturing a rich set of transition dynamics. The three hyperparameters control the time scale of the dynamics, the sparsity of the underlying state-transition matrix, and the expected number of distinct hidden states in a finite sequence. In this framework it is also natural to allow the alphabet of emitted symbols to be infinite - consider, for example, symbols being possible words appearing in English text.

Author(s): Beal, MJ. and Ghahramani, Z. and Rasmussen, CE.
Book Title: Advances in Neural Information Processing Systems 14
Journal: Advances in Neural Information Processing Systems 14
Pages: 577-584
Year: 2002
Month: September
Day: 0
Editors: Dietterich, T.G. , S. Becker, Z. Ghahramani
Publisher: MIT Press

Department(s): Empirical Inference
Bibtex Type: Conference Paper (inproceedings)

Event Name: Fifteenth Annual Neural Information Processing Systems Conference (NIPS 2001)
Event Place: Vancouver, BC, Canada

Address: Cambridge, MA, USA
Digital: 0
ISBN: 0-262-04208-8
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik

Links: PDF
Web

BibTex

@inproceedings{2286,
  title = {The Infinite Hidden Markov Model},
  author = {Beal, MJ. and Ghahramani, Z. and Rasmussen, CE.},
  journal = {Advances in Neural Information Processing Systems 14},
  booktitle = {Advances in Neural Information Processing Systems 14},
  pages = {577-584},
  editors = {Dietterich, T.G. , S. Becker, Z. Ghahramani},
  publisher = {MIT Press},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  address = {Cambridge, MA, USA},
  month = sep,
  year = {2002},
  month_numeric = {9}
}