Latent Classification Models

TL;DR: This paper proposes a new set of models for classification in continuous domains, termed latent classification models, and presents algorithms for learning both the parameters and the structure of a latent classification model, and demonstrates empirically that the accuracy of the proposed model is significantly higher than theuracy of other probabilistic classifiers.

Abstract: One of the simplest, and yet most consistently well-performing set of classifiers is the Naive Bayes models. These models rely on two assumptions: (i) All the attributes used to describe an instance are conditionally independent given the class of that instance, and (ii) all attributes follow a specific parametric family of distributions. In this paper we propose a new set of models for classification in continuous domains, termed latent classification models. The latent classification model can roughly be seen as combining the Naive Bayes model with a mixture of factor analyzers, thereby relaxing the assumptions of the Naive Bayes classifier. In the proposed model the continuous attributes are described by a mixture of multivariate Gaussians, where the conditional dependencies among the attributes are encoded using latent variables. We present algorithms for learning both the parameters and the structure of a latent classification model, and we demonstrate empirically that the accuracy of the proposed model is significantly higher than the accuracy of other probabilistic classifiers.