Cluster analysis is the automated search for groups of related observations in a dataset. Most clustering done in practice is based largely on heuristic but intuitively reasonable procedures, and most clustering methods available in commercial software are also of this type. However, there is little systematic guidance associated with these methods for solving important practical questions that arise in cluster analysis, such as how many clusters are there, which clustering method should be used, and how should outliers be handled. We review a general methodology for model-based clustering that provides a principled statistical approach to these issues. We also show that this can be useful for other problems in multivariate analysis, such as discriminant analysis and multivariate density estimation. We give examples from medical diagnosis, minefield detection, cluster recovery from noisy data, and spatial density estimation. Finally, we mention limitations of the methodology and discuss recent development...

https://calvino.polito.it/~gasparini/teaching/fraley2002.pdf

Model-Based Clustering, Discriminant Analysis, and Density Estimation

Most problems in frequentist statistics involve optimization of a function such as a likelihood or a sum of squares. EM algorithms are among the most effective algorithms for maximum likelihood estimation because they consistently drive the likelihood uphill by maximizing a simple surrogate function for the log-likelihood. Iterative optimization of a surrogate function as exemplified by an EM algorithm does not necessarily require missing data. Indeed, every EM algorithm is a special case of the more general class of MM optimization algorithms, which typically exploit convexity rather than missing data in majorizing or minorizing an objective function. In our opinion, MM algorithms deserve to be part of the standard toolkit of professional statisticians. This article explains the principle behind MM algorithms, suggests some methods for constructing them, and discusses some of their attractive features. We include numerous examples throughout the article to illustrate the concepts described. In addition t...

/pdf/a-tutorial-on-mm-algorithms-4yt18j1td5.pdf

A Tutorial on MM Algorithms

This paper gives an overview of the majorization-minimization (MM) algorithmic framework, which can provide guidance in deriving problem-driven algorithms with low computational cost. A general introduction of MM is presented, including a description of the basic principle and its convergence results. The extensions, acceleration schemes, and connection to other algorithmic frameworks are also covered. To bridge the gap between theory and practice, upperbounds for a large number of basic functions, derived based on the Taylor expansion, convexity, and special inequalities, are provided as ingredients for constructing surrogate functions. With the pre-requisites established, the way of applying MM to solving specific problems is elaborated by a wide range of applications in signal processing, communications, and machine learning.

Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning

The well-known EM algorithm is an optimization transfer algorithm that depends on the notion of incomplete or missing data By invoking convexity arguments, one can construct a variety of other optimization transfer algorithms that do not involve missing data These algorithms all rely on a majorizing or minorizing function that serves as a surrogate for the objective function Optimizing the surrogate function drives the objective function in the correct direction This article illustrates this general principle by a number of specific examples drawn from the statistical literature Because optimization transfer algorithms often exhibit the slow convergence of EM algorithms, two methods of accelerating optimization transfer are discussed and evaluated in the context of specific problems

http://lear.inrialpes.fr/people/oneata/reading_group/lange2000.pdf

Optimization Transfer Using Surrogate Objective Functions

We consider the problem of variable or feature selection for model-based clustering. The problem of comparing two nested subsets of variables is recast as a model comparison problem and addressed using approximate Bayes factors. A greedy search algorithm is proposed for finding a local optimum in model space. The resulting method selects variables (or features), the number of clusters, and the clustering model simultaneously. We applied the method to several simulated and real examples and found that removing irrelevant variables often improved performance. Compared with methods based on all of the variables, our variable selection method consistently yielded more accurate estimates of the number of groups and lower classification error rates, as well as more parsimonious clustering models and easier visualization of results.

https://bigdata.unl.edu/documents/ASA_Workshop_Materials/Variable%20Selection%20for%20Model-Based%20Clustering.pdf

Variable Selection for Model-Based Clustering

The standard latent class model is a finite mixture of indirectly observed multinomial distributions, each of which is assumed to exhibit statistical independence. Latent class analysis has been applied in a wide variety of research contexts, including studies of mobility, educational attainment, agreement, and diagnostic accuracy, and as measurement error models in social research. One of the attractive features of the latent class model in these settings is that the parameters defining the individual multinomials are readily interpretable marginal probabilities, conditional on the unobserved latent variable(s), that are often of substantive interest. There are, however, settings where the local-independence axiom is not supported, and hence it is useful to consider some form of local dependence. In this paper we consider a family of models defined in terms of finite mixtures of multinomial models where the multinomials are parameterized in terms of a set of models for the univariate marginal distributions and for marginal associations. Local dependence is introduced through the models for marginal associations, and the standard latent class model obtains as a special case. Three examples are analyzed with the models to illustrate their utility in analyzing complex cross-classifications.

https://www.jstor.org/stable/pdfplus/270972.pdf

Latent class marginal models for cross-classifications of counts

Log-linear models are useful for analyzing cross-classifications of counts arising in sociology, but it has been argued that in some cases, an alternative approach for formulating models—one based on simultaneously modeling univariate marginal logits and marginal associations—can lead to models that are more directly relevant for addressing the kinds of questions arising in those cases. In this article, the authors explore some of the similarities and differences between the log-linear models approach to modeling categorical data and a marginal modeling appraoch. It has been noted in past literature that the model of statistical independence is conveniently represented within both approaches to specifying models for cross-classifications of counts. The authors examine further the extent to which the two families of models overlap, as well as some important differences. The authors do not present a complete characterization of the conditions describing the intersection of the two families of models but cov...

/pdf/specifications-of-models-for-cross-classified-counts-2ukekh19sz.pdf

Specifications of Models for Cross-Classified Counts Comparisons of the Log-Linear Models and Marginal Models Perspectives

Latent class analysis has been applied in medical research to assessing the sensitivity and specificity of diagnostic tests/diagnosticians. In these applications, a dichotomous latent variable corresponding to the unobserved true disease status of the patients is assumed. Associations among multiple diagnostic tests are attributed to the unobserved heterogeneity induced by the latent variable, and inferences for the sensitivities and specificities of the diagnostic tests are made possible even though the true disease status is unknown. However, a shortcoming of this approach to analyses of diagnostic tests is that the standard assumption of conditional independence among the diagnostic tests given a latent class is contraindicated by the data in some applications. In the present paper, models incorporating dependence among the diagnostic tests given a latent class are proposed. The models are parameterized so that the sensitivities and specificities of the diagnostic tests are simple functions of model parameters, and the usual latent class model obtains as a special case. Marginal models are used to account for the dependencies within each latent class. An accelerated EM gradient algorithm is demonstrated to obtain maximum likelihood estimates of the parameters of interest, as well as estimates of the precision of the estimates.

Latent Variable Modeling of Diagnostic Accuracy

Most problems in computational statistics involve optimization of an objective function such as a loglikelihood, a sum of squares, or a log posterior function. The EM algorithm is one of the most effective algorithms for maximization because it iteratively transfers maximization from a complex function to a simple, surrogate function. This theoretical perspective clarifies the operation of the EM algorithm and suggests novel generalizations. Besides simplifying maximization, optimization transfer usually leads to highly stable algorithms with well-understood local and global convergence properties. Although convergence can be excruciatingly slow, various devices exist for accelerating it. Beginning with the EM algorithm, we review in this paper several optimization transfer algorithms of substantial utility in medical statistics.

/pdf/em-algorithms-without-missing-data-3i7io2y7hd.pdf

EM algorithms without missing data.

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Ilsoon Yang

Author Tools

Chat about Author

Papers

Latent class marginal models for cross-classifications of counts

Specifications of Models for Cross-Classified Counts Comparisons of the Log-Linear Models and Marginal Models Perspectives

Latent Variable Modeling of Diagnostic Accuracy

EM algorithms without missing data.