Model complexity control and compression using discriminative growth functions
Xunying Liu,Mark J. F. Gales +1 more
- 17 May 2004
- Vol. 1, pp 797-800
TL;DR: In this paper further experiments are carried out using a recently proposed criterion based on marginalizing a maximum mutual information (MMI) growth function for model compression, showing a reduction in word error rate.
read more
Abstract: State-of-the-art large vocabulary speech recognition systems are highly complex. Many techniques affect both system complexity and recognition performance. The need to determine the appropriate complexity without having to build each possible system has led to the development of automatic complexity control criteria. In this paper further experiments are carried out using a recently proposed criterion based on marginalizing a maximum mutual information (MMI) growth function. The use of this criterion is much detailed for determining the appropriate dimensionality in a multiple HLDA system and the number of components per state. A scheme for also using this criterion for model compression is described. Experimental results on a spontaneous telephone speech recognition task are described. Initial system compression experiments are inconclusive. However, comparing a standard state-of-the-art system with one generated using complexity control shows a reduction in word error rate.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Adaptation of Hybrid ANN/HMM Models Using Linear Hidden Transformations and Conservative Training
Roberto Gemello,Franco Mana,Stefano Scanzio,Pietro Laface,R. De Mori +4 more
- 14 May 2006
TL;DR: A new solution, called conservative training, is proposed that compensates for the lack of adaptation samples in certain classes that outperforms the use of transformations in the feature space and yields even better results when combined with linear input transformations.
Adaptation of Artificial Neural Networks Avoiding Catastrophic Forgetting
D. Albesano,Roberto Gemello,Pietro Laface,Franco Mana,Stefano Scanzio +4 more
- 30 Oct 2006
TL;DR: The results show that the combination of the proposed approaches mitigates the catastrophic forgetting effects, and always outperforms the use of the classical transformations in the feature space.
Automatic Model Complexity Control Using Marginalized Discriminative Growth Functions
Xunying Liu,Mjf Gales +1 more
TL;DR: Experimental results showed that marginalized discriminative growth functions outperforms manually tuned systems and conventional complexity control techniques, such as Bayesian information criterion (BIC), in terms of WER.
Automatic model complexity control using marginalized discriminative growth functions
Xunying Liu,Mjf Gales +1 more
- 16 Sep 2003
TL;DR: Experimental results on a spontaneous speech recognition task show that marginalized the MMI growth function outperforms data likelihood and standard Bayesian schemes in terms of both recognition performance ranking error and word error.
Towards Efficient and Robust Automatic Speech Recognition: Decoding Techniques and Discriminative Training
Janne Pylkkönen
- 01 Jan 2013
TL;DR: This thesis points out theoretical connections of the Baum-Welch algorithm to general constrained optimization and proposes new control methods for the algorithm, which are shown to improve the robustness of the acoustic models in several large vocabulary speech recognition tasks.
References
Estimating the Dimension of a Model
TL;DR: In this paper, the problem of selecting one of a number of models of different dimensions is treated by finding its Bayes solution, and evaluating the leading terms of its asymptotic expansion.
Estimating the dimension of a model
Gideon Schwarz
- 01 Jan 2005
TL;DR: In this paper, the problem of selecting one of a number of models of different dimensions is treated by finding its Bayes solution, and evaluating the leading terms of its asymptotic expansion.
40.6K
The minimum description length principle in coding and modeling
TL;DR: The normalized maximized likelihood, mixture, and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms.
Large scale discriminative training of hidden Markov models for speech recognition
Philip C. Woodland,Daniel Povey +1 more
TL;DR: It is shown that HMMs trained with MMIE benefit as much as MLE-trained HMMs from applying model adaptation using maximum likelihood linear regression (MLLR), which has allowed the straightforward integration of MMIe- trained HMMs into complex multi-pass systems for transcription of conversational telephone speech.
396
•Proceedings Article
A novel loss function for the overall risk criterion based discriminative training of HMM models.
Janez Kaiser,Bogomir Horvat,Zdravko Kacic +2 more
- 01 Jan 2000
TL;DR: Using HMM, trained with the proposed method, a decrease of word recognition error rate of up to 17.3% has been achieved for the phoneme recognition task on the TIMIT database.
101