Training Conditional Random Fields with Multivariate Evaluation Measures
Jun Suzuki,Erik McDermott,Hideki Isozaki +2 more
- 17 Jul 2006
- pp 217-224
TL;DR: This paper proposes a framework for training Conditional Random Fields to optimize multivariate evaluation measures, including non-linear measures such as F-score, derived from an error minimization approach that provides a simple solution for directly optimizing any evaluation measure.
read more
Abstract: This paper proposes a framework for training Conditional Random Fields (CRFs) to optimize multivariate evaluation measures, including non-linear measures such as F-score. Our proposed framework is derived from an error minimization approach that provides a simple solution for directly optimizing any evaluation measure. Specifically focusing on sequential segmentation tasks, i.e. text chunking and named entity recognition, we introduce a loss function that closely reflects the target evaluation measure for these tasks, namely, segmentation F-score. Our experiments show that our method performs better than standard CRF training.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Optimal thresholding of classifiers to maximize F1 measure
Zachary C. Lipton,Charles Elkan,Balakrishnan Naryanaswamy +2 more
- 15 Sep 2014
TL;DR: This paper provides new insight into maximizing F1 measures in the context of binary classification and also in the contexts of multilabel classification.
Joint Mention Extraction and Classification with Mention Hypergraphs
TL;DR: A novel model is presented for the task of joint mention extraction and classification that is able to effectively capture overlapping mentions with unbounded lengths and can be extended to additionally capture mention heads explicitly in a joint manner under the same time complexity.
•Proceedings Article
Softmax-Margin CRFs: Training Log-Linear Models with Cost Functions
Kevin Gimpel,Noah A. Smith +1 more
- 02 Jun 2010
TL;DR: A method of incorporating task-specific cost functions into standard conditional log-likelihood (CLL) training of linear structured prediction models for named-entity recognition is described and experimental results comparing with several commonly-used methods are presented.
•Posted Content
Thresholding Classifiers to Maximize F1 Score
TL;DR: For any classifier that produces a real-valued output, the relationship between the best achievable F1 score and the decision-making threshold that achieves this optimum is derived and the results can be surprising.
•Proceedings Article
An Exact Algorithm for F-Measure Maximization
Krzysztof Dembczyński,Willem Waegeman,Weiwei Cheng,Eyke Hüllermeier +3 more
- 12 Dec 2011
TL;DR: This paper presents an algorithm which is not only computationally efficient but also exact, regardless of the underlying distribution, and illustrates its practical performance by means of experimental results for multi-label classification.
References
•Proceedings Article
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
John Lafferty,Andrew McCallum,Fernando Pereira +2 more
- 28 Jun 2001
TL;DR: This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
A Formal Basis for the Heuristic Determination of Minimum Cost Paths
TL;DR: How heuristic information from the problem domain can be incorporated into a formal mathematical theory of graph searching is described and an optimality property of a class of search strategies is demonstrated.
12.7K
On the limited memory BFGS method for large scale optimization
Dong C. Liu,Jorge Nocedal +1 more
TL;DR: The numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence, and the convergence properties are studied to prove global convergence on uniformly convex problems.
Introduction to the CoNLL-2003 shared task: language-independent named entity recognition
Erik Tjong Kim Sang,Fien De Meulder +1 more
- 31 May 2003
TL;DR: The CoNLL-2003 shared task on NER as mentioned in this paper was the first NER task with language-independent named entity recognition (NER) data sets and evaluation method, and a general overview of the systems that participated in the task and their performance.