Open AccessPosted Content
EntEval: A Holistic Evaluation Benchmark for Entity Representations
TL;DR: This work proposes EntEval: a test suite of diverse tasks that require nontrivial understanding of entities including entity typing, entity similarity, entity relation prediction, and entity disambiguation, and develops training techniques for learning better entity representations by using natural hyperlink annotations in Wikipedia.
read more
Abstract: Rich entity representations are useful for a wide class of problems involving entities. Despite their importance, there is no standardized benchmark that evaluates the overall quality of entity representations. In this work, we propose EntEval: a test suite of diverse tasks that require nontrivial understanding of entities including entity typing, entity similarity, entity relation prediction, and entity disambiguation. In addition, we develop training techniques for learning better entity representations by using natural hyperlink annotations in Wikipedia. We identify effective objectives for incorporating the contextual information in hyperlinks into state-of-the-art pretrained language models and show that they improve strong baselines on multiple EntEval tasks.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Interpretable Entity Representations through Large-Scale Typing
Yasumasa Onoe,Greg Durrett +1 more
- 01 Apr 2020
TL;DR: This paper presents an approach to creating entity representations that are human readable and achieve high performance on entity-related tasks out of the box, and shows that these embeddings can be post-hoc modified through a small number of rules to incorporate domain knowledge and improve performance.
Mining Knowledge for Natural Language Inference from Wikipedia Categories
Mingda Chen,Zewei Chu,Karl Stratos,Kevin Gimpel +3 more
- 01 Nov 2020
TL;DR: WikiNLI is introduced: a resource for improving model performance on NLI and LE tasks, and it is shown that it can improve strong baselines such as BERT and RoBERTa by pretraining them on WikiNLI and transferring the models on downstream tasks.
•Posted Content
Exploring Neural Entity Representations for Semantic Information
Andrew Runge,Eduard Hovy +1 more
TL;DR: This paper evaluated a diverse set of eight neural entity embedding methods on a set of simple probing tasks, demonstrating which methods are able to remember words used to describe entities, learn type, relationship and factual information, and identify how frequently an entity is mentioned.
•Posted Content
Evaluation Benchmarks and Learning Criteria for Discourse-Aware Sentence Representations
TL;DR: This work proposes DiscoEval, a test suite of tasks to evaluate whether sentence representations include broader context information, and proposes a variety of training objectives that makes use of natural annotations from Wikipedia to build sentence encoders capable of modeling discourse.
References
Long short-term memory
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
99K
Glove: Global Vectors for Word Representation
Jeffrey Pennington,Richard Socher,Christopher D. Manning +2 more
- 01 Oct 2014
TL;DR: A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
•Posted Content
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Yinhan Liu,Myle Ott,Naman Goyal,Jingfei Du,Mandar Joshi,Danqi Chen,Omer Levy,Michael Lewis,Luke Zettlemoyer,Veselin Stoyanov +9 more
TL;DR: It is found that BERT was significantly undertrained, and can match or exceed the performance of every model published after it, and the best model achieves state-of-the-art results on GLUE, RACE and SQuAD.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin,Ming-Wei Chang,Kenton Lee,Kristina Toutanova +3 more
- 11 Oct 2018
TL;DR: BERT as mentioned in this paper pre-trains deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
24.6K
Deep contextualized word representations
Matthew E. Peters,Mark Neumann,Mohit Iyyer,Matt Gardner,Christopher Clark,Kenton Lee,Luke Zettlemoyer +6 more
- 15 Feb 2018
TL;DR: This paper introduced a new type of deep contextualized word representation that models both complex characteristics of word use (e.g., syntax and semantics), and how these uses vary across linguistic contexts (i.e., to model polysemy).