Word usage

Abstract: This paper presents a novel approach for automatically generating image descriptions: visual detectors, language models, and multimodal similarity models learnt directly from a dataset of image captions. We use multiple instance learning to train visual detectors for words that commonly occur in captions, including many different parts of speech such as nouns, verbs, and adjectives. The word detector outputs serve as conditional inputs to a maximum-entropy language model. The language model learns from a set of over 400,000 image descriptions to capture the statistics of word usage. We capture global semantics by re-ranking caption candidates using sentence-level features and a deep multimodal similarity model. Our system is state-of-the-art on the official Microsoft COCO benchmark, producing a BLEU-4 score of 29.1%. When human judges compare the system captions to ones written by other people on our held-out test set, the system captions have equal or better quality 34% of the time.

Abstract: We propose a new computational approach for tracking and detecting statistically significant linguistic shifts in the meaning and usage of words. Such linguistic shifts are especially prevalent on the Internet, where the rapid exchange of ideas can quickly change a word's meaning. Our meta-analysis approach constructs property time series of word usage, and then uses statistically sound change point detection algorithms to identify significant linguistic shifts. We consider and analyze three approaches of increasing complexity to generate such linguistic property time series, the culmination of which uses distributional characteristics inferred from word co-occurrences. Using recently proposed deep neural language models, we first train vector representations of words for each time period. Second, we warp the vector spaces into one unified coordinate system. Finally, we construct a distance-based distributional time series for each word to track its linguistic displacement over time. We demonstrate that our approach is scalable by tracking linguistic change across years of micro-blogging using Twitter, a decade of product reviews using a corpus of movie reviews from Amazon, and a century of written books using the Google Book Ngrams. Our analysis reveals interesting patterns of language usage change commensurate with each medium.

Abstract: As a language evolves, grammatical rules emerge and exceptions die out Lieberman et al have calculated the rate at which a language grows more regular, based on 1,200 years of English usage Of 177 irregular verbs, 79 became regular in the last millennium And the trend follows a simple rule: a verb's half-life scales as the square root of its frequency Irregular verbs that are 100 times as rare regularize 10 times faster The emergence of a rule (such as adding –ed for the past tense) spells death for exceptional forms The cover graphic makes the point: verb size corresponds to usage frequency, so large verbs stay at the top, and small verbs fall to the bottom 'Wed', the next irregular verb to go, is on the brink In a separate study, Pagel et al looked at changing word meanings Across the Indo-European languages, words like 'tail' or 'bird' evolve rapidly and are expressed by many unrelated words Others, like 'two', are expressed by closely related word forms across the whole language family Data from over 80 modern languages show that the more a word is used, the less it changes During language evolution, rules emerge and exceptions decline A quantitative study measures the rate at which a human language becomes more regular over time Specifically, the regularization of English verbs over the last 1200 years was studied, and it was found that half-life of a verb scales as the square root of its frequency, meaning that irregular verbs that are 100 times as rare regularize ten times faster Human language is based on grammatical rules1,2,3,4 Cultural evolution allows these rules to change over time5 Rules compete with each other: as new rules rise to prominence, old ones die away To quantify the dynamics of language evolution, we studied the regularization of English verbs over the past 1,200 years Although an elaborate system of productive conjugations existed in English’s proto-Germanic ancestor, Modern English uses the dental suffix, ‘-ed’, to signify past tense6 Here we describe the emergence of this linguistic rule amidst the evolutionary decay of its exceptions, known to us as irregular verbs We have generated a data set of verbs whose conjugations have been evolving for more than a millennium, tracking inflectional changes to 177 Old-English irregular verbs Of these irregular verbs, 145 remained irregular in Middle English and 98 are still irregular today We study how the rate of regularization depends on the frequency of word usage The half-life of an irregular verb scales as the square root of its usage frequency: a verb that is 100 times less frequent regularizes 10 times as fast Our study provides a quantitative analysis of the regularization process by which ancestral forms gradually yield to an emerging linguistic rule

Abstract: This study examines how first language and the type of writing task affect undergraduates’ word usage from source readings in their English writing. Of 87 participating university undergraduates, 39 were native English speakers from a 1st-year writing course in a North American university, whereas 48 were 3rd-year Chinese students learning English as a second language in a university in China. Using two preselected source texts, half of the students in each group completed a summary task; the other half completed an opinion task. Students’ drafts and the source texts were compared to identify exact or near verbatim retention of strings of words from sources with or without acknowledgement. A two-way ANOVA indicated that both task and first language had an effect on the amount of words borrowed. The study found that students who did the summary task borrowed more words than those who wrote the opinion essays, and Chinese students used source texts mostly without citing references for either task.