Proceedings Article10.48550/arXiv.2205.01500
Meta Learning for Natural Language Processing: A Survey
Hung-yi Lee,Shang-Wen Li,Ngoc T. Vu +2 more
- 03 May 2022
pp 666-684
28
TL;DR: The goal with this survey paper is to offer researchers pointers to relevant meta-learning works in NLP and attract more attention from the NLP community to drive future innovation.
read more
Abstract: Deep learning has been the mainstream technique in the natural language processing (NLP) area. However, deep learning requires many labeled data and is less generalizable across domains. Meta-learning is an arising field in machine learning. It studies approaches to learning better learning algorithms and aims to improve algorithms in various aspects, including data efficiency and generalizability. The efficacy of meta-learning has been shown in many NLP tasks, but there is no systematic survey of these approaches in NLP, which hinders more researchers from joining the field. Our goal with this survey paper is to offer researchers pointers to relevant meta-learning works in NLP and attract more attention from the NLP community to drive future innovation. This paper first introduces the general concepts of meta-learning and the common approaches. Then we summarize task construction settings, applications of meta-learning for various NLP problems and review the development of meta-learning in the NLP community.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
TL;DR: Mooler et al. as mentioned in this paper presented a comprehensive and practical guide for practitioners and end-users working with large language models (LLMs) in their downstream natural language processing (NLP) tasks.
A Survey of Adversarial Defences and Robustness in NLP
Shreyansh Goyal,Sumanth Doddapaneni,Mitesh M.Khapra,Balaraman Ravindran +3 more
TL;DR: In the past few years, it has become increasingly evident that deep neural networks are not resilient enough to withstand adversarial perturbations in input data, leaving them vulnerable to attack as mentioned in this paper .
99
AugGPT: Leveraging ChatGPT for Text Data Augmentation
Haixing Dai,Zheng Liu,Wenxiong Liao,Yihan Cao,Zihao Wu,Lin Zhao,Shaochen Xu,Wei Li,Ninghao Liu,Sheng Li,Dajiang Zhu,Hongmin Cai,Lichao Sun,Quanzheng Li,Dinggang Shen,Tianming Liu,Xiang (Robert) Li +16 more
- 25 Feb 2023
TL;DR: This article proposed a text data augmentation approach based on ChatGPT (named AugGPT), which rephrases each sentence in the training samples into multiple conceptually similar but semantically different samples.
77
Learn From Model Beyond Fine-Tuning: A Survey
Hongling Zheng,Li Shen,An Quang Tang,Yong Luo,Han Hu,Bo Du,Dacheng Tao +6 more
TL;DR: A comprehensive review of the current methods based on FM from the perspective of LFM is given to help readers better understand the current research status and ideas, and highlights several critical areas for future exploration and addressing open issues that require further attention.
Federated Learning and Meta Learning: Approaches, Applications, and Directions
Xiaonan Liu,Yansha Deng,Arumugam Nallanathan,Mehdi Bennis +3 more
- 24 Oct 2022
TL;DR: This tutorial presents a comprehensive review of FL, meta learning, and federated meta learning (FedMeta), and their applications over wireless networks and analyzes the relationships among these learning algorithms and examine their advantages and disadvantages in real-world applications.
20
References
•Posted Content
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TL;DR: A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
81.7K
•Posted Content
Distilling the Knowledge in a Neural Network
TL;DR: This work shows that it can significantly improve the acoustic model of a heavily used commercial system by distilling the knowledge in an ensemble of models into a single model and introduces a new type of ensemble composed of one or more full models and many specialist models which learn to distinguish fine-grained classes that the full models confuse.
21.2K
•Posted Content
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Colin Raffel,Noam Shazeer,Adam Roberts,Katherine Lee,Sharan Narang,Michael Matena,Yanqi Zhou,Wei Li,Peter J. Liu +8 more
TL;DR: This systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks and achieves state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more.
•Proceedings Article
Model-agnostic meta-learning for fast adaptation of deep networks
Chelsea Finn,Pieter Abbeel,Sergey Levine +2 more
- 06 Aug 2017
TL;DR: An algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning problems, including classification, regression, and reinforcement learning is proposed.
•Book
Foundations of Statistical Natural Language Processing
Christopher D. Manning,Hinrich Schütze +1 more
- 28 May 1999
TL;DR: This foundational text is the first comprehensive introduction to statistical natural language processing (NLP) to appear and provides broad but rigorous coverage of mathematical and linguistic foundations, as well as detailed discussion of statistical methods, allowing students and researchers to construct their own implementations.