Journal Article10.48550/arXiv.2302.03499
Exploring Data Augmentation for Code Generation Tasks
Pinzhen Chen,Gerasimos Lampouras +1 more
TL;DR: The authors proposed and adapted augmentation methods that yield consistent improvements in code translation and summarization by up to 6.9% and 7.5% respectively, and showed that their methods work orthogonally and show benefits in output code style and numeric consistency.
read more
Abstract: Advances in natural language processing, such as transfer learning from pre-trained language models, have impacted how models are trained for programming language tasks too. Previous research primarily explored code pre-training and expanded it through multi-modality and multi-tasking, yet the data for downstream tasks remain modest in size. Focusing on data utilization for downstream tasks, we propose and adapt augmentation methods that yield consistent improvements in code translation and summarization by up to 6.9% and 7.5% respectively. Further analysis suggests that our methods work orthogonally and show benefits in output code style and numeric consistency. We also discuss test data imperfections.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Data Augmentation Approaches for Source Code Models: A Survey
TL;DR: A comprehensive and integrative survey of data augmentation for source code can be found in this article , where the authors systematically compile and encapsulate existing literature to provide a comprehensive overview of the field.
Research and application of artificial intelligence based webshell detection model: A literature review
Mingrui Ma,Lansheng Han,Chunjie Zhou +2 more
- 28 Apr 2024
TL;DR: AI-based webshell detection research lacks a standardized methodology. This paper reviews the progress of AI-based webshell detection research, dividing it into three stages. The paper analyzes the main characteristics and core algorithms of each stage and identifies pain points and challenges. It also predicts future development trends.
1st Place Solution to Odyssey Emotion Recognition Challenge Task1: Tackling Class Imbalance Problem
Mingjie Chen,H.L. Zhang,Yuanchao Li,Jiachen Luo,Wen Wu,Zihan Ma,Peter Bell,CL Lai,Joshua D. Reiss,Lin Wang,Philip C. Woodland,Chen Xie,H H Phan,Thomas Hain +13 more
- 30 May 2024
TL;DR: The presented system tackles class imbalance problem in speech emotion recognition by introducing focal loss and prior-based class weights. It achieved top-1 performance on the Odyssey 2024 Emotion Recognition Challenge Task-1, with a Macro-F1 score of 35.69% and an accuracy of 37.32%.
Neural Machine Translation for Code Generation
KC Dharma,Clayton T. Morrison +1 more
TL;DR: A survey of NMT for code generation can be found in this article , where a variety of different input scenarios have been explored, including generating code based on natural language description, lower-level representations such as binary or assembly (neural decompilation), partial representations of source code (code completion and repair), and source code in another language (code translation).
References
•Posted Content
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TL;DR: A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.
81.7K
Bleu: a Method for Automatic Evaluation of Machine Translation
Kishore Papineni,Salim Roukos,Todd Ward,Wei-Jing Zhu +3 more
- 06 Jul 2002
TL;DR: This paper proposed a method of automatic machine translation evaluation that is quick, inexpensive, and language-independent, that correlates highly with human evaluation, and that has little marginal cost per run.
•Posted Content
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Yinhan Liu,Myle Ott,Naman Goyal,Jingfei Du,Mandar Joshi,Danqi Chen,Omer Levy,Michael Lewis,Luke Zettlemoyer,Veselin Stoyanov +9 more
TL;DR: It is found that BERT was significantly undertrained, and can match or exceed the performance of every model published after it, and the best model achieves state-of-the-art results on GLUE, RACE and SQuAD.
Improving Neural Machine Translation Models with Monolingual Data
Rico Sennrich,Barry Haddow,Alexandra Birch +2 more
- 12 Aug 2016
TL;DR: The authors used target-side monolingual data for NMT and obtained state-of-the-art performance for several NMT tasks, while only using parallel data for training.
•Posted Content
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Zhangyin Feng,Daya Guo,Duyu Tang,Nan Duan,Xiaocheng Feng,Ming Gong,Linjun Shou,Bing Qin,Ting Liu,Daxin Jiang,Ming Zhou +10 more
TL;DR: This work develops CodeBERT with Transformer-based neural architecture, and trains it with a hybrid objective function that incorporates the pre-training task of replaced token detection, which is to detect plausible alternatives sampled from generators.
1.8K