Large NLP models have recently shown impressive performance in language understanding tasks, typically evaluated by their finetuned performance.Alternatively, probing has received increasing attention as being a lightweight method for interpreting the intrinsic mechanisms of large NLP models.In probing, post-hoc classifiers are trained on "out-ofdomain" datasets that diagnose specific abilities.While probing the language models has led to insightful findings, they appear disjointed from the development of models.This paper explores the utility of probing deep NLP models to extract a proxy signal widely used in model development -the fine-tuning performance.We find that it is possible to use the accuracies of only three probing tests to predict the fine-tuning performance with errors 40% -80% smaller than baselines.We further discuss possible avenues where probing can empower the development of deep NLP models. 

pdf/predicting-fine-tuning-performance-with-probing-2ubbcfw4in.pdf

Predicting Fine-Tuning Performance with Probing

Aspect-based Sentiment Analysis (ABSA) is a complex model within the domain of Sentiment Analysis (SA) tasks which deals with classifying the sentiments related to particular aspects (or targets) in the given text. ABSA task has gained popularity due to its various sub-tasks related to the aspect-based sentiment analysis task. This work provides a comparative study of various approaches used to solve the ABSA task using the BERT technique. The selected approaches include a fine-tuned BERT model, adversarial training using BERT (Bidirectional Encoder Representations from Transformers) and the incorporation of disentangled attention in BERT or the DeBERTa for the ABSA task. One of the challenges faced during implementation of the ABSA task is that it requires an in-depth understanding about the language. Experiment results indicate that the approach, which uses the fine-tuned BERT model yields the best mean F1 score of 85.65 and the best mean accuracy score of 85.98 is yielded by the DeBERTa model. 

Comparative Analysis on Aspect-based Sentiment using BERT

Advances in large language models (LLMs) have revolutionized the natural language processing field. However, the text generated by LLMs can result in various issues, such as fake news, misinformation, and social media spam. In addition, detecting machine-generated text is becoming increasingly difficult because it produces text that resembles human writing. We propose a new method for effectively detecting machine-generated text by applying adversarial training (AT) to pre-trained language models (PLMs), such as Bidirectional Encoder Representations from Transformers (BERT). We generated adversarial examples that appeared to have been modified by humans and applied them to the PLMs to improve the model’s detection capabilities. The proposed method was validated on various datasets and experiments. It showed improved performance compared to traditional fine-tuning methods, with an average reduction in the probability of misclassification of machine-generated text by about 10%. We demonstrated the robustness of the model when generated with input tokens of different lengths and under different training data ratios. We suggested future research directions for applying AT to different languages and language model types. This study opens new possibilities for applying AT to the problem of machine-generated text detection and classification and contributes to building more effective detection models.

Enhancing Machine-Generated Text Detection: Adversarial Fine-Tuning of Pre-Trained Language Models

Adversarial training (AT) is one of the most reliable methods for defending against adversarial attacks in machine learning. Variants of this method have been used as regularization mechanisms to achieve SOTA results on NLP benchmarks, and they have been found to be useful for transfer learning and continual learning. We search for the reasons for the effectiveness of AT by contrasting vanilla and adversarially fine-tuned BERT models. We identify partial preservation of BERT's syntactic abilities during fine-tuning as the key to the success of AT. We observe that adversarially fine-tuned models remain more faithful to BERT's language modeling behavior and are more sensitive to the word order. As concrete examples of syntactic abilities, an adversarially fine-tuned model could have an advantage of up to 38% on anaphora agreement and up to 11% on dependency parsing. Our analysis demonstrates that vanilla fine-tuning oversimplifies the sentence representation by focusing heavily on one or a few label-indicative words. AT, however, moderates the effect of these influential words and encourages representational diversity. This allows for a more hierarchical representation of a sentence and leads to the mitigation of BERT's loss of syntactic abilities.

How Does Adversarial Fine-Tuning Benefit BERT?

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Hao Yang

Author Tools

Chat about Author

Papers

How Does Adversarial Fine-Tuning Benefit BERT?