Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks, since the release of ChatGPT in November 2022. LLMs' ability of general-purpose language understanding and generation is acquired by training billions of model's parameters on massive amounts of text data, as predicted by scaling laws \cite{kaplan2020scaling,hoffmann2022training}. The research area of LLMs, while very recent, is evolving rapidly in many different ways. In this paper, we review some of the most prominent LLMs, including three popular LLM families (GPT, LLaMA, PaLM), and discuss their characteristics, contributions and limitations. We also give an overview of techniques developed to build, and augment LLMs. We then survey popular datasets prepared for LLM training, fine-tuning, and evaluation, review widely used LLM evaluation metrics, and compare the performance of several popular LLMs on a set of representative benchmarks. Finally, we conclude the paper by discussing open challenges and future research directions.

Large Language Models: A Survey

The potential for an intelligent transportation system (ITS) has been made possible by the growth of the Internet of things (IoT) and artificial intelligence (AI), resulting in the integration of IoT and ITS—known as the Internet of vehicles (IoV). To achieve the goal of automatic driving and efficient mobility, IoV is now combined with modern communication technologies (such as 5G) to achieve intelligent connected vehicles (ICVs). However, IoV is challenged with security risks in the following five (5) domains: ICV security, intelligent device security, service platform security, V2X communication security, and data security. Numerous AI models have been developed to mitigate the impact of intrusion threats on ICVs. On the other hand, the rise in explainable AI (XAI) results from the requirement to inject confidence, transparency, and repeatability into the development of AI for the security of ICV and to provide a safe ITS. As a result, the scope of this review covered the XAI models used in ICV intrusion detection systems (IDSs), their taxonomies, and outstanding research problems. The results of the study show that XAI though in its infancy of application to ICV, is a promising research direction in the quest for improving the network efficiency of ICVs. The paper further reveals that XAI increased transparency will foster its acceptability in the automobile industry.

/pdf/explainable-artificial-intelligence-xai-for-intrusion-3jso39k4.pdf

Explainable Artificial Intelligence (XAI) for Intrusion Detection and Mitigation in Intelligent Connected Vehicles: A Review


Purpose
The application of artificial intelligence chatbots is an emerging trend in educational technology studies for its multi-faceted advantages. However, the existing studies rarely take a perspective of educational technology application to evaluate the application of chatbots to educational contexts. This study aims to bridge the research gap by taking an educational perspective to review the existing literature on artificial intelligence chatbots.


Design/methodology/approach
This study combines bibliometric analysis and citation network analysis: a bibliometric analysis through visualization of keyword, authors, organizations and countries and a citation network analysis based on literature clustering.


Findings
Educational applications of chatbots are still rising in post-COVID-19 learning environments. Popular research issues on this topic include technological advancements, students’ perception of chatbots and effectiveness of chatbots in different educational contexts. Originating from similar technological and theoretical foundations, chatbots are primarily applied to language education, educational services (such as information counseling and automated grading), health-care education and medical training. Diversifying application contexts demonstrate specific purposes for using chatbots in education but are confronted with some common challenges. Multi-faceted factors can influence the effectiveness and acceptance of chatbots in education. This study provides an extended framework to facilitate extending artificial intelligence chatbot applications in education.


Research limitations/implications
The authors have to acknowledge that this study is subjected to some limitations. First, the literature search was based on the core collection on Web of Science, which did not include some existing studies. Second, this bibliometric analysis only included studies published in English. Third, due to the limitation in technological expertise, the authors could not comprehensively interpret the implications of some studies reporting technological advancements. However, this study intended to establish its research significance by summarizing and evaluating the effectiveness of artificial intelligence chatbots from an educational perspective.


Originality/value
This study identifies the publication trends of artificial intelligence chatbots in educational contexts. It bridges the research gap caused by previous neglection of treating educational contexts as an interconnected whole which can demonstrate its characteristics. It identifies the major application contexts of artificial intelligence chatbots in education and encouraged further extending of applications. It also proposes an extended framework to consider that covers three critical components of technological integration in education when future researchers and instructors apply artificial intelligence chatbots to new educational contexts.


A bibliometric analysis of artificial intelligence chatbots in educational contexts

Modern Automatic Speech Recognition (ASR) systems can achieve high performance in terms of recognition accuracy. However, a perfectly accurate transcript still can be challenging to read due to grammatical errors, disfluency, and other errata common in spoken communication. Many downstream tasks and human readers rely on the output of the ASR system; therefore, errors introduced by the speaker and ASR system alike will be propagated to the next task in the pipeline. In this work, we propose a novel NLP task called ASR post-processing for readability (APR) that aims to transform the noisy ASR output into a readable text for humans and downstream tasks while maintaining the semantic meaning of the speaker. In addition, we describe a method to address the lack of task-specific data by synthesizing examples for the APR task using the datasets collected for Grammatical Error Correction (GEC) followed by text-to-speech (TTS) and ASR. Furthermore, we propose metrics borrowed from similar tasks to evaluate performance on the APR task. We compare fine-tuned models based on several open-sourced and adapted pre-trained models with the traditional pipeline method. Our results suggest that finetuned models improve the performance on the APR task significantly, hinting at the potential benefits of using APR systems. We hope that the read, understand, and rewrite approach of our work can serve as a basis that many NLP tasks and human readers can benefit from.

pdf/improving-readability-for-automatic-speech-recognition-2y6aqzv67c.pdf

Improving Readability for Automatic Speech Recognition Transcription

The widespread propagation of misinformation on social media platforms poses a significant concern, prompting substantial endeavors within the research community to develop robust detection solutions. Individuals often place unwavering trust in social networks, often without discerning the origins and authenticity of the information disseminated through these platforms. Hence, the identification of media-rich fake news necessitates an approach that adeptly leverages multimedia elements and effectively enhances detection accuracy. The ever-changing nature of cyberspace highlights the need for measures that may effectively resist the spread of media-rich fake news while protecting the integrity of information systems. This study introduces a robust approach for fake news detection, utilizing three publicly available datasets: WELFake, FakeNewsNet, and FakeNewsPrediction. We integrated FastText word embeddings with various Machine Learning and Deep Learning methods, further refining these algorithms with regularization and hyperparameter optimization to mitigate overfitting and promote model generalization. Notably, a hybrid model combining Convolutional Neural Networks and Long Short-Term Memory, enriched with FastText embeddings, surpassed other techniques in classification performance across all datasets, registering accuracy and F1-scores of 0.99, 0.97, and 0.99, respectively. Additionally, we utilized state-of-the-art transformer-based models such as BERT, XLNet, and RoBERTa, enhancing them through hyperparameter adjustments. These transformer models, surpassing traditional RNN-based frameworks, excel in managing syntactic nuances, thus aiding in semantic interpretation. In the concluding phase, explainable AI modeling was employed using Local Interpretable Model-Agnostic Explanations, and Latent Dirichlet Allocation to gain deeper insights into the model's decision-making process. 

Advancing Fake News Detection: Hybrid Deep Learning With FastText and Explainable AI

The success of Transformer architecture has seen increased interest in machine translation (MT). The translation quality of neural network-based MT transcends that of translations derived using statistical methods. This growth in MT research has entailed the development of accurate automatic evaluation metrics that allow us to track the performance of MT. However, automatically evaluating and comparing MT systems is a challenging task. Several studies have shown that traditional metrics (e.g., BLEU, TER) show poor performance in capturing semantic similarity between MT outputs and human reference translations. To date, to improve performance, various evaluation metrics have been proposed using the Transformer architecture. However, a systematic and comprehensive literature review on these metrics is still missing. Therefore, it is necessary to survey the existing automatic evaluation metrics of MT to enable both established and new researchers to quickly understand the trend of MT evaluation over the past few years. In this survey, we present the trend of automatic evaluation metrics. To better understand the developments in the field, we provide the taxonomy of the automatic evaluation metrics. Then, we explain the key contributions and shortcomings of the metrics. In addition, we select the representative metrics from the taxonomy, and conduct experiments to analyze related problems. Finally, we discuss the limitation of the current automatic metric studies through the experimentation and our suggestions for further research to improve the automatic evaluation metrics.

/pdf/a-survey-on-evaluation-metrics-for-machine-translation-2vypl9cc.pdf

A Survey on Evaluation Metrics for Machine Translation

The machine translation system aims to translate source language into target language. Recent studies on MT systems mainly focus on neural machine translation. One factor that significantly affects the performance of NMT is the availability of high-quality parallel corpora. However, high-quality parallel corpora concerning Korean are relatively scarce compared to those associated with other high-resource languages, such as German or Italian. To address this problem, AI Hub recently released seven types of parallel corpora for Korean. In this study, we conduct an in-depth verification of the quality of corresponding parallel corpora through Linguistic Inquiry and Word Count (LIWC) and several relevant experiments. LIWC is a word-counting software program that can analyze corpora in multiple ways and extract linguistic features as a dictionary base. To the best of our knowledge, this study is the first to use LIWC to analyze parallel corpora in the field of NMT. Our findings suggest the direction of further research toward obtaining the improved quality parallel corpora through our correlation analysis in LIWC and NMT performance.

/pdf/empirical-analysis-of-parallel-corpora-and-in-depth-analysis-1x4e1bi5.pdf

Empirical Analysis of Parallel Corpora and In-Depth Analysis Using LIWC

Fake News (disinformation with malicious intent) has emerged as a major social problem. To address this issue, previous studies mainly utilized single information, the news content, to detect fake news. However, using only news content in training is insufficient. Moreover, most studies did not consider the propagation aspect of fake news as a training feature. Thus, in an attempt to incorporate the ability to learn representation based on textual information and social context, this study proposed a fake news detection algorithm that thoroughly utilizes user graph in Korean fake news and dataset construction methods. In addition, a training strategy was proposed for utilizing user graph in Korean fake news detection through comparative and ablation studies. The experimental results showed that K-FANG outperformed the baseline in detecting fake news. Moreover, user engagements were found to be useful for detecting fake news even if the data contained hate speech. Finally, the validity of using stance information by expanding its class and controlling the class imbalance issues was also verified. This study provided useful implications for utilizing user information in fake news detection.

/pdf/utilization-strategy-of-user-engagements-in-korean-fake-news-2yscigq8.pdf

Utilization Strategy of User Engagements in Korean Fake News Detection

Generative commonsense reasoning refers to the ability of a language model to generate a sentence with a given concept-set based on compositional generalization and commonsense reasoning. In the CommonGen challenge, which evaluates the capability of generative commonsense reasoning, language models continue to exhibit low performances and struggle to leverage knowledge representation from humans. Therefore, we propose PU-GEN to leverage human-centered knowledge in language models to enhance compositional generalization and commonsense reasoning considering the human language generation process. To incorporate human-centered knowledge, PU -GEN reinterprets two linguistic philosophies from Wittgenstein: p icture theory and u se theory . First, we retrieve scene knowledge to reflect picture theory such that a model can describe a general situation as if it were being painted. Second, we extend relational knowledge to consider use theory for understanding various contexts. PU-GEN demonstrates superior performance in qualitative and quantitative evaluations over baseline models in CommonGen and generates convincing evidence for CommonsenseQA. Moreover, it outperforms the state-of-the-art model used in the previous CommonGen challenge. • Reinterpretation of generative commonsense reasoning via Wittgenstein’s picture and use theory. • We extend the limited knowledge in the CommonGen by leveraging scene and relational knowledge. • PU-GEN is a knowledge-enhanced language model for developing generative commonsense reasoning. • PU-GEN can be implemented in another commonsense downstream task: Commonsense Question Answering. 

PU-GEN: Enhancing generative commonsense reasoning for language models with human-centered knowledge

Existing methods of training domain-specialized neural machine translation (DS-NMT) models are based on the pretrain-finetuning approach (PFA). In this study, we reinterpret existing methods based on the perspective of cognitive science related to cross language speech perception. We propose the cross communication method (CCM), a new DS-NMT training approach. Inspired by the learning method of infants, we perform DS-NMT training by configuring and training DC and GC concurrently in batches. Quantitative and qualitative analysis of our experimental results show that CCM can achieve superior performance compared to the conventional methods. Additionally, we conducted an experiment considering the DS-NMT service to meet industrial demands.

/pdf/mimicking-infants-bilingual-language-acquisition-for-domain-10x20iha.pdf

Mimicking Infants’ Bilingual Language Acquisition for Domain Specialized Neural Machine Translation

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Chanjun Park

Author Tools

Chat about Author