Proceedings Article10.1109/ASIANCON55314.2022.9909423
The Transformers’ Ability to Implement for Solving Intricacies of Language Processing
Paras Nath Singh,Sagarika Behera +1 more
- 26 Aug 2022
pp 1-7
4
TL;DR: This proposal implements Natural Language Processing introducing POS (Parts of Speech) Tags, Bigrams and implementing pipeline, summarizer, paraphrasing, sentiment-analysis using transformers.
read more
Abstract: AI (Artificial Intelligence) tools have reached a surprising level of linguistic frequency. The best and biggest (little complex) of these are based on an architecture called the transformers. A transformer in AI is a tool that acts as a kind of training model to train the pre-trained models. Available pre-trained models can reduce computational cost, carbon foot print and saves time. A transformer model can be used for different modalities such as text, images, audio and even for multi-model including video classification and visual question answering. Latest version of transformers (Transformers 4.18.0) has been released by Pypi.org on 6th Apr 2022 as part of the Hugging Face logo which claims SOTA (state-of-the-art), NLP (Natural Language Processing for Tensorflow 2.0 and PyTorch. This paper is one the first papers using this latest release of transformer tool for NLP (Natural Language Processing) in Python. Using latest Transformers this paper overviews, represents implements its powerful tools BERT (Bidirectional Encoder Representation from Transformers), GPT (Generative Pre-Trained), BART (Bidirectional Abstractive Representation from Transformers), & other utilities of highly efficient linear algebra libraries of Python for Natural Language Processing. So, this proposal implements Natural Language Processing introducing POS (Parts of Speech) Tags, Bigrams and implementing pipeline, summarizer, paraphrasing, sentiment-analysis using transformers. For sentiment Analysis multilingual (English & French) have been selected & matching sentiment results are 94% to 98 % for four test cases which are far better than conventional methods.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Analyzing Embedding Models for Embedding Vectors in Vector Databases
Paras Nath Singh,Sreya Talasila,Shivaraj Veerappa Banakar +2 more
- 08 Dec 2023
TL;DR: Vector embeddings are frequently used for analyzing and exploring unstructured data. Vector databases are specifically designed to handle vector embeddings and provide quick and precise nearest-neighbor search, clustering, and similarity matching.
5
Multilingual Education through Optical Character Recognition (OCR) and AI
Hritik Singh,S Obaidat Mohammad,A Mustapha M Yaseen,Mayuri H. Molawade,Sagar Mohite,Veena Jadhav,Rohini Jadhav +6 more
- 25 Apr 2024
TL;DR: The system aims to democratize access to educational resources by extracting text from English PDFs using OCR and then translating and teaching the content in the user’s preferred language through AI.
1
Paraphrase Generation Model Using Transformer Based Architecture
TL;DR: This study proposes a paraphrase generation model utilizing a transformer-based architecture, enhancing language understanding and generation capabilities, and achieving state-of-the-art results in paraphrasing tasks by leveraging self-attention mechanisms and encoder-decoder frameworks.
Automated Business Report Summarization Using Transformer Model
B Faizal,Sajimon Abraham,Sijo Thomas +2 more
- 14 Mar 2024
TL;DR: This paper proposes a transformer-based model for automated business report summarization, handling both tabular and text data, to alleviate the challenging task of summarizing financial reports with visual representations.
References
•Posted Content
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
TL;DR: This work proposes a method to pre-train a smaller general-purpose language representation model, called DistilBERT, which can be fine-tuned with good performances on a wide range of tasks like its larger counterparts, and introduces a triple loss combining language modeling, distillation and cosine-distance losses.
7.3K
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context.
Zihang Dai,Zhilin Yang,Yiming Yang,Jaime G. Carbonell,Quoc V. Le,Ruslan Salakhutdinov +5 more
- 09 Jan 2019
TL;DR: This work proposes a novel neural architecture Transformer-XL that enables learning dependency beyond a fixed length without disrupting temporal coherence, which consists of a segment-level recurrence mechanism and a novel positional encoding scheme.
•Posted Content
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
TL;DR: The authors proposed a self-supervised loss that focuses on modeling inter-sentence coherence, and showed it consistently helps downstream tasks with multientence inputs, achieving state-of-the-art results on the GLUE, RACE, and \squad benchmarks.
4.1K
•Posted Content
Cross-lingual Language Model Pretraining.
Guillaume Lample,Alexis Conneau +1 more
TL;DR: This work proposes two methods to learn cross-lingual language models (XLMs): one unsupervised that only relies on monolingual data, and one supervised that leverages parallel data with a new cross-lingsual language model objective.
2.7K
•Posted Content
Reformer: The Efficient Transformer
TL;DR: The Reformer as discussed by the authors uses locality-sensitive hashing to improve the efficiency of Transformers and achieves state-of-the-art results on a number of tasks, but training these models can be prohibitively costly.
1.5K