Generative intelligence relies heavily on the integration of vision and language. Much of the research has focused on image captioning, which involves describing images with meaningful sentences. Typically, when generating sentences that describe the visual content, a language model and a vision encoder are commonly employed. Because of the incorporation of object areas, properties, multi-modal connections, attentive techniques, and early fusion approaches like bidirectional encoder representations from transformers (BERT), these components have experienced substantial advancements over the years. This research offers a reference to the body of literature, identiﬁes emerging trends in an area that blends computer vision as well as natural language processing in order to maximize their complementary effects, and identiﬁes the most signiﬁcant technological improvements in architectures employed for image captioning. It also discusses various problem variants and open challenges. This comparison allows for an objective assessment of different techniques, architectures, and training strategies by identifying the most signiﬁcant technical innovations, and offers valuable insights into the current landscape of image captioning research.

Deep Learning Approaches for Image Captioning: Opportunities, Challenges and Future Potential

Inspired by recent work on compression with and for humans, the success of transform-based approaches to information processing, and the rise of powerful language-based AI, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">textual transform coding . It shares some of its key properties with traditional transform-based coding underlying much of our current multimedia compression technologies. It can form the basis for compression at bit rates until recently considered uselessly low, and for boosting human satisfaction from reconstructions at more traditional bit rates.

Toward Textual Transform Coding*

The automatic identification of handwritten digits by computers or other devices is referred to as "handwritten digit recognition technology," and it has a wider range of potential applications in the processing of bank bills, financial statements, and postage stamps. This study examines the methods KNN, and CNN, and their use in handwritten digit recognition using samples from the MNIST handwritten digit database. In order to achieve the best results for each method, this work rewrites KNN with Python and CNN with Tensorflow throughout the training phase. The benefits and drawbacks of the two AI based techniques employed for handwriting recognition the results has been in terms of recognition rate. 

KNN and the CNN for Handwritten Digit Recognition: A comparative study

The ability to receive and comprehend visual content is crucial for many aspects of human existence, including education, entertainment, and communication. However, millions of people worldwide experience some form of impairment that restricts their ability to access visual data. This research outlines the following objectives: generating detailed image descriptions using a Multi-Modal caption generation system and Optical Character Recognition (OCR), translating these descriptions into low-resourced and morphologically complex Indian languages, and then converting them into speech using text-to-speech technology. This approach caters to users who are visually challenged. This paper provide a comprehensive analysis of the datasets used, the models proposed, the metrics analyzed, and the issues related to image captioning and translation that have been explored. This model aims to enable people with visual impairments to understand visual content efficiently and in their own local language, rather than relying on English. In conclusion, this paper identify potential future directions and unresolved issues in the fields of image captioning and translation.

Hearing Beyond Sight: Image Captioning and Translation with Speech Synthesis for the Visually Impaired

Abstract Captioning an image involves using a combination of vision and language models to describe the image in an expressive and concise sentence. Successful captioning task requires extracting as much information as possible from the corresponding image. One of these key pieces of information is the topic to which the image belongs. The state-of-the-art methods used topic modeling depending only on caption text in order to extract these topics. The problem with extracting the topics using topic modeling only on caption text is that it lacks the consideration of the image’s semantic information. Instead, concept modeling extracts the concepts directly from the images in addition to considering the corresponding caption text. Concept modeling can be used in image captioning to extremely capture the image contexts and benefit from it to produce more accurate descriptions. In this paper, novel image captioning models are proposed by utilizing the concept modeling technique. The first concept-based model is proposed by utilizing LSTM as a decoder while the second model is proposed in association with new multi-encoder transformer architecture. Standard metrics have been used to evaluate the proposed models using Microsoft COCO and Flickr30K datasets. The proposed models outperformed the related work methods with reduced computational complexity. 

Novel concept-based image captioning models using LSTM and multi-encoder transformer architecture

Image captioning task is highly used in many realworld applications. The captioning task is concerned with understanding the image using computer vision methods. Then, natural language processing methods are used to produce a description for the image. Different approaches were proposed to solve this task, and deep learning attention-based models have been proven to be the state-of-the-art. A survey on attentionbased models for image captioning is presented in this paper including new categories that were not included in other survey papers. The attention-based approaches are classified into four main categories, further classified into subcategories. All categories and subcategories of the attention-based approaches are discussed in detail. Furthermore, the state-of-the-art approaches are compared and the accuracy improvements are stated especially in the transformer-based models, and a summary of the benchmark datasets and the main performance metrics is presented. Keywords—Image captioning; attention model; deep learning; computer vision; natural language processing

/pdf/a-survey-on-attention-based-models-for-image-captioning-1un3wbec.pdf

A Survey on Attention-Based Models for Image Captioning

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of {limit} columns in the table. Remove at least 1 column to add or create another one.

Mona M. Soliman

Author Tools

Chat about Author

Papers

A Survey on Attention-Based Models for Image Captioning