Proceedings Article10.1109/ICCCIS51004.2021.9397159
An Approach to identify Captioning Keywords in an Image using LIME
Siddharth Sahay,Nikita Omare,K. K. Shukla +2 more
- 19 Feb 2021
26
TL;DR: In this paper, explainable AI techniques such as LIME (Local Interpretable Model-Agnostic Explanations) are employed to explain the predictions of complex image captioning models.
read more
Abstract: Machine Learning models are being increasingly deployed to tackle real-world problems in various domains like healthcare, crime and education among many others. However, most of the models are practically "black-boxes": although they may provide accurate results, they are unable to provide any conclusive reasoning for those results. In order for these decisions to be trusted, they must be explainable. Explainable AI, or XAI refers to methods and techniques in the application of AI such that the results of the solution are understandable by human experts. This paper focuses on the task of Image Captioning, and tries to employ XAI techniques such as LIME (Local Interpretable Model-Agnostic Explanations) to explain the predictions of complex image captioning models. It visually depicts the part of the image corresponding to a particular word in the caption, thereby justifying why the model predicted that word.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Automatic COVID-19 prediction using explainable machine learning techniques
TL;DR: In this paper , the authors analyzed automatic COVID-19 detection using machine learning techniques to build an intelligent web application, where they used explainable AI with the LIME framework to interpret the prediction results.
54
Food fraud detection using explainable artificial intelligence
TL;DR: In this paper , a case study was performed on a food fraud dataset using adulteration/fraud notifications retrieved from the Rapid Alert System for Food and Feed system and economically motivated adulterations database.
29
Explainability for Medical Image Captioning
19 Apr 2022
TL;DR: In this article , an explainable module for medical image captioning is presented, which provides a sound interpretation of the attention-based encoder-decoder model by explaining the correspondence between visual features and semantic features.
International Journal of Cognitive Computing in Engineering
TL;DR: In this paper , a voting classifier was used to determine the likeliness of death of a patient in a COVID-19 pandemic, which is a novel coronavirus that turned into a pandemic and caused enormous casualties.
4
References
•Proceedings Article
ROUGE: A Package for Automatic Evaluation of Summaries
Chin-Yew Lin
- 25 Jul 2004
TL;DR: Four different RouGE measures are introduced: ROUGE-N, ROUge-L, R OUGE-W, and ROUAGE-S included in the Rouge summarization evaluation package and their evaluations.
•Proceedings Article
METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments
Satanjeev Banerjee,Alon Lavie +1 more
- 01 Jun 2005
TL;DR: METEOR is described, an automatic metric for machine translation evaluation that is based on a generalized concept of unigram matching between the machineproduced translation and human-produced reference translations and can be easily extended to include more advanced matching strategies.
CIDEr: Consensus-based image description evaluation
Ramakrishna Vedantam,C. Lawrence Zitnick,Devi Parikh +2 more
- 07 Jun 2015
TL;DR: A novel paradigm for evaluating image descriptions that uses human consensus is proposed and a new automated metric that captures human judgment of consensus better than existing metrics across sentences generated by various sources is evaluated.
•Posted Content
Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI.
Alejandro Barredo Arrieta,Natalia Díaz-Rodríguez,Javier Del Ser,Javier Del Ser,Adrien Bennetot,Adrien Bennetot,Siham Tabik,Alberto Barbado,Salvador García,Sergio Gil-Lopez,Daniel Molina,Richard Benjamins,Raja Chatila,Francisco Herrera +13 more
TL;DR: Previous efforts to define explainability in Machine Learning are summarized, establishing a novel definition that covers prior conceptual propositions with a major focus on the audience for which explainability is sought, and a taxonomy of recent contributions related to the explainability of different Machine Learning models are proposed.
5.4K
Deep visual-semantic alignments for generating image descriptions
Andrej Karpathy,Li Fei-Fei +1 more
- 07 Jun 2015
TL;DR: A model that generates natural language descriptions of images and their regions based on a novel combination of Convolutional Neural Networks over image regions, bidirectional Recurrent Neural networks over sentences, and a structured objective that aligns the two modalities through a multimodal embedding is presented.