The rising popularity of explainable artificial intelligence (XAI) to understand high-performing black boxes raised the question of how to evaluate explanations of machine learning (ML) models. While interpretability and explainability are often presented as a subjectively validated binary property, we consider it a multi-faceted concept. We identify 12 conceptual properties, such as Compactness and Correctness, that should be evaluated for comprehensively assessing the quality of an explanation. Our so-called Co-12 properties serve as categorization scheme for systematically reviewing the evaluation practices of more than 300 papers published in the last 7 years at major AI and ML conferences that introduce an XAI method. We find that 1 in 3 papers evaluate exclusively with anecdotal evidence, and 1 in 5 papers evaluate with users. This survey also contributes to the call for objective, quantifiable evaluation methods by presenting an extensive overview of quantitative XAI evaluation methods. Our systematic collection of evaluation methods provides researchers and practitioners with concrete tools to thoroughly validate, benchmark and compare new and existing XAI methods. The Co-12 categorization scheme and our identified evaluation methods open up opportunities to include quantitative metrics as optimization criteria during model training in order to optimize for accuracy and interpretability simultaneously.

/pdf/from-anecdotal-evidence-to-quantitative-evaluation-methods-a-7wshq99m.pdf

From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI

Online streaming feature selection (OSFS) has attracted extensive attention during the past decades. Current approaches commonly assume that the feature space of fixed data instances dynamically increases without any missing data. However, this assumption does not always hold in many real applications. Motivated by this observation, this study aims to implement online feature selection from sparse streaming features, i.e., features flow in one by one with missing data as instance count remains fixed. To do so, this study proposes a latent-factor-analysis-based online sparse-streaming-feature selection algorithm (LOSSA). Its main idea is to apply latent factor analysis to pre-estimate missing data in sparse streaming features before conducting feature selection, thereby addressing the missing data issue effectively and efficiently. Theoretical and empirical studies indicate that LOSSA can significantly improve the quality of OSFS when missing data are encountered in target instances.

A Latent Factor Analysis-Based Approach to Online Sparse Streaming Feature Selection

Crash data augmentation using variational autoencoder.

Online streaming feature selection (OSFS) has attracted extensive attention during the past decades. Current approaches commonly assume that the feature space of fixed data instances dynamically increases without any missing data. However, this assumption does not always hold in many real applications. Motivated by this observation, this study aims to implement online feature selection from sparse streaming features, i.e., features flow in one by one with missing data as instance count remains fixed. To do so, this study proposes a latent-factor-analysis-based online sparse-streaming-feature selection algorithm (LOSSA). Its main idea is to apply latent factor analysis to pre-estimate missing data in sparse streaming features before conducting feature selection, thereby addressing the missing data issue effectively and efficiently. Theoretical and empirical studies indicate that LOSSA can significantly improve the quality of OSFS when missing data are encountered in target instances. 

Internet of Things (IoT) is proliferating in our real world, and it is a promising technology that serves a very comfortable service to the users. IoT’s underlying technology is to connect to central Cloud Computing (CC), which is a huge data center collecting the generated data by IoT devices, and it is located in different areas on demand. However, cloud computing lacks data transmission because of the infrastructure and limitations of networks which enormously decrease its performance. Therefore, a new paradigm has been founded to act as a middleware between the Cloud and IoT, termed Fog Computing (FC) Technology. Considering Fog as a cloud extension that provides computing service at the edge of the network, Fog Computing placement enables this technology to deal with numerous data locally. In this study, we surveyed Fog computing with in-depth analysis and covered the latest studies to address and overcome the existing challenges in FC. We reviewed fog computing technology conceptually and defined it based on the existing studies in the literature, together with its architecture, applications, advantages, and open issues with optimization methods being performed to obtain the optimal services.

/pdf/fog-computing-advancement-concept-architecture-applications-4h8gzf3o1a.pdf

Fog Computing Advancement: Concept, Architecture, Applications, Advantages, and Open Issues

We study the problem of online learning with varying feature spaces. The problem is challenging because, unlike traditional online learning problems, varying feature spaces can introduce new features or stop having some features without following a pattern. Other existing methods such as online streaming feature selection (Wu et al. 2013), online learning from trapezoidal data streams (Zhang et al. 2016), and learning with feature evolvable streams (Hou, Zhang, and Zhou 2017) are not capable to learn from arbitrarily varying feature spaces because they make assumptions about the feature space dynamics. In this paper, we propose a novel online learning algorithm OLVF to learn from data with arbitrarily varying feature spaces. The OLVF algorithm learns to classify the feature spaces and the instances from feature spaces simultaneously. To classify an instance, the algorithm dynamically projects the instance classifier and the training instance onto their shared feature subspace. The feature space classifier predicts the projection confidences for a given feature space. The instance classifier will be updated by following the empirical risk minimization principle and the strength of the constraints will be scaled by the projection confidences. Afterwards, a feature sparsity method is applied to reduce the model complexity. Experiments on 10 datasets with varying feature spaces have been conducted to demonstrate the performance of the proposed OLVF algorithm. Moreover, experiments with trapezoidal data streams on the same datasets have been conducted to show that OLVF performs better than the state-of-the-art learning algorithm (Zhang et al. 2016).

/pdf/online-learning-from-data-streams-with-varying-feature-gch2575p2j.pdf

Online Learning from Data Streams with Varying Feature Spaces

Learning with streaming data has received extensive attention during the past few years. Existing approaches assume the feature space is fixed or changes by following explicit regularities, limiting their applicability in dynamic environments where the data streams are described by an arbitrarily varying feature space. To handle such capricious data streams, we in this paper develop a novel algorithm, named OCDS (Online learning from Capricious Data Streams), which does not make any assumption on feature space dynamics. OCDS trains a learner on a universal feature space that establishes relationships between old and new features, so that the patterns learned in the old feature space can be used in the new feature space. Specifically, the universal feature space is constructed by leveraging the relatednesses among features. We propose a generative graphical model to model the construction process, and show that learning from the universal feature space can effectively improve performance with theoretical analysis. The experimental results demonstrate that OCDS achieves conspicuous performance on synthetic and real datasets.

/pdf/online-learning-from-capricious-data-streams-a-generative-2ev0vozxel.pdf

Online Learning from Capricious Data Streams: A Generative Approach.

Learning with streaming data has received extensive attention during the past few years. Existing approaches assume that the feature space is fixed or changes by following explicit regularities, limiting their applicability in real-time applications. For example, in a smart healthcare platform, the feature space of the patient data varies when different medical service providers use nonidentical feature sets to describe the patients’ symptoms. To fill the gap, we in this article propose a novel learning paradigm, namely, Generative Learning With Streaming Capricious (GLSC) data, which does not make any assumption on the feature space dynamics. In other words, GLSC handles the data streams with a varying feature space, where each arriving data instance can arbitrarily carry new features and/or stop carrying partial old features. Specifically, GLSC trains a learner on a universal feature space that establishes relationships between old and new features, so that the patterns learned in the old feature space can be used in the new feature space. The universal feature space is constructed by leveraging the relatednesses among features. We propose a generative graphical model to model the construction process, and show that learning from the universal feature space can effectively improve the performance with theoretical guarantees. The experimental results demonstrate that GLSC achieves conspicuous performance on both synthetic and real data sets.

Toward Mining Capricious Data Streams: A Generative Approach

Video transcoding is the process of converting a video to the format supported by the viewer's device. Video transcoding requires huge storage and computational resources, thus, many video stream providers choose to carry it out on the cloud. Video streaming providers generally need to prepare several formats of the same video (termed pre-transcoding) and stream the appropriate format to the viewer. However, pre-transcoding requires enormous storage space and imposes a significant cost to the stream provider. More importantly, pre-transcoding proven to be inefficient due to the long-tail access pattern to video streams in a repository. To reduce the incurred cost, in this research, we propose a method to partially pre-transcode video streams and re-transcode the rest of it in an on-demand manner. We will develop a method to strike a trade-off between pre-transcoding and on-demand transcoding of video streams to reduce the overall cost. Experimental results show the efficiency of our approach, particularly, when a high percentage of videos are accessed frequently. In such repositories, the proposed approach reduces the incurred cost by up to 70\%.

Cost Efficient Repository Management for Cloud-Based On-Demand Video Streaming

Video transcoding is the process of converting a video to the format supported by the viewer's device. Video transcoding requires a huge storage and computational resources, thus, many video stream providers choose to carry it out on the cloud. Video streaming providers generally need to prepare several formats of the same video (termed pre-transcoding) and stream the appropriate format to the viewer. However, pre-transcoding requires enormous storage space and imposes a significant cost to the stream provider. More importantly, pre-transcoding proven to be inefficient due to long-tail access pattern to video streams in a repository. To reduce the incurred cost, in this research, we propose a method to partially pre-transcode video streams and re-transcode the rest of it in an on-demand manner. We will develop a method to strike a trade-off between pre-transcoding and on-demand transcoding of video streams to reduce the overall cost. Experimental results show the efficiency of our approach, particularly, when a high percentage of videos are accessed frequently. In such repositories, the proposed approach reduces the incurred cost by up to 70%.

Cost Efficient Repository Management for Cloud-Based On-demand Video Streaming

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Ege Beyazit

Author Tools

Chat about Author