Deep Learning (DL) has recently achieved tremendous success. A variety of DL frameworks and platforms play a key role to catalyze such progress. However, the differences in architecture designs and implementations of existing frameworks and platforms bring new challenges for DL software development and deployment. Till now, there is no study on how various mainstream frameworks and platforms influence both DL software development and deployment in practice. To fill this gap, we take the first step towards understanding how the most widely-used DL frameworks and platforms support the DL software development and deployment. We conduct a systematic study on these frameworks and platforms by using two types of DNN architectures and three popular datasets. (1) For development process, we investigate the prediction accuracy under the same runtime training configuration or same model weights/biases. We also study the adversarial robustness of trained models by leveraging the existing adversarial attack techniques. The experimental results show that the computing differences across frameworks could result in an obvious prediction accuracy decline, which should draw the attention of DL developers. (2) For deployment process, we investigate the prediction accuracy and performance (refers to time cost and memory consumption) when the trained models are migrated/quantized from PC to real mobile devices and web browsers. The DL platform study unveils that the migration and quantization still suffer from compatibility and reliability issues. Meanwhile, we find several DL software bugs by using the results as a benchmark. We further validate the results through bug confirmation from stakeholders and industrial positive feedback to highlight the implications of our study. Through our study, we summarize practical guidelines, identify challenges and pinpoint new research directions, such as understanding the characteristics of DL frameworks and platforms, avoiding compatibility and reliability issues, detecting DL software bugs, and reducing time cost and memory consumption towards developing and deploying high quality DL systems effectively.

An empirical study towards characterizing deep learning development and deployment across different frameworks and platforms

Deep learning (DL) is moving its step into a growing number of mobile software applications. These software applications, named as DL based mobile applications (abbreviated as mobile DL apps) integrate DL models trained using large-scale data with DL programs. A DL program encodes the structure of a desirable DL model and the process by which the model is trained using training data. Due to the increasing dependency of current mobile apps on DL, software engineering (SE) for mobile DL apps has become important. However, existing efforts in SE research community mainly focus on the development of DL models and extensively analyze faults in DL programs. In contrast, faults related to the deployment of DL models on mobile devices (named as deployment faults of mobile DL apps) have not been well studied. Since mobile DL apps have been used by billions of end users daily for various purposes including for safety-critical scenarios, characterizing their deployment faults is of enormous importance. To fill in the knowledge gap, this paper presents the first comprehensive study to date on the deployment faults of mobile DL apps. We identify 304 real deployment faults from Stack Overflow and GitHub, two commonly used data sources for studying software faults. Based on the identified faults, we construct a fine-granularity taxonomy consisting of 23 categories regarding to fault symptoms and distill common fix strategies for different fault symptoms. Furthermore, we suggest actionable implications and research avenues that can potentially facilitate the deployment of DL models on mobile devices.

pdf/an-empirical-study-on-deployment-faults-of-deep-learning-1huchewgn5.pdf

An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications

Deep Learning (DL) has recently achieved tremendous success. A variety of DL frameworks and platforms play a key role to catalyze such progress. However, the differences in architecture designs and implementations of existing frameworks and platforms bring new challenges for DL software development and deployment. Till now, there is no study on how various mainstream frameworks and platforms influence both DL software development and deployment in practice. To fill this gap, we take the first step towards understanding how the most widely-used DL frameworks and platforms support the DL software development and deployment. We conduct a systematic study on these frameworks and platforms by using two types of DNN architectures and three popular datasets. (1) For development process, we investigate the prediction accuracy under the same runtime training configuration or same model weights/biases. We also study the adversarial robustness of trained models by leveraging the existing adversarial attack techniques. The experimental results show that the computing differences across frameworks could result in an obvious prediction accuracy decline, which should draw the attention of DL developers. (2) For deployment process, we investigate the prediction accuracy and performance (refers to time cost and memory consumption) when the trained models are migrated/quantized from PC to real mobile devices and web browsers. The DL platform study unveils that the migration and quantization still suffer from compatibility and reliability issues. Meanwhile, we find several DL software bugs by using the results as a benchmark. We further validate the results through bug confirmation from stakeholders and industrial positive feedback to highlight the implications of our study. Through our study, we summarize practical guidelines, identify challenges and pinpoint new research directions.

An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms

Sequential recommendation techniques are considered to be a promising way of providing better user experience in mobile commerce by learning sequential interests within user historical interaction behaviors. However, the recently increasing focus on privacy concerns, such as the General Data Protection Regulation (GDPR), can significantly affect the deployment of state-of-the-art sequential recommendation techniques, because user behavior data are no longer allowed to be arbitrarily used without the user’s explicit permission. To address the issue, this paper proposes DeepRec, an on-device deep learning framework of mining interaction behaviors for sequential recommendation without sending any raw data or intermediate results out of the device, preserving user privacy maximally. DeepRec constructs a global model using data collected before GDPR and fine-tunes a personal model continuously on individual mobile devices using data collected after GDPR. DeepRec employs the model pruning and embedding sparsity techniques to reduce the computation and network overhead, making the model training process practical on computation-constraint mobile devices. Evaluation results show that DeepRec can achieve comparable recommendation accuracy to existing centralized recommendation approaches with small computation overhead and up to 10x reduction in network overhead.

DeepRec: On-device Deep Learning for Privacy-Preserving Sequential Recommendation in Mobile Commerce

CPU is a powerful, pervasive, and indispensable platform for running deep learning (DL) workloads in systems ranging from mobile to extreme-end servers. In this article, we present a survey of techniques for optimizing DL applications on CPUs. We include the methods proposed for both inference and training and those offered in the context of mobile, desktop/server, and distributed systems. We identify the areas of strength and weaknesses of CPUs in the field of DL. This article will interest practitioners and researchers in the area of artificial intelligence, computer architecture, mobile systems, and parallel computing. 

A Survey of Deep Learning on CPUs: Opportunities and Co-Optimizations

Recently, several JavaScript-based deep learning frameworks have emerged, making it possible to perform deep learning tasks directly in browsers. However, little is known on what and how well we can do with these frameworks for deep learning in browsers. To bridge the knowledge gap, in this paper, we conduct the first empirical study of deep learning in browsers. We survey 7 most popular JavaScript-based deep learning frameworks, investigating to what extent deep learning tasks have been supported in browsers so far. Then we measure the performance of different frameworks when running different deep learning tasks. Finally, we dig out the performance gap between deep learning in browsers and on native platforms by comparing the performance of TensorFlow.js and TensorFlow in Python. Our findings could help application developers, deep-learning framework vendors and browser vendors to improve the efficiency of deep learning in browsers.

Moving Deep Learning into Web Browser: How Far Can We Go?

The web browser is one of the major channels to access the Internet on mobile devices. Based on the smartphone usage logs from millions of real-world Android users, it is interesting to find that about 38% users have more than one browser on their devices. However, it is unclear whether the quality of browsing experiences are different when visiting the same webpage on different browsers. In this paper, we collect 3-week consecutive traces of 337 popular webpages on three popular mobile browsers: Chrome, Firefox, and Opera. We first use a list of metrics and conduct an empirical study to measure the differences of these metrics on different browsers. Then, we explore the variety of loading time and cache performance of different browsers when visiting the same webpage, which has a great impact on the browsing experience. Furthermore, we try to find which metrics have significant effect on the differences, investigating the possible causes. Finally, according to our findings, we give some recommendations to web developers, browser vendors, and end users.

Understanding Quality of Experiences on Different Mobile Browsers

Mobile extended reality (XR) has developed rapidly in recent years. Compared with the app-based XR, XR in web browsers has the advantages of being lightweight and cross-platform, providing users with a pervasive experience. Therefore, many frameworks are emerging to support the development of XR in web browsers. However, little has been known about how well these frameworks perform and how complex XR apps modern web browsers can support on mobile devices. To fill the knowledge gap, in this paper, we conduct an empirical study of mobile XR in web browsers. We select seven most popular web-based XR frameworks and investigate their runtime performance, including 3D rendering, camera capturing, and real-world understanding. We find that current frameworks have the potential to further enhance their performance by increasing GPU utilization or improving computing parallelism. Besides, for 3D scenes with good rendering performance, developers can feel free to add camera capturing with little influence on performance to support augmented reality (AR) and mixed reality (MR) applications. Based on our findings, we draw several practical implications to provide better XR support in web browsers.

Demystifying Mobile Extended Reality in Web Browsers: How Far Can We Go?

Modern mobile operating systems support displaying Web pages in native mobile applications. When an app user navigates to a specific location containing a Web page, the Web page will be loaded and rendered from within the app. Such kind of Web browsing, as we call embedded Web browsing, is different from traditional Web browsing, which involves typing a URL on a browser and loading the Web page. However, little has been known about the prevalence or performance of such embedded Web pages. In this paper, we conduct, to the best of our knowledge, the first measurement study on embedded Web browsing on Android. Our study on 22,521 popular Android apps shows that 57.9 and 73.8 percent of apps embed Web pages on two popular app markets, that is, Google Play and Wandoujia, respectively. We design and implement EWProfiler, a tool that can automatically search for embedded Web pages inside apps, trigger page loads, and retrieve performance metrics to analyze the embedded Web browsing performance at scale. Based on 445 embedded Web pages obtained by EWProfiler in 99 popular apps from the two app markets, we investigate the characteristics and performance of embedded Web pages, and find that embedded Web pages significantly impede the app user experience. We investigate the effectiveness of three techniques, i.e., separating the browser kernel to a different process, loading pages from the local storage, and pre-rendering, to optimize the performance of embedded Web browsing. We believe that our findings could draw the attention of Web developers, browser vendors, app developers, and mobile OS vendors together toward a better performance of embedded Web browsing.

Characterizing Embedded Web Browsing in Mobile Apps

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Deyu Tian

Author Tools

Chat about Author

Papers

Moving Deep Learning into Web Browser: How Far Can We Go?

Moving Deep Learning into Web Browser: How Far Can We Go?

Understanding Quality of Experiences on Different Mobile Browsers

Demystifying Mobile Extended Reality in Web Browsers: How Far Can We Go?

Characterizing Embedded Web Browsing in Mobile Apps