Convolutional neural network (CNN), with ability of feature learning and nonlinear mapping, has demonstrated its effectiveness in prognostics and health management (PHM). However, explanation on the physical meaning of a CNN architecture has rarely been studied. In this paper, a novel wavelet driven deep neural network termed as WaveletKernelNet (WKN) is presented, where a continuous wavelet convolutional (CWConv) layer is designed to replace the first convolutional layer of the standard CNN. This enables the first CWConv layer to discover more meaningful filters. Furthermore, only the scale parameter and translation parameter are directly learned from raw data at this CWConv layer. This provides a very effective way to obtain a customized filter bank, specifically tuned for extracting defect-related impact component embedded in the vibration signal. In addition, three experimental verification using data from laboratory environment are carried out to verify effectiveness of the proposed method for mechanical fault diagnosis. The results show the importance of the designed CWConv layer and the output of CWConv layer is interpretable. Besides, it is found that WKN has fewer parameters, higher fault classification accuracy and faster convergence speed than standard CNN. 

/pdf/waveletkernelnet-an-interpretable-deep-neural-network-for-1v19nitk.pdf

WaveletKernelNet: An Interpretable Deep Neural Network for Industrial Intelligent Diagnosis

Due to the proliferation of large-scale remote-sensing (RS) archives with multiple annotations, multilabel RS scene classification and retrieval are becoming increasingly popular. Although some recent deep learning-based methods are able to achieve promising results in this context, the lack of research on how to learn embedding spaces under the multilabel assumption often makes these models unable to preserve complex semantic relations pervading aerial scenes, which is an important limitation in RS applications. To fill this gap, we propose a new graph relation network (GRN) for multilabel RS scene categorization. Our GRN is able to model the relations between samples (or scenes) by making use of a graph structure which is fed into network learning. For this purpose, we define a new loss function called scalable neighbor discriminative loss with binary cross entropy (SNDL-BCE) that is able to embed the graph structures through the networks more effectively. The proposed approach can guide deep learning techniques (such as convolutional neural networks) to a more discriminative metric space, where semantically similar RS scenes are closely embedded and dissimilar images are separated from a novel multilabel viewpoint. To achieve this goal, our GRN jointly maximizes a weighted leave-one-out $K$ -nearest neighbors ( $K$ NN) score in the training set, where the weight matrix describes the contributions of the nearest neighbors associated with each RS image on its class decision, and the likelihood of the class discrimination in the multilabel scenario. An extensive experimental comparison, conducted on three multilabel RS scene data archives, validates the effectiveness of the proposed GRN in terms of $K$ NN classification and image retrieval. The codes of this article will be made publicly available for reproducible research in the community.

/pdf/graph-relation-network-modeling-relations-between-scenes-for-2efsjrua3t.pdf

Graph Relation Network: Modeling Relations Between Scenes for Multilabel Remote-Sensing Image Classification and Retrieval

Domain adaptation addresses the learning problem where the training data are sampled from a source joint distribution (source domain), while the test data are sampled from a different target joint distribution (target domain). Because of this joint distribution mismatch, a discriminative classifier naively trained on the source domain often generalizes poorly to the target domain. In this article, we therefore present a Joint Distribution Invariant Projections (JDIP) approach to solve this problem. The proposed approach exploits linear projections to directly match the source and target joint distributions under the $L^{2}$ -distance. Since the traditional kernel density estimators for distribution estimation tend to be less reliable as the dimensionality increases, we propose a least square method to estimate the $L^{2}$ -distance without the need to estimate the two joint distributions, leading to a quadratic problem with analytic solution. Furthermore, we introduce a kernel version of JDIP to account for inherent nonlinearity in the data. We show that the proposed learning problems can be naturally cast as optimization problems defined on the product of Riemannian manifolds. To be comprehensive, we also establish an error bound, theoretically explaining how our method works and contributes to reducing the target domain generalization error. Extensive empirical evidence demonstrates the benefits of our approach over state-of-the-art domain adaptation methods on several visual data sets.

Domain Adaptation by Joint Distribution Invariant Projections

Learning to detect anomaly events in crowd scenes from synthetic data

Adder Neural Networks (ANNs) which only contain additions bring us a new way of developing deep neural networks with low energy consumption. Unfortunately, there is an accuracy drop when replacing all convolution filters by adder filters. The main reason here is the optimization difficulty of ANNs using $\ell_1$-norm, in which the estimation of gradient in back propagation is inaccurate. In this paper, we present a novel method for further improving the performance of ANNs without increasing the trainable parameters via a progressive kernel based knowledge distillation (PKKD) method. A convolutional neural network (CNN) with the same architecture is simultaneously initialized and trained as a teacher network, features and weights of ANN and CNN will be transformed to a new space to eliminate the accuracy drop. The similarity is conducted in a higher-dimensional space to disentangle the difference of their distributions using a kernel based method. Finally, the desired ANN is learned based on the information from both the ground-truth and teacher, progressively. The effectiveness of the proposed method for learning ANN with higher performance is then well-verified on several benchmarks. For instance, the ANN-50 trained using the proposed PKKD method obtains a 76.8\% top-1 accuracy on ImageNet dataset, which is 0.6\% higher than that of the ResNet-50.

/pdf/kernel-based-progressive-distillation-for-adder-neural-mgzv8851mu.pdf

Kernel Based Progressive Distillation for Adder Neural Networks

Precision medicine is devoted to discovering personalized therapy for complex and difficult diseases like cancer. Many machine learning approaches have been developed for drug response prediction towards precision medicine. Notwithstanding, genetic profiles based multi-view graph learning schemes have not yet been explored for drug response prediction in previous works. Furthermore, multi-scale latent feature fusion is not considered sufficiently in the existing frameworks of graph neural networks (GNNs). Previous works on drug response prediction mainly depend on sequence data or single-view graph data. In this paper, we propose to construct multi-view graph by means of multi-omics data and STRING protein-protein association data, and develop a new architecture of GNNs for drug response prediction in cancer. Specifically, we propose hybrid multi-view and multi-scale graph duplex-attention networks (HMM-GDAN), in which both multi-view self-attention mechanism and view-level attention mechanism are devised to capture the complementary information of views and emphasize on the importance of each view collaboratively, and rich multi-scale features are constructed and integrated to further form high-level representations for better prediction. Experiments on GDSC2 dataset verify the superiority of the proposed HMM-GDAN when compared with state-of-the-art baselines. The effectiveness of multi-view and multi-scale strategies is demonstrated by the ablation study. 

HMM-GDAN: Hybrid multi-view and multi-scale graph duplex-attention networks for drug response prediction in cancer.

Robust multiple subspaces transfer for heterogeneous domain adaptation

Subspace clustering aims to segment data sampled from a union of subspaces in visual data tasks. Structured Sparse Subspace Clustering (SSSC) model is a unified optimization framework, which proves successful in learning both the self representation of the data and their subspace segmentation. However, SSSC involves solving non-convex subproblems and hence it may be stuck into bad local minima such that clustering performance degrades. In this paper, we propose a self-paced subspace clustering algorithm to tackle this problem, which learns subspace segmentation of data by progressing from 'easy' to 'complex' examples under a novel self-paced regularizer. Experiments on the real-world human face datasets verify the effectiveness of the proposed algorithm.

Self-Paced Subspace Clustering

Graph neural networks (GNNs) could directly deal with the data of graph structure. Current GNNs are confined to the spatial domain and learn real low-dimensional embeddings in graph classification tasks. In this article, we explore frequency domain-oriented complex GNNs in which the node's embedding in each layer is a complex vector. The difficulty lies in the design of graph pooling and we propose a mirror-connected design with two crucial problems: parameter reduction problem and complex gradient backpropagation problem. To deal with the former problem, we propose the notion of squared singular value pooling (SSVP) and prove that the representation power of SSVP followed by a fully connected layer with nonnegative weights is exactly equivalent to that of a mirror-connected layer. To resolve the latter problem, we provide an alternative feasible method to solve singular values of complex embeddings with a theoretical guarantee. Finally, we propose a mixture of pooling strategies in which first-order statistics information is employed to enrich the last low-dimensional representation. Experiments on benchmarks demonstrate the effectiveness of the complex GNNs with mirror-connected layers.

Frequency Domain-Oriented Complex Graph Neural Networks for Graph Classification.

Convex–Concave Tensor Robust Principal Component Analysis

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Youfa Liu

Author Tools

Chat about Author