Few-shot learning (FSL) has emerged as an effective learning method and shows great potential. Despite the recent creative works in tackling FSL tasks, learning valid information rapidly from just a few or even zero samples remains a serious challenge. In this context, we extensively investigated 200+ FSL papers published in top journals and conferences in the past three years, aiming to present a timely and comprehensive overview of the most recent advances in FSL with a fresh perspective, and to provide an impartial comparison of the strengths and weaknesses of existing work. To avoid conceptual confusion, we first elaborate and contrast a set of relevant concepts including few-shot learning, transfer learning, and meta-learning. Then, we inventively extract prior knowledge related to few-shot learning in the form of a pyramid, which summarizes and classifies previous work in detail from the perspective of challenges. Furthermore, to enrich this survey, we present in-depth analysis and insightful discussions of recent advances in each subsection. What’s more, taking computer vision as an example, we highlight the important application of FSL, covering various research hotspots. Finally, we conclude the survey with unique insights into technology trends and potential future research opportunities to guide FSL follow-up research.

pdf/a-comprehensive-survey-of-few-shot-learning-evolution-2nzpufgr.pdf

A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities

Meta-learning offers an effective solution to learn new concepts under scarce supervision through an episodic-training scheme: a series of target-like tasks sampled from base classes are sequentially fed into a meta-learner to extract cross-task knowledge, which can facilitate the quick acquisition of task-specific knowledge of the target task with few samples. Despite its noticeable improvements, the episodic-training strategy samples tasks randomly and uniformly, without considering their hardness and quality, which may not progressively improve the meta-leaner’s generalization. In this paper, we propose Progressive Meta-learning using tasks from easy to hard. First, based on a predefined curriculum, we develop a Curriculum-Based Meta-learning (CubMeta) method. CubMeta is in a stepwise manner, and in each step, we design a BrotherNet module to establish harder tasks and an effective learning scheme for obtaining an ensemble of stronger meta-learners. Then we move a step further to propose an end-to-end Self-Paced Meta-learning (SepMeta) method. The curriculum in SepMeta is effectively integrated as a regularization term into the objective so that the meta-learner can measure the hardness of tasks adaptively, according to what the model has already learned. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed methods. Our code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/nobody-777</uri> . 

Progressive Meta-Learning With Curriculum

Meta-learning offers an effective solution to learn new concepts under scarce supervision through an episodic-training scheme: a series of target-like tasks sampled from base classes are sequentially fed into a meta-learner to extract cross-task knowledge, which can facilitate the quick acquisition of task-specific knowledge of the target task with few samples. Despite its noticeable improvements, the episodic-training strategy samples tasks randomly and uniformly, without considering their hardness and quality, which may not progressively improve the meta-leaner’s generalization. In this paper, we propose Progressive Meta-learning using tasks from easy to hard. First, based on a predefined curriculum, we develop a Curriculum-Based Meta-learning (CubMeta) method. CubMeta is in a stepwise manner, and in each step, we design a BrotherNet module to establish harder tasks and an effective learning scheme for obtaining an ensemble of stronger meta-learners. Then we move a step further to propose an end-to-end Self-Paced Meta-learning (SepMeta) method. The curriculum in SepMeta is effectively integrated as a regularization term into the objective so that the meta-learner can measure the hardness of tasks adaptively, according to what the model has already learned. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed methods. Our code is available at https://github.com/nobody-777.

Condition monitoring plays a vital role in ensuring the reliability and optimal performance of various engineering systems. Traditional methods for condition monitoring rely on physics-based models and statistical analysis techniques. However, these approaches often face challenges in dealing with complex systems and the limited availability of accurate physical models. In recent years, physics-informed machine learning (PIML) has emerged as a promising approach for condition monitoring, combining the strengths of physics-based modelling and data-driven machine learning. This study presents a comprehensive overview of PIML techniques in the context of condition monitoring. The central concept driving PIML is the incorporation of known physical laws and constraints into machine learning algorithms, enabling them to learn from available data while remaining consistent with physical principles. Through fusing domain knowledge with data-driven learning, PIML methods offer enhanced accuracy and interpretability in comparison to purely data-driven approaches. In this comprehensive survey, detailed examinations are performed with regard to the methodology by which known physical principles are integrated within machine learning frameworks, as well as their suitability for specific tasks within condition monitoring. Incorporation of physical knowledge into the ML model may be realized in a variety of methods, with each having its unique advantages and drawbacks. The distinct advantages and limitations of each methodology for the integration of physics within data-driven models are detailed, considering factors such as computational efficiency, model interpretability, and generalizability to different systems in condition monitoring and fault detection. Several case studies and works of literature utilizing this emerging concept are presented to demonstrate the efficacy of PIML in condition monitoring applications. From the literature reviewed, the versatility and potential of PIML in condition monitoring may be demonstrated. Novel PIML methods offer an innovative solution for addressing the complexities of condition monitoring and associated challenges. This comprehensive survey helps form the foundation for future work in the field. As the technology continues to advance, PIML is expected to play a crucial role in enhancing maintenance strategies, system reliability, and overall operational efficiency in engineering systems. 

Physics-informed machine learning: A comprehensive review on applications in anomaly detection and condition monitoring

Neural architecture search (NAS) has seen significant attention throughout the computational intelligence research community and has pushed forward the state-of-the-art of many neural models to address grid-like data such as texts and images. However, little work has been done on Graph Neural Network (GNN) models dedicated to unstructured network data. Given the huge number of choices and combinations of components such as aggregators and activation functions, determining the suitable GNN model for a specific problem normally necessitates tremendous expert knowledge and laborious trials. In addition, the moderate change of hyperparameters such as the learning rate and dropout rate would dramatically impact the learning capacity of a GNN model. In this paper, we propose a novel framework through the evolution of individual models in a large GNN architecture searching space. Instead of simply optimizing the model structures, an alternating evolution process is performed between GNN model structures and hyperparameters to dynamically approach the optimal fit of each other. Experiments and validations demonstrate that evolutionary NAS is capable of matching existing state-of-the-art reinforcement learning methods for both transductive and inductive graph representation learning and node classification. 

http://manuscript.elsevier.com/S0950705122003525/pdf/S0950705122003525.pdf

Genetic-GNN: Evolutionary architecture search for Graph Neural Networks

Power flow calculation in quasi-steady states is the basis of cascading failure analysis. However, in recent years, data-driven analysis methods that are based on sufficient data have put forward higher requirements on the speed of power flow calculation. To build a more accurate and efficient neural network for power flow calculation, a physics-informed graph neural network-based model is proposed for faster calculation. Via minimizing the physics-informed loss function and using a pre-training/fine-tuning method, the proposed model is trained to follow the physical equations directly and can generalize to dynamic power networks. Physics-informed <inline-formula><tex-math notation="LaTeX">$LOSS$</tex-math></inline-formula> makes the proposed model more interpretable, since the calculation error can be evaluated by <inline-formula><tex-math notation="LaTeX">$LOSS$</tex-math></inline-formula>. Then cascading failures are simulated with the proposed model, and a pre-set factor <inline-formula><tex-math notation="LaTeX">$\zeta$</tex-math></inline-formula> is introduced to balance the speed and accuracy of simulations. Finally, the accuracy of cascading failure simulations with the proposed model is verified in the IEEE 39-bus system, the 118-bus system, the 300-bus system, and a real-world French system. Experimental results show that compared with AC power flow, the proposed physics-informed graph neural network-based power flow model can reduce the simulation time significantly while maintaining high accuracy if <inline-formula><tex-math notation="LaTeX">$\zeta$</tex-math></inline-formula> is properly pre-set.

Cascading Failure Analysis Based on a Physics-Informed Graph Neural Network

Abstract Temporal network link prediction is an important task in the field of network science, and has a wide range of applications in practical scenarios. Revealing the evolutionary mechanism of the network is essential for link prediction, and how to effectively utilize the historical information for temporal links and efficiently extract the high-order patterns of network structure remains a vital challenge. To address these issues, in this paper, we propose a novel temporal link prediction model with adjusted sigmoid function and 2-simplex structure (TLPSS). The adjusted sigmoid decay mode takes the active, decay and stable states of edges into account, which properly fits the life cycle of information. Moreover, the latent matrix sequence is introduced, which is composed of simplex high-order structure, to enhance the performance of link prediction method since it is highly feasible in sparse network. Combining the life cycle of information and simplex high-order structure, the overall performance of TLPSS is achieved by satisfying the consistency of temporal and structural information in dynamic networks. Experimental results on six real-world datasets demonstrate the effectiveness of TLPSS, and our proposed model improves the performance of link prediction by an average of 15% compared to other baseline methods. 

/pdf/temporal-link-prediction-via-adjusted-sigmoid-function-and-2-1y2g8xap.pdf

Temporal link prediction via adjusted sigmoid function and 2-simplex structure

—Graph embedding provides a feasible methodology to conduct pattern classiﬁcation for graph-structured data by mapping each data into the vectorial space. Various pioneering works are essentially coding method that concentrates on a vectorial representation about the inner properties of a graph in terms of the topological constitution, node attributions, link relations, etc. However, the classiﬁcation for each targeted data is a qualitative issue based on understanding the overall discrepancies within the dataset scale. From the statistical point of view, these discrepancies manifest a metric distribution over the dataset scale if the distance metric is adopted to measure the pairwise similarity or dissimilarity. Therefore, we present a novel embedding strategy named MetricDistribution2vec to extract such distribution characteristics into the vectorial representation for each data. We demonstrate the application and effectiveness of our representation method in the supervised prediction tasks on extensive real-world structural graph datasets. The results have gained some unexpected increases compared with a surge of baselines on all the datasets, even if we take the lightweight models as classiﬁers. Moreover, the proposed methods also conducted experiments in Few-Shot classiﬁcation scenarios, and the results still show attractive discrimination in rare training samples based inference.

pdf/metric-distribution-to-vector-constructing-data-snugz2gn.pdf

Metric Distribution to Vector: Constructing Data Representation via Broad-Scale Discrepancies

Despite recent success on various tasks, deep learning techniques still perform poorly on adversarial examples with small perturbations. While optimization-based methods for adversarial attacks are well-explored in the ﬁeld of computer vision, it is impractical to directly ap-ply them in natural language processing due to the discrete nature of the text. To address the problem, we propose a uniﬁed framework to extend the existing optimization-based adversarial attack methods in the vision domain to craft textual adversarial samples. In this framework, continuously optimized perturbations are added to the embedding layer and ampliﬁed in the forward propagation process. Then the ﬁnal perturbed latent representations are decoded with a masked language model head to obtain potential adversarial samples. In this paper, we instantiate our framework with an attack algorithm named T extual P rojected G radient D escent ( T-PGD ). We ﬁnd our algorithm effective even using proxy gradient information. Therefore, we perform the more challenging transfer black-box attack and conduct comprehensive experiments to evaluate our attack algorithm with several models on three benchmark datasets. Experimental results demonstrate that our method achieves an overall better performance and produces more ﬂuent and grammatical adversarial samples compared to strong baseline methods. All the code and data will be made public.

Bridge the Gap Between CV and NLP! An Optimization-based Textual Adversarial Attack Framework

Despite the proliferation of generative models, achieving fast sampling during inference without compromising sample diversity and quality remains challenging. Existing models such as Denoising Diffusion Probabilistic Models (DDPM) deliver high-quality, diverse samples but are slowed by an inherently high number of iterative steps. The Denoising Diffusion Generative Adversarial Networks (DDGAN) attempted to circumvent this limitation by integrating a GAN model for larger jumps in the diffusion process. However, DDGAN encountered scalability limitations when applied to large datasets. To address these limitations, we introduce a novel approach that tackles the problem by matching implicit and explicit factors. More specifically, our approach involves utilizing an implicit model to match the marginal distributions of noisy data and the explicit conditional distribution of the forward diffusion. This combination allows us to effectively match the joint denoising distributions. Unlike DDPM but similar to DDGAN, we do not enforce a parametric distribution for the reverse step, enabling us to take large steps during inference. Similar to the DDPM but unlike DDGAN, we take advantage of the exact form of the diffusion process. We demonstrate that our proposed method obtains comparable generative performance to diffusion-based models and vastly superior results to models with a small number of sampling steps.

pdf/semi-implicit-denoising-diffusion-models-siddms-k5o2tw29.pdf

Semi-Implicit Denoising Diffusion Models (SIDDMs)

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Wei Wei

Author Tools

Chat about Author