Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
Laura von Rueden,Sebastian Mayer,Katharina Beckh,Bogdan Georgiev,Sven Giesselbach,Raoul Heese,Birgit Kirsch,Julius Pfrommer,Annika Pick,Rajkumar Ramamurthy,Michal Walczak,Jochen Garcke,Christian Bauckhage,Jannis Schuecker +13 more
TL;DR: A definition and proposed concept for informed machine learning is provided, which illustrates its building blocks and distinguishes it from conventional machine learning, and a taxonomy is introduced that serves as a classification framework forinformed machine learning approaches.
read more
Abstract: Despite its great success, machine learning can have its limits when dealing with insufficient training data. A potential solution is the additional integration of prior knowledge into the training process which leads to the notion of informed machine learning. In this paper, we present a structured overview of various approaches in this field. We provide a definition and propose a concept for informed machine learning which illustrates its building blocks and distinguishes it from conventional machine learning. We introduce a taxonomy that serves as a classification framework for informed machine learning approaches. It considers the source of knowledge, its representation, and its integration into the machine learning pipeline. Based on this taxonomy, we survey related research and describe how different knowledge representations such as algebraic equations, logic rules, or simulation results can be used in learning systems. This evaluation of numerous papers on the basis of our taxonomy uncovers key methods in the field of informed machine learning.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Figures
Table 1: Illustrative Overview of Knowledge Representations in the Informed Machine Learning Taxonomy. Each representation type is illustrated by a simple or prominent example in order to give a first intuitive understanding. 
Figure 1: Information Flow in Informed Machine Learning. The informed machine learning pipeline requires a hybrid information source with two components: Data and prior knowledge. In conventional machine learning knowledge is used for data preprocessing and feature engineering, but this process is deeply intertwined with the learning pipeline (*). In contrast, in informed machine learning prior knowledge comes from an independent source, is given by formal representations (e.g., by knowledge graphs, simulation results, or logic rules), and is explicitly integrated. 
Table 3: References Classified by Knowledge Representation and (Path to) Knowledge Integration. 
Table 2: References Classified by Knowledge Representation and (Path from) Knowledge Source. 
Figure 2: Taxonomy of Informed Machine Learning. This taxonomy serves as a classification framework for informed machine learning and structures approaches according to the three above analysis questions about the knowledge source, knowledge representation and knowledge integration. Based on a comparative and iterative literature survey, we identified for each dimension a set of elements that represent a spectrum of different approaches. The size of the elements reflects the relative count of papers. We combine the taxonomy with a Sankey diagram in which the paths connect the elements across the three dimensions and illustrate the approaches that we found in the analyzed papers. The broader the path, the more papers we found for that approach. Main paths (at least four or more papers with the same approach across all dimensions) are highlighted in darker grey and represent central approaches of informed machine learning. ![Figure 6: Steps of Rules-to-Network Translation [53]. Simple example for integrating rules into a KBANN.](/figures/figure6-1-1njbuauauz50.png)
Figure 6: Steps of Rules-to-Network Translation [53]. Simple example for integrating rules into a KBANN.
![Figure 6: Steps of Rules-to-Network Translation [53]. Simple example for integrating rules into a KBANN.](/figures/figure6-1-1njbuauauz50.webp)
Citations
Explainable Machine Learning for Scientific Insights and Discoveries
TL;DR: In this paper, the authors provide a survey of recent scientific works that incorporate machine learning and the way that explainable machine learning is used in combination with domain knowledge from the application areas.
•Book
Computational Science and Engineering
Gilbert Strang
- 01 Jan 2007
TL;DR: Computational Science and Engineering (CSE) is the multi-disciplinary field of computer-based modelling and simulation for studying scientific phenomena and engineering designs.
560
•Posted Content
Integrating Physics-Based Modeling with Machine Learning: A Survey
Jared Willard,Xiaowei Jia,Shaoming Xu,Michael Steinbach,Vipin Kumar +4 more
- 01 Jan 2020
TL;DR: An overview of techniques to integrate machine learning with physics-based modeling and classes of methodologies used to construct physics-guided machine learning models and hybrid physics-machine learning frameworks from a machine learning standpoint is provided.
422
Prognostics and Health Management (PHM): Where are we and where do we (need to) go in theory and practice
01 Feb 2022
TL;DR: In this article , the authors point out main challenges and directions of advancements, for full deployment of condition-based and predictive maintenance in practice, for the purpose of improving safe and reliable design, operation and maintenance of complex socio-cyber-physical systems.
338
Prognostics and Health Management (PHM): Where are we and where do we (need to) go in theory and practice
Enrico Zio,Enrico Zio +1 more
TL;DR: This paper aims at pointing out main challenges and directions of advancements, for full deployment of condition-based and predictive maintenance in practice, for Prognostics and Health Management and its benefits in practice.
292
References
•Proceedings Article
Interaction networks for learning about objects, relations and physics
Peter W. Battaglia,Razvan Pascanu,Matthew Lai,Danilo Jimenez Rezende,Koray Kavukcuoglu +4 more
- 05 Dec 2016
TL;DR: The interaction network is introduced, a model which can reason about how objects in complex systems interact, supporting dynamical predictions, as well as inferences about the abstract properties of the system, and is implemented using deep neural networks.
1.2K
Very deep convolutional networks for text classification
Alexis Conneau,Holger Schwenk,Loïc Barrault,Yann LeCun +3 more
- 03 Apr 2017
TL;DR: Very deep convolutional networks (VDCNN) as mentioned in this paper have been applied to text classification. And they have achieved state-of-the-art performance on several public text classification tasks.
Robots that can adapt like animals
Antoine Cully,Jeff Clune,Danesh Tarapore,Danesh Tarapore,Danesh Tarapore,Jean-Baptiste Mouret +5 more
TL;DR: An intelligent trial-and-error algorithm is introduced that allows robots to adapt to damage in less than two minutes in large search spaces without requiring self-diagnosis or pre-specified contingency plans, and may shed light on the principles that animals use to adaptation to injury.
•Book
First-Order Logic and Automated Theorem Proving
Melvin Fitting,David Gries +1 more
- 31 Jul 2012
TL;DR: This monograph on classical logic presents fundamental concepts and results in a rigorous mathematical style and is intended for those interested in computer science and mathematics at the beginning graduate level.
1.2K
•Posted Content
Learning from Simulated and Unsupervised Images through Adversarial Training
TL;DR: This work develops a method for S+U learning that uses an adversarial network similar to Generative Adversarial Networks (GANs), but with synthetic images as inputs instead of random vectors, and makes several key modifications to the standard GAN algorithm to preserve annotations, avoid artifacts, and stabilize training.
1K