Shortening product development cycles and fully customisable products pose major challenges for production systems. These not only have to cope with an increased product diversity but also enable h...

https://www.tandfonline.com/doi/pdf/10.1080/00207543.2021.1973138

Deep reinforcement learning in production systems: a systematic literature review

To facilitate the personalized smart manufacturing paradigm with cognitive automation capabilities, Deep Reinforcement Learning (DRL) has attracted ever-increasing attention by offering an adaptive and flexible solution. DRL takes the advantages of both Deep Neural Networks (DNN) and Reinforcement Learning (RL), by embracing the power of representation learning, to make precise and fast decisions when facing dynamic and complex situations. Ever since the first paper of DRL was published in 2013, its applications have sprung up across the manufacturing field with exponential publication growth year by year. However, there still lacks any comprehensive review of the DRL in the field of smart manufacturing. To fill this gap, a systematic review process was conducted, with 261 relevant publications selected to date (20-Oct-2022), to gain a holistic understanding of the development, application, and challenges of DRL in smart manufacturing along the whole engineering lifecycle. First, the concept and development of DRL are summarized. Then, the typical DRL applications are analyzed in the four engineering lifecycle stages: design, manufacturing, distribution, and maintenance. Finally, the challenges and future directions are illustrated, especially emerging DRL-related technologies and solutions that can improve the manufacturing system’s deployment feasibility, cognitive capability, and learning efficiency, respectively. It is expected that this work can provide an insightful guide to the research of DRL in the smart manufacturing field and shed light on its future perspectives. 

Deep reinforcement learning in smart manufacturing: A review and prospects

This study aims to present an overall review of the recent research status regarding Machine Learning (ML) applications in machining processes. In the current industrial systems, processes require the capacity to adapt to manufacturing conditions continuously, guaranteeing high performance in terms of production quality and equipment availability. Artificial Intelligence (AI) offers new opportunities to develop and integrate innovative solutions in conventional machine tools to reduce undesirable effects during operational activities. In particular, the significant increase of the computational capacity may permit the application of complex algorithms to big data volumes in a short time, expanding the potentialities of ML techniques. ML applications are present in several contexts of machining processes, from roughness quality prediction to tool condition monitoring. This review focuses on recent applications and implications, classifying the main problems that may be solved using ML related to the machining quality, energy consumption and conditional monitoring. Finally, a discussion on the advantages and limits of ML algorithms is summarized for future investigations.

Recent Advances on Machine Learning Applications in Machining Processes

Service Oriented, Holonic and Multi-Agent Manufacturing Systems for Industry of the Future: Proceedings of SOHOMA 2022

Abstract The main driver of industry 4.0 is data and information for realizing resilient manufacturing systems through data-based decision-making. This requires the data flow from different machinery and its processing. Therefore, a methodical survey on the logical and technical requirements for enabling an efficient data flow can be beneficial since it fastens the process of establishing the data flow and gives production and software engineers a common understanding of the data’s meaning. Hence, data, related metadata like the unit or the frequency of recordings, and technical information, e.g., the machine’s interface or the communication protocol, are necessary. Based on the classical value stream mapping (VSM) as a lean management tool, the first approach for a methodical gathering of data, value stream mapping 4.0, was established in 2016. Since VSM 4.0 provides technical properties for specifying the data flow, some shortcomings were identified, particularly related to technical information. This contribution introduces the VSM 4.0 plus that solved the identified shortcomings by adding new properties. Compared to VSM 4.0, VSM 4.0 plus offers detailed specifications of data points, includes technical information, e.g., used communication protocols, adds information regarding data processing, and enhances the storage and usage dimension by providing more details. Through the added properties, VSM 4.0 plus allows domain and software engineers more efficient collaboration since all required information for establishing the data flow and its processing are united in one methodology. To verify the practicability, the VSM 4.0 plus was applied within several manufacturing-related use cases. 

/pdf/extension-of-value-stream-mapping-4-0-for-comprehensive-1aegyrh1.pdf

Extension of value stream mapping 4.0 for comprehensive identification of data and information flows within the manufacturing domain

In dynamic production environments, bringing flexibility to production planning and control (PPC) against continuously changing requirements is key to fulfill production objectives. Such environments often involve various information systems that make the integration of knowledge on the current state of production more challenging. In this paper, we present a conceptual approach of Digital Shadows that allow for a holistic data view on PPC-specific tasks based on an ontology, serving as a knowledge base of semantic-enriched data and production constraints. An exemplary real-world application for PPC in injection molding demonstrates the benefits of the concept. • Stakeholders in PPC are overwhelmed with information from less interlinked IT system. • Digital Shadows are a promising approach to support decisions in PPC. • Concept of Digital Shadows for Production Planning and Control. • Decision support provided through purpose driven information generation. • Exemplary concept application for minimizing rejection rates in injection molding. 

Towards digital shadows for production planning and control in injection molding

Modern manufacturing is increasingly data-driven. Yet there are a number of applications traditionally performed by humans because of their capabilities to think analytically, learn from previous experience and adapt. With the appearance of Deep Reinforcement Learning (RL) many of these applications can be partly or completely automated. In this paper we aim at finding an optimal clamping position for a workpiece (WP) with the help of deep RL. Traditionally, a human expert chooses a clamping position that leads to an efficient, high quality machining without axis limit violations or collisions. This decision is hard to automate because of the variety of WP geometries and possible ways to manufacture them. We investigate whether the use of RL can aid in finding a near-optimal WP clamping position, even for unseen WPs during training. We develop a use case representing a simplified problem of clamping position optimisation, formalise it as a Markov Decision Process (MDP) and conduct a number of RL experiments to demonstrate the applicability of the approach in terms of training stability and quality of the solutions. First evaluations of the concept demonstrate the capability of a trained RL agent to find a near-optimal clamping position for an unseen WP with a small number of iterations required.

Using Reinforcement Learning for Optimization of a Workpiece Clamping Position in a Machine Tool.

Data-driven decision support for process quality improvements

The advent of AlphaGo and its successors marked the beginning of a new paradigm in playing games using artificial intelligence. This was achieved by combining Monte Carlo tree search, a planning procedure, and deep learning. While the impact on the domain of games has been undeniable, it is less clear how useful similar approaches are in applications beyond games and how they need to be adapted from the original methodology. We review 129 peer-reviewed articles detailing the application of neural Monte Carlo tree search methods in domains other than games. Our goal is to systematically assess how such methods are structured in practice and if their success can be extended to other domains. We find applications in a variety of domains, many distinct ways of guiding the tree search using learned policy and value functions, and various training methods. Our review maps the current landscape of algorithms in the family of neural monte carlo tree search as they are applied to practical problems, which is a first step towards a more principled way of designing such algorithms for specific problems and their requirements.

pdf/beyond-games-a-systematic-review-of-neural-monte-carlo-tree-ofy2vkn3.pdf

Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applications

Causal inference is a fundamental research topic for discovering the cause–effect relationships in many disciplines. Inferring causality means identifying asymmetric relations between two variables. In real-world systems, e.g., finance, healthcare, and industrial processes, time series data from sensors and other data sources offer an especially good basis to infer causal relationships. Therefore, many different time series causal inference algorithms have been proposed in recent years. However, not all algorithms are equally well-suited for a given dataset. For instance, some approaches may only be able to identify linear relationships, while others are applicable for non-linearities. Algorithms further vary in their sensitivity to noise and their ability to infer causal information from coupled vs. non-coupled time series. As a consequence, different algorithms often generate different causal relationships for the same input. In order to achieve a more robust causal inference result, this publication proposes a novel data-driven two-phase multi-split causal ensemble model to combine the strengths of different causality base algorithms. In comparison to existing approaches, the proposed ensemble method reduces the influence of noise through a data partitioning scheme in a first phase. To achieve this, the data are initially divided into several partitions and the base causal inference algorithms are applied to each partition. Subsequently, Gaussian mixture models are used to identify the causal relationships derived from the different partitions that are likely to be valid. In the second phase, the identified relationships from each base algorithm are then merged based on three combination rules. The proposed ensemble approach is evaluated using multiple metrics, among them a newly developed evaluation index for causal ensemble approaches. We perform experiments using three synthetic datasets with different volumes and complexity, which have been specifically designed to test causality detection methods under different circumstances while knowing the ground truth causal relationships. In these experiments, our causality ensemble outperforms each of its base algorithms. In practical applications, the use of the proposed method could hence lead to more robust and reliable causality results.

/pdf/a-data-driven-two-phase-multi-split-causal-ensemble-model-7j7o6qv8.pdf

A Data-Driven Two-Phase Multi-Split Causal Ensemble Model for Time Series

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Daniel Lütticke

Author Tools

Chat about Author