Safe human–robot collaboration for industrial settings: a survey

The domain of industrial automation faces challenges, such as shortened product life cycles, shortage of skilled labor, and increased complexity. Addressing these issues necessitates innovative solutions, one of which is the Digital Twin, being a virtual counterpart of a physical asset. Central to the quality of a Digital Twin is the data it harnesses. While current Digital Twins primarly draw data from their corresponding physical assets, future interconnected production environments promise an influx of additional data from external devices. However, it remains uncertain how existing Digital Twins incorporate and leverage such data. In this systematic literature review, drawing from a pool of 1107 unique publications, we analyzed 141 works to shed light on data utilization in industrial Digital Twins. We categorized these publications based on Digital Twin types and classified them according to various criteria regarding different characteristics of data. Our findings reveal that the majority of Digital Twins predominantly rely on structured data sourced directly from their associated assets, often employing proprietary integration methods. Facing the trends towards agile and interconnected production ecosystems, as well as an increasing amount of unstructured data, we assert that current Digital Twins are not equipped to meet forthcoming demands in the industrial domain. Consequently, we propose necessary adaptations to fully unleash the potential of Digital Twins and outline future research fields, including automated data integration and evaluation.

Data Integration for Digital Twins in Industrial Automation: A Systematic Literature Review

The programming of industrial robots necessitates specialized expertise and significant time and effort, particularly for small batch sizes. However, with the increasing demand for agility in production, the solutions used for robot programming have evolved significantly. Intuitive robot programming systems based on diverse concepts have been introduced to facilitate rapid deployment of robot systems. One such approved concept is no-code robot programming with finger-based gesture. In this concept, non-expert users draw a robot path via finger movement, which is subsequently translated into robot programming language to facilitate the corresponding movement. A significant challenge associated with this method is the valid replication of the corresponding robot Tool Center Point (TCP) orientation. Reachability issues, non-compliant hand contortions, and sensor occlusions make it difficult to directly derive the robot’s TCP orientation from the finger’s orientation. This work presents two novel approaches for estimating robot orientation using numerical analysis and point cloud information for finger-based robot programming without requiring prior knowledge. The first approach utilizes numerical analysis to estimate the relative robot orientation based on the geometry of the trajectory. In contrast, the second approach uses point-cloud to derive the robot orientation based on the object contour. Input shaping algorithms are employed and evaluated to reduce the divergence in the orientation estimations. Experiments demonstrate the effectiveness of the proposed approach utilizing a low-cost camera as a cost-efficient alternative to existing no-code programming strategies, potentially accelerating the real-world deployment of robotic applications in industrial environments.

Enhanced No-Code Finger-Gesture-Based Robot Programming: Simultaneous Path and Contour Awareness for Orientation Estimation

Multimodal user interfaces promise natural and intuitive human–machine interactions. However, is the extra effort for the development of a complex multisensor system justified, or can users also be satisfied with only one input modality? This study investigates interactions in an industrial weld inspection workstation. Three unimodal interfaces, including spatial interaction with buttons augmented on a workpiece or a worktable, and speech commands, were tested individually and in a multimodal combination. Within the unimodal conditions, users preferred the augmented worktable, but overall, the interindividual usage of all input technologies in the multimodal condition was ranked best. Our findings indicate that the implementation and the use of multiple input modalities is valuable and that it is difficult to predict the usability of individual input modalities for complex systems.

/pdf/assessing-the-value-of-multimodal-interfaces-a-study-on-1m7ecdlz.pdf

Assessing the Value of Multimodal Interfaces: A Study on Human–Machine Interaction in Weld Inspection Workstations

pdf/improving-safety-in-human-robot-collaboration-towards-ap2nbzqfw5my.pdf

Improving safety in human–robot collaboration towards sustainable production in Industry 5.0

In the era of Industry 4.0 and agile manufacturing, the conventional methodologies for risk assessment, risk reduction, and safety procedures may not fulfill the End-User requirements, especially the SMEs with their product diversity and changeable production lines and processes. This work proposes a novel approach for planning and implementing safe and flexible Human-Robot-Interaction (HRI) workspaces using multilayer HRI operation modes. The collaborative operation modes are grouped in different clusters and categorized at various levels systematically. In addition to that, this work proposes a safety-related finite-state machine for describing the transitions between these modes dynamically and properly. The proposed approach is integrated into a new dynamic risk assessment tool as a promising solution toward a new safety horizon in line with industry 4.0.

/pdf/towards-safety4-0-a-novel-approach-for-flexible-human-robot-430x9av7.pdf

Towards safety4.0: A novel approach for flexible human-robot-interaction based on safety-related dynamic finite-state machine with multilayer operation modes

According to the standard ISO/TS 15066, a human-robot shared workspace could be divided into different zones with various collaborative modes. Usually, this division is based on the distance between human and robot, the level of human-robot interaction “HRI”, and the risk assessment. These factors define the robot velocity and the required sensors for ensuring human safety in every zone and every operation mode separately. This procedure could generally ensure human safety during interaction with industrial robots. However, productivity, efficiency and diversity of the shared tasks have been overtaken. This work proposes a system that simultaneously can use heterogeneous safety sensors in all collaborative operation modes. Furthermore, it presents situational zone-based robot control with the help of 3D advanced selection matrices. Every robot-subspace could be vision-, force-, or position-controlled in every zone, depending on the tasks and safety requirements. Using the proposed system, heavy-duty and collaborative robot “cobots” can perform various shared tasks with humans safely and efficiently. The proposed approach is tested in one use case in the automotive industry.

Situational zone-based robot control for heterogeneous safety sensors in agile HRI applications

Manual annotation for human action recognition with content semantics using 3D Point Cloud (3D-PC) in industrial environments consumes a lot of time and resources. This work aims to recognize, analyze, and model human actions to develop a framework for automatically extracting content semantics. Main Contributions of this work: 1. design a multi-layer structure of various DNN classifiers to detect and extract humans and dynamic objects using 3D-PC preciously, 2. empirical experiments with over 10 subjects for collecting datasets of human actions and activities in one industrial setting, 3. development of an intuitive GUI to verify human actions and its interaction activities with the environment, 4. design and implement a methodology for automatic sequence matching of human actions in 3D-PC. All these procedures are merged in the proposed framework and evaluated in one industrial Use-Case with flexible patch sizes. Comparing the new approach with standard methods has shown that the annotation process can be accelerated by 5.2 times through automation.

/pdf/a-novel-approach-for-automatic-annotation-of-human-actions-2cxauplu.pdf

A novel approach for automatic annotation of human actions in 3D point clouds for flexible collaborative tasks with industrial robots

Accurate detection of the human body and its limbs in real-time is one of the challenges toward human-robot collaboration (HRC) technology in the factory of the future. In this work, a new algorithm has been developed to fuse thermal, depth, and color information. Starting with the calibration of the sensors to each other, proceeding with data fusion and detection of human body parts, and ending with pose estimation. The proposed approach has been tested in various HRC scenarios. It has shown a significant decrease in errors and noise in industrial environments. Furthermore, it produces not only a higher positioning accuracy of the head and hands compared to the state of art algorithms (e.g., OpenPose), but it is also faster. Hence, such an algorithm could be joined with digital twins for safe and efficient human-robot collaboration.

Fusion of depth, color, and thermal images towards digital twins and safe human interaction with a robot in an industrial environment

Popular Tasks used by Researchers

How can I help with your research?

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

We encountered an error while loading the panel. Please try refreshing the page.

Something went wrong

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

Anyone with the link can view this chat. You can switch it back to private anytime.

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Ibrahim Al Naser

Author Tools

Chat about Author

Papers

Towards safety4.0: A novel approach for flexible human-robot-interaction based on safety-related dynamic finite-state machine with multilayer operation modes

Situational zone-based robot control for heterogeneous safety sensors in agile HRI applications

A novel approach for automatic annotation of human actions in 3D point clouds for flexible collaborative tasks with industrial robots

Fusion of depth, color, and thermal images towards digital twins and safe human interaction with a robot in an industrial environment