Robotic systems are increasingly a part of everyday life. Characteristics of robotic systems such as interaction with the physical world, and integration of hardware and software components, differentiate robotic systems from conventional software systems. Although numerous studies have investigated the challenges of software testing in practice, no such study has focused on testing of robotic systems. In this paper, we conduct a qualitative study to better understand the testing practices used by the robotics community, and identify the challenges faced by practitioners when testing their systems. We identify a total of 12 testing practices and 9 testing challenges from our participants’ responses. We group these challenges into 3 major themes: Real-world complexities, Community and standards, and Component integration. We believe that further research on addressing challenges described with these three major themes can result in higher adoption of robotics testing practices, more testing automation, and higher-quality robotic systems.

A Study on Challenges of Testing Robotic Systems

Neural networks have had discernible achievements in a wide range of applications. The wide-spread adoption also raises the concern of their dependability and reliability. Similar to traditional decision-making programs, neural networks can have defects that need to be repaired. The defects may cause unsafe behaviors, raise security concerns or unjust societal impacts. In this work, we address the problem of repairing a neural network for desirable properties such as fairness and the absence of backdoor. The goal is to construct a neural network that satisfies the property by (minimally) adjusting the given neural network's parameters (i.e., weights). Specifically, we propose CARE (CAusality-based REpair), a causality-based neural network repair technique that 1) performs causality-based fault localization to identify the ‘guilty’ neurons and 2) optimizes the parameters of the identified neurons to reduce the misbehavior. We have empirically evaluated CARE on various tasks such as backdoor removal, neural network repair for fairness and safety properties. Our experiment results show that CARE is able to repair all neural networks efficiently and effectively. For fairness repair tasks, CARE successfully improves fairness by 61.91 % on average. For backdoor removal tasks, CARE reduces the attack success rate from over 98% to less than 1 %. For safety property repair tasks, CARE reduces the property violation rate to less than 1 %. Results also show that thanks to the causality-based fault localization, CARE's repair focuses on the misbehavior and preserves the accuracy of the neural networks.

/pdf/causality-based-neural-network-repair-fjs3tqpd.pdf

Causality-Based Neural Network Repair

Drones are increasingly adopted to serve a smart city through their ability to render quick and adaptive services. They are also known as unmanned aerial vehicles (UAVs) and are deployed to conduct area surveillance, monitor road networks for traffic, deliver goods and observe environmental phenomena. Cyber threats posed through compromised drones contribute to sabotage in a smart city’s airspace, can prove to be catastrophic to its operations, and can also cause fatalities. In this contribution, we propose a machine learning-based approach for detecting hijacking, GPS signal jamming and denial of service (DoS) attacks that can be carried out against a drone. A detailed machine learning-based classification of drone datasets for the DJI Phantom 4 model, compromising both normal and malicious signatures, is conducted, and results obtained yield advisory to foster futuristic opportunities to safeguard a drone system against such cyber threats.

/pdf/securing-the-smart-city-airspace-drone-cyber-attack-1eq7iif2.pdf

Securing the Smart City Airspace: Drone Cyber Attack Detection through Machine Learning

Actual causality is increasingly well understood. Recent formal approaches, proposed by Halpern and Pearl, have made this concept mature enough to be amenable to automated reasoning. Actual causality is especially vital for building accountable, explainable systems. Among other reasons, causality reasoning is computationally hard due to the requirements of counterfactuality and the minimality of causes. Previous approaches presented either inefficient or restricted, and domain-specific, solutions to the problem of automating causality reasoning. In this paper, we present a novel approach to formulate different notions of causal reasoning, over binary acyclic models, as optimization problems, based on quantifiable notions within counterfactual computations. We contribute and compare two compact, non-trivial, and sound integer linear programming (ILP) and Maximum Satisfiability (MaxSAT) encodings to check causality. Given a candidate cause, both approaches identify what a minimal cause is. Also, we present an ILP encoding to infer causality without requiring a candidate cause. We show that both notions are efficiently automated. Using models with more than $8000$ variables, checking is computed in a matter of seconds, with MaxSAT outperforming ILP in many cases. In contrast, inference is computed in a matter of minutes.

From Checking to Inference: Actual Causality Computations as Optimization Problems

Testing plays an essential role in ensuring the safety and quality of cyberphysical systems (CPSs). One of the main challenges in automated and software-in-the-loop simulation testing of CPSs is defining effective oracles that can check that a given system conforms to expectations of desired behavior. Manually specifying such oracles can be tedious, complex, and error-prone, and so techniques for automatically learning oracles are attractive. Characteristics of CPSs, such as limited or no access to source code, behavior that is non-deterministic and sensitive to noise, and that the system may respond differently to input based on its context introduce considerable challenges for automated oracle learning. We present Mithra, a novel, unsupervised oracle learning technique for CPSs that operates on existing telemetry data. It uses a three-step multivariate time series clustering to discover the set of unique, correct behaviors for a CPS, which it uses to construct robust oracles. We instantiate our proposed technique for ArduPilot, a popular, open-source autopilot software. On a set of 24 bugs, we show that Mithra effectively identifies buggy executions with few false positives and outperforms AR-SI, a state-of-the-art CPS oracle learning technique. We demonstrate Mithra’s wider applicability by applying it to an autonomous racer built for the Robot Operating System. 

Mithra: Anomaly Detection as an Oracle for Cyberphysical Systems

As capabilities of cyber-physical systems (CPS) increase, the interaction of software and physical components becomes more complicated. When a CPS encounters an incident, the increased complexity makes diagnosis a challenging task for traditional diagnostic approaches. To overcome this problem, we split the diagnostic procedure into three steps, namely: (1) type causality, (2) detection and (3) actual causality analyses. We then utilize various technologies to automate each step. Fault trees are extracted from the four variable model of a CPS. This results in modular and human-readable fault trees. Moreover, CPS logs are mapped to the instances of the fault tree nodes using time series analysis techniques. Through examples of unmanned aerial vehicles (UAV), we demonstrate that our framework can diagnose a wide range of scenarios including software, sensor, and actuator failures.

Diagnosis of Safety Incidents for Cyber-Physical Systems: A UAV Example

The rapid deployment of digital systems into all aspects of daily life requires embedding social constructs into the digital world. Because of the complexity of these systems, there is a need for technical support to understand their actions. Social concepts, such as explainability, accountability, and responsibility rely on a notion of actual causality. Encapsulated in the Halpern and Pearl’s (HP) definition, actual causality conveniently integrates into the socio-technical world if operationalized in concrete applications. To the best of our knowledge, theories of actual causality such as the HP definition are either applied in correspondence with domain-specific concepts (e.g., a lineage of a database query) or demonstrated using straightforward philosophical examples. On the other hand, there is a lack of explicit automated actual causality theories and operationalizations for helping understand the actions of systems. Therefore, this paper proposes a unifying framework and an interactive platform (Actual Causality Canvas) to address the problem of operationalizing actual causality for different domains and purposes. We apply this framework in such areas as aircraft accidents, unmanned aerial vehicles, and artificial intelligence (AI) systems for purposes of forensic investigation, fault diagnosis, and explainable AI. We show that with minimal effort, using our general-purpose interactive platform, actual causality reasoning can be integrated into these domains.

Actual Causality Canvas: A General Framework for Explanation-Based Socio-Technical Constructs

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Ehsan Zibaei

Author Tools

Chat about Author

Papers

Diagnosis of Safety Incidents for Cyber-Physical Systems: A UAV Example

Actual Causality Canvas: A General Framework for Explanation-Based Socio-Technical Constructs