Abstract: The digital twin is a rather new industrial control and automation systems concept. While the approach so far has gained interest mainly due to capabilities to make advanced simulations and optimizations, recently the possibilities for enhanced security have got attention within the research community. In this article, we discuss how a digital twin replication model and corresponding security architecture can be used to allow data sharing and control of security-critical processes. We identify design-driving security requirements for digital twin based data sharing and control. We show that the proposed state synchronization design meets the expected digital twin synchronization requirements and give a high-level design and evaluation of other security components of the architecture. We also make performance evaluations of a proof of concept for protected software upgrade using the proposed digital twin design. Our new security framework provides a foundation for future research work in this promising new area.

Abstract: This paper presents the design, implementation, and evaluation of the PyTorch distributed data parallel module. Py-Torch is a widely-adopted scientific computing package used in deep learning research and applications. Recent advances in deep learning argue for the value of large datasets and large models, which necessitates the ability to scale out model training to more computational resources. Data parallelism has emerged as a popular solution for distributed training thanks to its straightforward principle and broad applicability. In general, the technique of distributed data parallelism replicates the model on every computational resource to generate gradients independently and then communicates those gradients at each iteration to keep model replicas consistent. Despite the conceptual simplicity of the technique, the subtle dependencies between computation and communication make it non-trivial to optimize the distributed training efficiency. As of v1.5, PyTorch natively provides several techniques to accelerate distributed data parallel, including bucketing gradients, overlapping computation with communication, and skipping gradient synchronization. Evaluations show that, when configured appropriately, the PyTorch distributed data parallel module attains near-linear scalability using 256 GPUs.

Abstract: This paper is mainly concerned with the synchronization problem of reaction–diffusion neural networks (RDNNs) with delays and its direct application in image secure communications. An adaptive control is designed without a sign function in which the controller gain matrix is a function of time. The synchronization criteria are established for an error model derived from master–slave models through solving the set of linear matrix inequalities derived by constructing the suitable novel Lyapunov–Krasovskii functional candidate, Green’s formula, and Wirtinger’s inequality. If the proposed sufficient conditions are satisfied, then the global asymptotic synchronization of the error model is guaranteed. The numerical illustrations are provided to demonstrate the validity of the derived synchronization criteria. In addition, the role of system parameters is picturized through the chaotic nature of RDNNs and those unprecedented solutions is utilized to promote better security of image transactions. As is evident, the enhancement of image encryption algorithm is designed with two levels, namely, image watermarking and diffusion process. The contributions of this paper are discussed as concluding remarks.

Abstract: Synchronization processes play critical roles in the functionality of a wide range of both natural and man-made systems. Recent work in physics and neuroscience highlights the importance of higher-order interactions between dynamical units, i.e., three- and four-way interactions in addition to pairwise interactions, and their role in shaping collective behavior. Here we show that higher-order interactions between coupled phase oscillators, encoded microscopically in a simplicial complex, give rise to added nonlinearity in the macroscopic system dynamics that induces abrupt synchronization transitions via hysteresis and bistability of synchronized and incoherent states. Moreover, these higher-order interactions can stabilize strongly synchronized states even when the pairwise coupling is repulsive. These findings reveal a self-organized phenomenon that may be responsible for the rapid switching to synchronization in many biological and other systems that exhibit synchronization without the need of particular correlation mechanisms between the oscillators and the topological structure. While first order phase transitions between incoherence and synchronization are critical for collective behavior in various oscillator system application, e.g., the brain and power grids, such transitions typically require finely tuned properties. In this work the authors show that first order phase transitions and bistability can emerge naturally as a consequence of the presence of higher-order interactions between oscillators.

Abstract: The industrial Internet of Things (IoT) is a trend of factory development and a basic condition of intelligent factory. It is very important to ensure the security of data transmission in industrial IoT. Applying a new chaotic secure communication scheme to address the security problem of data transmission is the main contribution of this paper. The scheme is proposed and studied based on the synchronization of different-structure fractional-order chaotic systems with different order. The Lyapunov stability theory is used to prove the synchronization between the fractional-order drive system and the response system. The encryption and decryption process of the main data signals is implemented by using the n-shift encryption principle. We calculate and analyze the key space of the scheme. Numerical simulations are introduced to show the effectiveness of theoretical approach we proposed.

Abstract: Most existing finite-/fixed-time synchronization control schemes are nonsmooth or discontinuous, and the settling time is estimated with conservatism. It is due to the utilization of signum function or fraction power state feedback. This brief considers the problem of prespecified-time cluster synchronization of complex networks with a smooth control protocol. The synchronization time is independent of any control parameters or any systems’ initial conditions, which is actually uniformly prescribed according to task requirements without any estimations. Moreover, the cluster synchronization can maintain after the specified time, and the smooth control input can always keep uniformly bounded in an infinite time interval as well. Finally, one numerical example is provided to illustrate the effectiveness of the proposed protocol and design method.

Abstract: Opinion entity extraction is a fundamental task in fine-grained opinion mining. Related studies generally extract aspects and/or opinion expressions without recognizing the relations between them. However, the relations are crucial for downstream tasks, including sentiment classification, opinion summarization, etc. In this paper, we explore Aspect-Opinion Pair Extraction (AOPE) task, which aims at extracting aspects and opinion expressions in pairs. To deal with this task, we propose Synchronous Double-channel Recurrent Network (SDRN) mainly consisting of an opinion entity extraction unit, a relation detection unit, and a synchronization unit. The opinion entity extraction unit and the relation detection unit are developed as two channels to extract opinion entities and relations simultaneously. Furthermore, within the synchronization unit, we design Entity Synchronization Mechanism (ESM) and Relation Synchronization Mechanism (RSM) to enhance the mutual benefit on the above two channels. To verify the performance of SDRN, we manually build three datasets based on SemEval 2014 and 2015 benchmarks. Extensive experiments demonstrate that SDRN achieves state-of-the-art performances.

Abstract: Digital twin, an emerging representation of cyberphysical systems, has attracted increasing attentions very recently. It opens the way to real-time monitoring and synchronization of real-world activities with the virtual counterparts. In this study, we develop a digital twin paradigm using an advanced driver assistance system (ADAS) for connected vehicles. By leveraging vehicle-to-cloud (V2C) communication, on-board devices can upload the data to the server through cellular network. The server creates a virtual world based on the received data, processes them with the proposed models, and sends them back to the connected vehicles. Drivers can benefit from this V2C based ADAS, even if all computations are conducted on the cloud. The cooperative ramp merging case study is conducted, and the field implementation results show the proposed digital twin framework can benefit the transportation systems regarding mobility and environmental sustainability with acceptable communication delays and packet losses.

Abstract: Cyber attacks pose severe threats on synchronization of multi-agent systems. Deception attack, as a typical type of cyber attack, can bypass the surveillance of the attack detection mechanism silently, resulting in a heavy loss. Therefore, the problem of mean-square bounded synchronization in multi-agent systems subject to deception attacks is investigated in this paper. The control signals can be replaced with false data from controller-to-actuator channels or the controller. The success of the attack is measured through a stochastic variable. A distributed impulsive controller using a pinning strategy is redesigned, which ensures that mean-square bounded synchronization is achieved in the presence of deception attacks. Some sufficient conditions are derived, in which upper bounds of the synchronization error are given. Finally, two numerical simulations with symmetric and asymmetric network topologies are given to illustrate the theoretical results.

Abstract: In the paper, we investigate the exponential stability of nonlinear delayed systems with destabilizing and stabilizing delayed impulses, respectively. Specifically, the study can be divided into tw...

Abstract: Neural cryptography is a public key exchange algorithm based on the principle of neural network synchronization. By using the learning algorithm of a neural network, the two neural networks update their own weight through exchanging output from each other. Once the synchronization is completed, the weights of the two neural networks are the same. The weights of the neural network can be used for the secret key. However, all the existing works are based on the real-valued neural network model. There are seldom works studying the neural cryptography based on a complex-valued neural network model. In this technical note, a neural cryptography based on the complex-valued tree parity machine network (CVTPM) is proposed. The input, output, and weights of CVTPM are a complex value, which can be considered as an extension of TPM. There are two advantages of the CVTPM: 1) the security of CVTPM is higher than that of TPM with the same hidden units, input neurons, and synaptic depths and 2) the two parties with the CVTPM can exchange two group keys in one neural synchronization process. A series of numerical simulation experiments is provided to verify our results.

Abstract: Distributed deep learning becomes very common to reduce the overall training time by exploiting multiple computing devices (e.g., GPUs/TPUs) as the size of deep models and data sets increases. However, data communication between computing devices could be a potential bottleneck to limit the system scalability. How to address the communication problem in distributed deep learning is becoming a hot research topic recently. In this paper, we provide a comprehensive survey of the communication-efficient distributed training algorithms in both system-level and algorithmic-level optimizations. In the system-level, we demystify the system design and implementation to reduce the communication cost. In algorithmic-level, we compare different algorithms with theoretical convergence bounds and communication complexity. Specifically, we first propose the taxonomy of data-parallel distributed training algorithms, which contains four main dimensions: communication synchronization, system architectures, compression techniques, and parallelism of communication and computing. Then we discuss the studies in addressing the problems of the four dimensions to compare the communication cost. We further compare the convergence rates of different algorithms, which enable us to know how fast the algorithms can converge to the solution in terms of iterations. According to the system-level communication cost analysis and theoretical convergence speed comparison, we provide the readers to understand what algorithms are more efficient under specific distributed environments and extrapolate potential directions for further optimizations.

Abstract: In this paper, to investigate the exponential synchronization of stochastic neural networks, a new periodically intermittent discrete observation control (PIDOC) is first proposed. Different from the existing periodically intermittent control, our control in control time is feedback control based on discrete-time state observations (FCDSOs) instead of a continuous-time one. By employing the Lyapunov method, graph theory, and theory of differential inclusions, the exponential synchronization of stochastic neural networks with a discontinuous right-hand side is realized by PIDOC and some sufficient conditions are presented. Especially, when control width tends to control period, PIDOC will be reduced to a general FCDSO and we give some detailed discussions. Then, we provide some corollaries about synchronization in mean square, asymptotical synchronization in mean square, and exponential synchronization of stochastic neural networks under FCDSO. Finally, some numerical simulations are provided to demonstrate our analytical results.

Abstract: The authors introduce an analytical framework to compute the stability of synchronization in populations of phase oscillators with higher-order interactions up to any order, and arbitrary complex topology.

Abstract: To realize high-performance contouring tracking of dual-linear-motor-driven (DLMD) gantry, both motion coordination of multiaxes and synchronization of redundant actuators are indispensable factors. However, earlier contouring methods of multiaxes systems only focus on motion coordination, leading to certain inherent limitations and uncertainties when applied to DLMD gantry. Especially, the usually ignored synchronization issue may directly affect the normal operation of system. To this end, this paper proposes a novel contouring control method by integrating both motion coordination between axes and synchronization of redundant actuators. With the knowledge of the complete system dynamics, corresponding modification and extension have been done within the framework of global task coordinate frame (GTCF). Subsequently, both associated issues are effectively dealt with in the task space to simultaneously obtain efficient motion coordination ability and high-level synchronization, along with more accurate contouring error calculation. The proposed method is compared with the earlier GTCF-based contouring control method without consideration of a synchronization issue. The experimental results fully indicate the effectiveness and superiority of the proposed method to indeed guarantee and further improve the contouring performance.

Abstract: This paper investigates exponential stability of fractional-order impulsive control systems (FICSs) and exponential synchronization of fractional-order Cohen–Grossberg neural networks (FCGNNs). First, under the framework of the generalized Caputo fractional-order derivative, some new results for fractional-order calculus are established by mainly using L’Hospital’s rule and Laplace transform. Besides, FICSs are translated into impulsive differential equations with fractional-order via utilizing the definition of Dirac function, which reveals that the effect of impulsive control on fractional systems is dependent of the order of the addressed systems. Furthermore, exponential stability of FICSs is proposed and some novel criteria are obtained by applying average impulsive interval and the method of induction. As an application of the stability for FICSs, exponential synchronization of FCGNNs is considered and several synchronization conditions are established under impulsive control. Finally, several numerical examples are provided to illustrate the effectiveness of the derived results.

Abstract: To tackle the logistics challenges faced by enterprises using unmanned aerial vehicles (UAV) with human-driven vans for parcel deliveries, we introduce the two-echelon vehicle routing problem with time windows and mobile satellites (2E-VRP-TM), which, when solved, optimizes delivery routes for a fleet of van-UAV combinations. Typically, one van carries several UAVs. The first echelon involves time-window-driven parcel deliveries using vans from a distribution center (DC) to customers. The second echelon involves UAVs being dispatched from mobile-satellite vans to serve customers with time windows and directly delivering parcels from the DC. When the first-echelon vehicles park at customer locations and wait for second-echelon vehicle departures and returns, the first-echelon vehicles are used as mobile satellites. We develop a vehicle-flow formulation, in which the mobile-satellite synchronization constraints are included to ensure the echelon interaction. We provide an adaptive large neighborhood search heuristic. Computational experiments evaluate the validity of the 2E-VRP-TM formulation and the effectiveness of the heuristic.

Abstract: In this paper, spatial diffusions are introduced to fractional-order coupled networks and the problem of synchronization is investigated for fractional-order coupled neural networks with reaction-diffusion terms. First, a new fractional-order inequality is established based on the Caputo partial fractional derivative. To realize asymptotical synchronization, two types of adaptive coupling weights are considered, namely: 1) coupling weights only related to time and 2) coupling weights dependent on both time and space. For each type of coupling weights, based on local information of the node’s dynamics, an edge-based fractional-order adaptive law and an edge-based fractional-order pinning adaptive scheme are proposed. Furthermore, some new analytical tools, including the method of contradiction, L’Hopital rule, and Barbalat lemma are developed to establish adaptive synchronization criteria of the addressed networks. Finally, an example with numerical simulations is provided to illustrate the validity and effectiveness of the theoretical results.

Abstract: This article is devoted to the cluster synchronization issue of coupled fractional-order neural networks. By introducing the stability theory of fractional-order differential systems and the framework of Filippov regularization, some sufficient conditions are derived for ascertaining the asymptotic and finite-time cluster synchronization of coupled fractional-order neural networks, respectively. In addition, the upper bound of the settling time for finite-time cluster synchronization is estimated. Compared with the existing works, the results herein are applicable for fractional-order systems, which could be regarded as an extension of integer-order ones. A numerical example with different cases is presented to illustrate the validity of theoretical results.

Abstract: This article concentrates on the synchronization problem of memristive complex-valued neural networks (CVNNs) with time delays via the pinning control method. Different from general control schemes, the pinning control is beneficial to reduce the control cost by pinning the fractional nodes instead of all ones. By separating the complex-valued system into two equivalent real-valued systems and employing the Lyapunov functional as well as some inequality techniques, the asymptotic synchronization criterion is given to guarantee the realization of synchronization of memristive CVNNs. Meanwhile, sufficient conditions for exponential synchronization of the considered systems is also proposed. Finally, the validity of our proposed results is verified by a numerical example.

Abstract: Low power wide area (LPWA) wireless networks based on the LoRa physical layer have attracted huge attention in recent years, both from industry and from academic researchers. While this rising popularity is due to this technology’s demonstrated effectiveness and low cost, unfortunately, due to their complexity, the timing and frequency synchronization algorithms required to detect LoRa-modulated frames, in the context of minimum sampling rate optimum receivers, have received little attention. The aim of this paper is to fill this gap and describe how robust frame detection can be performed while focusing on minimal complexity implementations of the proposed algorithms. The ultimate goal is to propose frame detection techniques applicable to recently proposed ultra-low power software-defined receivers.

Abstract: Network telemetry is essential for administrators to monitor massive data traffic in a network-wide manner. Existing telemetry solutions often face the dilemma between resource efficiency (i.e., low CPU, memory, and bandwidth overhead) and full accuracy (i.e., error-free and holistic measurement). We break this dilemma via a network-wide architectural design OmniMon, which simultaneously achieves resource efficiency and full accuracy in flow-level telemetry for large-scale data centers. OmniMon carefully coordinates the collaboration among different types of entities in the whole network to execute telemetry operations, such that the resource constraints of each entity are satisfied without compromising full accuracy. It further addresses consistency in network-wide epoch synchronization and accountability in error-free packet loss inference. We prototype OmniMon in DPDK and P4. Testbed experiments on commodity servers and Tofino switches demonstrate the effectiveness of OmniMon over state-of-the-art telemetry designs.

Abstract: We propose Federated Accelerated Stochastic Gradient Descent (FedAc), a principled acceleration of Federated Averaging (FedAvg, also known as Local SGD) for distributed optimization. FedAc is the first provable acceleration of FedAvg that improves convergence speed and communication efficiency on various types of convex functions. For example, for strongly convex and smooth functions, when using $M$ workers, the previous state-of-the-art FedAvg analysis can achieve a linear speedup in $M$ if given $M$ rounds of synchronization, whereas FedAc only requires $M^{\frac{1}{3}}$ rounds. Moreover, we prove stronger guarantees for FedAc when the objectives are third-order smooth. Our technique is based on a potential-based perturbed iterate analysis, a novel stability analysis of generalized accelerated SGD, and a strategic tradeoff between acceleration and stability.