Showing papers on "Synchronization (computer science) published in 2017"
Proceedings Article•
27 Mar 2017TL;DR: A new, general geo-distributed ML system, Gaia, is introduced that decouples the communication within a data center from the communication between data centers, enabling different communication and consistency models for each.
Abstract: Machine learning (ML) is widely used to derive useful information from large-scale data (such as user activities, pictures, and videos) generated at increasingly rapid rates, all over the world. Unfortunately, it is infeasible to move all this globally-generated data to a centralized data center before running an ML algorithm over it--moving large amounts of raw data over wide-area networks (WANs) can be extremely slow, and is also subject to the constraints of privacy and data sovereignty laws. This motivates the need for a geo-distributed ML system spanning multiple data centers. Unfortunately, communicating over WANs can significantly degrade ML system performance (by as much as 53.7× in our study) because the communication overwhelms the limited WAN bandwidth.
Our goal in this work is to develop a geo-distributed ML system that (1) employs an intelligent communication mechanism over WANs to efficiently utilize the scarce WAN bandwidth, while retaining the accuracy and correctness guarantees of an ML algorithm; and (2) is generic and flexible enough to run a wide range of ML algorithms, without requiring any changes to the algorithms.
To this end, we introduce a new, general geo-distributed ML system, Gaia, that decouples the communication within a data center from the communication between data centers, enabling different communication and consistency models for each. We present a new ML synchronization model, Approximate Synchronous Parallel (ASP), whose key idea is to dynamically eliminate insignificant communication between data centers while still guaranteeing the correctness of ML algorithms. Our experiments on our prototypes of Gaia running across 11 Amazon EC2 global regions and on a cluster that emulates EC2 WAN bandwidth show that Gaia provides 1.8-53.5× speedup over two state-of-the-art distributed ML systems, and is within 0.94-1.40× of the speed of running the same ML algorithm on machines on a local area network (LAN).
427 citations
TL;DR: It is shown by theoretic proof that the estimation bound of the settling time given in this paper is less conservative and more accurate compared with the classical results.
391 citations
TL;DR: This paper focuses on the fixed-time synchronization control methodology for a class of delayed memristor-based recurrent neural networks, based on Lyapunov functionals, analytical techniques, and together with novel control algorithms that can be adjusted to desired values regardless of the initial conditions.
Abstract: This paper focuses on the fixed-time synchronization control methodology for a class of delayed memristor-based recurrent neural networks. Based on Lyapunov functionals, analytical techniques, and together with novel control algorithms, sufficient conditions are established to achieve fixed-time synchronization of the master and slave memristive systems. Moreover, the settling time of fixed-time synchronization is estimated, which can be adjusted to desired values regardless of the initial conditions. Finally, the corresponding simulation results are included to show the effectiveness of the proposed methodology derived in this paper.
322 citations
TL;DR: It is shown that the leader-following consensus problem with stochastic sampling can be transferred into a master-slave synchronization problem with only one master system and two slave systems.
Abstract: This paper is concerned with sampled-data leader-following consensus of a group of agents with nonlinear characteristic. A distributed consensus protocol with probabilistic sampling in two sampling periods is proposed. First, a general consensus criterion is derived for multiagent systems under a directed graph. A number of results in several special cases without transmittal delays or with the deterministic sampling are obtained. Second, a dimension-reduced condition is obtained for multiagent systems under an undirected graph. It is shown that the leader-following consensus problem with stochastic sampling can be transferred into a master–slave synchronization problem with only one master system and two slave systems. The problem solving is independent of the number of agents, which greatly facilitates its application to large-scale networked agents. Third, the network design issue is further addressed, demonstrating the positive and active roles of the network structure in reaching consensus. Finally, two examples are given to verify the theoretical results.
309 citations
Posted Content•
TL;DR: In this paper, an end-to-end trainable method for visual-inertial odometry is presented, which performs fusion of the data at an intermediate feature representation level.
Abstract: In this paper we present an on-manifold sequence-to-sequence learning approach to motion estimation using visual and inertial sensors. It is to the best of our knowledge the first end-to-end trainable method for visual-inertial odometry which performs fusion of the data at an intermediate feature-representation level. Our method has numerous advantages over traditional approaches. Specifically, it eliminates the need for tedious manual synchronization of the camera and IMU as well as eliminating the need for manual calibration between the IMU and camera. A further advantage is that our model naturally and elegantly incorporates domain specific information which significantly mitigates drift. We show that our approach is competitive with state-of-the-art traditional methods when accurate calibration data is available and can be trained to outperform them in the presence of calibration and synchronization errors.
278 citations
9 May 2017
TL;DR: A heterogeneity-aware algorithm that uses a constant learning rate schedule for updates before adding them to the global parameter allows us to suppress stragglers' harm on robust convergence and theoretically prove the valid convergence of both approaches.
Abstract: We study distributed machine learning in heterogeneous environments in this work. We first conduct a systematic study of existing systems running distributed stochastic gradient descent; we find that, although these systems work well in homogeneous environments, they can suffer performance degradation, sometimes up to 10x, in heterogeneous environments where stragglers are common because their synchronization protocols cannot fit a heterogeneous setting. Our first contribution is a heterogeneity-aware algorithm that uses a constant learning rate schedule for updates before adding them to the global parameter. This allows us to suppress stragglers' harm on robust convergence. As a further improvement, our second contribution is a more sophisticated learning rate schedule that takes into consideration the delayed information of each update. We theoretically prove the valid convergence of both approaches and implement a prototype system in the production cluster of our industrial partner Tencent Inc. We validate the performance of this prototype using a range of machine-learning workloads. Our prototype is 2-12x faster than other state-of-the-art systems, such as Spark, Petuum, and TensorFlow; and our proposed algorithm takes up to 6x fewer iterations to converge.
253 citations
1 Jan 2017
TL;DR: Chaos and Complexity Theory for Management: Nonlinear DynamicsFractional-Order Nonlinear SystemsBifurcation and Chaos in FractionalOrder Systems
232 citations
TL;DR: A novel event-triggered control law is proposed which could greatly reduce the number of control updates for synchronization tasks of coupled switched neural networks involving embedded microprocessors with limited on-board resources.
Abstract: This paper addresses the issue of synchronization of switched delayed neural networks with communication delays via event-triggered control. For synchronizing coupled switched neural networks, we propose a novel event-triggered control law which could greatly reduce the number of control updates for synchronization tasks of coupled switched neural networks involving embedded microprocessors with limited on-board resources. The control signals are driven by properly defined events, which depend on the measurement errors and current-sampled states. By using a delay system method, a novel model of synchronization error system with delays is proposed with the communication delays and event-triggered control in the unified framework for coupled switched neural networks. The criteria are derived for the event-triggered synchronization analysis and control synthesis of switched neural networks via the Lyapunov-Krasovskii functional method and free weighting matrix approach. A numerical example is elaborated on to illustrate the effectiveness of the derived results.
194 citations
1 Mar 2017
TL;DR: An extensive study of the scheme's four key design decisions: concurrency control protocol, version storage, garbage collection, and index management is conducted and identifies the fundamental bottlenecks of each design choice.
Abstract: Multi-version concurrency control (MVCC) is currently the most popular transaction management scheme in modern database management systems (DBMSs). Although MVCC was discovered in the late 1970s, it is used in almost every major relational DBMS released in the last decade. Maintaining multiple versions of data potentially increases parallelism without sacrificing serializability when processing transactions. But scaling MVCC in a multi-core and in-memory setting is non-trivial: when there are a large number of threads running in parallel, the synchronization overhead can outweigh the benefits of multi-versioning.To understand how MVCC perform when processing transactions in modern hardware settings, we conduct an extensive study of the scheme's four key design decisions: concurrency control protocol, version storage, garbage collection, and index management. We implemented state-of-the-art variants of all of these in an in-memory DBMS and evaluated them using OLTP workloads. Our analysis identifies the fundamental bottlenecks of each design choice.
183 citations
TL;DR: The concept of dynamic consensus is introduced and results on robust stability which assess the conditions for practical asymptotic synchronization of networked systems and characterize their collective behavior are presented.
Abstract: We present an analysis framework for the study of synchronization of heterogeneous nonlinear systems interconnected over networks described by directed graphs. Heterogeneous systems may have totally different dynamical models, albeit of the same dimension, or may possess equal models with different lumped parameters. We show that their behavior, when network-interconnected, is fully characterized in terms of two properties whose study may be recasted in terms of the stability analysis of two corresponding interconnected dynamical systems that evolve in orthogonal spaces: on one hand, we have the so-called emergent dynamics and, on the other, the synchronization error dynamics. Based on this premise, we introduce the concept of dynamic consensus and we present results on robust stability which assess the conditions for practical asymptotic synchronization of networked systems and characterize their collective behavior. To illustrate our main theoretical findings we broach a brief case-study on mutual synchronization of heterogeneous chaotic oscillators.
177 citations
TL;DR: It is shown numerically, and it is proved analytically, that the multilayer coupling induces a spontaneous explosive synchronization and a heterogeneous distribution of allocations, otherwise not present in the two systems considered separately.
Abstract: We introduce a framework to intertwine dynamical processes of different nature, each with its own distinct network topology, using a multilayer network approach. As an example of collective phenomena emerging from the interactions of multiple dynamical processes, we study a model where neural dynamics and nutrient transport are bidirectionally coupled in such a way that the allocation of the transport process at one layer depends on the degree of synchronization at the other layer, and vice versa. We show numerically, and we prove analytically, that the multilayer coupling induces a spontaneous explosive synchronization and a heterogeneous distribution of allocations, otherwise not present in the two systems considered separately. Our framework can find application to other cases where two or more dynamical processes such as synchronization, opinion formation, information diffusion, or disease spreading, are interacting with each other.
TL;DR: The adaptive control theory and the robust output regulation theory are used to solve the output synchronization problem for a heterogeneous network where the non-identical followers are affected by parameter perturbations and controlled to achieve output synchronization with an uncertain leader.
TL;DR: In this article, a memristor is used to connect two neurons and the phase synchronization in electrical activities is discussed, and the dependence of coupling intensity on phase synchronization of neuron is investigated and the effect of electromagnetic induction is considered.
Abstract: Synapse plays an important role in signal exchange and information encoding between neurons. Electric and chemical synapses are often used to investigate the synchronization in electrical activities of neurons. In this paper, memristor is used to connect two neurons and the phase synchronization in electrical activities is discussed. Inter-spike interval (ISI) is calculated from the sampled time series for membrane potential, and the dependence of coupling intensity on phase synchronization of neuron is investigated and the effect of electromagnetic induction is considered. Furthermore, the synchronization stability of network is detected under noise; a statistical synchronization factor is also calculated. It is found synchronization can be enhanced under memristor coupling and appropriate noise is also helpful for synchronization stability.
TL;DR: In this paper, an event-based leader-following strategy for synchronization of multi-agent systems (MASs) is considered and a model-based approach is adopted to predict the relative inter-node states between intermittent communications.
TL;DR: To accomplish the target of fixed-time synchronization, a novel feedback control procedure is designed for the slave neural networks by means of the Filippov discontinuity theories and Lyapunov stability theories, and an upper bound of the settling time is acquired.
TL;DR: A new Lyapunov functional is constructed to directly propose the asymptotic stability of the inertial neural networks, and some new stability criteria are derived by means of Barbalat Lemma.
TL;DR: It is revealed that synchronization(anti-synchronization) is faster as the order increases, and the relationship between the order and synchronization (anti- synchronized) is demonstrated numerically.
Abstract: This paper is concerned with the issues of synchronization and anti-synchronization for fractional chaotic financial system with market confidence by taking advantage of active control approach. Some sufficient conditions are derived to guarantee the synchronization and anti-synchronization for the proposed fractional system. Moreover, the relationship between the order and synchronization(anti-synchronization) is demonstrated numerically. It reveals that synchronization(anti-synchronization) is faster as the order increases. Finally, two illustrative examples are exploited to verify the efficiency of the obtained theoretical results.
TL;DR: In this paper, the authors derived sufficient conditions for lag synchronization of the addressed chaotic neural networks in terms of linear matrix inequalities (LMIs) based on the established LMIs and designed a hybrid controller including sampled-data controller and impulsive controller.
Abstract: In the framework of sampled-data control, this paper deals with the lag synchronization of chaotic neural networks with time delay meanwhile taking the impulsive control into account. By constructing a proper Lyapunov function and employing the impulsive control theory, some sufficient conditions for lag synchronization of the addressed chaotic neural networks are derived in terms of linear matrix inequalities (LMIs). The hybrid controller including sampled-data controller and impulsive controller is designed based on the established LMIs. A numerical example is provided to demonstrate the effectiveness and advantage of the obtained results.
TL;DR: This survey looks into the details of synchronization over IEEE 802.11 with a particular focus on the infrastructure mode which is most relevant for industrial use cases and highlights the different parameters which affect the performance of clock synchronization over WLAN and compares the performances of existing synchronization methods to analyze their shortcomings.
Abstract: Just like Ethernet before, IEEE 802.11 is now transcending the borders of its usage from the office environment toward real-time communication on the factory floor. However, similar to Ethernet, the availability of synchronized clocks to coordinate and control communication and distributed real-time services is not a built-in feature in WLAN. Over the years, this has led to the design and use of a wide variety of customized protocols with varying complexity and precision, both for wired and wireless networks, in accordance with the increasingly demanding requirements from real-time applications. This survey looks into the details of synchronization over IEEE 802.11 with a particular focus on the infrastructure mode which is most relevant for industrial use cases. It highlights the different parameters which affect the performance of clock synchronization over WLAN and compares the performance of existing synchronization methods to analyze their shortcomings. Finally, it identifies new trends and directions for future research as well as features for wireless clock synchronization which will be required by the applications in the near future.
19 Aug 2017
TL;DR: This paper presents Rudra, a parameter server based distributed computing framework tuned for training large-scale deep neural networks, and introduces a new learning rate modulation strategy to counter the effect of stale gradients and proposes a new synchronization protocol that can effectively bound the staleness in gradients, improve runtime performance and achieve good model accuracy.
Abstract: Deep learning with a large number of parameters requires distributed training, where model accuracy and runtime are two important factors to be considered. However, there has been no systematic study of the tradeoff between these two factors during the model training process. This paper presents Rudra, a parameter server based distributed computing framework tuned for training large-scale deep neural networks. Using variants of the asynchronous stochastic gradient descent algorithm we study the impact of synchronization protocol, stale gradient updates, minibatch size, learning rates, and number of learners on runtime performance and model accuracy. We introduce a new learning rate modulation strategy to counter the effect of stale gradients and propose a new synchronization protocol that can effectively bound the staleness in gradients, improve runtime performance and achieve good model accuracy. Our empirical investigation reveals a principled approach for distributed training of neural networks: the mini-batch size per learner should be reduced as more learners are added to the system to preserve the model accuracy. We validate this approach using commonly-used image classification benchmarks: CIFAR10 and ImageNet.
TL;DR: The results extend and improve the exponential impulsive control theory of neural network system with time-varying delays and reaction-diffusion terms to the fractional-order case and provide a conceptually straightforward mathematical representation of rather complex processes.
TL;DR: In this paper, the authors studied the pinning impulsive synchronization problem for a class of complex dynamical networks with time-varying delay and derived a sufficient verifiable criterion for the synchronization of delayed complex dynamic networks with small delay.
Abstract: In this paper, a robust mathematical method is proposed to study a new hybrid synchronization type, which is a combining generalized synchronization and inverse generalized synchronization. The method is based on Laplace transformation, Lyapunov stability theory of integer‐order systems and stability theory of linear fractional systems. Sufficient conditions are derived to demonstrate the coexistence of generalized synchronization and inverse generalized synchronization between different dimensional incommensurate fractional chaotic systems. Numerical test of the method is used. Copyright © 2016 John Wiley & Sons, Ltd.
TL;DR: This paper investigates the controllability and synchronization problems for identical-hierarchy mixed-valued logical control networks and resorts to Perron–Frobenius theorem to guarantee group-controllability and simultaneously-controllingability.
Abstract: This paper investigates the controllability and synchronization problems for identical-hierarchy mixed-valued logical control networks. The logical network considered is hierarchical, and Boolean network is a special case of logical network. Here, identical-hierarchy means that there are identical number of nodes in each layer of logical network and corresponding nodes have the same dimension for any two layers of logical networks. Meanwhile, in each layer of logical networks, the dimensions of nodes are distinct, and it is called a mixed-valued logical network. First, the controllability problem is investigated and two notions of controllability are presented, i.e., group-controllability and simultaneously-controllability. By resorting to Perron–Frobenius theorem, some necessary and sufficient criteria are obtained to guarantee group-controllability and simultaneously-controllability, respectively. Second, based on the algebraic representation of the studied model, synchronization problems are analytically discussed for two types of controls, i.e., free control sequences and state-output feedback control. Finally, two numerical examples are presented to show the validness of our main results.
TL;DR: In this article, the authors briefly review phase reduction theory, which is a simple and powerful method for analyzing the synchronization properties of limit-cycle oscillators exhibiting spontaneous rhythms, including the phase locking of an oscillator to a periodic external forcing and the mutual synchronization of interacting oscillators.
Abstract: Systems of dynamical elements exhibiting spontaneous rhythms are found in various fields of science and engineering, including physics, chemistry, biology, physiology, and mechanical and electrical engineering. Such dynamical elements are often modeled as nonlinear limit-cycle oscillators. In this article, we briefly review phase reduction theory, which is a simple and powerful method for analyzing the synchronization properties of limit-cycle oscillators exhibiting rhythmic dynamics. Through phase reduction theory, we can systematically simplify the nonlinear multi-dimensional differential equations describing a limit-cycle oscillator to a one-dimensional phase equation, which is much easier to analyze. Classical applications of this theory, i.e., the phase locking of an oscillator to a periodic external forcing and the mutual synchronization of interacting oscillators, are explained. Further, more recent applications of this theory to the synchronization of non-interacting oscillators induced by common noise and the dynamics of coupled oscillators on complex networks are discussed. We also comment on some recent advances in phase reduction theory for noise-driven oscillators and rhythmic spatiotemporal patterns.
TL;DR: By constructing a suitable Lyapunov-Krasovskii functional (LKF), which uses more information of the delay bounds, global asymptotic synchronization criteria for the considered system are established in the form of LMIs.
22 Mar 2017
TL;DR: The synchronization requirements and challenges from 5G in terms of the new technologies, new services, and the new architecture are discussed, and a new time reference model is suggested and the budget of the fronthaul is presented for C-RAN.
Abstract: This article discusses the synchronization requirements and challenges from 5G in terms of the new technologies, new services, and the new architecture. Time synchronization via transport networks will be essential. ±130ns is proposed as the new goal of network limits for 5G. Higher precision requirements can be achieved through the same network cluster in a relatively limited area. A new time reference model is suggested, and in particular, the budget of the fronthaul is presented for C-RAN. Corresponding solutions for enhancing time accuracy are presented. New technologies of the time source and the time transport need to be introduced. SDOs need closer cooperation to define the specification of time synchronization for 5G.
TL;DR: New testable algebraic criteria for ensuring the existence and global asymptotic stability of the system equilibrium point are obtained by employing the Kakutani's fixed point theorem of set-valued maps, the comparison theorem, and the stability criterion for FO linear systems with multiple delays.
TL;DR: The architecture, and the benefits, of a distributed middleware prototype supporting a new generation of smart-factory-enabled applications with special attention paid to simulation tools are presented.