TL;DR: In this paper, the authors present an update scheme called HOGWILD!, which allows processors access to shared memory with the possibility of overwriting each other's work, which achieves a nearly optimal rate of convergence.
Abstract: Stochastic Gradient Descent (SGD) is a popular algorithm that can achieve state-of-the-art performance on a variety of machine learning tasks. Several researchers have recently proposed schemes to parallelize SGD, but all require performance-destroying memory locking and synchronization. This work aims to show using novel theoretical analysis, algorithms, and implementation that SGD can be implemented without any locking. We present an update scheme called HOGWILD! which allows processors access to shared memory with the possibility of overwriting each other's work. We show that when the associated optimization problem is sparse, meaning most gradient updates only modify small parts of the decision variable, then HOGWILD! achieves a nearly optimal rate of convergence. We demonstrate experimentally that HOGWILD! outperforms alternative schemes that use locking by an order of magnitude.
TL;DR: This work proposes that processes underlying working and long-term memory might interact in the medial temporal lobe and proposes that this is accomplished by neural operations involving phase–phase and phase–amplitude synchronization.
Abstract: In recent years, studies ranging from single-unit recordings in animals to electroencephalography and magnetoencephalography studies in humans have demonstrated the pivotal role of phase synchronization in memory processes. Phase synchronization - here referring to the synchronization of oscillatory phases between different brain regions - supports both working memory and long-term memory and acts by facilitating neural communication and by promoting neural plasticity. There is evidence that processes underlying working and long-term memory might interact in the medial temporal lobe. We propose that this is accomplished by neural operations involving phase-phase and phase-amplitude synchronization. A deeper understanding of how phase synchronization supports the flexibility of and interaction between memory systems may yield new insights into the functions of phase synchronization in general.
TL;DR: This work aims to show using novel theoretical analysis, algorithms, and implementation that SGD can be implemented without any locking, and presents an update scheme called HOGWILD! which allows processors access to shared memory with the possibility of overwriting each other's work.
Abstract: Stochastic Gradient Descent (SGD) is a popular algorithm that can achieve state-of-the-art performance on a variety of machine learning tasks. Several researchers have recently proposed schemes to parallelize SGD, but all require performance-destroying memory locking and synchronization. This work aims to show using novel theoretical analysis, algorithms, and implementation that SGD can be implemented without any locking. We present an update scheme called HOGWILD! which allows processors access to shared memory with the possibility of overwriting each other's work. We show that when the associated optimization problem is sparse, meaning most gradient updates only modify small parts of the decision variable, then HOGWILD! achieves a nearly optimal rate of convergence. We demonstrate experimentally that HOGWILD! outperforms alternative schemes that use locking by an order of magnitude.
TL;DR: This talk will introduce arrays of optomechanical cells, and discuss the first theoretical results on the nonlinear dynamics of such a setup, using a mean-field approach based on the Lindblad master equation, as well as semiclassical Langevin equations.
Abstract: Optomechanical systems couple light stored inside an optical cavity to the motion of a mechanical mode. Recent experiments have demonstrated setups, such as photonic crystal structures, that in principle allow one to confine several optical and vibrational modes on a single chip. Here we start to investigate the collective nonlinear dynamics in arrays of coupled optomechanical cells. We show that such ``optomechanical arrays'' can display synchronization, and that they can be described by an effective Kuramoto-type model.
TL;DR: In this article, the concept of phase cohesiveness is introduced as an analysis tool and performance index for synchronization in the finite-dimensional Kuramoto model and a necessary and sufficient condition on the critical coupling strength to achieve synchronization is presented.
Abstract: The celebrated Kuramoto model captures various synchronization phenomena in biological and man-made dynamical systems of coupled oscillators. It is well known that there exists a critical coupling strength among the oscillators at which a phase transition from incoherency to synchronization occurs. This paper features four contributions. First, we characterize and distinguish the different notions of synchronization used throughout the literature and formally introduce the concept of phase cohesiveness as an analysis tool and performance index for synchronization. Second, we review the vast literature providing necessary, sufficient, implicit, and explicit estimates of the critical coupling strength in the finite- and infinite-dimensional cases and for both first-order and second-order Kuramoto models. Third, we present the first explicit necessary and sufficient condition on the critical coupling strength to achieve synchronization in the finite-dimensional Kuramoto model for an arbitrary distribution of...
TL;DR: In this article, the car-following models are used for time-delay feedback and coupling in Coupled Time-delay systems and transition from anticipatory to Lag Synchronization via complete synchronization.
Abstract: Delay Differential Equations.- Linear Stability and Bifurcation Analysis.- Bifurcation and Chaos in Time-delayed Piecewise Linear.- A Few Other Interesting Chaotic Delay Differential Equations.- Implications of Delay Feebdack: Amplitude Death and Other Effects.- Recent Developments on Delay Feedback/Coupling: Complex.- Complete Synchronization in Coupled Time-delay Systems.- Transition from Anticipatory to Lag Synchronization via Complete.- Intermittency Transition to Generalized Snychronization.- Transition from Phase to Generalized Synchronization.- DTM Induced Oscillating Synchronization.- Exact Solutions of Certain Time Delay Systems: The Car-following Models.- A Computing Lyapunov Exponents for Time-delay systems.- B A Brief Introduction to Synchronization in Chaotic Dynamical Systems.- C Recurrence Analysis.- References.- Glossary.- Index.
TL;DR: In this paper, a cooperative tracking adaptive controller is designed based on each node maintaining a neural network parametric approximator and suitably tuning it to guarantee stability and performance, and a Lyapunov-based proof shows the ultimate boundedness of the tracking error.
TL;DR: It is suggested that the considerable intellectual effort needed for designing efficient algorithms for multi-core architectures may be most fruitfully expended in designing portable algorithms, once and for all, for such a bridging model.
TL;DR: In this paper, a review article summarizes the various experimental and theoretical studies of hydrodynamic synchronization, and put them in a framework which draws parallels between the different systems and suggests useful directions for further research.
Abstract: After a long gap following the classic work of Taylor, there have recently been several studies dealing with hydrodynamic synchronization. It is now apparent that synchronization driven by hydrodynamic interactions is not only possible, but relevant to the efficiency of pumping by arrays of cilia and to bacterial swimming. Recent work has included experiments demonstrating synchronization, both in model systems and between bacterial flagella. The effect has been demonstrated in model swimmers and pumps, and large scale simulations have been used to investigate synchronization of cilia and of sperm cells. In this review article, we summarize the various experimental and theoretical studies of hydrodynamic synchronization, and put them in a framework which draws parallels between the different systems and suggests useful directions for further research.
TL;DR: In this article, an apparatus, system, and method are disclosed for data block usage information synchronization for a non-volatile storage volume. But the method is not suitable for the synchronization of multiple data blocks.
Abstract: An apparatus, system, and method are disclosed for data block usage information synchronization for a non-volatile storage volume. The method includes referencing first data block usage information for data blocks of a non-volatile storage volume managed by a storage manager. The first data block usage information is maintained by the storage manager. The method also includes synchronizing second data block usage information managed by a storage controller with the first data block usage information maintained by the storage manager. The storage manager maintains the first data block usage information separate from second data block usage information managed by the storage controller.
TL;DR: In this paper, a system for configuring data synchronization between two devices is presented, where the user interface elicits information from a user regarding a personal information manager (PIM) that is installed on the second device.
Abstract: A system is provided for configuring data synchronization between two devices. The first device generates a user configuration interface to be presented on the second device. The user interface elicits information from a user regarding a personal information manager (PIM) that is installed on the second device. This information is transmitted to the first device, which downloads to the second device a synchronization application package based on the PIM. The user interface installs the synchronization application on the second device, obtains information regarding data to be synchronized, and transmits that information to the synchronization application.
TL;DR: Several platforms for embedded systems, including microcontrollers, microprocessors, field-programmable gate arrays, digital signal processors, and application-specific integrated circuits are discussed and compared and examples of real-life design decisions specific to development of such systems are presented.
Abstract: This paper presents a survey on embedded systems design and applications. Several platforms for embedded systems, including microcontrollers, microprocessors, field-programmable gate arrays, digital signal processors, and application-specific integrated circuits are discussed and compared. A survey of embedded system-based industrial applications is presented. Examples of real-life design decisions specific to development of such systems are also presented. The carefully selected three design case study examples include industrial control of wind tunnel with emphasis on actuator control, a mobile robot navigation system with emphasis on integration and synchronization of several subsystems, and optimized implementation of computationally intensive control system on a small microcontroller system.
TL;DR: In this paper, the system and methods for synchronizing workspaces in a web-based collaboration environment with local folders on computers of collaborators of the workspaces are disclosed, and conflicts between edits made by different collaborators to the same file are flagged and a notification is sent to the author attempting to make edits to a version of a file that has already been updated by another collaborator.
Abstract: Systems and methods for synchronizing workspaces in a web-based collaboration environment with local folders on computers of collaborators of the workspaces are disclosed. Collaborators of a workspace can edit copies of work items either at the collaboration environment server or at the computer. Updates to all synchronized folders are automatically performed. Conflicts between edits made by different collaborators to the same file are flagged, and a notification is sent to the author attempting to make edits to a version of a file that has already been updated by another collaborator.
TL;DR: In this article, a client device with a display and in a method for displaying messages at the client device receives message information from a server system, the message information representing a set of messages.
Abstract: At a client device with a display and in a method for displaying messages at the client device, the client device receives message information from a server system, the message information representing a set of messages. In accordance with a determination that the set of messages include one or more unread priority messages, where priority messages are messages that satisfy predefined message importance criteria, the device presents a new mail notification. In accordance with a determination that the set of messages do not include any unread priority messages, the device foregoes presenting a new mail notification.
TL;DR: The sampled-data synchronization control problem is investigated for a class of general complex networks with time-varying coupling delays and the desired sampled- data feedback controllers are designed in terms of the solution to certain linear matrix inequalities (LMIs) that can be solved effectively by using available software.
TL;DR: New delay-dependent synchronization criteria are established to not only guarantee stable synchronization of both transmitter and receiver systems but also recover the transmitted original signal at the receiver.
Abstract: In this paper, a synchronization method of Lur’e systems for chaotic secure communication systems with interval time-varying delay feedback control is proposed. To increase communication security, the transmitted message is encrypted with the techniques of N-shift cipher and public key. Based on Lyapunov method and linear matrix inequality (LMI) formulation, new delay-dependent synchronization criteria are established to not only guarantee stable synchronization of both transmitter and receiver systems but also recover the transmitted original signal at the receiver. Throughout a numerical example, the validity and superiority of the proposed method are shown.
TL;DR: This work uses a high-gain methodology to construct linear decentralized controllers for consensus, in networks with identical but general multi-input linear time-invariant (LTI) agents and quitegeneral time- Invariant and time-varying observation topologies.
Abstract: We use a high-gain methodology to construct linear decentralized controllers for consensus, in networks with identical but general multi-input linear time-invariant (LTI) agents and quitegeneral time-invariant and time-varying observation topologies.
TL;DR: A new formalism is given for read-modify-write (RMW) synchronization operations, used to extend the memory reference combining mechanism introduced in the NYU Ultracomputer, to arbitrary RMW operations.
Abstract: A new formalism is given for read-modify-write (RMW) synchronization operations. This formalism is used to extend the memory reference combining mechanism introduced in the NYU Ultracomputer, to arbitrary RMW operations. A formal correctness proof of this combining mechanism is given. General requirements for the practicality of combining are discussed. Combining is shown to be practical for many useful memory access operations. This includes memory updates of the form mem_val := mem_val op val, where op need not be associative, and a variety of synchronization primitives. The computation involved is shown to be closely related to parallel prefix evaluation.
TL;DR: In this paper, the authors consider synchronization of coupled dynamical systems when different types of interactions are simultaneously present, and they show that the linear stability problem can be reduced in a master stability function form.
Abstract: We consider synchronization of coupled dynamical systems when different types of interactions are simultaneously present. We assume that a set of dynamical systems are coupled through the connections of two or more distinct networks (each of which corresponds to a distinct type of interaction), and we refer to such a system as a hypernetwork. Applications include neural networks formed of both electrical gap junctions and chemical synapses, the coordinated motion of shoals of fishes communicating through both vision and flow sensing, and hypernetworks of coupled chaotic oscillators. We first analyze the case of a hypernetwork formed of $m=2$ networks. We look for necessary and sufficient conditions for synchronization. We attempt at reducing the linear stability problem in a master stability function form, i.e., at decoupling the effects of the coupling functions from the structure of the networks. Unfortunately, we are unable to obtain a reduction in a master stability function form for the general case. However, we show that such a reduction is possible in three cases of interest: (i) the Laplacian matrices associated with the two networks commute; (ii) one of the two networks is unweighted and fully connected; (iii) one of the two networks is such that the coupling strength from node $i$ to node $j$ is a function of $j$ but not of $i$. Furthermore, we define a class of networks such that if either one of the two coupling networks belongs to this class, the reduction can be obtained independently of the other network. As an example of interest, we study synchronization of a neural hypernetwork for which the connections can be either chemical synapses or electrical gap junctions. We propose a generalization of our stability results to the case of hypernetworks formed of $m\geq 2$ networks.
TL;DR: In this article, the synchronization of application state between one or more instances of an application on multiple devices is discussed, such that a user can configure a new email account within a first instance of an email application on a laptop device.
Abstract: One or more techniques and/or systems are provided for synchronizing application state between one or more instances of an application on one or more devices. In particular, an application state, such as application settings, may be synchronized between a first instance of an application on a first device and a second instance of the application on a second device. For example, a user may configure a new email account within a first instance of an email application on a laptop device. The new email account may be synchronized with a second instance of the email application on a tablet device. During synchronization, various conflict resolution, version verifications, and/or consistency verifications may be performed. In this way, the user may be provided with a seamless and consistent experience between the laptop and tablet device.
TL;DR: The robot’s mechanism and motion-control is discussed, which uses physics simulation and animation principles to achieve both expressivity and safety, and an interactive improvisation system based on the notion of physical gestures for both musical and visual expression is presented.
Abstract: Shimon is a interactive robotic marimba player, developed as part of our ongoing research in Robotic Musicianship. The robot listens to a human musician and continuously adapts its improvisation and choreography, while playing simultaneously with the human. We discuss the robot's mechanism and motion-control, which uses physics simulation and animation principles to achieve both expressivity and safety. We then present an interactive improvisation system based on the notion of physical gestures for both musical and visual expression. The system also uses anticipatory action to enable real-time improvised synchronization with the human player.
We describe a study evaluating the effect of embodiment on one of our improvisation modules: antiphony, a call-and-response musical synchronization task. We conducted a 3×2 within-subject study manipulating the level of embodiment, and the accuracy of the robot's response. Our findings indicate that synchronization is aided by visual contact when uncertainty is high, but that pianists can resort to internal rhythmic coordination in more predictable settings. We find that visual coordination is more effective for synchronization in slow sequences; and that occluded physical presence may be less effective than audio-only note generation.
Finally, we test the effects of visual contact and embodiment on audience appreciation. We find that visual contact in joint Jazz improvisation makes for a performance in which audiences rate the robot as playing better, more like a human, as more responsive, and as more inspired by the human. They also rate the duo as better synchronized, more coherent, communicating, and coordinated; and the human as more inspired and more responsive.
TL;DR: KILO TM is proposed, a novel hardware TM design for GPUs that scales to 1000s of concurrent transactions that uses word-level, value-based conflict detection to avoid broadcast communication and reduce on-chip storage overhead.
Abstract: Graphics processor units (GPUs) are designed to efficiently exploit thread level parallelism (TLP), multiplexing execution of 1000s of concurrent threads on a relatively smaller set of single-instruction, multiple-thread (SIMT) cores to hide various long latency operations. While threads within a CUDA block/OpenCL workgroup can communicate efficiently through an intra-core scratchpad memory, threads in different blocks can only communicate via global memory accesses. Programmers wishing to exploit such communication have to consider data-races that may occur when multiple threads modify the same memory location. Recent GPUs provide a form of inter-block communication through atomic operations for single 32-bit/64-bit words. Although fine-grained locks can be constructed from these atomic operations, synchronization using locks is prone to deadlock. In this paper, we propose to solve these problems by extending GPUs to support transactional memory (TM). Major challenges include supporting 1000s of concurrent transactions and committing non-conflicting transactions in parallel. We propose KILO TM, a novel hardware TM design for GPUs that scales to 1000s of concurrent transactions. Without cache coherency hardware to depend on, it uses word-level, value-based conflict detection to avoid broadcast communication and reduce on-chip storage overhead. It employs speculative validation using a novel bloom filter organization to increase transaction commit parallelism. For a set of TM-enhanced GPU applications, KILO TM captures 59% of the performance of fine-grained locking, and is on average 128x faster than executing all transactions serially, for an estimated hardware area overhead of 0.5% of a commercial GPU.
TL;DR: In this paper, a determination is made at a first computing device of whether there is a synchronization conflict between a change to a first synchronized item at the first computer and a change made by a second computer at the second computer.
Abstract: A determination is made at a first computing device of whether there is a synchronization conflict between a change to a first synchronized item at the first computing device and a change to a second synchronized item at a second computing device. If a synchronization conflict exists, rule-based conflict resolution is performed to automatically resolve the synchronization conflict. The rule-based conflict resolution is based at least in part on a type of the change made to the first synchronized item, a type of the change made to the second synchronized item, and a set of multiple rules. A notification of the resolution is then sent to the second computing device.
TL;DR: This paper mainly investigates the lag synchronization of complex networks via pinning control without assuming the symmetry and irreducibility of the coupling matrix, sufficient conditions of lag synchronization are obtained by adding controllers to a part of nodes.
Abstract: This paper mainly investigates the lag synchronization of complex networks via pinning control. Without assuming the symmetry and irreducibility of the coupling matrix, sufficient conditions of lag synchronization are obtained by adding controllers to a part of nodes. Particularly, the following two questions are solved: (1) How many controllers are needed to pin a coupled complex network to a homogeneous solution? (2) How should we distribute these controllers? Finally, a simple example is provided to demonstrate the effectiveness of the theory.
TL;DR: A class of Cohen-Grossberg neural networks with time-varying delays are studied by designing a periodically intermittent controller and some novel and effective exponential synchronization criteria are derived by applying some analysis techniques.
TL;DR: In this paper, the uplink synchronization establishment in a base station which operates a plurality of component carriers according to one embodiment of the present description, is performed in that the base station is connected to a user equipment.
Abstract: Uplink synchronization establishment in a base station which operates a plurality of component carriers according to one embodiment of the present description, is performed in that the base station is connected to a user equipment, sets component carrier aggregation information, generates an uplink timing groups in the set component carrier aggregation, and transmits information on the thus-generated uplink timing groups to the user equipment.
TL;DR: In this paper, cluster synchronization problem is studied for an array of coupled stochastic delayed neural networks by using pinning control strategy and two specific methods are given to achieve desired cluster synchronization pattern.
TL;DR: With the hypothesis that the coupling network topology contains a directed spanning tree and each of the strongly connected components is detail-balanced, it is proved that finite-time synchronization can be achieved via pinning control.
Abstract: This paper investigates finite-time synchronization of an array of coupled neural networks via discontinuous controllers. Based on Lyapunov function method and the discontinuous version of finite-time stability theory, some sufficient criteria for finite-time synchronization are obtained. Furthermore, we propose switched control and adaptive tuning parameter strategies in order to reduce the settling time. In addition, pinning control scheme via a single controller is also studied in this paper. With the hypothesis that the coupling network topology contains a directed spanning tree and each of the strongly connected components is detail-balanced, we prove that finite-time synchronization can be achieved via pinning control. Finally, some illustrative examples are given to show the validity of the theoretical results.