Abstract: In this technical note, we consider the cooperative output regulation of linear multi-agent systems. The overall system consists of two groups of subsystems. While the first group of subsystems can access the exogenous signal, the second cannot. As a result, the problem cannot be solved by the decentralized approach. By devising a distributed observer, we can solve the problem by a dynamic full information distributed control scheme. The problem can also be viewed as a generalization of some results of the leader-following consensus problem of multi-agent systems.

Abstract: In this paper, we consider the cooperative output regulation of linear multi-agent systems under switching network. The problem can be viewed as a generalization of the leader-following consensus problem of multi-agent systems. Due to the limited information exchanges of different subsystems, the problem cannot be solved by the decentralized approach and is not allowed to be solved by the centralized control. By devising a distributed observer network, we can solve the problem by both dynamic state feedback control and dynamic measurement output feedback control. As an application of our main result, we show that a special case of our results leads to the solution of the leader-following consensus problem of linear multi-agent systems.

Abstract: This paper addresses the problem of ensuring trustworthy computation in a linear consensus network. A solution to this problem is relevant for several tasks in multi-agent systems including motion coordination, clock synchronization, and cooperative estimation. In a linear consensus network, we allow for the presence of misbehaving agents, whose behavior deviate from the nominal consensus evolution. We model misbehaviors as unknown and unmeasurable inputs affecting the network, and we cast the misbehavior detection and identification problem into an unknown-input system theoretic framework. We consider two extreme cases of misbehaving agents, namely faulty (non-colluding) and malicious (Byzantine) agents. First, we characterize the set of inputs that allow misbehaving agents to affect the consensus network while remaining undetected and/or unidentified from certain observing agents. Second, we provide worst-case bounds for the number of concurrent faulty or malicious agents that can be detected and identified. Precisely, the consensus network needs to be 2k+1 (resp. k+1) connected for k malicious (resp. faulty) agents to be generically detectable and identifiable by every well behaving agent. Third, we quantify the effect of undetectable inputs on the final consensus value. Fourth, we design three algorithms to detect and identify misbehaving agents. The first and the second algorithm apply fault detection techniques, and affords complete detection and identification if global knowledge of the network is available to each agent, at a high computational cost. The third algorithm is designed to exploit the presence in the network of weakly interconnected subparts, and provides local detection and identification of misbehaving agents whose behavior deviates more than a threshold, which is quantified in terms of the interconnection structure.

Abstract: Reinforcement Learning was originally developed for Markov Decision Processes (MDPs). It allows a single agent to learn a policy that maximizes a possibly delayed reward signal in a stochastic stationary environment. It guarantees convergence to the optimal policy, provided that the agent can sufficiently experiment and the environment in which it is operating is Markovian. However, when multiple agents apply reinforcement learning in a shared environment, this might be beyond the MDP model. In such systems, the optimal policy of an agent depends not only on the environment, but on the policies of the other agents as well. These situations arise naturally in a variety of domains, such as: robotics, telecommunications, economics, distributed control, auctions, traffic light control, etc. In these domains multi-agent learning is used, either because of the complexity of the domain or because control is inherently decentralized. In such systems it is important that agents are capable of discovering good solutions to the problem at hand either by coordinating with other learners or by competing with them. This chapter focuses on the application reinforcement learning techniques in multi-agent systems. We describe a basic learning framework based on the economic research into game theory, and illustrate the additional complexity that arises in such systems. We also described a representative selection of algorithms for the different areas of multi-agent reinforcement learning research.

Abstract: Smart and energy-efficient buildings have recently become a trend for future building industry. The major challenge in the control system design for such a building is to minimize the power consumption without compromising the customers comfort. For this purpose, a hierarchical multiagent control system with an intelligent optimizer is proposed in this study. Four types of agents, which are switch agent, central coordinator-agent, local controller-agent, and load agent, cooperate with each other to achieve the overall control goals. Particle swarm optimization (PSO) is utilized to optimize the overall system and enhance the intelligence of the integrated building and microgrid system. A Graphical User Interface (GUI) based platform is designed for customers to input their preferences and monitor the results. Two sets of case studies are carried out and corresponding simulation results are presented in this paper.

Abstract: In this paper, an agent based intelligent energy management system is proposed to facilitate power trading among microgrids and allow customers to participate in demand response. The proposed intelligence uses demand response, and diversity in electricity consumption patterns of the customers and availability of power from distributed generators as the vital means in managing power in the system. A new priority index is proposed for customers participating in the market based on frequency and size of load participating in demand response. In order to validate the proposed method, a case study with two interconnected microgrids is simulated. Based on extensive simulation results of the system developed using Java Agent DEvelopment framework (JADE), it has been found that multi-agent based demand response is successful in reducing the system peak in addition to cost benefit for the customers with high priority index.

Abstract: In this paper we describe a verification system for multi-agent programs. This is the first comprehensive approach to the verification of programs developed using programming languages based on the BDI (belief-desire-intention) model of agency. In particular, we have developed a specific layer of abstraction, sitting between the underlying verification system and the agent programming language, that maps the semantics of agent programs into the relevant model-checking framework. Crucially, this abstraction layer is both flexible and extensible; not only can a variety of different agent programming languages be implemented and verified, but even heterogeneous multi-agent programs can be captured semantically. In addition to describing this layer, and the semantic mapping inherent within it, we describe how the underlying model-checker is driven and how agent properties are checked. We also present several examples showing how the system can be used. As this is the first system of its kind, it is relatively slow, so we also indicate further work that needs to be tackled to improve performance.

Abstract: A decentralized control method is developed to enable a group of agents to achieve a desired global configuration while maintaining global network connectivity and avoiding obstacles, using only local feedback and no radio communication between the agents for navigation. By modeling the interaction among the agents as a graph, and given a connected initial graph with a desired neighborhood between agents, the developed method ensures the desired communication links remain connected for all time. To guide the agents to a desired configuration while avoiding obstacles, a decentralized controller is developed based on the navigation function formalism. By proving that the proposed controller is a qualified navigation function, convergence to the desired formation is guaranteed.

Abstract: This paper proposes a fully distributed multiagent-based reinforcement learning method for optimal reactive power dispatch. According to the method, two agents communicate with each other only if their corresponding buses are electrically coupled. The global rewards that are required for learning are obtained with a consensus-based global information discovery algorithm, which has been demonstrated to be efficient and reliable. Based on the discovered global rewards, a distributed Q-learning algorithm is implemented to minimize the active power loss while satisfying operational constraints. The proposed method does not require accurate system model and can learn from scratch. Simulation studies with power systems of different sizes show that the method is very computationally efficient and able to provide near-optimal solutions. It can be observed that prior knowledge can significantly speed up the learning process and decrease the occurrences of undesirable disturbances. The proposed method has good potential for online implementation.

Abstract: This article addresses the consensus problem for cooperative multiple agents with nonlinear dynamics on a fixed directed information network, where each agent can only communicate with its neighbours intermittently. A class of control algorithms is first introduced, using only intermittent relative local information. By combining tools from switching systems and Lyapunov stability theory, some sufficient conditions are established for consensus of multi-agent systems without any external disturbances under a fixed strongly connected topology. Theoretical analyses are further provided for consensus of multi-agent systems in the presence of external disturbances. It is shown that a finite ℒ2-gain performance index for the closed-loop multi-agent systems can be guaranteed if the coupling strength of the network is larger than a threshold value determined by the average communication rate and the generalised algebraic connectivity of the strongly connected topology. The results are then extended to consensus ...

Abstract: In this paper, distributed games for large-population multiagent systems with random time-varying parameters are investigated, where the agents are coupled via their individual costs and the structure parameters are a family of independent Markov chains with identical generators. The cost function of each agent is a long-run average tracking-type functional with an unknown mean field coupling nonlinear term as “reference signal.” To reduce the computational complexity, the mean field approach is applied to construct distributed strategies. The population statistics effect (PSE) is used to approximate the average effect of all the agents, and the distributed strategies are given through solving a Markov jump tracking problem. Here the PSE is a deterministic quantity and can be obtained by solving the Stackelberg equilibrium of an auxiliary two-player game. It is shown that the closed-loop system is uniformly stable, and the distributed strategies are asymptotically optimal in the sense of Nash equilibrium,...

Abstract: In real manufacturing environments, the control of system elements such as automated guided vehicles has some difficulties when planning operations dynamically. Multi agent-based systems, a newly maturing area of distributed artificial intelligence, provide some effective mechanisms for the management of such dynamic operations in manufacturing environments. This paper proposes a multi-agent based scheduling approach for automated guided vehicles and machines within a manufacturing system. The proposed multi-agent based approach works under a real-time environment and generates feasible schedules using negotiation/bidding mechanisms between agents. This approach is tested on off-line scheduling problems from the literature. The results show that our approach is capable of generating good schedules in real time comparable with the optimization algorithms and the frequently used dispatching rules.

Abstract: We propose an optimal control framework for persistent monitoring problems where the objective is to control the movement of multiple cooperating agents to minimize an uncertainty metric in a given mission space. In a one-dimensional mission space, we show that the optimal solution is for each agent to move at maximal speed from one switching point to the next, possibly waiting some time at each point before reversing its direction. Thus, the solution is reduced to a simpler parametric optimization problem: determining a sequence of switching locations and associated waiting times at these switching points for each agent. This amounts to a hybrid system which we analyze using Infinitesimal Perturbation Analysis (IPA) to obtain a complete on-line solution through a gradient-based algorithm. We also show that the solution is robust with respect to the uncertainty model used. This establishes the basis for extending this approach to a two-dimensional mission space.

Abstract: Recommender systems in e-Tourism normally focus on helping tourists to select appropriate destinations. A related problem that has been less explored in the literature is how to provide personalised recommendations of cultural and leisure activities when the tourist has already arrived at the destination. This paper presents a novel recommendation system, [email protected], which addresses this issue. Its agent-based modular design permits to model different kinds of activities in a flexible way, and allows the implementation of a location-aware front-end in the mobile device of the user. Special care has been put in the recommendation engine, implemented via a specialised Recommender Agent. It incorporates a mixture of content-based and collaborative recommendation strategies, thus avoiding the drawbacks of each individual method, and is able to perform recommendations in heterogeneous scenarios. Recommendations take into account user profiles which are implicitly updated after the analysis of user actions (e.g., queries, evaluations). The system has been successfully deployed and tested in the World Heritage-listed city of Tarragona.

Abstract: This article considers the distributed robust control problems of uncertain linear multi-agent systems with undirected communication topologies. It is assumed that the agents have identical nominal dynamics while subject to different norm-bounded parameter uncertainties, leading to weakly heterogeneous multi-agent systems. Distributed controllers are designed for both continuous- and discrete-time multi-agent systems, based on the relative states of neighbouring agents and a subset of absolute states of the agents. It is shown for both the continuous- and discrete-time cases that the distributed robust control problems under such controllers in the sense of quadratic stability are equivalent to the H ∞ control problems of a set of decoupled linear systems having the same dimensions as a single agent. A two-step algorithm is presented to construct the distributed controller for the continuous-time case, which does not involve any conservatism and meanwhile decouples the feedback gain design from the commun...

Abstract: This paper considers the sampled-data average consensus problem for multi-agent systems with first order continuous dynamics. The communication channels among the agents are constrained in which the exchanged information is digital rather than analogue. In this paper, the logarithmic quantizer is applied to the communication channels. A distributed consensus protocol is proposed based on sampled measurements. It is proved that as long as the quantization levels are dense enough, the proposed protocol is robust to the logarithmic quantization, i.e. all the states of the agents are uniformly bounded and the gap between the state of each agent and the average value of the initial conditions converges to zero as the density of quantization levels goes to infinity. An example is given to demonstrate the effectiveness of the protocol.

Abstract: How can an agent bootstrap up from a low-level representation to autonomously learn high-level states and actions using only domain-general knowledge? In this paper, we assume that the learning agent has a set of continuous variables describing the environment. There exist methods for learning models of the environment, and there also exist methods for planning. However, for autonomous learning, these methods have been used almost exclusively in discrete environments. We propose attacking the problem of learning high-level states and actions in continuous environments by using a qualitative representation to bridge the gap between continuous and discrete variable representations. In this approach, the agent begins with a broad discretization and initially can only tell if the value of each variable is increasing, decreasing, or remaining steady. The agent then simultaneously learns a qualitative representation (discretization) and a set of predictive models of the environment. These models are converted into plans to perform actions. The agent then uses those learned actions to explore the environment. The method is evaluated using a simulated robot with realistic physics. The robot is sitting at a table that contains a block and other distractor objects that are out of reach. The agent autonomously explores the environment without being given a task. After learning, the agent is given various tasks to determine if it learned the necessary states and actions to complete them. The results show that the agent was able to use this method to autonomously learn to perform the tasks.

Abstract: Distributed problem solving is a subfield within multiagent systems, where agents are assumed to be part of a team and collaborate with each other to reach a common goal. In this article, we illustrate the motivations for distributed problem solving and provide an overview of two distributed problem solving models, namely distributed constraint satisfaction problems (DCSPs) and distributed constraint optimization problems (DCOPs), and some of their algorithms.

Abstract: This paper describes the design of an agent-based modelling framework for high performance computing. Rather than a collection of methods that require parallel programming expertise the framework presented allows modellers to concentrate on the model while the framework handles the efficient execution of simulations. The framework uses a state machine based representation of agents that allows a statically calculated optimal ordering of agent execution and parallel communication routines. Some experiments with the current implementation and the results of using a simple communication dominant model for benchmarking performance are reported. The model with half a million agents is used to show that a parallel efficiency of above 80% is achievable when distributed over 432 processors. Future improvements are discussed including data dependency analysis, vector operations over agents, and dynamic task scheduling.

Abstract: In this chapter, we use the frequency-domain method to develop variable scalable consensus conditions for low-order agent systems with diverse communication and input delays. High-order consensus is also defined and investigated for a class of high-order heterogeneous agent systems.