In ant societies, and, more in general, in insect societies, the activities of the individuals, as well as of the society as a whole, are not regulated by any explicit form of centralized control On the other hand, adaptive and robust behaviors transcending the behavioral repertoire of the single individual can be easily observed at society level These complex global behaviors are the result of self-organizing dynamics driven by local interactions and communications among a number of relatively simple individualsThe simultaneous presence of these and other fascinating and unique characteristics have made ant societies an attractive and inspiring model for building new algorithms and new multi-agent systems In the last decade, ant societies have been taken as a reference for an ever growing body of scientific work, mostly in the fields of robotics, operations research, and telecommunicationsAmong the different works inspired by ant colonies, the Ant Colony Optimization metaheuristic (ACO) is probably the most successful and popular one The ACO metaheuristic is a multi-agent framework for combinatorial optimization whose main components are: a set of ant-like agents, the use of memory and of stochastic decisions, and strategies of collective and distributed learningIt finds its roots in the experimental observation of a specific foraging behavior of some ant colonies that, under appropriate conditions, are able to select the shortest path among few possible paths connecting their nest to a food site The pheromone, a volatile chemical substance laid on the ground by the ants while walking and affecting in turn their moving decisions according to its local intensity, is the mediator of this behaviorAll the elements playing an essential role in the ant colony foraging behavior were understood, thoroughly reverse-engineered and put to work to solve problems of combinatorial optimization by Marco Dorigo and his co-workers at the beginning of the 1990'sFrom that moment on it has been a flourishing of new combinatorial optimization algorithms designed after the first algorithms of Dorigo's et al, and of related scientific eventsIn 1999 the ACO metaheuristic was defined by Dorigo, Di Caro and Gambardella with the purpose of providing a common framework for describing and analyzing all these algorithms inspired by the same ant colony behavior and by the same common process of reverse-engineering of this behavior Therefore, the ACO metaheuristic was defined a posteriori, as the result of a synthesis effort effectuated on the study of the characteristics of all these ant-inspired algorithms and on the abstraction of their common traitsThe ACO's synthesis was also motivated by the usually good performance shown by the algorithms (eg, for several important combinatorial problems like the quadratic assignment, vehicle routing and job shop scheduling, ACO implementations have outperformed state-of-the-art algorithms)The definition and study of the ACO metaheuristic is one of the two fundamental goals of the thesis The other one, strictly related to this former one, consists in the design, implementation, and testing of ACO instances for problems of adaptive routing in telecommunication networksThis thesis is an in-depth journey through the ACO metaheuristic, during which we have (re)defined ACO and tried to get a clear understanding of its potentialities, limits, and relationships with other frameworks and with its biological background The thesis takes into account all the developments that have followed the original 1999's definition, and provides a formal and comprehensive systematization of the subject, as well as an up-to-date and quite comprehensive review of current applications We have also identified in dynamic problems in telecommunication networks the most appropriate domain of application for the ACO ideas According to this understanding, in the most applicative part of the thesis we have focused on problems of adaptive routing in networks and we have developed and tested four new algorithmsAdopting an original point of view with respect to the way ACO was firstly defined (but maintaining full conceptual and terminological consistency), ACO is here defined and mainly discussed in the terms of sequential decision processes and Monte Carlo sampling and learningMore precisely, ACO is characterized as a policy search strategy aimed at learning the distributed parameters (called pheromone variables in accordance with the biological metaphor) of the stochastic decision policy which is used by so-called ant agents to generate solutions Each ant represents in practice an independent sequential decision process aimed at constructing a possibly feasible solution for the optimization problem at hand by using only information local to the decision stepAnts are repeatedly and concurrently generated in order to sample the solution set according to the current policy The outcomes of the generated solutions are used to partially evaluate the current policy, spot the most promising search areas, and update the policy parameters in order to possibly focus the search in those promising areas while keeping a satisfactory level of overall explorationThis way of looking at ACO has facilitated to disclose the strict relationships between ACO and other well-known frameworks, like dynamic programming, Markov and non-Markov decision processes, and reinforcement learning In turn, this has favored reasoning on the general properties of ACO in terms of amount of complete state information which is used by the ACO's ants to take optimized decisions and to encode in pheromone variables memory of both the decisions that belonged to the sampled solutions and their qualityThe ACO's biological context of inspiration is fully acknowledged in the thesis We report with extensive discussions on the shortest path behaviors of ant colonies and on the identification and analysis of the few nonlinear dynamics that are at the very core of self-organized behaviors in both the ants and other societal organizations We discuss these dynamics in the general framework of stigmergic modeling, based on asynchronous environment-mediated communication protocols, and (pheromone) variables priming coordinated responses of a number of ``cheap' and concurrent agentsThe second half of the thesis is devoted to the study of the application of ACO to problems of online routing in telecommunication networks This class of problems has been identified in the thesis as the most appropriate for the application of the multi-agent, distributed, and adaptive nature of the ACO architectureFour novel ACO algorithms for problems of adaptive routing in telecommunication networks are throughly described The four algorithms cover a wide spectrum of possible types of network: two of them deliver best-effort traffic in wired IP networks, one is intended for quality-of-service (QoS) traffic in ATM networks, and the fourth is for best-effort traffic in mobile ad hoc networksThe two algorithms for wired IP networks have been extensively tested by simulation studies and compared to state-of-the-art algorithms for a wide set of reference scenarios The algorithm for mobile ad hoc networks is still under development, but quite extensive results and comparisons with a popular state-of-the-art algorithm are reported No results are reported for the algorithm for QoS, which has not been fully tested The observed experimental performance is excellent, especially for the case of wired IP networks: our algorithms always perform comparably or much better than the state-of-the-art competitorsIn the thesis we try to understand the rationale behind the brilliant performance obtained and the good level of popularity reached by our algorithms More in general, we discuss the reasons of the general efficacy of the ACO approach for network routing problems compared to the characteristics of more classical approaches Moving further, we also informally define Ant Colony Routing (ACR), a multi-agent framework explicitly integrating learning components into the ACO's design in order to define a general and in a sense futuristic architecture for autonomic network controlMost of the material of the thesis comes from a re-elaboration of material co-authored and published in a number of books, journal papers, conference proceedings, and technical reports The detailed list of references is provided in the Introduction

Ant colony optimization and its application to adaptive routing in telecommunication networks

Recently, mobile networking systems have been designed with more complexity of infrastructure and higher diversity of associated devices and resources, as well as more dynamical formations of networks, due to the fast development of current Internet and mobile communication industry. In such emerging mobile heterogeneous networks (HetNets), there are a large number of technical challenges focusing on the efficient organization, management, maintenance, and optimization, over the complicated system resources. In particular, HetNets have attracted great interest from academia and industry in deploying more effective solutions based on artificial intelligence (AI) techniques, e.g., machine learning, bio-inspired algorithms, fuzzy neural network, and so on, because AI techniques can naturally handle the problems of large-scale complex systems, such as HetNets towards more intelligent and automatic-evolving ones. In this paper, we discuss the state-of-the-art AI-based techniques for evolving the smarter HetNets infrastructure and systems, focusing on the research issues of self-configuration, self-healing, and self-optimization, respectively. A detailed taxonomy of the related AI-based techniques of HetNets is also shown by discussing the pros and cons for various AI-based techniques for different problems in HetNets. Opening research issues and pending challenges are concluded as well, which can provide guidelines for future research work.

Artificial Intelligence-Based Techniques for Emerging Heterogeneous Network: State of the Arts, Opportunities, and Challenges

In vehicular sensor networks (VSNs), an increase in the density of the vehicles on road and route jamming in the network causes delay in receiving the emergency alerts, which results in overall system performance degradation. In order to address this issue in VSNs deployed in dense urban regions, in this paper, we propose collaborative learning automata-based routing algorithm for sending information to the intended destination with minimum delay and maximum throughput. The learning automata (LA) stationed at the nearest access points (APs) in the network learn from their past experience and make routing decisions quickly. The proposed strategy consists of dividing the whole region into different clusters, based on which an optimized path is selected using collaborative LA having input parameters as vehicle density, distance from the nearest service unit, and delay. A theoretical expression for density estimation is derived, which is used for the selection of the “best” path by LA. The performance of the proposed scheme is evaluated with respect to metrics such as packet delivery delay (network delay), packet delivery ratio with varying node (vehicle) speed, transmission range, density of vehicle, and number of road side units/APs). The results obtained show that the proposed scheme performs better than the benchmark chosen in this study, as there is a 30% reduction in network delay and a 20% increase in packet delivery ratio.

Collaborative Learning Automata-Based Routing for Rescue Operations in Dense Urban Regions Using Vehicular Sensor Networks

One important approach of multiagent reinforcement learning (MARL) is equilibrium-based MARL, which is a combination of reinforcement learning and game theory. Most existing algorithms involve computationally expensive calculation of mixed strategy equilibria and require agents to replicate the other agents’ value functions for equilibrium computing in each state. This is unrealistic since agents may not be willing to share such information due to privacy or safety concerns. This paper aims to develop novel and efficient MARL algorithms without the need for agents to share value functions. First, we adopt pure strategy equilibrium solution concepts instead of mixed strategy equilibria given that a mixed strategy equilibrium is often computationally expensive. In this paper, three types of pure strategy profiles are utilized as equilibrium solution concepts: pure strategy Nash equilibrium, equilibrium-dominating strategy profile, and nonstrict equilibrium-dominating strategy profile. The latter two solution concepts are strategy profiles from which agents can gain higher payoffs than one or more pure strategy Nash equilibria. Theoretical analysis shows that these strategy profiles are symmetric meta equilibria. Second, we propose a multistep negotiation process for finding pure strategy equilibria since value functions are not shared among agents. By putting these together, we propose a novel MARL algorithm called negotiation-based Q-learning (NegoQ). Experiments are first conducted in grid-world games, which are widely used to evaluate MARL algorithms. In these games, NegoQ learns equilibrium policies and runs significantly faster than existing MARL algorithms (correlated Q-learning and Nash Q-learning). Surprisingly, we find that NegoQ also performs well in team Markov games such as pursuit games, as compared with team-task-oriented MARL algorithms (such as friend Q-learning and distributed Q-learning).

Multiagent Reinforcement Learning With Unshared Value Functions

A system for determining the state-of-health of batteries has an adaptive component The system tests a battery by measuring a number of electrochemical parameters and using fuzzy logic to compute a state-of-health for the battery The fuzzy logic computations are based upon membership functions which are preferably triangular The membership functions may be stored in a matrix The adaptive component adapts the system for use with new models of battery by obtaining a reliable measurement of SoH for a number of calibration batteries Electrochemical parameters of the calibration batteries are measured The adaptive component modifies a prototype matrix selected to match the chemistry type of the calibration batteries by scaling, translating and flexing its membership functions to achieve a matrix which specifies membership functions suitable for assessing state-of-health of the new model of battery

Fuzzy logic method and apparatus for battery state of health determination

Several approaches based on Computational Intelligence techniques that develop efficient solutions to some of the most significant traffic control problems of High Speed Networks have been proposed so far in the literature. In this chapter, using the experience obtained from already published works that regard the use of Reinforcement Learning Algorithms (RLA) in ATM networks, we try to form a general framework for encountering this kind of problems using RLA, making some general observations and remarks about the factors that affect considerably their performance, as well as classifying both the problems and the proposed solutions. Although this framework is developed using specific proposed mechanisms and Reinforcement Learning Algorithm, it can give some general but efficient guidelines that can be used irrespective of the RLA that is employed in each specific case, resulting in a better coping with this kind of problems.

The Use of Reinforcement Learning Algorithms in Traffic Control of High Speed Networks

Research: On the use of a stochastic estimator learning algorithm to the ATM routing problem: a methodology

In this study, we are concerned with a modularization of fuzzy relational equations that is converting a highly dimensional (multivariable) relational equation into a series of single input fuzzy relational equations. The problem originates from a need of handling (estimating) highly dimensional relational structures and is inherently associated with the curse of dimensionality present in relational fuzzy models. We propose a two-layer architecture and discuss a detailed optimization scheme leading to the determination of the fuzzy relations occurring there. Illustrative numerical studies are also included.

Modularization of fuzzy relational equations

Computational intelligence techniques have been successfully used for solving control problems in packet-switching network architectures. The introduction of active networking adds a high degree of flexibility in customizing the network infrastructure and introducing new functionality. Therefore, there is a clear need for investigating both the applicability of computational intelligence techniques in this new networking environment, as well as the provisions of active networking technology that computational intelligence techniques can exploit for improved operation. We report on the characteristics of these technologies, their synergy and on outline recent efforts in the design of a computational intelligence toolkit and its application to routing on a novel active networking environment.

Application of computational intelligence techniques in active networks

In this chapter the use of a Reinforcement Learning Algorithm for optimizing the routing in Asynchronous Transfer Mode (ATM) networks based on the Private Network-to-Network Interface (PNNI) standard is proposed. This algorithm aims at maximizing the network throughput (allocating efficiently the network resources) while guaranteeing the Quality of Service (QoS) requirements for each connection. In this study, large-scale networks are considered where it becomes necessary to be organized hierarchically so that a scale in terms of computation, communication, and storage requirements will be achieved. A comparative performance study of the proposed and other well-known routing schemes is demonstrated by means of simulation on an existing commercial network. Simulation results over a wide range of uniform, time-varying, and skewed loading conditions show the effectiveness of the proposed routing algorithm, and disclose the strength and weakness of the various schemes.

QoS-Based Hierarchical Routing in ATM Networks Using Reinforcement Learning Algorithms: A Methodology

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Athanasios V. Vasilakos

Author Tools

Chat about Author