We propose a fully distributed actor-critic architecture, named diffusion-distributed-actor-critic Diff-DAC, with application to multitask reinforcement learning (MRL). During the learning process, agents communicate their value and policy parameters to their neighbours, diffusing the information across a network of agents with no need for a central station. Each agent can only access data from its

Designing hierarchical reinforcement learning algorithms that exhibit safe behaviour is not only vital for practical applications but also facilitates a better understanding of an agent’s decisions. We tackle this problem in the options framework (Sutton, Precup & Singh, 1999), a particular way to specify temporally abstract actions which allow an agent to use sub-policies with start and end conditions

Argumentation and eXplainable Artificial Intelligence (XAI) are closely related, as in the recent years, Argumentation has been used for providing Explainability to AI. Argumentation can show step by step how an AI System reaches a decision; it can provide reasoning over uncertainty and can find solutions when conflicting information is faced. In this survey, we elaborate over the topics of Argumentation

Openness is a challenging property that may characterize multi-agent systems (MAS). It refers to their ability to deal with entities leaving and joining agent society over time. This property makes the MAS behaviour complex and difficult to study and analyze, hence the need for a representative model allowing its understanding. In this context, many models were defined in the literature and we propose

Statistical significance analysis, based on hypothesis tests, is a common approach for comparing classifiers. However, many studies oversimplify this analysis by simply checking the condition p-value < 0.05, ignoring important concepts such as the effect size and the statistical power of the test. This problem is so worrying that the American Statistical Association has taken a strong stand on the

Learning from human demonstration (LfD), among many speedup techniques for reinforcement learning (RL), has seen many successful applications. We consider one LfD technique called human–agent transfer (HAT), where a model of the human demonstrator’s decision function is induced via supervised learning and used as an initial bias for RL. Some recent work in LfD has investigated learning from observations

The aim of this article is to propose a model for the measurement of the strength of rhetorical arguments (i.e., threats, rewards, and appeals), which are used in persuasive negotiation dialogues when a proponent agent tries to convince his opponent to accept a proposal. Related articles propose a calculation based on the components of the rhetorical arguments, that is, the importance of the goal of

Multi-Robot System (MRS) is composed of a group of robots that work cooperatively. However, Multi-Agent System (MAS) is computational systems consisting of a group of agents that interact with each other to solve a problem. The central difference between MRS and MAS is that in the first case, the agent is a robot, and in the second, it is a software. Analyzing the scientific literature, it is possible

Simple ontology alignments, largely studied in the literature, link one single entity of a source ontology to one single entity of a target ontology. One of the limitations of these alignments is, however, their lack of expressiveness, which can be overcome by complex alignments, which are composed of correspondences involving logical constructors or transformation functions. While most work on complex

The quality of a dataset used for evaluating data linking methods, techniques, and tools depends on the availability of a set of mappings, called reference alignment, that is known to be correct. In particular, it is crucial that mappings effectively represent relations between pairs of entities that are indeed similar due to the fact that they denote the same object. Since the reliability of mappings

In multi-objective multi-agent systems (MOMASs), agents explicitly consider the possible trade-offs between conflicting objective functions. We argue that compromises between competing objectives in MOMAS should be analyzed on the basis of the utility that these compromises have for the users of a system, where an agent’s utility function maps their payoff vectors to scalar utility values. This utility-based

We are interested in understanding how socially desirable traits like sympathy, reciprocity, and fairness can survive in environments that include aggressive and exploitative agents. Social scientists have long theorized about ingrained motivational factors as explanations for departures from self-seeking behaviors by human subjects. Some of these factors, namely reciprocity, have also been studied

A Coordination Blockchain is a blockchain that coordinates activities of multiple private blockchains. This paper discusses the pros and cons of using Ethereum MainNet, the public Ethereum blockchain, as a Coordination Blockchain. The requirements Ethereum MainNet needs to fulfil to perform this role are analyzed within the context of Ethereum Private Sidechains, a private blockchain technology which

First-order expressive capabilities allow Bayesian networks (BNs) to model problem domains where the number of entities, their attributes, and their relationships can vary significantly between model instantiations. First-order BNs are well-suited for capturing knowledge representation dependencies, but literature on design patterns specific to first-order BNs is few and scattered. To identify useful

Trust between agents in multi-agent systems (MASs) is critical to encourage high levels of cooperation. Existing methods to assess trust and reputation use direct and indirect past experiences about an agent to estimate their future performance; however, these will not always be representative if agents change their behaviour over time. Real-world distributed networks such as online market places,

Blockchain is an emerging technology framework for creating and storing transaction in distributed ledgers with a high degree of security and reliability. In this paper, we present a blockchain-based platform to create and store contracts in between students and their higher education sponsors facilitated by intermediary brokers denoted as fundraisers. The sponsorship might be in any form, such as

Many real-world domains can benefit from adaptable decentralized task allocation through emergent specialization, especially in large teams of non-communicating agents. We begin with an existing bio-inspired response threshold reinforcement approach for decentralized task allocation and extend it to handle hierarchical task domains. We test the extension on self-deployment of a large team of non-communicating

Cloud robotics (CR) is currently a growing area in the robotic community. Indeed, the use of cloud computing to share data and resources of distributed robotic systems leads to the design and development of cloud robotic systems (CRS) which constitute useful technologies for a wide range of applications such as smart manufacturing, aid and rescue missions. However, in order to get coherent agent-to-cloud

In software architectures, architectural design decisions (ADDs) strongly influence the quality of the resulting software system. Wrong decisions lead to low-quality systems and are difficult to repair later on in the development process. As of today, little is known about the impact of certain ADDs for the development of architectures for blockchain-based systems. Thus, it is difficult to predict

The blockchain is an emerging technology which has the potential to improve many information systems. In this regard, the applications and the platform they are built on must be able to connect and communicate with each other. However, the current blockchain platforms have several limitations, such as lack of interoperability among different systems. The existing platforms of blockchain applications

The development of semi-automated and automated ontology alignment techniques is an important part of realizing the potential of the Semantic Web. Until very recently, most existing work in this area was focused on finding simple (1:1) equivalence correspondences between two ontologies. However, many real-world ontology pairs involve correspondences that contain multiple entities from each ontology

A humanoid robot developed to play multievent athletes like human has paved a way for interesting and popular robotics research. One of the great dreams is to develop a humanoid robot being able to challenge human athletes. Therefore, the challenge of humanoid robots to play archery against human is organized at Taichung, Taiwan, in HuroCup, FIRA 2018, on August 7th. The difficulties of developing

The software and hardware applications are clearly on the way of becoming an integral tool of business, communication and popular culture in many parts of the world. People are interacting with the environment via the Internet to perform physical activities remotely. These applications are hosted in the public or private servers under the control of the server admin. The users’ online usage data can

The Internet of Things (IoT) has recently emerged as an innovative technology capable of empowering various areas such as healthcare, agriculture, smart cities, smart homes and supply chain with real-time and state-of-the-art sensing capabilities. Due to the underlying potential of this technology, it already saw exponential growth in a wide variety of use-cases in multiple application domains. As

Blockchain technology has rapidly emerged as a decentralized trusted network to replace the traditional centralized intermediator. Especially, the smart contracts that are based on blockchain allow users to define the agreed behaviour among them, the execution of which will be enforced by the smart contracts. Based on this, we propose a decentralized booking system that uses the blockchain as the intermediator

Ontology matching aims at discovering mappings between the entities of two ontologies. It plays an important role in the integration of heterogeneous data sources that are described by ontologies. Interactive ontology matching involves domain experts in the matching process. In some approaches, the expert provides feedback about mappings between ontology entities, that is, these approaches select mappings

Nakamoto proposed a new solution to transact value via the internet. And since 2009, blockchain technology has expanded and diversified. It has, however, proven to be inefficient in the way it achieves its outcomes, especially through the proof of work protocol. Other developers are promoting alternative methods but, as yet, none has superseded proof of work. The competing protocols illuminate a key

The popularity of smart cars is increasing around the world as they offer a wide range of services and conveniences. These smart cars are equipped with a variety of sensors generating a large amount of data, many of which are critical. Besides, there are multiple parties involved in the lifespan of a smart car, such as manufacturers, car owners, government agencies, and third-party service providers

The aim of this critical review paper is threefold: (a) to provide an insight on the impact of ontology engineering methodologies (OEMs) to the evolution of living and reused ontologies, (b) to update the ontology engineering (OE) community on the status and trends in OEMs and of their use in practice and (c) to propose a set of recommendations for working ontologists to consider during the life cycle

Within the next decades, robots will need to be able to execute a large variety of tasks autonomously in a large variety of environments. To relax the resulting programming effort, a knowledge-enabled approach to robot programming can be adopted to organize information in re-usable knowledge pieces. However, for the ease of reuse, there needs to be an agreement on the meaning of terms. A common approach

This paper presents an approach for the enactment of policies in digital health based on our earlier work on the implementation of digital contracts in distributed systems. A formal policy model and an abstract policy language for the expression of healthcare policies are first proposed, leveraging the semantics of the ISO Reference Model for Open Distributed Processing enterprise language standard

There is an implicit assumption in machine learning techniques that each new task has no relation to the tasks previously learned. Therefore, tasks are often addressed independently. However, in some domains, particularly reinforcement learning (RL), this assumption is often incorrect because tasks in the same or similar domain tend to be related. In other words, even though tasks are quite different

The problem of traffic congestion incurs numerous social and economical repercussions and has thus become a central issue in every major city in the world. For this work we look at the transportation domain from a multiagent system perspective, where every driver can be seen as an autonomous decision-making agent. We explore how learning approaches can help achieve an efficient outcome, even when agents

Biodiversity research studies the variability and diversity of organisms, including variability within and between species with particular focus on the functional diversity of traits and their relationship to environment. Managing biodiversity data implies dealing with its heterogeneous nature using semantics and tailored ontologies. These are themselves differently conceived, and combining them in

This research demonstrates financial derivative trade of unprocessed materials, for the mining industry through legal smart contracts. Within the mining supply chain, a stock of mined resources can reside in a mineral stockpile for over twenty years without gaining financial interest and without undergoing the mineral extraction process to derive value from the asset. This research elaborates on a

Recent progress in reinforcement learning (RL) using self-play has shown remarkable performance with several board games (e.g., Chess and Go) and video games (e.g., Atari games and Dota2). It is plausible to hypothesize that RL, starting from zero knowledge, might be able to gradually approach a winning strategy after a certain amount of training. In this paper, we explore neural Monte Carlo Tree Search

Loyalty cards programs have been used by retailers to increase customer retention. Loyality cards provide means to identify a particular customer and to collect customer-specific data, thus enabling individualized marketing; however, operating a loyalty program is complicated for retailers since they require to manage balances, collections, and transfers of customers. This is exactly the same problem

Ontology alignment is an important and inescapable problem for the interconnections of two ontologies stating the same concepts. Ontology alignment evaluation initiative (OAEI) has been taken place for more than a decade to monitor and help the progress of the field and to compare systematically existing alignment systems. As of 2018, the evaluation of systems is partly transitioned to the HOBBIT platform

Distributed ledger technology (DLT) is regarded as a revolutionary solution that offers immutability, transparency, trust, and efficiency while ‘transcending law and regulation’. One of the potential applications of DLT is in the securities market. Share registration, settlement, regulatory compliance, information disclosure, payment systems, and market service requirements can be redesigned with the

The use of robots in performance arts is increasing. But, it is hard for robots to cope with unexpected circumstances during a performance, and it is almost impossible for robots to act fully autonomously in such situations. IROS-HAC is a new challenge in robotics research and a new opportunity for cross-disciplinary collaborative research. In this paper, we describe a practical method for generating

Network providers either attempt to handle massive distributed denial-of-service attacks themselves or redirect traffic to third-party scrubbing centers. If providers adopt the first option, it is sensible to counter such attacks in their infancy via provider collaborations deploying distributed security mechanisms across multiple domains in an attack path. This motivated our work presented in this

A large variety of issues influence the success of data mining on a given problem. Two primary and important issues are the representation and the quality of the dataset. Specifically, if much redundant and unrelated or noisy and unreliable information is presented, then knowledge discovery becomes a very difficult problem. It is well-known that data preparation steps require significant processing

Research on Multi-Agent Systems (MAS) has led to the development of several models, languages, and technologies for programming not only agents, but also their interaction, the application environment where they are situated, as well as the organization in which they participate. Research on those topics moved from agent-oriented programming towards multi-agent-oriented programming (MAOP). A MAS program

None

Recently, valuable knowledge that can be retrieved from a huge volume of datasets (called Big Data) set in motion the development of frameworks to process data based on parallel and distributed computing, including Apache Hadoop, Facebook Corona, and Microsoft Dryad. Apache Hadoop is an open source implementation of Google MapReduce that attracted strong attention from the research community both in

Generalized planning studies the representation, computation and evaluation of solutions that are valid for multiple planning instances. These are topics studied since the early days of AI. However, in recent years, we are experiencing the appearance of novel formalisms to compactly represent generalized planning tasks, the solutions to these tasks (called generalized plans) and efficient algorithms

In this paper, we describe how predictive models can be positively exploited in abstract argumentation. In particular, we present two main sets of results. On one side, we show that predictive models are effective for performing algorithm selection in order to determine which approach is better to enumerate the preferred extensions of a given argumentation framework. On the other side, we show that

Social networks occupy a ubiquitous and pervasive place in the life of their users. The substantial amount of content generated and shared by social networking users offers new research opportunities across a wide variety of disciplines, including media and communication studies, linguistics, sociology, psychology, information and computer sciences, or education. This situation, in combination with

One challenge faced by reinforcement learning (RL) agents is that in many environments the reward signal is sparse, leading to slow improvement of the agent’s performance in early learning episodes. Potential-based reward shaping can help to resolve the aforementioned issue of sparse reward by incorporating an expert’s domain knowledge into the learning through a potential function. Past work on reinforcement

An increase in volumes of data and a shift towards live data enabled a stronger focus on resource-intensive tasks which run continuously over long periods. A Grid has potential to offer the required resources for these tasks, while considering a fair and balanced allocation of resources among multiple client agents. Taking this into account, a Grid might be unwilling to allocate its resources for long

Deep reinforcement learning (deep RL) has achieved superior performance in complex sequential tasks by using deep neural networks as function approximators to learn directly from raw input images. However, learning directly from raw images is data inefficient. The agent must learn feature representation of complex states in addition to learning a policy. As a result, deep RL typically suffers from

In the reinforcement learning context, a landmark is a compact information which uniquely couples a state, for problems with hidden states. Landmarks are shown to support finding good memoryless policies for Partially Observable Markov Decision Processes (POMDP) which contain at least one landmark. SarsaLandmark, as an adaptation of Sarsa(λ), is known to promise a better learning performance with the

Among an array of techniques proposed to speed-up reinforcement learning (RL), learning from human demonstration has a proven record of success. A related technique, called Human-Agent Transfer, and its confidence-based derivatives have been successfully applied to single-agent RL. This article investigates their application to collaborative multi-agent RL problems. We show that a first-cut extension

In order to enable robots to interact with humans in a natural way, they need to be able to autonomously learn new tasks. The most natural way for humans to tell another agent, which can be a human or robot, to perform a task is via natural language. Thus, natural human–robot interactions also require robots to understand natural language, i.e. extract the meaning of words and phrases. To do this,

One way to address this low sample efficiency of reinforcement learning (RL) is to employ human expert demonstrations to speed up the RL process (RL from demonstration or RLfD). The research so far has focused on demonstrations from a single expert. However, little attention has been given to the case where demonstrations are collected from multiple experts, whose expertise may vary on different aspects

User validation is one of the challenges facing the ontology alignment community, as there are limits to the quality of the alignments produced by automated alignment algorithms. In this paper, we present a broad study on user validation of ontology alignments that encompasses three distinct but inter-related aspects: the profile of the user, the services of the alignment system, and its user interface

In this study, we designed a localization and obstacle avoidance system for humanoid robots in the Federation of International Robot-soccer Association (FIRA) HuroCup united soccer competition event. The localization is implemented by using grid points, gait, and steps to determine the positions of each robot. To increase the localization accuracy and eliminate the accumulated distance errors resulting

The current fourth industrial revolution, or ‘Industry 4.0’ (I4.0), is driven by digital data, connectivity, and cyber systems, and it has the potential to create impressive/new business opportunities. With the arrival of I4.0, the scenario of various intelligent systems interacting reliably and securely with each other becomes a reality which technical systems need to address. One major aspect of

This paper presents our sketch drawing artist humanoid robot research. One of the limitations of the existing artist humanoid robot is the lack of feedback on the error that occurs during the drawing process. The contribution of this research is the development of a humanoid robot artist with drawing error correction capability. Based on our previous work with open-loop control pen-and-ink humanoid

The team-orienteering problem (TOP) has broad applicability. Examples of possible uses are in factory and automation settings, robot sports teams, and urban search and rescue applications. We chose the rescue domain as a guiding example throughout this paper. Hence, this paper explores a practical variant of TOP with time window (TOPTW) for rescue applications by humanoid robots called TOPTWR. Due