This paper proposes a visual teleoperation with human–robot posture-consistent based on deep neural network. A multi-stage structure of visual teleoperation network, in which the angles of robotic joints are obtained from human, is deduced. Furthermore, a novel human–robot posture-consistent mapping method is developed to generate dataset of the visual teleoperation network by solving constrained nonlinear

Novel applications of soft pneumatic actuation in minimally invasive surgery (MIS) are proposed due to its relatively safe robot–environment interactions. Although the inherent compliance of soft robots makes them suitable for surgery, their low force output and complicated system response and behavior may limit their potential as practical MIS instruments. In this paper, three lobster-inspired antagonistic

Vehicle tracking is an attractive problem in the field of public transportation with several research projects conducted using Kalman filter (KF) to tackle this. While a driver may act on his own decision, there exist parameters affecting his behavior so called situation assesment such as neighboring drivers, possible obstacles, or alternative routs changing over time. In this paper, utilizing online

A methodology for planning the sequence of tasks for a harvesting robot is presented. The fruit targets are situated at unknown locations and must be detected by the robot through a sequence of sensing tasks. Once the targets are detected, the robot must execute a harvest action at each target location. The traveling salesman paradigm (TSP) is used to plan the sequence of sensing and harvesting tasks

This paper is concentrated on path planning for robots working in a dynamic environment to satisfy real-time needs. An efficient bias-goal factor RRT (EBG-RRT), which is multiple-query sampling-based replanning algorithm, is proposed with rapid response and high success rate. Specifically, a relay node method is proposed to get a position where the robot and dynamic obstacles will be no-collision and

Teach and Repeat (T&R) refers to the technology that allows a robot to autonomously follow a previously traversed route, in a natural scene and using only its onboard sensors. In this paper we present a Visual-Inertial Teach and Repeat (VI-T&R) algorithm that uses stereo and inertial data and targets Unmanned Aerial Vehicles with limited on-board computational resources. We propose a tightly-coupled

Visual localization is a challenging problem, especially over the long run, since places can exhibit significant variation due to dynamic environmental and seasonal changes. To tackle this problem, we propose a visual place recognition method based on directed acyclic graph matching and feature maps extracted from deep convolutional neural networks (DCNN). Furthermore, in order to find the best subset

To achieve temporal and spatial correspondence between multiple robotic manipulators, the system must correctly analyze the coordinated path required for a specific task. Based on manipulator kinematics analysis, we first studied the kinematic constraints between the end-effectors of cooperative manipulators, and deduced the multi-manipulator cooperative kinematics constraint equations in the Cartesian

This paper addresses multi-robot task scheduling for two robot types arising from heterogeneous robotic order fulfillment systems. The heterogeneous multi-robot system comprises two types of robots with specialized and complementary capabilities to achieve long-cycle and multi-station order fulfillment tasks on a logistic network. This problem is extremely challenging because of innate complex-schedule

Autonomous navigation of mobile robot via classic neural network (NN) models are no more valid in terms of efficiency and accuracy due to the development of new advanced techniques. However, the necessity of finding an implementable Recursive Neural Network (RNN) model to predict the motor control of the robot with both speed and accuracy constraints still remains stagnant because of the nonlinearity

Automated image completion and masking have been emerged as a subject of keen interest due to their impact on image modification and interpretation. The current state-of-the-art approaches require a fixed format of missing parts and are ineffective for handling corrupted images. Besides, they focus exclusively on the image completion without taking into consideration the image masking as an inverse

The design of high-level decision-making systems is a topical problem in the field of autonomous driving. In this paper, we combine traditional rule-based strategies and reinforcement learning (RL) with the goal of achieving transparency and robustness. On the one hand, the use of handcrafted rule-based controllers allows for transparency, i.e., it is always possible to determine why a given decision

Mobile service robots possess high potential of providing numerous assistances in the working areas. In an attempt to develop a mobile service robot which is dynamically balanced for faster movement and taller manipulation capability, we designed and prototyped J4.alpha, which is intended for swift navigation and nimble manipulation. Previously, we devised a pure visual method based on a supervised

Optics-based systems may provide high spatial and temporal resolution for close range object detection in underwater environments. By using a monocular camera on a low cost underwater vehicle manipulator system, objects can be tracked by the vehicle and handled by the manipulator. In this paper, a monocular camera is used to detect an object of interest through object detection. Spatial features of

Fixed-Wing UAVs (Unmanned Aerial Vehicles) flocking is still a challenging problem due to the kinematics complexity and environmental dynamics. In this paper, we solve the leader–followers flocking problem using a novel deep reinforcement learning algorithm that can generate roll angle and velocity commands by training an end-to-end controller in continuous state and action spaces. Specifically, we

This work considers the problem of locating a single robot given a set of squared noisy range difference measurements to a set of points (anchors) whose positions are known. In the sequel, localization problem is solved in the Least-Squares (LS) sense by writing the robot position in polar/spherical coordinates. This representation transforms the original nonconvex/multimodal cost function into the

This paper presents a fully hardware synchronized mapping robot with support for a hardware synchronized external tracking system, for super-precise timing and localization. Nine high-resolution cameras and two 32-beam 3D Lidars were used along with a professional, static 3D scanner for ground truth map collection. With all the sensors calibrated on the mapping robot, three datasets are collected to

In a typical operation mode, a wheel loader frequently accelerates and decelerates, and the curvature of the driving path is inconsistent. In the past, autonomous vehicle trajectory planning has not considered the related changes in the velocity of the vehicle. Therefore, the trajectory tracking control process has seldom considered the impact of curving paths on the trajectory tracking performance

Due to its vast applicability, the semantic interpretation of regions or entities increasingly attracts the attention of scholars within the robotics community. The paper at hand introduces a novel unsupervised technique to semantically identify the position of an autonomous agent in unknown environments. When the robot explores a certain path for the first time, community detection is achieved through

Rapidly-exploring Randomized Trees (RRT) is a kind of probabilistically complete exploration algorithm based on the tree structure. It has been widely used in the robotic navigation since it guarantees the complete discovery and the exploration of environment maps through robots. In the present study, the RRT algorithm is extended to propose an optimization-based map exploration strategy for multiple

Robotic obstacle avoidance is an important issue in robotic navigation for unknown or partially known, dynamic environments. A good number of techniques have already been proposed to navigate obstacles in this kind of environment. They include a series of velocity space methods that have been successful implemented in several applications. They formulate the problem as one of constrained optimization

The interaction between robots and humans is of great relevance for the field of neurorobotics as it can provide insights on how humans perform motor control and sensor processing and on how it can be applied to robotics. We propose a spiking neural network (SNN) to trigger finger motion reflexes on a robotic hand based on human surface Electromyography (sEMG) data. The first part of the network takes

Place recognition is an essential component to address the problem of visual navigation and SLAM. The long-term place recognition is challenging as the environment exhibits significant variations across different times of the days, months, and seasons. In this paper, we view appearance changes as multiple domains and propose a Feature Disentanglement Network (FDNet) based on a convolutional auto-encoder

A fuel-efficient control strategy for a manipulator-equipped spacecraft is presented. The strategy uses the thrusters, the reaction wheels, and the arm drives in a coordinated way to limit the use of the thrusters and achieve ideally zero fuel consumption in contact-free maneuvering. The thrusters are activated automatically only after contact, to stabilize the inertial motion of the system. The controller

In human–robot cooperation, the information interaction plays a key role. Most of the information interaction rely on Border Gateway Protocol (BGP), which is a vital route protocol on networks. However, the BGP is susceptible to the prefix interception attacks because the rightful origin of each prefix cannot be verified in BGP. For this reason, we propose a novel and effective route selection method

This paper proposes a novel control system design for a two-wheeled service robot that follows a person as an assistant without knowing the person’s destination. For this kind of service robot, the key skill is to realize human-friendly movement. However, appropriate motion always changed depending on the situation. For instance, when the robot is close and person turns toward it, it is important to

This paper proposes a general approach to the problem of extrinsic calibration of multiple sensors of varied modalities. This is of particular relevance for intelligent vehicles, which are complex systems that often encompass several sensors of different modalities. Our approach is seamlessly integrated with the Robot Operating System (ROS) framework, and allows for the interactive positioning of sensors

The task of detecting 3D objects is important in various robotic applications. The existing deep learning-based detection techniques have achieved impressive performances. However, these techniques are limited to being run on a graphics processing unit (GPU) in a real-time environment. To achieve real-time 3D object detection with limited computational resources, we propose an efficient detection method

Robotic skill learning suffers from the diversity and complexity of robotic tasks in continuous domains, making the learning of transferable skills one of the most challenging issues in this area, especially for the case where robots differ in terms of structure. Aiming at making the policy easier to be generalized or transferred, the graph neural networks (GNN) was previously employed to incorporate

This paper addresses the problem of obtaining an Earth-fixed trajectory and map (ETM), with the associated uncertainty, using the sensor-based map provided by a globally asymptotically/exponentially stable (GES) SLAM filter. The algorithm builds on an optimization problem with a closed-form solution, and its uncertainty description is derived resorting to perturbation theory. The combination of the

Learning from human skills has become one of the popular inspirations in grasp prediction and evaluation, but lack of effective methods on groups of grasp points for multi-fingered dexterous hands yields an open challenge. When facing an object, humans firstly predict a variety of options for grasps, which can be concerned as a complex multi-valued problem. After prediction, humans evaluate grasps

With the development of the robotic technology, Autonomous Mobile Manipulator (AMM) is increasingly used in more applications. Reasonable motion planning for AMM to maintain high manipulation capability is the prerequisite for the success of the mobile manipulation task. In this paper, the Capability Map (CM) of AMM that gives the distribution of the manipulability in cartesian space is first built

This paper presents the manipulability analysis of free-floating multi-arm space robots. Evaluation of manipulator capability is useful both in the design and in the operation phase. After capturing a target, closed kinematic chains are formed with multi-arm cooperative manipulating a common object. Owing to the dynamic coupling effect, the manipulability analysis of free-floating systems is more complex

Limited communication range, together with mobility of robots, makes it crucial to design the control plans such that connectivity of a multi-robot network is maintained. Recently, many local and global connectivity maintenance schemes have been proposed to preserve connectivity of a robotic network. The traditional local connectivity maintenance method (LCM) is known to preserve every existing link

In this paper, we developed an underactuated robotic finger with three joints having a branching tendon mechanism and a sensing system to estimate the wrench applied to the fingertip based on the moment-equivalent point (MEP). We proposed the design to combine the branching tendon mechanism and the principle of wrench sensing based on the MEP. The proposed system realized the measurement of the wrench

Currently, most social robots interact with their surroundings or humans through sensors that are integral parts of the robots, which limits the usability of the sensors, human–robot interaction, and interchangeability. A wearable sensor garment that fits many robots is needed in many applications. This article presents an affordable wearable sensor vest, and an open-source software architecture with

We propose a means of omni-directional contact detection using accelerometers instead of tactile sensors for object shape estimation using touch. Unlike tactile sensors, our contact-based detection method tends to induce a degree of uncertainty with false-positive contact data because the sensors may react not only to actual contact but also to the unstable behavior of the robot. Therefore, it is crucial

The motion coordination problem for a fleet of Autonomous Guided Vehicles (AGVs) in a confined industrial facility is addressed. The working scenario involves a group of AGVs that is tasked to transport without collisions to predefined locations within the industrial facility. We introduce a centralized motion coordination controller that utilizes a dynamic priority logic to resolve motion conflicts

An essential capability of humans is the effortless identification of useful tasks based on visual cues in everyday situations. Objects and their surroundings are integrated and processed to differentiate plausible from implausible actions. In this work, we study how to teach this ability to robots. In contrast to many tasks in computer vision where the goal is an accurate description (object labels

We present a novel method to localize the vehicle on an easily accessible geo-referenced satellite image based on LiDAR. We first design a neural network to extract and compare the spatial-discriminative feature maps of the satellite image patch and the LiDAR points, and obtain the probability of correspondence. Then based on the outputs of the network, a particle filter is used to obtain the probability

This paper presents an optimal communication relay positioning method to improve the communication performance of mobile multi-node networks in complex environments. The communication channel quality prediction between nodes is of primary concern to find the optimal relay node positions while considering the uncertain and dynamic nature of environments. To this end, the learning-based or the distance

The tightness inspection for the slot wedges is significant for the safe operation of large generators. One of the traditional methods is analysis of the acoustic signals of knocking on the surface of the slot wedge by inspection experts. Nowadays the slot wedge inspecting robot is an effective way to measure the tightness of the slot wedges and classify the level of the slot wedges into different

Designing a robot system with reasoning and learning ability has gradually become a research focus in robotics research field. Recently, Skill Transfer Learning (STL), i.e., the ability of transferring human skills to robots, has become a research thrust for autonomous robots and human–robot cooperation. It provides the following benefits: (i) the skill transfer learning system with independent decision-making

The design of a phase-based robust oscillator for wearable robots, that could assist humans performing periodic or repetitive tasks, is presented in this paper. The bounds on perturbations, that guaranteed the stability of the output for the phase oscillator controller, were identified and the Lyapunov redesign method was applied to construct a robust controller using a bounding function. The robust

An important envisaged application of legged robots is the exploration and mapping of extreme environments with an unknown terrain. Corin is a hexapod designed at the University of Manchester, which is able to perform motions using footholds on surfaces perpendicular to the ground plane. This allows it to be able to navigate through confined and narrow spaces. The hexapod requires an accurate estimate

With the rapid development of computer vision, vision-based simultaneous localization and mapping (vSLAM) plays an increasingly important role in the field of unmanned driving. However, traditional SLAM methods based on a monocular camera only perform well in simple indoor environments or urban environments with obvious structural features. In off-road environments, the situation that SLAM encounters

This work details the design and simulation results of a bioinspired minimalist algorithm based on C. elegans, using autonomous agents to forage for attractant energy sources. The robotic agents are energy-constrained and depend on the energy they forage to recharge their batteries, which is significant as the foraging task is one of the canonical testbeds for cooperative robotics. The algorithm consists

Intelligent vehicle is an inevitable trend in urban transportation development. Whether for driver assistance systems or advanced driverless systems, safety issues are one of the cores of these systems. This paper proposes a scene safety evaluation method based on binocular camera. By proposing a comprehensive safety evaluation model, the uncertainty of the scene can be represented with a single value

Wall-climbing robots have been widely applied in the inspection of smooth walls. However, only a few adhesion methods have been developed for robots that will allow them to climb cliffs and dusty, high-altitude, rough walls (constructed using coarse concrete, bricks, and stones, etc.) that may be subjected to vibrations. This paper proposes a suitable adhesion method that employs grappling-hook-like

In this article, we show how visual constraints such as homographies and fundamental matrices can be integrated tightly into the locomotion controller of a humanoid robot to drive it from one configuration to another (pose-regulation), only by means of images. The visual errors generated by these constraints are stacked as terms of the objective function of a Quadratic Program so as to specify the

Legged-wheeled robots combine the advantages of efficient wheeled mobility with the capability of adapting to real-world terrains through the legged locomotion. Thanks to their hybrid mobility skill, they can excel in many application scenarios where other mobile platforms are not suitable for. However, the improved versatility of their mobility increases the number of constraints in their motion control

The hydrodynamic model can be used to predict velocity of underwater vehicles in still water. However, there are few economical and effective methods for estimating the hydrodynamic parameters of irregular-shaped underwater vehicles. Thus, this paper proposes a hybrid estimation strategy, which contains a rough estimation using a least squares (LS) based algorithm and a precise estimation using an

In recent years, we have witnessed the proliferation of so-called collaborative robots or cobots, that are designed to work safely along with human operators. These cobots typically use the “program from demonstration” paradigm to record and replay trajectories, rather than the traditional source-code based programming approach. While this requires less knowledge from the operator, the basic functionality

We investigate Model Predictive Control (MPC) schemes without stabilizing constraints or costs for the set-point stabilization of holonomic mobile robots. Herein, we ensure closed-loop asymptotic stability using the concept of cost controllability. To this end, we derive a growth bound on the finite-horizon value function in terms of the running costs evaluated at the current state, which is then used

This work presents a series of demonstrations of our self-reconfigurable modular robots (SRMR) “Roombots” in the context of adaptive and assistive furniture. In literature, simulations are often ahead of what currently can be demonstrated in hardware with such systems due to significant challenges in transferring them to the real world. Here, we describe how Roombots tackled these difficulties in real

A 3UPU-I parallel sensor with six division-force limbs and six standard force sensors is developed for measuring robotic wrist load. Its measuring approach and performances are studied and evaluated. A prototype of the developed parallel sensor is built up and its merits are analyzed. A statics equation among the forces of the six standard force sensors and the wrist load is established, and a mapped

Despite the increment of researches related to Social Assistive Robotics (SAR), achieving a plausible Robot-Assisted Diagnosis (RAD) for Children with Autism Spectrum Disorders (CwASD) remains a considerable challenge to the clinical and robotics community. The work of specialists regarding ASD diagnosis is hard and labor-intensive due to the condition’s manifestations are inherently heterogeneous

Most designs of wearable robots are based on human biomechanical statistics, engineering experience or individual experiments. Despite great successes, few of them consider the human–robot integration and individual differences between users. Additionally, the design periods, cost and safety also need to be further improved. Learning from the natural driving mechanism of human body, we propose a general

Recent research on automotive driving has developed an efficient end-to-end learning mode that directly maps visual input to control commands. However, it models distinct driving variations in a single network, which increases learning complexity and is less adaptive for modular integration. In this paper, we re-investigate human’s driving style and propose to learn an intermediate driving intention