Deep reinforcement learning (RL) has become one of the most popular topics in artificial intelligence research. It has been widely used in various fields, such as end-to-end control, robotic control, recommendation systems, and natural language dialogue systems. In this survey, we systematically categorize the deep RL algorithms and applications, and provide a detailed review over existing deep RL

We investigate the solution and stability of continuous-time cross-dimensional linear systems (CCDLSs) with dimension bounded by V-addition and V-product. Using the integral iteration method, the solution to CCDLSs can be obtained. Based on the new algebraic expression of the solution and the Jordan decomposition method of matrix, a necessary and sufficient condition is derived for judging whether

Unfortunately the corresponding author’s ORCID was incorrect. It should be: Fan ZHANG, https://orcid.org/0000-0001-7456-8377

General purpose graphics processing units (GPGPUs) can be used to improve computing performance considerably for regular applications. However, irregular memory access exists in many applications, and the benefits of graphics processing units (GPUs) are less substantial for irregular applications. In recent years, several studies have presented some solutions to remove static irregular memory access

Large-scale datasets are driving the rapid developments of deep convolutional neural networks for visual sentiment analysis. However, the annotation of large-scale datasets is expensive and time consuming. Instead, it is easy to obtain weakly labeled web images from the Internet. However, noisy labels still lead to seriously degraded performance when we use images directly from the web for training

Run-length limited (RLL) codes can facilitate reliable data transmission and provide flicker-free illumination in visible light communication (VLC) systems. We propose novel high-rate RLL codes, which can improve error performance and mitigate flicker. Two RLL coding schemes are developed by designing the finite-state machine to further enhance the coding gain by improving the minimum Hamming distance

Opinion question machine reading comprehension (MRC) requires a machine to answer questions by analyzing corresponding passages. Compared with traditional MRC tasks where the answer to every question is a segment of text in corresponding passages, opinion question MRC is more challenging because the answer to an opinion question may not appear in corresponding passages but needs to be deduced from

The weighting subspace fitting (WSF) algorithm performs better than the multi-signal classification (MUSIC) algorithm in the case of low signal-to-noise ratio (SNR) and when signals are correlated. In this study, we use the random matrix theory (RMT) to improve WSF. RMT focuses on the asymptotic behavior of eigenvalues and eigenvectors of random matrices with dimensions of matrices increasing at the

A robust polynomial observer is designed based on passive synchronization of a given class of fractional-order Colpitts (FOC) systems with mismatched uncertainties and disturbances. The primary objective of the proposed observer is to minimize the effects of unknown bounded disturbances on the estimation of errors. A more practicable output-feedback passive controller is proposed using an adaptive

In dense traffic unmanned aerial vehicle (UAV) ad-hoc networks, traffic congestion can cause increased delay and packet loss, which limit the performance of the networks; therefore, a traffic balancing strategy is required to control the traffic. In this study, we propose TQNGPSR, a traffic-aware Q-network enhanced geographic routing protocol based on greedy perimeter stateless routing (GPSR), for

Similarity measure has long played a critical role and attracted great interest in various areas such as pattern recognition and machine perception. Nevertheless, there remains the issue of developing an efficient two-dimensional (2D) robust similarity measure method for images. Inspired by the properties of subspace, we develop an effective 2D image similarity measure technique, named transformation

Developing new saturable absorbers for use in the mid-infrared region has practical significance for short-pulsed lasers and related scientific and industrial applications. The performance of gold nanorods (GNRs) as saturable absorbers at novel mid-infrared wavelengths needs to be evaluated even though these benefit from ultrafast nonlinear responses and broadband saturable absorption. Passive Q-switching

We built a Tm:Ho co-doped fiber laser using a Ti3C2 MXene material as a saturable absorber (SA). The formation of vector solitons (VSs) and noise-like pulses (NLPs) was observed. The SA was prepared by dripping a Ti3C2 solution on a side-polished D-shaped fiber and then naturally vaporized. The VS is characterized by two coexisting sets of Kelly sidebands. By modulating the polarization controller

An artificial intelligence enhanced star identification algorithm is proposed for star trackers in lost-in-space mode. A convolutional neural network model based on Vgg16 is used in the artificial intelligence algorithm to classify star images. The training dataset is constructed to achieve the networks’ optimal performance. Simulation results show that the proposed algorithm is highly robust to many

Generally, the distributed bundle adjustment (DBA) method uses multiple worker nodes to solve the bundle adjustment problems and overcomes the computation and memory storage limitations of a single computer. However, the performance considerably degrades owing to the overhead introduced by the additional block partitioning step and synchronous waiting. Therefore, we propose a low-overhead consensus

Smartphone, as a smart device with multiple built-in sensors, can be used for collecting information (e.g., vibration and location). In this paper, we propose an approach for using the smartphone as a sensing platform to obtain real-time data on vehicle acceleration, velocity, and location through the development of the corresponding application software and thereby achieve the green concept based

In this review, we introduce some chip-based waveguide biosensing and imaging techniques, which significantly reduce the complexity of the entire system. These techniques use a well-confined evanescent field to interact with the surrounding materials and achieve high sensitivity sensing and high signal-to-noise ratio (SNR) super-resolution imaging. The fabrication process of these chips is simple and

The multi-objective optimization problem has been encountered in numerous fields such as high-speed train head shape design, overlapping community detection, power dispatch, and unmanned aerial vehicle formation. To address such issues, current approaches focus mainly on problems with regular Pareto front rather than solving the irregular Pareto front. Considering this situation, we propose a many-objective

Long-term coherent integration can remarkably improve the ability of detection and motion parameter estimation of radar for maneuvering targets. However, the linear range migration, quadratic range migration (QRM), and Doppler frequency migration within the coherent processing interval seriously degrade the detection and estimation performance. Therefore, an efficient and noise-resistant coherent integration

The recommendation task with a textual corpus aims to model customer preferences from both user feedback and item textual descriptions. It is highly desirable to explore a very deep neural network to capture the complicated nonlinear preferences. However, training a deeper recommender is not as effortless as simply adding layers. A deeper recommender suffers from the gradient vanishing/exploding issue

Crowd density estimation, in general, is a challenging task due to the large variation of head sizes in the crowds. Existing methods always use a multi-column convolutional neural network (MCNN) to adapt to this variation, which results in an average effect in areas with different densities and brings a lot of noise to the density map. To address this problem, we propose a new method called the segmentation-aware

Permanent magnet synchronous motor (PMSM) has been widely used in position control applications. Its performance is not satisfactory due to internal uncertainties and external load disturbances. To enhance the control performance of PMSM systems, a new method that has fast response and good robustness is proposed in this study. First, a modified integral terminal sliding mode controller is developed

In this study, we focus mainly on the problem of finding the minimum-length path through a set of circular regions by a fixed-wing unmanned aerial vehicle. Such a problem is referred to as the Dubins traveling salesman problem with neighborhoods (DTSPN). Algorithms developed in the literature for solving DTSPN either are computationally demanding or generate low-quality solutions. To achieve a better

We investigate the stability of Boolean networks (BNs) with impulses triggered by both states and random factors. A hybrid index model is used to describe impulsive BNs. First, several necessary and sufficient conditions for forward completeness are obtained. Second, based on the stability criterion of probabilistic BNs and the forward completeness criterion, the necessary and sufficient conditions

Images are widely used by companies to advertise their products and promote awareness of their brands. The automatic synthesis of advertising images is challenging because the advertising message must be clearly conveyed while complying with the style required for the product, brand, or target audience. In this study, we proposed a data-driven method to capture individual design attributes and the

Crowd counting has been applied to a variety of applications such as video surveillance, traffic monitoring, assembly control, and other public safety applications. Context information, such as perspective distortion and background interference, is a crucial factor in achieving high performance for crowd counting. While traditional methods focus merely on solving one specific factor, we aggregate sufficient

We consider optimal two-impulse space interception problems with multiple constraints. The multiple constraints are imposed on the terminal position of a space interceptor, impulse and impact instants, and the component-wise magnitudes of velocity impulses. These optimization problems are formulated as multi-point boundary value problems and solved by the calculus of variations. Slackness variable

Passive bistatic radar detects targets by exploiting available local broadcasters and communication transmissions as illuminators, which are not designed for radar. The signal usually contains a time-varying structure, which may result in high-level range ambiguity sidelobes. Because the mismatched filter is effective in suppressing sidelobes, it can be used in a passive bistatic radar. However, due

We present a double-layered control algorithm to plan the local trajectory for automated trucks equipped with four hub motors. The main layer of the proposed control algorithm consists of a main layer nonlinear model predictive control (MLN-MPC) controller and a secondary layer nonlinear MPC (SLN-MPC) controller. The MLN-MPC controller is applied to plan a dynamically feasible trajectory, and the SLN-MPC

With the growing amount of information and data, object-oriented storage systems have been widely used in many applications, including the Google File System, Amazon S3, Hadoop Distributed File System, and Ceph, in which load balancing of metadata plays an important role in improving the input/output performance of the entire system. Unbalanced load on the metadata server leads to a serious bottleneck

Dynamic channel assignment (DCA) plays a key role in extending vehicular ad-hoc network capacity and mitigating congestion. However, channel assignment under vehicular direct communication scenarios faces mutual influence of large-scale nodes, the lack of centralized coordination, unknown global state information, and other challenges. To solve this problem, a multiagent reinforcement learning (RL)

Image classification is an important application of deep learning. In a typical classification task, the classification accuracy is strongly related to the features that are extracted via deep learning methods. An autoencoder is a special type of neural network, often used for dimensionality reduction and feature extraction. The proposed method is based on the traditional autoencoder, incorporating

We propose a multi-focus image fusion method, in which a fully convolutional network for focus detection (FD-FCN) is constructed. To obtain more precise focus detection maps, we propose to add skip layers in the network to make both detailed and abstract visual information available when using FD-FCN to generate maps. A new training dataset for the proposed network is constructed based on dataset CIFAR-10

The collaborative filtering technology used in traditional recommendation systems has a problem of data sparsity. The traditional matrix decomposition algorithm simply decomposes users and items into a linear model of potential factors. These limitations have led to the low accuracy in traditional recommendation algorithms, thus leading to the emergence of recommendation systems based on deep learning

Precise web page classification can be achieved by evaluating features of web pages, and the structural features of web pages are effective complements to their textual features. Various classifiers have different characteristics, and multiple classifiers can be combined to allow classifiers to complement one another. In this study, a web page classification method based on heterogeneous features and

Infrared Earth sensors are widely used in attitude-determination and control systems of satellites. The main deficiency of static infrared Earth sensors is the requirement of a small field of view (FOV). A typical FOV for a static infrared Earth sensor is about 20° to 30°, which may not be sufficient for low-Earth-orbiting micro-satellites. A novel compact infrared Earth sensor with an FOV of nearly

With the development of computer science, more and more hardware implementations can be reproduced by software programming, bringing compact, cheap, and fast components to imaging instrumentation. In recent years, computational methods have been introduced into spectral detection, and computational spectrum acquisition implementations have emerged. This paper highlights the advantages of computational

Traceability link recovery (TLR) is an important and costly software task that requires humans establish relationships between source and target artifact sets within the same project. Previous research has proposed to establish traceability links by machine learning approaches. However, current machine learning approaches cannot be well applied to projects without traceability information (links),

Unfortunately the second author’s name has been misspelt. It should be read: Abbas NIKNAFS.

The fractional logistic map holds rich dynamics and is adopted to generate chaotic series. A watermark image is then encrypted and inserted into the original images. Since the encryption image takes the fractional order within (0, 1], it increases the key space and becomes difficult to attack. This study provides a robust watermark method in the protection of the copyright of hardware, images, and

Multi-focus image fusion is an increasingly important component in image fusion, and it plays a key role in imaging. In this paper, we put forward a novel multi-focus image fusion method which employs fractional-order derivative and intuitionistic fuzzy sets. The original image is decomposed into a base layer and a detail layer. Furthermore, a new fractional-order spatial frequency is built to reflect

An autonomous five-dimensional (5D) system with offset boosting is constructed by modifying the well-known three-dimensional autonomous Liu and Chen system. Equilibrium points of the proposed autonomous 5D system are found and its stability is analyzed. The proposed system includes Hopf bifurcation, periodic attractors, quasi-periodic attractors, a one-scroll chaotic attractor, a double-scroll chaotic

With the rapid increase of the size of applications and the complexity of the supercomputer architecture, topology-aware process mapping becomes increasingly important. High communication cost has become a dominant constraint of the performance of applications running on the supercomputer. To avoid a bad mapping strategy which can lead to terrible communication performance, we propose an optimized

In this study, we focus on the controllability of fractional-order damped systems in linear and nonlinear cases with multiple time-varying delays in control. For the linear system based on the Mittag-Leffler matrix function, we define a controllability Gramian matrix, which is useful in judging whether the system is controllable or not. Furthermore, in two special cases, we present serval equivalent

A novel color image encryption algorithm based on dynamic deoxyribonucleic acid (DNA) encoding and chaos is presented. A three-neuron fractional-order discrete Hopfield neural network (FODHNN) is employed as a pseudo-random chaotic sequence generator. Its initial value is obtained with the secret key generated by a five-parameter external key and a hash code of the plain image. The external key includes

In this study, we discuss mainly the image denoising and texture retention issues. Usually, the time-fractional derivative has an adjustable fractional order to control the diffusion process, and its memory effect can nicely retain the image texture when it is applied to image denoising. Therefore, we design a new Rudin-Osher-Fatemi model with a time-fractional derivative based on a classical one,

We introduce the fractional-order global optimal backpropagation machine, which is trained by an improved fractional-order steepest descent method (FSDM). This is a fractional-order backpropagation neural network (FBPNN), a state-of-the-art fractional-order branch of the family of backpropagation neural networks (BPNNs), different from the majority of the previous classic first-order BPNNs which are

Rapidly increasing capacities, decreasing costs, and improvements in computational power, storage, and communication technologies have led to the development of many applications that carry increasingly large amounts of traffic on the global networking infrastructure. Smart devices lead to emerging technologies and play a vital role in rapid evolution. Smart devices have become a primary 24/7 need

With the rapid development of electronic information technology, digital signature has become an indispensable part of our lives. Traditional public key certificate cryptosystems cannot overcome the limitations of certificate management. Identity-based cryptosystems can avoid the certificate management issues. The development of quantum computers has brought serious challenges to traditional cryptography

Visual servo control rules that refer to the control methods of robot motion planning using image data acquired from the camera mounted on the robot have been widely applied to the motion control of robotic arms or mobile robots. The methods are usually classified as image-based visual servo, position-based visual servo, and hybrid visual servo (HVS) control rules. Mobile manipulation enhances the

Symbolic execution is an effective way of systematically exploring the search space of a program, and is often used for automatic software testing and bug finding. The program to be analyzed is usually compiled into a binary or an intermediate representation, on which symbolic execution is carried out. During this process, compiler optimizations influence the effectiveness and efficiency of symbolic

The applications of artificial intelligence (AI) and machine learning (ML) technologies in wireless communications have drawn significant attention recently. AI has demonstrated real success in speech understanding, image identification, and natural language processing domains, thus exhibiting its great potential in solving problems that cannot be easily modeled. AI techniques have become an enabler

To address indeterminism in the bilevel knapsack problem, an uncertain bilevel knapsack problem (UBKP) model is proposed. Then, an uncertain solution for UBKP is proposed by defining the PE Nash equilibrium and PE Stackelberg-Nash equilibrium. To improve the computational efficiency of the uncertain solution, an evolutionary algorithm, the improved binary wolf pack algorithm, is constructed with one

We present a real-time monocular simultaneous localization and mapping (SLAM) system with a new distributed structure for multi-UAV collaboration tasks. The system is different from other general SLAM systems in two aspects: First, it does not aim to build a global map, but to estimate the latest relative position between nearby vehicles; Second, there is no centralized structure in the proposed system

We use the advanced proximal policy optimization (PPO) reinforcement learning algorithm to optimize the stochastic control strategy to achieve speed control of the “model-free” quadrotor. The model is controlled by four learned neural networks, which directly map the system states to control commands in an end-to-end style. By introducing an integral compensator into the actor-critic framework, the

We propose multi-objective social learning pigeon-inspired optimization (MSLPIO) and apply it to obstacle avoidance for unmanned aerial vehicle (UAV) formation. In the algorithm, each pigeon learns from the better pigeon but not necessarily the global best one in the update process. A social learning factor is added to the map and compass operator and the landmark operator. In addition, a dimension-dependent

Coordinating multiple unmanned aerial vehicles (multi-UAVs) is a challenging technique in highly dynamic and sophisticated environments. Based on digital pheromones as well as current mainstream unmanned system controlling algorithms, we propose a strategy for multi-UAVs to acquire targets with limited prior knowledge. In particular, we put forward a more reasonable and effective pheromone update mechanism

With the development of sensor fusion technologies, there has been a lot of research on intelligent ground vehicles, where obstacle detection is one of the key aspects of vehicle driving. Obstacle detection is a complicated task, which involves the diversity of obstacles, sensor characteristics, and environmental conditions. While the on-road driver assistance system or autonomous driving system has

Lower-limb assisted exoskeletons are widely researched for movement assistance or rehabilitation training. Due to advantages of compliance with human body and lightweight, some cable-driven prototypes have been developed, but most of these can assist only unidirectional movement. In this paper we present an untethered cable-driven ankle exoskeleton that can achieve plantarflexion-dorsiflexion bidirectional

Accurate acceleration acquisition is a critical issue in the robotic exoskeleton system, but it is difficult to directly obtain the acceleration via the existing sensing systems. The existing algorithm-based acceleration acquisition methods put more attention on finite-time convergence and disturbance suppression but ignore the error constraint and initial state irrelevant techniques. To this end,