Semantic segmentation is one of the important technologies in autonomous driving, and ensuring its real-time and high performance is of utmost importance for the safety of pedestrians and passengers. To improve its performance using deep neural networks that operate in real-time, the authors propose a simple and efficient method called ADFNet using accumulated decoder features, ADFNet operates by only

Multiple research studies have recently demonstrated deep networks can generate realistic-looking textures and stylised images from a single texture example. However, they suffer from some drawbacks. Generative adversarial networks are in general difficult to train. Multiple feature variations, encoded in their latent representation, require a priori information to generate images with specific features

Recent studies using deep neural networks have shown remarkable success in style transfer, especially for artistic and photo-realistic images. However, these methods cannot solve more sophisticated problems. The approaches using global statistics fail to capture small, intricate textures and maintain correct texture scales of the artworks, and the others based on local patches are defective on global

Video data are of two different intrinsic modes, in-frame and temporal. It is beneficial to incorporate static in-frame features to acquire dynamic features for video applications. However, some existing methods such as recurrent neural networks do not have a good performance, and some other such as 3D convolutional neural networks (CNNs) are both memory consuming and time consuming. This study proposes

Most existing text detection methods are mainly motivated by deep learning-based object detection approaches, which may result in serious overlapping between detected text lines, especially in dense text scenarios. It is because text boxes are not commonly overlapped, as different from general objects in natural scenes. Moreover, text detection requires higher localisation accuracy than object detection

Locality preserving projections (LPP) method is a classical manifold learning method for dimensionality reduction. However, LPP is sensitive to outliers since squared L2-norm may exaggerate the distance of outliers. Besides, the normalisation constraint of LPP may impair its robustness during embedding. Motivated by this observation, the authors propose a novel robust LPP using angle-based adaptive

Motion history image (MHI) is a spatio-temporal template that temporal motion information is collapsed into a single image where intensity is a function of recency of motion. Also, it consists of spatial information. Energy image (EI) based on the magnitude of optical flow is a temporal template that shows only temporal information of motion. Each video can be described in these templates. So, four

The task of referring expression comprehension (REC) is to localise an image region of a specific object described by a natural language expression, and all existing REC methods assume that the object described by the referring expression must be located in the given image. However, this assumption is not correct in some real applications. For example, a visually impaired user might tell his robot

In this study, the authors focus on improving the spatio–temporal representation ability of three-dimensional (3D) convolutional neural networks (CNNs) in the video domain. They observe two unfavourable issues: (i) the convolutional filters only dedicate to learning local representation along input channels. Also they treat channel-wise features equally, without emphasising the important features;

Unsupervised domain adaption aims to reduce the divergence between the source domain and the target domain. The final objective is to learn domain-invariant features from both domains that get the minimised expected error on the target domain. The divergence between domains which is also called domain shift is mainly between the distributions of domains' samples. Additionally, the label shift is also

Existing style transfer methods have achieved great success in artwork generation by transferring artistic styles onto everyday photographs while keeping their contents unchanged. Despite this success, these methods have one inherent limitation: they cannot produce newly created image contents, lacking creativity and flexibility. On the other hand, generative adversarial networks (GANs) can synthesise

Fisher kernels derived from stochastic probabilistic models such as restricted and deep Boltzmann machines have shown competitive visual classification results in comparison to widely popular deep discriminative models. This genre of Fisher kernels bridges the gap between shallow and deep learning paradigm by inducing the characteristics of deep architecture into Fisher kernel, further deployed for

A pedestrian segmentation algorithm in the presence of cast shadows is presented in this study. The novelty of this algorithm lies in the fusion of multi-view and multi-plane homographic projections of foregrounds and the use of the fused data to guide colour clustering. This brings about an advantage over the existing binocular algorithms in that it can remove cast shadows while keeping pedestrians’

This study proposes a new method to evaluate the performance of algorithms for moving object detection (MODA) in video sequences. The proposed method is based on human performance metric intervals, instead of ideal metric values (0 or 1) which are commonly used in the literature. These intervals are proposed to establish a more reliable evaluation and comparison, and to identify areas of improvement

Deep-learning-based pedestrian detectors can enhance the capabilities of smart camera systems in a wide spectrum of machine vision applications including video surveillance, autonomous driving, robots and drones, smart factory, and health monitoring. However, such complex paradigms do not scale easily and are not traditionally implemented in resource-constrained smart cameras for on-device processing

In this study, the authors discuss a novel approach for multi-camera-based unobtrusive identification and tracking of occupants in wide-area, multi-building scenarios. Considering the scalability issues in adopting a centralised approach to monitor wide-area scenarios, they proposed a distributed approach to occupant identification and tracking. The key technical idea underlying their approach is to

In the tracking-by-detection approach of online multiple object tracking (MOT), a major challenge is how to associate object detections on the new video frame with previously tracked objects. Two important aspects that directly influence the performance of MOT are quality of detection and accuracy in data association. The authors propose an efficient and unified MOT framework for improved object detection

High-density crowd counting in natural scenes is an extremely difficult and challenging research subject in computer vision. Although the algorithm based on the convolutional neural network has achieved significantly better results than the traditional algorithm, most of them tend to focus on the local features of images, and difficult to obtain the rich global contextual dependencies. To solve this

Swarms of drones are being more and more used in many practical scenarios, such as surveillance, environmental monitoring, search and rescue in hardly-accessible areas and so on. While a single drone can be guided by a human operator, the deployment of a swarm of multiple drones requires proper algorithms for automatic task-oriented control. In this study, the authors focus on visual coverage optimisation

In this work, the authors propose a novel decentralised coordination scheme for autonomous ground vehicles to enable map building and path planning with a network of smart overhead cameras. Decentralised indoor smart camera mapping and hierarchical navigation supports the automatic generation of waypoint graphs for each camera in an environment and allows path planning through the environment across

Effective acquisition, analysis and reconstruction of multi-modal data such as colour and multi-/hyper-spectral imagery is crucial in smart camera applications, where wavelet-based coding and compression of images are highly demanded. Many existing discrete wavelet filtering banks have fixed coefficients hence their performance is highly dependent on the signal/image being processed. To tackle this

Stereo image completion (SIC) is to fill holes existing in a pair of stereo images. SIC is more complicated than single image repairing, which needs to complete the pair of images while keeping their stereoscopic consistency. In recent years, deep learning has been introduced into single image repairing but seldom used for SIC. The authors present a novel deep learning-based approach for SIC. In their

Hyperspectral imaging systems provide dense spectral information on the scene under investigation by collecting data from a high number of contiguous bands of the electromagnetic spectrum. The low spatial resolutions of these sensors frequently give rise to the mixing problem in remote sensing applications. Several unmixing approaches are developed in order to handle the challenging mixing problem

Transferring a photo to a stylised image with beautiful texture has become one of the most popular topics in computer vision and the application of image processing. Controlling the stroke size of the texture is one of the challenging problems in this task. Recent representative methods for such problem introduce a pyramid model to regulate receptive fields in the network. Meanwhile, dilated convolutions

Human emotion recognition is studied by means of unimodal channels over the last decade. However, efforts continue to answer tempting questions about how variant modalities can complement each other. This study proposes a multimodal approach using three-dimensional (3D) convolutional neural networks (CNNs) to model human emotion through a modality-referenced system while investigating the solution

Pose variation has been one of the challenges of face recognition. To solve this challenge, the authors propose a classification algorithm using supervised subspace learning and non-local representation (SSLNR). In SSLNR, they first propose a supervised subspace learning algorithm (SSLA). SSLA includes three different terms. The first term is the difference term, which can reduce the intra-class differences

Conventional AlexNet has the problems of slow training speed, single characteristic scale and low recognition accuracy. To solve these problems, a convolutional neural network identification model based on Inception module and dilated convolution is proposed in this study. The inception module combined with dilated convolution, could extract disease characteristics at different scales and increase

Scene graph generation is to recognise objects and their semantic relationships in an image and can help computers understand visual scene. To improve relationship prediction, geometry information is essential and usually incorporated into relationship features. Existing methods use coordinates of objects to encode their spatial layout. However, in this way, they neglect the context of objects. In

The authors present a framework for interactive design of new image stylisations using a wide range of predefined filter blocks. Both novel and off-the-shelf image filtering and rendering techniques are extended and combined to allow the user to unleash their creativity to intuitively invent, modify, and tune new styles from a given set of filters. In parallel to this manual design, they propose a

Colour grading is an essential step in movie post-production, which is done in the industry by experienced artists on expensive edit hardware and software suites. This paper presents a review of the advances made to automate this process. The review looks in particular at how the state-of-the-art in optimal transport and deep learning has advanced some of the fundamental problems of colour transfer

Frame interpolation is the process of synthesising a new frame in-between existing frames in an image sequence. It has emerged as a key algorithmic module in motion picture effects. In the context of this special issue, this study provides a review of the technology used to create in-between frames and presents a Bayesian framework that generalises frame interpolation algorithms using the concept of

Portrait matting is of vital importance for many applications such as portrait editing, background replacement, ecommerce demonstration, and augmented reality. The portrait matt can be accessed by predicting the α value of the original picture. Previous deep matting methods usually adopt a segmentation network to tackle portrait matting tasks. However, these traditional methods will introduce unpleasant

An end-to-end pipeline for the creation of high-quality animatable volumetric video of human performances is presented. Going beyond the application of free-viewpoint video, the authors allow re-animation and alteration of an actor's performance through the enrichment of the captured data with semantics and animation properties. Hybrid geometry- and video-based animation methods are applied that allow

Creating realistic animations of human faces is still a challenging task in computer graphics. While computer graphics (CG) models capture much variability in a small parameter vector, they usually do not meet the necessary visual quality. This is due to the fact, that geometry-based animation often does not allow fine-grained deformations and fails in difficult areas (mouth, eyes) to produce realistic

This study presents an offline mobile diagnosis system for citrus pests and diseases by compression convolutional neural network. Recently, with the growth of labelled data, the deep neural network incites the revolutionary change with a quantum leap in various fields. Benefiting from the backpropagation method, the proper network structure can automatically extract high-level representations and find

This study proposes a three-stream model using two different types of deep convolutional neural networks (CNNs): (i) a spatial stream with a CNN on images; (ii) a ResNet (residual network) on optical flows; and, (iii) a ResNet on the concatenation of motion features. This model is applied to four datasets: (i) UCF Sports; (ii) Youtube Sports; (iii) SBU action interaction; and (iv) a subset of the UCF-1M

With the development of deep learning, the performance of object detection has made great progress. However, there are still some challenging problems, such as the detection accuracy of small objects and the efficiency of the detector. This study proposes an accurate and fast single shot multibox detector, which includes context comprehensive enhancement (CCE) module and feature enhancement module

Video-based person re-identification aims to verify the pedestrian identity from image sequences. The sequences are captured by cameras located in different directions at different times. Existing studies have certain limitations in the case of occlusions and pose variations. To solve the aforementioned problems, this study proposes a new two-stage framework, from which the key-image-based fusion spatial–temporal

Nowadays, a large number of deep convolutional neural network (CNN) models are applied to image classification tasks. However, the authors find that the most widely used evaluation indicator, the Top- N Accuracy indicator, cannot discriminate these models effectively. In this study, they propose a new indicator called Maximum-Spanning-Confusion-Tree indicator to solve this problem. The Maximum-Spa

Action detection and recognition are popular subjects of research in the field of computer vision. The task of action detection can be regarded as the sum of action location and recognition. Action features described by using information concerning the human skeleton have the advantages of robustness against external factors and requiring a small amount of calculation. This study proposes a skeleton-based

Complex inverse synthetic aperture radar (ISAR) object recognition is a critical and challenging problem in computer vision tasks. An efficient complex object recognition method for ISAR images is proposed based on adaptive deep relation learning. (i) An adaptive multimodal mechanism is proposed to greatly improve the multimodal sampling and transformation capabilities of convolutional neural networks

Predicting distant future trajectories of agents in a dynamic scene is challenging because the future trajectory of an agent is affected not only by their past trajectory but also the scene contexts. To tackle this problem, the authors propose a model based on recurrent neural networks, and a novel method for training this model. The proposed model is based on an encoder–decoder architecture where

Adversarial examples (AEs) against deep neural networks (DNNs) raise wide concerns about the robustness of DNNs. Existing detection mechanisms are often limited to a given attack algorithm. Therefore, it is highly desirable to develop a robust detection approach that remains effective for a large group of attack algorithms. In addition, most of the existing defences only perform well for small images

Automatic image annotation (AIA) is an image retrieval mechanism to extract relative semantic tags from visual content. So far, the improvement of accuracy in newly developed such methods have been about 1 or 2% in the F1-score and the architectures seem to have room for improvement. Therefore, the authors designed a more detailed architecture for AIA and suggested new algorithms for its main parts

LIDAR (light detection and ranging) based real-time 3D perception is crucial for applications such as autonomous driving. However, most of the convolutional neural network (CNN) based methods are time-consuming and computation-intensive. These drawbacks are mainly attributed to the highly variable density of LIDAR point cloud and the complexity of their pipelines. To find a balance between speed and

The low-rank tensor completion problem, which aims to recover the missing data from partially observable data. However, most of the existing tensor completion algorithms based on Tucker decomposition cannot avoid using singular value decomposition (SVD) operation to calculate the Tucker factors, so they are not suitable for the completion of large-scale data. To solve this problem, they propose a new

Polyps are a group of cells growing on the inner surface of the colon. Over time, some polyps can lead to colon cancer, which is often fatal if found in its later stages. Colon cancer can be prevented if the polyps are identified and removed in their early stages. Colonoscopy is a very effective screening method to remove polyps and it largely prevents colon cancer. However, some polyps may not be

In order to improve the accuracy and robustness of optical flow computation under large displacements and motion occlusions, the authors present in this study a large displacement flow field estimation approach using similarity transformation-based dense correspondence, named STDC-Flow approach. First, the authors compute an initial nearest-neighbour field by using the STDC-Flow of the consecutive

Training of the convolution neural network (CNN) is a problem of global optimisation. This study proposed a hybrid modified particle swarm optimisation (MPSO) and conjugate gradient (CG) algorithm for efficient training of CNN. The training involves MPSO–CG to avoid trapping in local minima. Particularly, improvements in the MPSO by introducing a novel approach for control parameters, improved parameters

Face recognition systems perform accurately in a controlled environment, but an unconstrained environment dramatically degrades their performance. In this study, a novel pose-invariant face recognition system is proposed based on the occlusion free regions. This method utilises a gallery set of frontal face images and can handle large pose variations. For a 2D probe face image with an arbitrary pose

Scale variation of a target frequently appears in tasks of visual tracking. Accurate scale estimation is challenging due to deformation, occlusion, rotation, change in the view angle and diversity of tracking object categories. Most tracking methods employ an exhaustive search of scales to estimate the target scales. However, only finite and discrete scales are usually searched due to the expensive