Predicting pedestrian trajectory is useful in many applications, such as autonomous driving and unmanned vehicles. However, it is a challenging task because of the complexity of the interactions among pedestrians and the environment. Most existing works employ long short-term memory networks to learn pedestrian behaviors, but their prediction accuracy is not good, and their computing speed is relatively

Predicting future actions is a key in visual understanding, surveillance, and human behavior analysis. Current methods for video-based prediction are primarily using single-view data, while in the real world multiple cameras and produced videos are readily available, which may potentially benefit the action prediction tasks. However, it may bring up a new challenge: subjects in the videos are more

In the last few years, squared fiducial markers have become a popular and efficient tool to solve monocular localization and tracking problems at a very low cost. Nevertheless, marker detection is affected by noise and blur: small camera movements may cause image blurriness that prevents marker detection. The contribution of this paper is two-fold. First, it proposes a novel approach for estimating

Recognizing artworks in a cultural site using images acquired from the user's point of view (First Person Vision) allows to build interesting applications for both the visitors and the site managers. However, current object detection algorithms working in fully supervised settings need to be trained with large quantities of labeled data, whose collection requires a lot of times and high costs in order

Clothing compatibility learning has gained increasing research attention due to the fact that a properly coordinated outfit can represent personality and improve an individual's appearance greatly. In this paper, we propose a Compatibility Matrix-Regularized Generative Adversarial Network (CMRGAN) for compatible item generation. In particular, we utilize a multi-modal embedding to transform the image

In this paper, we propose a Motion Saliency based multi-stream Multiplier ResNets (MSM-ResNets) for action recognition. The proposed MSM-ResNets model consists of three interactive streams: the appearance stream, motion stream and motion saliency stream. Similar to conventional two-stream CNNs models, the appearance stream and motion stream are responsible for capturing the appearance information and

As a practical pursuit of quantified uniqueness, biometrics explores the parameters that make us who we are and provides the tools we need to secure the integrity of that identity. In our culture of constant connectivity, an increasing reliance on biometrically secured mobile devices is transforming them into a target for bad actors. While no system will ever prevent all forms of intrusion, even state

How to effectively utilize and fuse deep features has become a critical point for salient object detection. Most existing methods usually adopt the convolutional features based on U-shape structures and fuse multi-scale convolutional features without fully considering the different characteristics between high-level features and low-level features. Furthermore, existing salient object detection methods

This paper presents an accurate non-iterative method for the Perspective-n-Point problem(PnP). Our main idea is to mitigate scale bias by multiplying an independent inverse average depth variable onto the object space error. The introduced variable is of order 2 in the objective function and the optimality conditions constitute a polynomial system with three third-order and one first-order unknowns

Recently, plenty of deep learning methods have been proposed to handle point clouds. Almost all of them input the entire point cloud and ignore the information redundancy lying in point clouds. This paper addresses this problem by extracting the Reeb graph from point clouds, which is a much more informative and compact representation of point clouds, and then filter the graph with deep graph convolution

Human-Robot Interaction (HRI) provides assisted services in different real-time applications. The robotic systems identify objects through digital visualization wherein a three-dimensional (3D) image is converged to a plane-based projection. The projection is analyzed using the co-ordinates and identification points for recognizing the object. In such a converging process, the misidentification of

Face frontalization aims to normalize profile faces to frontal ones for pose-invariant face recognition. Current works have achieved promising results in face frontalization by using deep learning techniques. However, training deep models of face frontalization usually needs paired training data which is undoubtedly costly and time-consuming to acquire. To address this issue, we propose a Pose Conditional

Driven by business intelligence applications for rating attraction of products in shops, a new problem — salient object grading is studied in this paper. In computer vision, plenty of salient object detection approaches have been proposed, while most existing studies detect objects in a binary manner: salient or not. This paper focuses on a new problem setting that requires detecting all salient objects

An enormous amount of sensing devices (scalar or multimedia) collect and generate information (in the form of events) over the Internet of Things (IoT). Present research on IoT mainly focus on the processing of scalar sensor data events and barely considers the challenges posed by multimedia based events. In this paper, we systematically review the existing solutions available for the Internet of Multimedia

Perception systems, to a large extent, rely on neural networks. Commonly, the training of neural networks uses a finite amount of data. The usual assumption is that an appropriate training dataset is available, which covers all relevant domains. This abstract will follow the example of different lighting conditions in autonomous driving scenarios. In real-world datasets, a single source domain, such

Existing stereo matching networks based on deep learning lack multi-level and multi-module attention and integration for feature information. Therefore, we propose an attention-guided aggregation stereo matching network to encode and integrate information multiple times. Specifically, we design a residual network based on the 2D channel attention block to adaptively calibrate weight response, improving

Although refinement is commonly used in visual tasks to improve pre-obtained results, it has not been studied for Multiple Object Tracking (MOT) tasks. This could be attributed to two reasons: i) it has not been explored what kinds of errors should — and could — be reduced in MOT refinement; ii) the refinement target, namely, the tracklets, are intertwined and interactive in a 3D spatio-temporal space

Video surveillance systems are popular and used in public places such as market places, shopping malls, hospitals, banks, streets, education institutions, city administrative offices, and smart cities to enhance the safety of public lives and assets. Most of the time, the timely and accurate detection of video anomalies is the main objective of security applications. The video anomalies such as anomalous

In the past few years, video-based person re-identification (Re-ID) have attracted growing research attention. The crucial problem for this task is how to learn robust video feature representation, which can weaken the influence of factors such as occlusion, illumination, and background etc. A great deal of previous works utilize spatio-temporal information to represent pedestrian video, but the correlations

Pedestrian detection in images is a topic that has been studied extensively, but existing detectors designed for perspective images do not perform as successfully on images taken with top-view fisheye cameras, mainly due to the orientation variation of people in such images. In our proposed approach, several perspective views are generated from a fisheye image and then concatenated to form a composite

In view of the difficulty in training the algorithm of image oil painting style migration and reconstruction based on the generative adversarial network, and the loss gradient of generator and discriminator disappears, this paper proposes an improved generative adversarial network based on gradient penalty, and constructs the total variance loss function to carry out the research of image oil painting

Generalized zero-shot classification is predicting the labels of the test images coming from seen or unseen classes. The task is difficult because of the bias problem, that is, unseen samples are easily to be misclassified to seen classes. Many methods have handled the problem by training a generative adversarial network (GAN) to generate fake samples. However, the GAN model trained with seen samples

In order to study the application effect and function of industrial visual perception technology in smart city, the image processing and quality evaluation system was constructed by using convolutional neural network (CNN) and Internet of things (IoT) technology. The system was simulated, and then the quality performance of image and video obtained by using industrial visual perception technology was

Vehicle re-identification is an important part of intelligent transportation. Although much work has been done on this subject in recent years, vehicle re-identification is still a challenging task due to its obvious illumination change, high similarity between inter-class and great changes under different views. As discriminatory local areas and vehicle view information is the key to improving the

Human-robot interaction (HRI) is a multidisciplinary area that consists of several technologies that are used to create various smart city applications. The knowledge gain and analysis of the smart city environment improves response time. This paper introduces the dependable information processing (DIP) method for handling multi-attribute environmental information in a smart city application. Information

Deep learning-based object detection solutions emerged from computer vision has captivated full attention in recent years. The growing UAV market trends and interest in potential applications such as surveillance, visual navigation, object detection, and sensors-based obstacle avoidance planning have been holding good promises in the area of deep learning. Object detection algorithms implemented in

With the widespread use of mobile devices, iris recognition systems encounter more challenges, such as the vulnerability of Presentation Attack Detection (PAD). Recent works pointed out the contact lens attacks, especially images captured under the uncontrolled environment, as a hard task for iris PAD. In this paper, we propose a novel framework for detecting iris presentation attacks that especially

Multimodal image fusion aims to generate a fused image from different signals that captured by multimodal sensors. Although the images obtained by multimodal sensors have different appearances, the information included in these images might be redundant and noisy. In the previous studies, the fusion rule and their properties that guiding how to merge the features from multiple images is relatively

Vehicle model recognition (VMR) is a typical fine-grained classification task in computer vision. To improve the representation power of classical CNN networks for this special task, we focus on enhancing the subtle difference of features and their spatial encoding based on the attention mechanism, and then propose a novel architectural unit, which we term the “convolutional attention model” (CAM)

The limited capacity to recognize micro-expressions with subtle and rapid motion changes is a long-standing problem that presents a unique challenge for expression recognition systems and even for humans. The problem regarding micro-expression is less covered by research when compared to macro-expression. Nevertheless, micro-expression recognition (MER) is imperative to exploit the full potential of

In this paper, we propose an infrared and visible image fusion framework based on the consensus problem. Most current infrared and visible image fusion models aim to transfer only one characteristic of each source domain to the final fusion result. This mechanism limits the performances of fusion algorithms under different conditions. We present a general fusion framework based to solve the global

Recent advances in deep learning have shown excellent performance in various scene understanding tasks. However, in some complex environments or under challenging conditions, it is necessary to employ multiple modalities that provide complementary information on the same scene. A variety of studies have demonstrated that deep multimodal fusion for semantic image segmentation achieves significant performance

The application of computer vision (CV) in healthcare applications is familiar with the wireless and communication technology. CV methods are incorporated in the healthcare for providing programmed interactions towards patient monitoring. The requirements of systems are the analysis and detection of the images' visualization of patients. In this paper, a multi-modal visualization analysis (MMVA) method

Today, with the rapid development of science and technology, the era of big data has been proposed and triggered reforms in all walks of life. Face recognition is a biometric recognition method with the characteristics of non-contact, non mandatory, friendly and harmonious, which has a good application prospect in the fields of national security and social security. With the deepening of the research

Although many studies have investigated markerless motion capture, the technology has not been applied to real sports or concerts. In this paper, we propose a markerless motion capture method with spatiotemporal accuracy and smoothness from multiple cameras in wide-space and multi-person environments. The proposed method predicts each person's 3D pose and determines the bounding box of multi-camera

Crowd counting is an important surveillance application and receives significant attention from the computer vision community. Most of the current methods treat crowd counting by density map estimation and use the Fully Convolution Network (FCN) for prediction. The mainstream framework is to predict density maps and use the sum up the density maps to get the number of people. In such methods, the main

The detection of objects with multi-orientations and multi-scales in aerial images is receiving increasing attention because of numerous useful applications in computer vision, image understanding, satellite remote sensing and surveillance. However, such detection can be exceedingly challenging because of a birds eye view, multi-scale rotating objects with large aspect ratios, dense distributions and

Nowadays, the technology of using computer vision to calibrate objects is widely used, which has a huge market demand in many fields. This paper provides a calibration method of computer vision system based on dual attention neural network. This paper uses the camera to simulate human eyes to obtain three-dimensional images. After obtaining the three-dimensional images, the images are input into the

Facial expression recognition aims to accurately interpret facial muscle movements in affective states (emotions). Previous studies have proposed holistic analysis of the face, as well as the extraction of features pertained only to specific facial regions towards expression recognition. While classically the latter have shown better performances, we here explore this in the context of deep learning

Periocular region is considered as an important biometric trait due to its ease of collectability and high acceptability in our society. Recent advancements in surveillance applications require infra-red (IR) sensing equipments to be deployed in order to capture the activities occurring in low-light conditions. This gives rise to the problem of matching periocular images in heterogeneous environments

Traditionally, the P3P problem is solved by firstly transforming its 3 quadratic equations into a quartic one, then by locating the roots of the resulting quartic equation and verifying whether a root does really correspond to a true solution of the P3P problem itself. It is well known that a root of the quartic equation could correspond to 2, or 1, or even null solution at all to the P3P problem,

Surveillance camera networks are a useful infrastructure for various visual analytics applications, where high-level inferences and predictions could be made based on target tracking across the network. Most multi-camera tracking works focus on target re-identification and trajectory association problems to track the target. However, since camera networks can generate enormous amount of video data

Building extraction, which is a fundamental task in the community of remote sensing image analysis, has been extensively applied in various applications related to smart cities. Due to the complicated background information in urban areas, how to extract building footprints from high-resolution aerial images is challenging. The recent achievements of deep learning have shed light on building extraction

In biometrics, periocular recognition analysis is an essential constituent for identifying the human being. Among prevailing the modalities, ocular biometric traits such as iris, sclera and periocular eye movement have experienced noteworthy consciousness in the recent past. In this paper, we are presenting new multi-biometric fusion method called Phase Intensive Mutual Exclusive Distribution (PI-MED)

Face verification performance by human brain has been shown to be much better than most of the state-of-the-art approaches in computer vision. Performance improvement of automated face recognition (FR) systems that may equal or surpass the human intellect is the key goal of this research. We present our face verification system using facial mark (FM) combined with deep convolutional neural network

Generative adversarial networks (GANs) have been enjoying considerable success in recent years. However, mode collapse remains a major unsolved problem in training GANs and is one of the main obstacles hindering progress. In this paper, we present spectral regularization for GANs (SR-GANs), a new and robust method for combating the mode collapse problem in GANs. We first perform theoretical analysis

Biometric feature based human authentication systems provide various advantages over the classical authentication systems. Out of numerous biometric features in a human body, the fingerprint is the most commonly used biometric feature for the authentication of a person. Typically, the fingerprint-based systems rely on minutiae points information and use it directly as a user template. Several studies

We propose a novel framework for cross-modal zero-shot learning (ZSL) in the context of sketch-based image retrieval (SBIR). Conventionally, the SBIR schema mainly considers simultaneous mappings among the two image views and the semantic side information. Therefore, it is desirable to consider fine-grained classes mainly in the sketch domain using highly discriminative and semantically rich feature

Augmented and virtual reality deployment is finding increasing use in novel applications. Some of these emerging and foreseen applications allow the users to access sensitive information and functionalities. Head Mounted Displays (HMD) are used to enable such applications and they typically include eye facing cameras to facilitate advanced user interaction. Such integrated cameras capture iris and

A few-shot learning model generally consists of a feature extraction network and a classification module. In this paper, we propose an approach to improve few-shot image classification performance by increasing the representational capacity of the feature extraction network and improving the quality of the features extracted by it. The ability of the feature extraction network to extract highly discriminative

Existing efforts of saliency detection have achieved excellent performance in RGB images, thus to sufficiently exploit existing RGB saliency models and further do some extensions on them, we can transfer existing RGB saliency models to the similar research field, i.e. RGBD saliency detection, by introducing depth cues. Here, we construct a novel RGBD saliency model upon an existing RGB saliency model

The unsupervised person re-identification (re-ID) has become increasingly significant in the community because it is more scalable than the supervised method when dealing with the large-scale person re-ID. However, it is difficult to learn discriminative enough features from across-camera images without labelling information. To address this problem, we propose a joint patch and instance discrimination

There has been a continued interest in learning features that are generalizable across sensors and spectra for ocular biometrics. This is usually facilitated through a model that can learn features that are robust across pose, lighting conditions, spectra, and device sensor variations. In this paper, we propose an efficient deep learning-based feature extraction pipeline for learning the aforementioned