A major development in the area of image captioning consists of trying to incorporate visual attention in the design of language generative model. However, most previous studies only emphasize its role in enhancing visual composition at the current moment, while neglect its role in global sequence reasoning. This problem appears not only in captioning model, but also in reinforcement learning structure

With the rise of deep learning technology, person re-identification (Re-id) technology has been developed rapidly. During the training process, many recent methods are susceptible to target misalignment and without sufficient discriminative features. Aiming at these two problems, a simple and potent model is proposed by us. A new self-attention module and a multi-loss function with relative weight

A challenging problem confronted when designing a blind/no-reference (NR) stereoscopic image quality assessment (SIQA) algorithm is to simulate the quality assessment (QA) behavior of the human visual system (HVS) during binocular vision. An effective way to solve this problem is to estimate the quality of the merged single view created in the human brain which is also referred to as the cyclopean

Early intervention and treatment are crucial for individuals with autism spectrum disorder (ASD). However, it is challenging to identify individuals with ASD at an early age, i.e. under 3 years old, due to the lack of an effective and objective identification method. The mainstream clinical diagnosis relies on long-term observation of children’s behaviors, which is time-consuming and expensive, and

This paper proposes a novel light field image compression approach with viewpoint scalability and random access functionalities. Although current state-of-the-art image coding algorithms for light fields already achieve high compression ratios, there is a lack of support for such functionalities, which are important for ensuring compatibility with different displays/capturing devices, enhanced user

This work is about solving a challenging problem of estimating the full 3D hand pose when a hand interacts with an unknown object. Compared to isolated single hand pose estimation, occlusion and interference induced by the manipulated object and the clutter background bring more difficulties for this task. Our proposed Multi-Level Fusion Net focuses on extracting more effective features to overcome

Convolutional Neural Network (CNN) structures have been designed for in-loop filtering to improve video coding performance. These CNN models are usually trained through learning the correlations between the reconstructed and the original frames, which are then applied to every single reconstructed frame to improve the overall video quality. This direct model training and deployment strategy is effective

The release of the latest video coding standard, known as Versatile Video Coding (VVC), has created the need to convert current High Efficiency Video Coding (HEVC) content to this new standard. However, the traditional cascade transcoding pipeline is not effective due to the exorbitant computational complexity of VVC. With this in mind, this paper proposes a fast HEVC-VVC transcoder that implements

In this paper, we propose a new bio-inspired metric for classifying autism spectrum disorder (ASD) children and typically developed (TD) children. The model used in the Saliency4ASD Grand Challenge at ICME 2019 uses linear regression and prior probability to process distance and time data respectively. Unfortunately, this model performs unsatisfactorily because the visual attention characteristics

Autism Spectrum Disorder is a developmental disorder characterized by a deficit in social behaviour and specific interactions such as reduced eye contact and body gestures. Recent advancements in software and hardware multimedia technologies provide the tools for early detecting the presence of this disorder. In this paper we present an approach based on the combined use of machine learning and eye

Deterioration modelling of ageing phenomena on materials is an actively researched topic in computer graphics and vision, with a wide range of applications in domains such as cultural heritage, game programming, material science and virtual reality. As a result significant progress has been accomplished and existing methods are able to produce visually pleasing results that appear realistic. However

As early intervention is highly effective for young children with autism spectrum disorder (ASD), it is imperative to make accurate diagnosis as early as possible. ASD has often been associated with atypical visual attention and eye gaze data can be collected at a very early age. An automatic screening tool based on eye gaze data that could identify ASD risk offers the opportunity for intervention

Most existing convolutional neural network (CNN) based models designed for natural image quality assessment (IQA) employ image patches as training samples for data augmentation, and obtain final quality score by averaging all predicted scores of image patches. This brings two problems when applying these methods for screen content image (SCI) quality assessment. Firstly, SCI contains more complex content

Rate–distortion optimization (RDO) is utilized to select the optimal coding parameters in multi-view video coding (MVC), which employs a Lagrange multiplier to balance the relationship between the distortion and the bitrate. In this paper, an efficient RDO method for the dependent view (DV) in multi-view video (MVV) is proposed based on inter-view dependency. First of all, by investigating the sources

Based on discrete Fourier transforms and logistic-exponent-sine map, this paper investigates an encryption algorithm for multiple color images. In the encryption process, each color image represented in trinion matrix is performed by block-wise discrete trinion Fourier transforms. Then the first real matrix is constructed by splicing real and imaginary parts of transformed results. Followed by two-dimensional

Lightfield (LF) technology has attained significant attention in recent years due to its capability to capture much richer textural and geometric information in the scene compared to the classical 2D representation. The resampling and compression operations on LFs often lead to visual quality degradation, thus, sophisticated visual quality assessment methods play a crucial role to ensure a pleasant

Visual attention for the diagnosis of Autism Spectrum Disorder (ASD) which is a kind of mental disorder has attracted the interests of increasing number of researchers. Although multiple visual attention prediction models have been proposed, this problem is still open. In this paper, considering the shift of visual attention, we propose that an image can be viewed as a pseudo sequence. Besides, we

Recent developments of video super-resolution reconstruction often exploit spatial and temporal contexts from input frame sequence by making use of explicit motion estimation, e.g., optical flow, which may introduce accumulated errors and requires huge computations to obtain an accurate estimation. In this paper, we propose a novel multi-branch dilated convolution module for real-time frame alignment

Facial replay attacks have been a topic of interest in recent past due to the vulnerability of intrusive nature in biometric security systems. In order to build a robust biometric system many safeguard approaches have already been developed by the researchers to nullify spoofing activities like print and replay attacks. This paper proposes a comprehensive study on the application of Multidimensional

The High Efficiency Video Coding (HEVC) is adopted by various video applications in recent years. Because of its high computational demand, controlling the complexity of HEVC is of paramount importance to appeal to the varying requirements in many applications, including power-constrained video coding, video streaming, and cloud gaming. Most of the existing complexity control methods are only capable

Underwater image processing has played an important role in various fields such as submarine terrain scanning, submarine communication cable laying, underwater vehicles, underwater search and rescue. However, there are many difficulties in the process of acquiring underwater images. Specifically, the water body will selectively absorb part of the light when light travels through the water, resulting

Many low-light image enhancement methods ignore the characteristic of non-uniform low-light images, thereby causing the enhanced results to lose their naturalness. In this paper, a non-local similarity decomposition model based on the Retinex theory is proposed to obtain high-quality enhanced results for non-uniform low-light images. For the reflectance layer, we explore a weighted L1-norm regularization

This paper describes a system for people indoor localization using omnidirectional cameras and machine learning that significantly reduces the database annotation requirements for the training stage. The most prominent works for people detection are based on machine learning techniques that requires large databases with bounding box annotations (that enclose the people). In this work, a novel multiple

Objective image quality metrics try to estimate the perceptual quality of the given image by considering the characteristics of the human visual system. However, it is possible that the metrics produce different quality scores even for two images that are perceptually indistinguishable by human viewers, which have not been considered in the existing studies related to objective quality assessment.

In recent years, numerous facial expression recognition related applications had been commercialized in the market. Many of them achieved promising and reliable performance results in real-world applications. In contrast, the automated micro-expression recognition system relevant research analysis is still greatly lacking. This is because of the nature of the micro-expression that is usually appeared

We investigated whether watching two-dimensional television (2DTV) or three-dimensional television (3DTV) resulted in differences in the brain’s processing of sensory information. We divided 25 participants into 2DTV (n = 13) and 3DTV (n = 12) groups. Participants watched 2DTV or 3DTV for 1, 2, or 3 h on different days. Before and at the end of each session, electroencephalography (EEG) was recorded

Recent deep learning-based inpainting methods have shown significant improvements and generate plausible images. However, most of these methods may either synthesis unrealistic and blurry texture details or fail to capture object semantics. Furthermore, they employ huge models with inefficient mechanisms such as attention. Motivated by these observations, we propose a new end-to-end generative-based

Several deep supervised hashing techniques have been proposed to allow for extracting compact and efficient neural network representations for various tasks. However, many deep supervised hashing techniques ignore several information-theoretic aspects of the process of information retrieval, often leading to sub-optimal results. In this paper, we propose an efficient deep supervised hashing algorithm

Subspace clustering refers to clustering data points into their respective subspaces and finding a low-dimensional structure to fit each group of points. In subspace clustering, the inter-cluster correlation of data which is caused by noise such as illumination and background affects the performance of subspace clustering algorithms. To solve this problem, a new approach is proposed to detect the unusual

The median filtering heuristic is considered to be an indispensable tool for the currently popular variational optical flow computation. Its attractive advantages are that outliers reduction is attained while image edges and motion boundaries are preserved. However, it still may generate blurring at image edges and motion boundaries caused by large displacement, motion occlusion, complex texture, and

Due to the storage and retrieval efficiency of hashing, as well as the highly discriminative feature extraction by deep neural networks, deep cross-modal hashing retrieval has been attracting increasing attention in recent years. However, most of existing deep cross-modal hashing methods simply employ single-label to directly measure the semantic relevance across different modalities, but neglect the

Face aging (FA) for young faces refers to rendering the aging faces at target age for an individual, generally under 20s, which is an important topic of facial age analysis. Unlike traditional FA for adults, it is challenging to age children with one deep learning-based FA network, since there are deformations of facial shapes and variations of textural details. To alleviate the deficiency, a unified

Convolutional neural networks have achieved prominent performance in the image restoration field at the cost of massive network parameters and computations. Several model compression methods have been proposed, however, most of them are designed for high-level vision tasks, which originally have some tolerance for information loss. To make image restoration networks targeting for low-level vision tasks

With the increasing maturity of video editing technology, forgers are more inclined to transcode videos to High Efficiency Video Coding (HEVC) videos, as HEVC not only enables people to enjoy high-definition videos but also allows broadcasters to stream it more efficiently across networks. Therefore, to verify the originality and authenticity, it is of great significance to propose an algorithm for

Multi-focus image fusion aims to generate an image with all objects in focus by integrating multiple partially focused images. It is challenging to find an effective focus measure to evaluate the clarity of source images. In this paper, a novel multi-focus image fusion algorithm based on Geometrical Sparse Representation (GSR) over single images is proposed. The main novelty of this work is that it

The authenticity and copyright protection of volumetric medical images has become extremely important when these images are distributed online for diagnosis and education purpose. Compared to the authenticity and copyright protection of conventional images, there are two additional challenges for protecting the volumetric medical images. On one hand, the content of the protected medical images must

Haze is an atmospheric phenomenon which diminishes visibility in outdoor images. Algorithms based on dark channel prior (DCP) and haze line prior are found to be effective for dehazing images. These two methods make use of the Laplacian matrix, which is computationally complex, memory intensive and slow, thus making it impossible to use them on high-resolution (large) images. Multiple strategies have

The traditional visual secret sharing (VSS) schemes share a secret image into multiple shares, and all the shares have the same importance. However, it is occasionally necessary to differentiate participants based on their different levels of importance in certain applications. In previous works, some weighted VSS schemes were implemented only for the (k,n) threshold access structure and were not applicable

Multi-focus image fusion is the activity of synthesizing multiple images of different focusing settings to construct a fully focused image. Many of the latest methods for image fusion rarely consider the structural differences between the guidance image and the input image, and do not retain well the important source image features while producing a fully focused image. To address this issue, a method

This article proposes a spectral–spatial method for classification of hyperspectral images (HSIs) by modifying traditional Auto-Encoder based on Majorization Minimization (MM) technique. The proposed method consists of suggesting three main modifications. First, to construct weights of Auto-Encoder, similarity angle map(SAM) criterion is used as regularization term. It is useful to extract spectral

In the previous multiple histograms modification (MHM) based reversible data hiding (RDH) method, the prediction-error histograms are generated by a fixed manner, which may constrain the performance owing to the lack of adaptivity. In order to compensate this, we propose a deep neural networks (DNN) based method for dynamical multiple histograms generation. Through learning the prior knowledge, DNN

Generalized zero-shot classification (GZSC) is a challenging task to recognize seen and unseen samples from target domain by seen samples in source domain. Since the lack of unseen data, many methods train a generative adversarial network (GAN) to generate unseen samples. However, the GAN model trained by seen samples is not suitable for generating unseen samples. For dealing with this problem, we

Recent deep learning models outperform standard lossy image compression codecs. However, applying these models on a patch-by-patch basis requires that each image patch be encoded and decoded independently. The influence from adjacent patches is therefore lost, leading to block artefacts at low bitrates. We propose the Binary Inpainting Network (BINet), an autoencoder framework which incorporates binary

This paper considers the compressive sensing framework as a way of overcoming the spatio-angular trade-off inherent to light field acquisition devices. We present a novel method to reconstruct a full 4D light field from a sparse set of data samples or measurements. The approach relies on the assumption that sparse models in the 4D Fourier domain can efficiently represent light fields. The proposed

The recent studies showing that gaze features can be useful in the identification of Autism Spectrum Disorder (ASD), have opened a new domain where Visual Attention (VA) modeling could be of great help. In this sense, this paper presents a report of the Grand Challenge “Saliency4ASD: Visual attention modeling for Autism Spectrum Disorder”, organized at IEEE ICME’19, aiming at supporting the research

Due to the rapid development of free-viewpoint television (FVT), Depth-Image-Based Rendering (DIBR) technology has been widely used to synthesize images of virtual view-points. However, the types of distortions in the synthesized images are different from those of natural images, such as discontinuity, flickering, stretching, etc. To measure the distortion occurred in the synthesized images, we propose

Human action recognition in videos is still an important while challenging task. Existing methods based on RGB image or optical flow are easily affected by clutters and ambiguous backgrounds. In this paper, we propose a novel Pose-Guided Inflated 3D ConvNet framework (PI3D) to address this issue. First, we design a spatial–temporal pose module, which provides essential clues for the Inflated 3D ConvNet

Geometric art, an artwork that is structured by geometric shapes, was first made popular by the introduction of Cubism paintings by Pablo Picasso and Georges Braque in the early-20th-century. With the recent advancement in digital imaging technology coupled with the rising popularity of social media such as Instagram, automatic geometric abstraction that can automatically transform a photograph into

With increasing attentions being drawn to the underwater observation and utilization of marine resources in recent years, underwater image processing and analysis have become an active research hotspot. Different from the general images, marine environment is usually faced with some complicated situations such as underwater turbulence and diffusion, severe absorption and scattering of water body, various

For sensitive areas that even the slight distortion in images is not tolerated, we propose a prediction error expansion-based reversible data hiding algorithm which can embed reversible watermark in high dynamic range (HDR) image with low distortion. On the one hand, considering unique floating-point storage format and perceptual characteristics of HDR image, the multiple carriers are generated with

Stereoscopic 3D (S3D) visual quality prediction (VQP) is used to predict human perception of visual quality for S3D images accurately and automatically. Unlike that of 2D VQP, the quality prediction of S3D images is more difficult owing to complex binocular vision mechanisms. In this study, inspired by the binocular fusion and competition of the binocular visual system (BVS), we designed a blind deep

A new method for the polygonal approximation is presented. The method is based on the search for break points through a context-free grammar, that accepts digital straight segments with loss of information, as well as the decrease in the error committed employing the comparison of a tolerable error. We present an application of our method to different sets of objects widely used, as well as a comparison

Image plays an irreplaceable role compared with the text and sound in the underwater data collection and transmission researches. However, it suffers from the limited bandwidth of the underwater acoustic communication which cannot afford the large image data. Compressing the image data before transmission is an inevitable process in the underwater image communication. As usual, the natural image compression

Multi-focus image fusion is an effective method of information fusion that can take a series of source images and obtain a fused image where everything is in focus. In this paper, a multi-focus image fusion method based on image texture that adopts a modified Pulse-Coupled Neural Network (PCNN) approach is proposed. First, the texture of an image is obtained by means of image cartoon and texture decomposition

Every camera sensor leaves unique traces on the acquired images that can be thought of as a camera fingerprint. This work presents an efficient algorithm for clustering images based on their camera fingerprints. The algorithm performs a fast preliminary clustering based on a compressed representation of the camera fingerprints, then it refines the initial clusters using full-size fingerprints. The

No-reference/blind image quality assessment (NR-IQA/BIQA) algorithms play an important role in image evaluation, as they can assess the quality of an image automatically, only using the distorted image whose quality is being assessed. Among the existing NR-IQA/BIQA methods, natural scene statistic (NSS) models which can be expressed in different bandpass domains show good consistency with human subjective

Impressive progress has been made recently in image-to-image translation using generative adversarial networks (GANs). However, existing methods often fail in translating source images with noise to target domain. To address this problem, we joint image-to-image translation with image denoising and propose an enhanced generative adversarial network (EGAN). In particular, built upon pix2pix, we introduce

Super resolution (SR) is an attractive issue in image processing. In the synthetic aperture radar (SAR) image, speckle noise is a crucial problem that is multiplicative. Therefore, numerous custom SR methods considering additive Gaussian noise cannot respond to this image degradation model. The main contribution of this paper is to propose a novel variational convex optimization model for the single