Multimedia data analysis is at the heart of multimedia research, where the patterns, information, and knowledge are mined and recognized from the multimedia data for various applications. We have witnessed significant performance improvements in multimedia data analysis driven by the recent advances in artificial intelligence and deep learning. While most of the existing methods focus on analyzing

Presents the front cover for this issue of the publication.

Advetisement, IEEE.

Presents the table of contents for this issue of the publication.

Provides a listing of current staff, committee members and society officers.

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

Multimedia is one of the key drivers improving virtual reality and augmented reality (VR/AR), which are promising to reform human–computer interaction in the future with lower-cost and all-in-one headsets containing powerful hardware. Advances in multimedia research on video compression and human–computer interfaces have further enhanced the immersion and efficiency of experiences on the platform.

The most discriminative expression features are mostly concentrated in local key facial regions. Thus, we propose a simple and efficient framework that can learn more discriminative expression features from scrambled facial images. Specifically, we first divide the input image into local subregions of the same size and shuffle them randomly at a certain range to obtain the damaged image to increase

Considering the application of a sentiment analysis in decision-making and personalized advertising, we adopt it in tourism. Specifically, we perform a sentiment analysis on the posted Weibos about the passengers’ experience in civil aviation travel. Different travel events could influence passengers’ sentiment, e.g., flight delay may cause negative sentiment. Inspired by this observation, we propose

This work presents an approach for emotion recognition in video through the interaction of visual, audio, and language information in an end-to-end learning manner with three key points: 1) lightweight feature extractor, 2) attention strategy, and 3) adaptive loss. We proposed a lightweight deep architecture with approximately 1 MB, which for the most crucial part, accounts for feature extraction,

Presents a call for 2021 major award nominations.

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

Presents the IEEE Edcuation Awards nominations.

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

Advertisement.

Video surveillance is a crucial part of public safety and security systems. State-of-the-art techniques either use back-tracing for owner identification or there is no provision of reporting ownership of the abandoned objects. We provide a novel object association technique which very elegantly tackles this issue. We propose a convolution neural network based framework for abandoned object localization

Due to the booming popularity of online social networks, emojis have been widely used in online communication. As nonverbal language units, emojis help to convey emotions and express feelings. In this article, we focus on the sentiment-aware emoji insertion task, which predicts multiple emojis and their positions in a sentence conditioned on the plain texts and sentiment polarities. To facilitate future

Critical utility infrastructure like power grids are vast (in hundreds of kilometers), linear, and operated 24×7 throughout the year. Maintenance inspections using low-cost unmanned aerial vehicles and aerial imaging are therefore gaining popularity. To have low-cost framework, the quality of the camera used is not of high quality or a stereo-rig one. Also, the sensors used are limited in variety and

Emotion is precious, useful for many applications such as public opinion detection and psychological disease prediction. Emotion recognition on multimodal data has attracted extensive attention. Although modifying model structure or multimodal feature fusion methods have contributed a lot to emotion recognition, little attention is paid to mining implicit emotion relationship. In this article, implicit

Presents the front cover for this issue of the publication.

Advertisement, IEEE.

Presents the table of contents for this issue of this publication.

Provides a listing of current staff, committee members and society officers.

Advertisement, IEEE.

Coronavirus Disease 2019 (COVID-19) has been affecting most of the countries and impacting almost every aspect of people's lives. More than one hundred million confirmed cases and two million deaths have been reported due to COVID-19 as of February 2021. While our society suffers an unanticipated epidemic, researchers and engineers have developed various technologies to manage this global emergency

In this article, we propose a face spoofing detection method by learning to fuse high-frequency (HF) and low-frequency (LF) features, in an effort to improve the generalization capability and fill up the domain gap between training and testing when the antispoofing is practically conducted in unseen scenarios. In particular, the proposed face antispoofing model consists of two streams that extract

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

Sarcasm is a sophisticated linguistic phenomenon and commonly manifests on social media platforms, which poses a great challenge for opinion mining systems. Therefore, multimodal sarcasm detection, which aims to understand the implied sentiment in the video, has gained more and more attention. However, previous works mostly focus on multimodal feature fusion without explicitly modeling the incongruity

We present a learning model for multimodal context-aware emotion recognition. Our approach combines multiple human co-occurring modalities (such as facial, audio, textual, and pose/gaits) and two interpretations of context. To gather and encode background semantic information for the first context interpretation from the input image/video, we use a self-attention-based CNN to encode. Similarly, for

Recognizing emotion through facial expression has now been widely applied in our daily lives. Therefore, facial expression recognition (FER) is attracting increasing research interests in the field of artificial intelligence and multimedia. With the development of convolutional neural networks (CNN), end-to-end deep learning frameworks for FER have achieved great success on large-scale datasets. However

In this article, we propose a novel reinforcement learning network that keeps track of the gradual emotional changes from every utterance throughout the conversation and uses this information for each utterance’s emotion detection. Concretely, we first establish an agent and, then, utilize sliding windows to extract the accumulated emotional information before the current utterance. We define the concatenation

Fine-grained facial expression aims at changing the expression of an image without altering facial identity. Most current expression manipulation methods are based on a discrete expression label, which mainly manipulates holistic expression with details neglected. To handle the above mentioned problems, we propose an end-to-end expression-guided generative adversarial network (EGGAN), which synthesizes

The satisfied user ratio (SUR) for a given distortion level is the fraction of subjects that cannot perceive a quality difference between the original image and its compressed version. By predicting the SUR, one can determine the highest distortion level which allows to save bit rate while guaranteeing a good visual quality. We propose the first method to predict the SUR for symmetrically and asymmetrically

Microexpression (ME) spotting is a crucial step for emotion analysis to detect people’s true emotions. However, the short duration, small motion amplitude, and limited sample number make accurate spotting and ME-locating challenging. To address these problems, we make two contributions in this article, including an ME dataset SDU2 and a spotting method. The dataset SDU2 contains hybrid expressions

Through well-designed optimization paradigm and deep neural networks as feature extractor, deep reinforcement learning (DRL) algorithms learn optimal policy on discrete and continuous action space. However, such capability is restricted by the low sampling efficiency. By inspecting the importance of feature extraction in DRL, we find that state feature learning is one of the key obstacles for sampling

Single image dehazing is the key to enhancing image visibility in outdoor scenes, which facilitates human observation and computer recognition. The existing approaches generally utilize a one-shot strategy that indiscriminately applies the same filters to all local regions. However, due to neglecting inhomogeneous illumination and detail distortion, their dehazed results easily suffer from underfiltering

The rapid development and popularization of video surveillance highlight the critical and challenging problem, vehicle reidentification, which suffers from the limited interinstance discrepancy between different vehicle identities and large intrainstance differences of the same vehicle. In this article, we propose a novel multilevel attention network to hierarchically learn an efficient feature embedding

Although the text-to-image synthesis task has shown significant progress, it still remains a challenge in generating high-quality images. In this article, we first propose an attention-driven, cycle-refinement generative adversarial network, AGAN-v1, to bridge the domain gap between visual contents and semantic concepts by constructing spatial configurations of objects. The generation of image contours

Political statements are carefully crafted to garner public support for a particular ideology. These statements are often biased and sometimes misleading. Separating fact from fiction has proven to be a difficult task, generally accomplished by cross-checking political statements against an impartial and trustworthy news source. In this article, we make three contributions. First, we compile a novel

As the interest in deep learning tools continues to rise, new multimedia research fields begin to discover its potential. Both image and point cloud coding are good examples of technologies, where deep learning-based solutions have recently displayed very competitive performance. In this context, this article brings two novel contributions to the point cloud geometry coding state-of-the-art; first

Classically, the style of the generated music by deep learning models is usually governed by the training dataset. In this article, we improved this by proposing the continuous style embedding ${z}_{s}$ to the general formulation of variational autoencoder (VAE) to allow users to be able to condition on the style of the generated music. For this purpose, we explored and compared two different methods

In this article, we focus on a new concept called multichannel steganography, in which a sender is able to transmit different secret data to multiple receivers via the same cover image. First, we propose a general framework to achieve multichannel steganography based on modern steganographic paradigm. Then, we design two schemes to, respectively, achieve separable and sequential multichannel steganography

Video coding is more important than ever due to emergence of larger resolutions, new formats requiring higher bandwidths, prevalence of new video services, etc. Consequently, new codecs are designed to minimize transmission rates of such contents without sacrificing quality. This article proposes an intraprediction solution for future video codecs that save the bit rates of intramode signaling for

An ideal audio retrieval system identifies a short query snippet from a massive audio database with both robustness and efficiency. Unfortunately, none of the existing systems could robustly handle all distortions while being efficient. Enhanced sampling and counting (eSC), the state-of-the-art audio retrieval method proposed for Philips-like fingerprints, has achieved both high efficiency and strong

There has been a consensus of cross-component prediction in video compression that the interchannel redundancies can be effectively removed through a linear prediction model. To fully explore the similarities existing in different color components, we propose prediction with multicross component (PMC) as a new chroma intracoding mode for the emerging video coding standard AVS3. The proposed PMC enjoys

Presents the front cover for this issue of the publication.

Presents information on the HOST 2020 conference.

Presents the table of contents for this issue of the publication.

Presents a listing of the editorial board, board of governors, current staff, committee members, and/or society editors for this issue of the publication.

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

Presents the introductory editorial for this issue of the publication.

The thirteen papers in this special section focus on social media data analytics with deep intelligence. Big cross-model social media data analytics with deep intelligence aims to handle data sampling from multimodal deep spaces, so as to well characterize the big data. The addressed topic span from the range of human action recognition to affective computing, disaster detection, classification, retrieval

With the current global outbreak of COVID-19, an increasing number of people are suffering from negative mental states and mental disorders. We propose a multimodal psychological computational technology in a universal environment. We establish a mental health database following a naturalistic paradigm as well as a long-term ubiquitous interpretable psychological computing model based on prior knowledge

In this article, we develop an end-to-end clothing collocation learning framework based on a bidirectional long short-term memories (Bi-LTSM) model, and propose new feature extraction and fusion modules. The feature extraction module uses Inception V3 to extract low-level feature information and the segmentation branches of Mask Region Convolutional Neural Network (RCNN) to extract high-level semantic

Advertisement.

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

Advertisement.