当前位置: X-MOL 学术Cognit. Comput. › 论文详情
Dense-CaptionNet: a Sentence Generation Architecture for Fine-grained Description of Image Semantics
Cognitive Computation Pub Date : 2020-03-02 , DOI: 10.1007/s12559-019-09697-1
I. Khurram, M. M. Fraz, M. Shahzad, N. M. Rajpoot

Automatic image captioning, a highly challenging research problem, aims to understand and describe the contents of the complex scene in human understandable natural language. The majority of the recent solutions are based on holistic approaches where the scene is described as a whole, potentially losing the important semantic relationship of objects in the scene. We propose Dense-CaptionNet, a region-based deep architecture for fine-grained description of image semantics, which localizes and describes each object/region in the image separately and generates a more detailed description of the scene. The proposed network contains three components which work together to generate a fine-grained description of image semantics. Region descriptions and object relationships are generated by the first module, whereas the second one generates the attributes of objects present in the scene. The textual descriptions obtained as an output of the two modules are concatenated to feed as an input to the sentence generation module, which works on encoder-decoder formulation to generate a grammatically correct but single line, fine-grained description of the whole scene. The proposed Dense-CaptionNet is trained and tested using Visual Genome, MSCOCO, and IAPR TC-12 datasets. The results establish a new state-of-the-art when compared with the existing top performing methodologies, e.g., Up-Down-Captioner, Show, Attend and Tell, Semstyle, and Neural Talk, especially on complex scenes. The implementation has been shared on GitHub for other researchers: http://bit.ly/2VIhfrf

更新日期:2020-04-20

 

全部期刊列表>>
施普林格自然
欢迎访问IOP中国网站
GIANT
自然科研线上培训服务
ACS ES&T Engineering
自然职场线上招聘会
ACS ES&T Water
产业、创新与基础设施
材料学研究精选
胸腔和胸部成像专题
屿渡论文,编辑服务
何川
苏昭铭
陈刚
姜涛
李闯创
北大
刘立明
隐藏1h前已浏览文章
课题组网站
新版X-MOL期刊搜索和高级搜索功能介绍
ACS材料视界
天合科研
x-mol收录
上海纽约大学
曾林
天津大学
何振宇
史大永
吉林大学
卓春祥
张昊
杨中悦
试剂库存
down
wechat
bug