当前位置: X-MOL 学术Big Data Res. › 论文详情
PXDedup: Deduplicating Massive Visually Identical JPEG Image Data
Big Data Research ( IF 2.673 ) Pub Date : 2020-11-20 , DOI: 10.1016/j.bdr.2020.100171
Hengxiang Xie; Yuhui Deng; Hao Feng; Lei Si

The explosive growth of data brings a big challenge for the data storage and backup of data centers. Moreover, existing techniques of mobile phone make image become one of the main ways for information presentation. Most images are compressed to JPEG format, and the image data accounts for a large part of the data growth. To reduce the storage cost, data deduplication is proposed and has now become a requisite component of backup systems. However, traditional deduplication techniques based on binary stream are not efficient for JPEG files, for the image compression process breaks redundancy. This paper proposes a deduplication method for JPEG files from a new perspective: visual redundancy which is named PXDedup. Different from traditional deduplication techniques, PXDedup decompresses a JPEG file to an image first, and then partitions the image to chunks. The visually identical image chunks are regarded as redundant chunks and are eliminated. Besides, PXDedup recompresses unique image chunks before storing them on disk. This operation further reduces the data size by making use of the feature that a high quality JPEG image can be recompressed with a bit lower quality parameter without quality loss. Experimental results show that PXDedup achieves a good reduction ratio when making a backup of a dataset consisting of large amounts of JPEG files, especially when the dataset includes many JPEG images which have few visual differences.

更新日期:2020-11-23
全部期刊列表>>
ERIS期刊投稿
欢迎阅读创刊号
自然职场,为您触达千万科研人才
spring&清华大学出版社
城市可持续发展前沿研究专辑
Springer 纳米技术权威期刊征稿
全球视野覆盖
施普林格·自然新
chemistry
物理学研究前沿热点精选期刊推荐
自然职位线上招聘会
欢迎报名注册2020量子在线大会
化学领域亟待解决的问题
材料学研究精选新
GIANT
ACS ES&T Engineering
ACS ES&T Water
屿渡论文,编辑服务
ACS Publications填问卷
阿拉丁试剂right
苏州大学
林亮
南方科技大学
朱守非
内蒙古大学
杨小会
隐藏1h前已浏览文章
课题组网站
新版X-MOL期刊搜索和高级搜索功能介绍
ACS材料视界
上海纽约大学
浙江大学
廖矿标
天合科研
x-mol收录
试剂库存
down
wechat
bug