当前位置: X-MOL 学术Pattern Recogn. Lett. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
A Hybrid Tucker-VQ Ttensor Sketch decomposition model for coding and streaming real world light fields using stack of differently focused images
Pattern Recognition Letters ( IF 3.9 ) Pub Date : 2022-04-29 , DOI: 10.1016/j.patrec.2022.04.034
Joshitha Ravishankar 1 , Mansi Sharma 1 , Sally Khaidem 1
Affiliation  

Computational multi-view displays involving light fields are a fast emerging choice for 3D presentation of real-world scenes. Tensor autostereoscopic glasses-free displays use just few light attenuating layers in front of a backlight to output high quality light field. We propose three novel schemes, Focal Stack - Hybrid Tucker-TensorSketch Vector Quantization (FS-HTTSVQ), Focal Stack - Tucker-TensorSketch (FS-TTS), and Focal Stack - Tucker Alternating Least-Squares (FS-TALS), for efficient representation, streaming and coding of light fields using a stack of differently focused images. Working with a focal stack instead of the entire light field majorly reduces the data acquisition cost as well as the computation and processing cost. Extensive experiments with real world light field focal stacks demonstrate that proposed novel one-pass Tucker decomposition using TensorSketch with hybrid vector quantization in FS-HTTSVQ, compactly represents the approximated focal stack in codebook form for better transmission and streaming. Encoding with High Efficiency Video Coding (HEVC) eliminates all intrinsic redundancies present in the approximated focal stack. Resultant low-rank approximated and coded focal stack is then employed to analytically optimize layer patterns for the tensor display. The complete end-to-end light field processing pipelines flexibly work for multiple bitrates and are adaptable for a variety of multi-view autostereoscopic platforms. Our schemes exhibit note-worthy performances on focal stacks compared to direct encoding of an entire light field using a standard codec like HEVC.



中文翻译:

一种混合 Tucker-VQ Ttensor Sketch 分解模型,用于使用不同聚焦图像的堆栈对真实世界的光场进行编码和流式传输

涉及光场的计算多视图显示是现实世界场景 3D 呈现的快速新兴选择。张量自动立体无眼镜显示器仅在背光源前使用少量光衰减层来输出高质量的光场。我们提出了三种新方案,Focal Stack - Hybrid Tucker-TensorSketch Vector Quantization (FS-HTTSVQ)、Focal Stack - Tucker-TensorSketch (FS-TTS) 和 Focal Stack - Tucker Alternating Least-Squares (FS-TALS),以提高效率使用一堆不同聚焦的图像对光场进行表示、流式传输和编码。使用焦点堆栈而不是整个光场可以大大降低数据采集成本以及计算和处理成本。在 FS- HTTSVQ中具有混合矢量量化的 TensorSketch,以码本形式紧凑地表示近似焦点堆栈,以实现更好的传输和流式传输。使用高效视频编码 (HEVC) 进行编码消除了近似焦点堆栈中存在的所有内在冗余。然后使用得到的低秩近似和编码焦点堆栈来分析优化张量显示的层模式。完整的端到端光场处理流水线灵活适用于多种比特率,并适用于各种多视图自动立体平台。与使用 HEVC 等标准编解码器直接编码整个光场相比,我们的方案在焦点堆栈上表现出值得注意的性能。

更新日期:2022-04-29
down
wechat
bug