-
ANIM: Accurate Neural Implicit Model for Human Reconstruction from a single RGB-D image arXiv.cs.GR Pub Date : 2024-03-15 Marco Pesavento, Yuanlu Xu, Nikolaos Sarafianos, Robert Maier, Ziyan Wang, Chun-Han Yao, Marco Volino, Edmond Boyer, Adrian Hilton, Tony Tung
Recent progress in human shape learning, shows that neural implicit models are effective in generating 3D human surfaces from limited number of views, and even from a single RGB image. However, existing monocular approaches still struggle to recover fine geometric details such as face, hands or cloth wrinkles. They are also easily prone to depth ambiguities that result in distorted geometries along
-
HeadEvolver: Text to Head Avatars via Locally Learnable Mesh Deformation arXiv.cs.GR Pub Date : 2024-03-14 Duotun Wang, Hengyu Meng, Zeyu Cai, Zhijing Shao, Qianxi Liu, Lin Wang, Mingming Fan, Ying Shan, Xiaohang Zhan, Zeyu Wang
We present HeadEvolver, a novel framework to generate stylized head avatars from text guidance. HeadEvolver uses locally learnable mesh deformation from a template head mesh, producing high-quality digital assets for detail-preserving editing and animation. To tackle the challenges of lacking fine-grained and semantic-aware local shape control in global deformation through Jacobians, we introduce a
-
A New Split Algorithm for 3D Gaussian Splatting arXiv.cs.GR Pub Date : 2024-03-14 Qiyuan Feng, Gengchen Cao, Haoxiang Chen, Tai-Jiang Mu, Ralph R. Martin, Shi-Min Hu
3D Gaussian splatting models, as a novel explicit 3D representation, have been applied in many domains recently, such as explicit geometric editing and geometry generation. Progress has been rapid. However, due to their mixed scales and cluttered shapes, 3D Gaussian splatting models can produce a blurred or needle-like effect near the surface. At the same time, 3D Gaussian splatting models tend to
-
VisionGPT-3D: A Generalized Multimodal Agent for Enhanced 3D Vision Understanding arXiv.cs.GR Pub Date : 2024-03-14 Chris Kelly, Luhui Hu, Jiayin Hu, Yu Tian, Deshun Yang, Bang Yang, Cindy Yang, Zihao Li, Zaoshan Huang, Yuexian Zou
The evolution of text to visual components facilitates people's daily lives, such as generating image, videos from text and identifying the desired elements within the images. Computer vision models involving the multimodal abilities in the previous days are focused on image detection, classification based on well-defined objects. Large language models (LLMs) introduces the transformation from nature
-
Surface-aware Mesh Texture Synthesis with Pre-trained 2D CNNs arXiv.cs.GR Pub Date : 2024-03-11 Áron Samuel Kovács, Pedro Hermosilla, Renata G. Raidou
Mesh texture synthesis is a key component in the automatic generation of 3D content. Existing learning-based methods have drawbacks -- either by disregarding the shape manifold during texture generation or by requiring a large number of different views to mitigate occlusion-related inconsistencies. In this paper, we present a novel surface-aware approach for mesh texture synthesis that overcomes these
-
Inverse Garment and Pattern Modeling with a Differentiable Simulator arXiv.cs.GR Pub Date : 2024-03-11 Boyang Yu, Frederic Cordier, Hyewon Seo
The capability to generate simulation-ready garment models from 3D shapes of clothed humans will significantly enhance the interpretability of captured geometry of real garments, as well as their faithful reproduction in the virtual world. This will have notable impact on fields like shape capture in social VR, and virtual try-on in the fashion industry. To align with the garment modeling process standardized
-
Vertex Block Descent arXiv.cs.GR Pub Date : 2024-03-10 Anka He Chen, Ziheng Liu, Yin Yang, Cem Yuksel
We introduce vertex block descent, a block coordinate descent solution for the variational form of implicit Euler through vertex-level Gauss-Seidel iterations. It operates with local vertex position updates that achieve reductions in global variational energy with maximized parallelism. This forms a physics solver that can achieve numerical convergence with unconditional stability and exceptional computation
-
FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications arXiv.cs.GR Pub Date : 2024-03-11 Yuki Tatsukawa, I-Chao Shen, Anran Qi, Yuki Koyama, Takeo Igarashi, Ariel Shamir
Acquiring the desired font for various design tasks can be challenging and requires professional typographic knowledge. While previous font retrieval or generation works have alleviated some of these difficulties, they often lack support for multiple languages and semantic attributes beyond the training data domains. To solve this problem, we present FontCLIP: a model that connects the semantic understanding
-
Cyclic Polygon Plots arXiv.cs.GR Pub Date : 2024-03-08 Maksim Schreck, Peter Albers, Filip Sadlo
In this paper, we introduce the cyclic polygon plot, a representation based on a novel projection concept for multi-dimensional values. Cyclic polygon plots combine the typically competing requirements of quantitativeness, image-space efficiency, and readability. Our approach is complemented with a placement strategy based on its intrinsic features, resulting in a dimensionality reduction strategy
-
GSEdit: Efficient Text-Guided Editing of 3D Objects via Gaussian Splatting arXiv.cs.GR Pub Date : 2024-03-08 Francesco Palandra, Andrea Sanchietti, Daniele Baieri, Emanuele Rodolà
We present GSEdit, a pipeline for text-guided 3D object editing based on Gaussian Splatting models. Our method enables the editing of the style and appearance of 3D objects without altering their main details, all in a matter of minutes on consumer hardware. We tackle the problem by leveraging Gaussian splatting to represent 3D scenes, and we optimize the model while progressively varying the image
-
Finding Waldo: Towards Efficient Exploration of NeRF Scene Space arXiv.cs.GR Pub Date : 2024-03-07 Evangelos Skartados, Mehmet Kerim Yucel, Bruno Manganelli, Anastasios Drosou, Albert Saà-Garriga
Neural Radiance Fields (NeRF) have quickly become the primary approach for 3D reconstruction and novel view synthesis in recent years due to their remarkable performance. Despite the huge interest in NeRF methods, a practical use case of NeRFs has largely been ignored; the exploration of the scene space modelled by a NeRF. In this paper, for the first time in the literature, we propose and formally
-
Online Photon Guiding with 3D Gaussians for Caustics Rendering arXiv.cs.GR Pub Date : 2024-03-06 Jiawei Huang, Hajime Tanaka, Taku Komura, Yoshifumi Kitamura
In production rendering systems, caustics are typically rendered via photon mapping and gathering, a process often hindered by insufficient photon density. In this paper, we propose a novel photon guiding method to improve the photon density and overall quality for caustic rendering. The key insight of our approach is the application of a global 3D Gaussian mixture model, used in conjunction with an
-
Implicit-Explicit simulation of Mass-Spring-Charge Systems arXiv.cs.GR Pub Date : 2024-03-05 Zhiyuan Zhang, Zhaocheng Liu, Stefanos Papanicolopulos, Kartic Subr
Point masses connected by springs, or mass-spring systems, are widely used in computer animation to approximate the behavior of deformable objects. One of the restrictions imposed by these models is that points that are not topologically constrained (linked by a spring) are unable to interact with each other explicitly. Such interactions would introduce a new dimension for artistic control and animation
-
Towards Geometric-Photometric Joint Alignment for Facial Mesh Registration arXiv.cs.GR Pub Date : 2024-03-05 Xizhi Wang, Yaxiong Wang, Mengjian Li
This paper presents a Geometric-Photometric Joint Alignment(GPJA) method, for accurately aligning human expressions by combining geometry and photometric information. Common practices for registering human heads typically involve aligning landmarks with facial template meshes using geometry processing approaches, but often overlook photometric consistency. GPJA overcomes this limitation by leveraging
-
MagicClay: Sculpting Meshes With Generative Neural Fields arXiv.cs.GR Pub Date : 2024-03-04 Amir Barda, Vladimir G. Kim, Noam Aigerman, Amit H. Bermano, Thibault Groueix
The recent developments in neural fields have brought phenomenal capabilities to the field of shape generation, but they lack crucial properties, such as incremental control - a fundamental requirement for artistic work. Triangular meshes, on the other hand, are the representation of choice for most geometry related tasks, offering efficiency and intuitive control, but do not lend themselves to neural
-
VR Research at Fraunofer IGD, Darmstadt, Germany arXiv.cs.GR Pub Date : 2024-03-03 Wolfgang Felger, Martin Göbel, Dirk Reiners, Gabriel Zachmann
We present a historical outline of the research and developments of Virtual Reality at the Fraunhofer Institute for Computer Graphics (IGD) in Darmstadt, Germany, from 1990 through 2000.
-
DaReNeRF: Direction-aware Representation for Dynamic Scenes arXiv.cs.GR Pub Date : 2024-03-04 Ange Lou, Benjamin Planche, Zhongpai Gao, Yamin Li, Tianyu Luan, Hao Ding, Terrence Chen, Jack Noble, Ziyan Wu
Addressing the intricate challenge of modeling and re-rendering dynamic scenes, most recent approaches have sought to simplify these complexities using plane-based explicit representations, overcoming the slow training time issues associated with methods like Neural Radiance Fields (NeRF) and implicit representations. However, the straightforward decomposition of 4D dynamic scenes into multiple 2D
-
Spectrum AUC Difference (SAUCD): Human-aligned 3D Shape Evaluation arXiv.cs.GR Pub Date : 2024-03-03 Tianyu Luan, Zhong Li, Lele Chen, Xuan Gong, Lichang Chen, Yi Xu, Junsong Yuan
Existing 3D mesh shape evaluation metrics mainly focus on the overall shape but are usually less sensitive to local details. This makes them inconsistent with human evaluation, as human perception cares about both overall and detailed shape. In this paper, we propose an analytic metric named Spectrum Area Under the Curve Difference (SAUCD) that demonstrates better consistency with human evaluation
-
G3DR: Generative 3D Reconstruction in ImageNet arXiv.cs.GR Pub Date : 2024-03-01 Pradyumna Reddy, Ismail Elezi, Jiankang Deng
We introduce a novel 3D generative method, Generative 3D Reconstruction (G3DR) in ImageNet, capable of generating diverse and high-quality 3D objects from single images, addressing the limitations of existing methods. At the heart of our framework is a novel depth regularization technique that enables the generation of scenes with high-geometric fidelity. G3DR also leverages a pretrained language-vision
-
Hybrid Base Complex: Extract and Visualize Structure of Hex-dominant Meshes arXiv.cs.GR Pub Date : 2024-03-01 Lei Si, Haowei Cao, Guoning Chen
Hex-dominant mesh generation has received significant attention in recent research due to its superior robustness compared to pure hex-mesh generation techniques. In this work, we introduce the first structure for analyzing hex-dominant meshes. This structure builds on the base complex of pure hex-meshes but incorporates the non-hex elements for a more comprehensive and complete representation. We
-
Formalizing Feint Actions, and Example Studies in Two-Player Games arXiv.cs.GR Pub Date : 2024-03-03 Junyu Liu, Wangkai Jin, Xiangjun Peng
Feint actions refer to a set of deceptive actions, which enable players to obtain temporal advantages from their opponents. Such actions are regarded as widely-used tactic in most non-deterministic Two-player Games (e.g. boxing and fencing). However, existing literature does not provide comprehensive and concrete formalization on Feint actions, and their implications on Two-Player Games. We argue that
-
3D Gaussian Model for Animation and Texturing arXiv.cs.GR Pub Date : 2024-02-29 Xiangzhi Eric Wang, Zackary P. T. Sin
3D Gaussian Splatting has made a marked impact on neural rendering by achieving impressive fidelity and performance. Despite this achievement, however, it is not readily applicable to developing interactive applications. Real-time applications like XR apps and games require functions such as animation, UV-mapping, and model editing simultaneously manipulated through the usage of a 3D model. We propose
-
Learning a Generalized Physical Face Model From Data arXiv.cs.GR Pub Date : 2024-02-29 Lingchen Yang, Gaspard Zoss, Prashanth Chandran, Markus Gross, Barbara Solenthaler, Eftychios Sifakis, Derek Bradley
Physically-based simulation is a powerful approach for 3D facial animation as the resulting deformations are governed by physical constraints, allowing to easily resolve self-collisions, respond to external forces and perform realistic anatomy edits. Today's methods are data-driven, where the actuations for finite elements are inferred from captured skin geometry. Unfortunately, these approaches have
-
NToP: NeRF-Powered Large-scale Dataset Generation for 2D and 3D Human Pose Estimation in Top-View Fisheye Images arXiv.cs.GR Pub Date : 2024-02-28 Jingrui Yu, Dipankar Nandi, Roman Seidel, Gangolf Hirtz
Human pose estimation (HPE) in the top-view using fisheye cameras presents a promising and innovative application domain. However, the availability of datasets capturing this viewpoint is extremely limited, especially those with high-quality 2D and 3D keypoint annotations. Addressing this gap, we leverage the capabilities of Neural Radiance Fields (NeRF) technique to establish a comprehensive pipeline
-
Non-Euclidean Sliced Optimal Transport Sampling arXiv.cs.GR Pub Date : 2024-02-26 Baptiste Genest, Nicolas Courty, David Coeurjolly
In machine learning and computer graphics, a fundamental task is the approximation of a probability density function through a well-dispersed collection of samples. Providing a formal metric for measuring the distance between probability measures on general spaces, Optimal Transport (OT) emerges as a pivotal theoretical framework within this context. However, the associated computational burden is
-
CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and Editing arXiv.cs.GR Pub Date : 2024-02-27 Chufeng Xiao, Hongbo Fu
Personalization techniques for large text-to-image (T2I) models allow users to incorporate new concepts from reference images. However, existing methods primarily rely on textual descriptions, leading to limited control over customized images and failing to support fine-grained and local editing (e.g., shape, pose, and details). In this paper, we identify sketches as an intuitive and versatile representation
-
CharNeRF: 3D Character Generation from Concept Art arXiv.cs.GR Pub Date : 2024-02-27 Eddy Chu, Yiyang Chen, Chedy Raissi, Anand Bhojan
3D modeling holds significant importance in the realms of AR/VR and gaming, allowing for both artistic creativity and practical applications. However, the process is often time-consuming and demands a high level of skill. In this paper, we present a novel approach to create volumetric representations of 3D characters from consistent turnaround concept art, which serves as the standard input in the
-
Transparent Image Layer Diffusion using Latent Transparency arXiv.cs.GR Pub Date : 2024-02-27 Lvmin Zhang, Maneesh Agrawala
We present LayerDiffusion, an approach enabling large-scale pretrained latent diffusion models to generate transparent images. The method allows generation of single transparent images or of multiple transparent layers. The method learns a "latent transparency" that encodes alpha channel transparency into the latent manifold of a pretrained latent diffusion model. It preserves the production-ready
-
2+2D Texture for Full Positive Parallax Effect arXiv.cs.GR Pub Date : 2024-02-26 Alexandre Yip Gonçalves Dias, Marcelo Knörich Zuffo
The representation of parallax on virtual environment is still a problem to be studied. Common algorithms, such as Bump Mapping, Parallax Mapping and Displacement Mapping, treats this problem for small disparity between a real object and a simplified model. This work will introduce a new texture structure and one possible render algorithm able to display parallax for large disparities, it is an approach
-
Cinematographic Camera Diffusion Model arXiv.cs.GR Pub Date : 2024-02-25 Hongda Jiang, Xi Wang, Marc Christie, Libin Liu, Baoquan Chen
Designing effective camera trajectories in virtual 3D environments is a challenging task even for experienced animators. Despite an elaborate film grammar, forged through years of experience, that enables the specification of camera motions through cinematographic properties (framing, shots sizes, angles, motions), there are endless possibilities in deciding how to place and move cameras with characters
-
CharacterMixer: Rig-Aware Interpolation of 3D Characters arXiv.cs.GR Pub Date : 2024-02-23 Xiao Zhan, Rao Fu, Daniel Ritchie
We present CharacterMixer, a system for blending two rigged 3D characters with different mesh and skeleton topologies while maintaining a rig throughout interpolation. CharacterMixer also enables interpolation during motion for such characters, a novel feature. Interpolation is an important shape editing operation, but prior methods have limitations when applied to rigged characters: they either ignore
-
CMC: Few-shot Novel View Synthesis via Cross-view Multiplane Consistency arXiv.cs.GR Pub Date : 2024-02-26 Hanxin Zhu, Tianyu He, Zhibo Chen
Neural Radiance Field (NeRF) has shown impressive results in novel view synthesis, particularly in Virtual Reality (VR) and Augmented Reality (AR), thanks to its ability to represent scenes continuously. However, when just a few input view images are available, NeRF tends to overfit the given views and thus make the estimated depths of pixels share almost the same value. Unlike previous methods that
-
Mochi: Fast \& Exact Collision Detection arXiv.cs.GR Pub Date : 2024-02-22 Durga Keerthi Mandarapu, Nicholas James, Milind Kulkarni
Collision Detection (CD) has several applications across the domains such as robotics, visual graphics, and fluid mechanics. Finding exact collisions between the objects in the scene is quite computationally intensive. To quickly filter the object pairs that do not result in a collision, bounding boxes are built on the objects, indexed using a Bounding Volume Hierarchy(BVH), and tested for intersection
-
FrameNeRF: A Simple and Efficient Framework for Few-shot Novel View Synthesis arXiv.cs.GR Pub Date : 2024-02-22 Yan Xing, Pan Wang, Ligang Liu, Daolun Li, Li Zhang
We present a novel framework, called FrameNeRF, designed to apply off-the-shelf fast high-fidelity NeRF models with fast training speed and high rendering quality for few-shot novel view synthesis tasks. The training stability of fast high-fidelity models is typically constrained to dense views, making them unsuitable for few-shot novel view synthesis tasks. To address this limitation, we utilize a
-
TaylorGrid: Towards Fast and High-Quality Implicit Field Learning via Direct Taylor-based Grid Optimization arXiv.cs.GR Pub Date : 2024-02-22 Renyi Mao, Qingshan Xu, Peng Zheng, Ye Wang, Tieru Wu, Rui Ma
Coordinate-based neural implicit representation or implicit fields have been widely studied for 3D geometry representation or novel view synthesis. Recently, a series of efforts have been devoted to accelerating the speed and improving the quality of the coordinate-based implicit field learning. Instead of learning heavy MLPs to predict the neural implicit values for the query coordinates, neural voxels
-
MVD$^2$: Efficient Multiview 3D Reconstruction for Multiview Diffusion arXiv.cs.GR Pub Date : 2024-02-22 Xin-Yang Zheng, Hao Pan, Yu-Xiao Guo, Xin Tong, Yang Liu
As a promising 3D generation technique, multiview diffusion (MVD) has received a lot of attention due to its advantages in terms of generalizability, quality, and efficiency. By finetuning pretrained large image diffusion models with 3D data, the MVD methods first generate multiple views of a 3D object based on an image or text prompt and then reconstruct 3D shapes with multiview 3D reconstruction
-
Mip-Grid: Anti-aliased Grid Representations for Neural Radiance Fields arXiv.cs.GR Pub Date : 2024-02-22 Seungtae Nam, Daniel Rho, Jong Hwan Ko, Eunbyung Park
Despite the remarkable achievements of neural radiance fields (NeRF) in representing 3D scenes and generating novel view images, the aliasing issue, rendering "jaggies" or "blurry" images at varying camera distances, remains unresolved in most existing approaches. The recently proposed mip-NeRF has addressed this challenge by rendering conical frustums instead of rays. However, it relies on MLP architecture
-
Real-time High-resolution View Synthesis of Complex Scenes with Explicit 3D Visibility Reasoning arXiv.cs.GR Pub Date : 2024-02-20 Tiansong Zhou, Yebin Liu, Xuangeng Chu, Chengkun Cao, Changyin Zhou, Fei Yu, Yu Li
Rendering photo-realistic novel-view images of complex scenes has been a long-standing challenge in computer graphics. In recent years, great research progress has been made on enhancing rendering quality and accelerating rendering speed in the realm of view synthesis. However, when rendering complex dynamic scenes with sparse views, the rendering quality remains limited due to occlusion problems.
-
Persistent Homology-Driven Optimization of Effective Relative Density Range for Triply Periodic Minimal Surface arXiv.cs.GR Pub Date : 2024-02-19 Gao Depeng, Zhang Yuanzhi, Lin Hongwei
Triply periodic minimal surfaces (TPMSs) play a vital role in the design of porous structures, with applications in bone tissue engineering, chemical engineering, and the creation of lightweight models. However, fabrication of TPMSs via additive manufacturing is feasible only within a specific range of relative densities, termed the effective relative density range (EDR), outside of which TPMSs exhibit
-
Periodic Implicit Representation, Design and Optimization of Porous Structures Using Periodic B-splines arXiv.cs.GR Pub Date : 2024-02-19 Gao Depeng, Gao Yang, Lin Hongwei
Porous structures are intricate solid materials with numerous small pores, extensively used in fields like medicine, chemical engineering, and aerospace. However, the design of such structures using computer-aided tools is a time-consuming and tedious process.In this study, we propose a novel representation method and design approach for porous units that can be infinitely spliced to form a porous
-
DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generation arXiv.cs.GR Pub Date : 2024-02-19 Chong Zeng, Yue Dong, Pieter Peers, Youkang Kong, Hongzhi Wu, Xin Tong
This paper presents a novel method for exerting fine-grained lighting control during text-driven diffusion-based image generation. While existing diffusion models already have the ability to generate images under any lighting condition, without additional guidance these models tend to correlate image content and lighting. Moreover, text prompts lack the necessary expressional power to describe detailed
-
GaussianHair: Hair Modeling and Rendering with Light-aware Gaussians arXiv.cs.GR Pub Date : 2024-02-16 Haimin Luo, Min Ouyang, Zijun Zhao, Suyi Jiang, Longwen Zhang, Qixuan Zhang, Wei Yang, Lan Xu, Jingyi Yu
Hairstyle reflects culture and ethnicity at first glance. In the digital era, various realistic human hairstyles are also critical to high-fidelity digital human assets for beauty and inclusivity. Yet, realistic hair modeling and real-time rendering for animation is a formidable challenge due to its sheer number of strands, complicated structures of geometry, and sophisticated interaction with light
-
Deep Spectral Meshes: Multi-Frequency Facial Mesh Processing with Graph Neural Networks arXiv.cs.GR Pub Date : 2024-02-15 Robert Kosk, Richard Southern, Lihua You, Shaojun Bian, Willem Kokke, Greg Maguire
With the rising popularity of virtual worlds, the importance of data-driven parametric models of 3D meshes has grown rapidly. Numerous applications, such as computer vision, procedural generation, and mesh editing, vastly rely on these models. However, current approaches do not allow for independent editing of deformations at different frequency levels. They also do not benefit from representing deformations
-
GaussianObject: Just Taking Four Images to Get A High-Quality 3D Object with Gaussian Splatting arXiv.cs.GR Pub Date : 2024-02-15 Chen Yang, Sikuang Li, Jiemin Fang, Ruofan Liang, Lingxi Xie, Xiaopeng Zhang, Wei Shen, Qi Tian
Reconstructing and rendering 3D objects from highly sparse views is of critical importance for promoting applications of 3D vision techniques and improving user experience. However, images from sparse views only contain very limited 3D information, leading to two significant challenges: 1) Difficulty in building multi-view consistency as images for matching are too few; 2) Partially omitted or highly
-
DUDF: Differentiable Unsigned Distance Fields with Hyperbolic Scaling arXiv.cs.GR Pub Date : 2024-02-14 Miguel Fainstein, Viviana Siless, Emmanuel Iarussi
In recent years, there has been a growing interest in training Neural Networks to approximate Unsigned Distance Fields (UDFs) for representing open surfaces in the context of 3D reconstruction. However, UDFs are non-differentiable at the zero level set which leads to significant errors in distances and gradients, generally resulting in fragmented and discontinuous surfaces. In this paper, we propose
-
NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFs arXiv.cs.GR Pub Date : 2024-02-13 Michael Fischer, Zhengqin Li, Thu Nguyen-Phuoc, Aljaz Bozic, Zhao Dong, Carl Marshall, Tobias Ritschel
A Neural Radiance Field (NeRF) encodes the specific relation of 3D geometry and appearance of a scene. We here ask the question whether we can transfer the appearance from a source NeRF onto a target 3D geometry in a semantically meaningful way, such that the resulting new NeRF retains the target geometry but has an appearance that is an analogy to the source NeRF. To this end, we generalize classic
-
Regional Adaptive Metropolis Light Transport arXiv.cs.GR Pub Date : 2024-02-13 Hisanari Otsu, Killian Herveau, Johannes Hanika, Derek Nowrouzezahrai, Carsten Dachsbacher
The design of the proposal distributions, and most notably the kernel parameters, are crucial for the performance of Markov chain Monte Carlo (MCMC) rendering. A poor selection of parameters can increase the correlation of the Markov chain and result in bad rendering performance. We approach this problem by a novel path perturbation strategy for online-learning of state-dependent kernel parameters
-
Sophia-in-Audition: Virtual Production with a Robot Performer arXiv.cs.GR Pub Date : 2024-02-10 Taotao Zhou, Teng Xu, Dong Zhang, Yuyang Jiao, Peijun Xu, Yaoyu He, Lan Xu, Jingyi Yu
We present Sophia-in-Audition (SiA), a new frontier in virtual production, by employing the humanoid robot Sophia within an UltraStage environment composed of a controllable lighting dome coupled with multiple cameras. We demonstrate Sophia's capability to replicate iconic film segments, follow real performers, and perform a variety of motions and expressions, showcasing her versatility as a virtual
-
3D Gaussian as a New Vision Era: A Survey arXiv.cs.GR Pub Date : 2024-02-11 Ben Fei, Jingyi Xu, Rui Zhang, Qingyuan Zhou, Weidong Yang, Ying He
3D Gaussian Splatting (3D-GS) has emerged as a significant advancement in the field of Computer Graphics, offering explicit scene representation and novel view synthesis without the reliance on neural networks, such as Neural Radiance Fields (NeRF). This technique has found diverse applications in areas such as robotics, urban mapping, autonomous navigation, and virtual reality/augmented reality, just
-
Oriented-grid Encoder for 3D Implicit Representations arXiv.cs.GR Pub Date : 2024-02-09 Arihant Gaur, G. Dias Pais, Pedro Miraldo
Encoding 3D points is one of the primary steps in learning-based implicit scene representation. Using features that gather information from neighbors with multi-resolution grids has proven to be the best geometric encoder for this task. However, prior techniques do not exploit some characteristics of most objects or scenes, such as surface normals and local smoothness. This paper is the first to exploit
-
Collaborative Control for Geometry-Conditioned PBR Image Generation arXiv.cs.GR Pub Date : 2024-02-08 Shimon Vainer, Mark Boss, Mathias Parger, Konstantin Kutsy, Dante De Nigris, Ciara Rowles, Nicolas Perony, Simon Donné
Current 3D content generation builds on generative models that output RGB images. Modern graphics pipelines, however, require physically-based rendering (PBR) material properties. We propose to model the PBR image distribution directly to avoid photometric inaccuracies in RGB generation and the inherent ambiguity in extracting PBR from RGB. Existing paradigms for cross-modal finetuning are not suited
-
AvatarMMC: 3D Head Avatar Generation and Editing with Multi-Modal Conditioning arXiv.cs.GR Pub Date : 2024-02-08 Wamiq Reyaz Para, Abdelrahman Eldesokey, Zhenyu Li, Pradyumna Reddy, Jiankang Deng, Peter Wonka
We introduce an approach for 3D head avatar generation and editing with multi-modal conditioning based on a 3D Generative Adversarial Network (GAN) and a Latent Diffusion Model (LDM). 3D GANs can generate high-quality head avatars given a single or no condition. However, it is challenging to generate samples that adhere to multiple conditions of different modalities. On the other hand, LDMs excel at
-
M2fNet: Multi-modal Forest Monitoring Network on Large-scale Virtual Dataset arXiv.cs.GR Pub Date : 2024-02-07 Yawen Lu, Yunhan Huang, Su Sun, Tansi Zhang, Xuewen Zhang, Songlin Fei, Victor Chen
Forest monitoring and education are key to forest protection, education and management, which is an effective way to measure the progress of a country's forest and climate commitments. Due to the lack of a large-scale wild forest monitoring benchmark, the common practice is to train the model on a common outdoor benchmark (e.g., KITTI) and evaluate it on real forest datasets (e.g., CanaTree100). However
-
NeRF as Non-Distant Environment Emitter in Physics-based Inverse Rendering arXiv.cs.GR Pub Date : 2024-02-07 Jingwang Ling, Ruihan Yu, Feng Xu, Chun Du, Shuang Zhao
Physics-based inverse rendering aims to jointly optimize shape, materials, and lighting from captured 2D images. Here lighting is an important part of achieving faithful light transport simulation. While the environment map is commonly used as the lighting model in inverse rendering, we show that its distant lighting assumption leads to spatial invariant lighting, which can be an inaccurate approximation
-
GSN: Generalisable Segmentation in Neural Radiance Field arXiv.cs.GR Pub Date : 2024-02-07 Vinayak Gupta, Rahul Goel, Sirikonda Dhawal, P. J. Narayanan
Traditional Radiance Field (RF) representations capture details of a specific scene and must be trained afresh on each scene. Semantic feature fields have been added to RFs to facilitate several segmentation tasks. Generalised RF representations learn the principles of view interpolation. A generalised RF can render new views of an unknown and untrained scene, given a few views. We present a way to
-
VRMM: A Volumetric Relightable Morphable Head Model arXiv.cs.GR Pub Date : 2024-02-06 Haotian Yang, Mingwu Zheng, Chongyang Ma, Yu-Kun Lai, Pengfei Wan, Haibin Huang
In this paper, we introduce the Volumetric Relightable Morphable Model (VRMM), a novel volumetric and parametric facial prior for 3D face modeling. While recent volumetric prior models offer improvements over traditional methods like 3D Morphable Models (3DMMs), they face challenges in model learning and personalized reconstructions. Our VRMM overcomes these by employing a novel training framework
-
DARTS: Diffusion Approximated Residual Time Sampling for Low Variance Time-of-flight Rendering in Homogeneous Scattering Medium arXiv.cs.GR Pub Date : 2024-02-05 Qianyue He, Xin Jin, Haitian Jiang, Dongyu Du
Time-of-flight (ToF) devices have greatly propelled the advancement of various multi-modal perception applications. However, due to complexity in both sampling path construction and vertex connection in time domain, it is extremely challenging to accurately render time-resolved information in ToF device simulation, particularly in scenes involving complex geometric structures, diverse materials and
-
TensoSDF: Roughness-aware Tensorial Representation for Robust Geometry and Material Reconstruction arXiv.cs.GR Pub Date : 2024-02-05 Jia Li, Lu Wang, Lei Zhang, Beibei Wang
Reconstructing objects with realistic materials from multi-view images is problematic, since it is highly ill-posed. Although the neural reconstruction approaches have exhibited impressive reconstruction ability, they are designed for objects with specific materials (e.g., diffuse or specular materials). To this end, we propose a novel framework for robust geometry and material reconstruction, where
-
CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimization arXiv.cs.GR Pub Date : 2024-02-04 Jingyu Hu, Ka-Hei Hui, Zhengzhe Liu, Hao Zhang, Chi-Wing Fu
This paper introduces a new approach based on a coupled representation and a neural volume optimization to implicitly perform 3D shape editing in latent space. This work has three innovations. First, we design the coupled neural shape (CNS) representation for supporting 3D shape editing. This representation includes a latent code, which captures high-level global semantics of the shape, and a 3D neural
-
Surface Reconstruction Using Rotation Systems arXiv.cs.GR Pub Date : 2024-02-02 Ruiqi Cui, Emil Toftegaard Gæde, Eva Rotenberg, Leif Kobbelt, J. Andreas Bærentzen
Inspired by the seminal result that a graph and an associated rotation system uniquely determine the topology of a closed manifold, we propose a combinatorial method for reconstruction of surfaces from points. Our method constructs a spanning tree and a rotation system. Since the tree is trivially a planar graph, its rotation system determines a genus zero surface with a single face which we proceed