-
A Closest Point Method for PDEs on Manifolds with Interior Boundary Conditions for Geometry Processing ACM Trans. Graph. (IF 7.8) Pub Date : 2024-06-17 Nathan King, Haozhe Su, Mridul Aanjaneya, Steven Ruuth, Christopher Batty
Many geometry processing techniques require the solution of partial differential equations (PDEs) on manifolds embedded in \(\mathbb {R}^2 \) or \(\mathbb {R}^3 \), such as curves or surfaces. Such manifold PDEs often involve boundary conditions (e.g., Dirichlet or Neumann) prescribed at points or curves on the manifold’s interior or along the geometric (exterior) boundary of an open manifold. However
-
Analytic rotation-invariant modelling of anisotropic finite elements ACM Trans. Graph. (IF 7.8) Pub Date : 2024-05-28 Huancheng Lin, Floyd Mulenga Chitalu, Taku Komura
Anisotropic hyperelastic distortion energies are used to solve many problems in fields like computer graphics and engineering with applications in shape analysis, deformation, design, mesh parameterization, biomechanics and more. However, formulating a robust anisotropic energy that is low-order and yet sufficiently non-linear remains a challenging problem for achieving the convergence promised by
-
A Framework for Solving Parabolic Partial Differential Equations on Discrete Domains ACM Trans. Graph. (IF 7.8) Pub Date : 2024-05-28 Leticia Mattos Da Silva, Oded Stein, Justin Solomon
We introduce a framework for solving a class of parabolic partial differential equations on triangle mesh surfaces, including the Hamilton-Jacobi equation and the Fokker-Planck equation. PDE in this class often have nonlinear or stiff terms that cannot be resolved with standard methods on curved triangle meshes. To address this challenge, we leverage a splitting integrator combined with a convex optimization
-
Differentiable solver for time-dependent deformation problems with contact ACM Trans. Graph. (IF 7.8) Pub Date : 2024-05-22 Zizhou Huang, Davi Colli Tozoni, Arvi Gjoka, Zachary Ferguson, Teseo Schneider, Daniele Panozzo, Denis Zorin
We introduce a general differentiable solver for time-dependent deformation problems with contact and friction. Our approach uses a finite element discretization with a high-order time integrator coupled with the recently proposed incremental potential contact method for handling contact and friction forces to solve ODE- and PDE-constrained optimization problems on scenes with complex geometry. It
-
View-Independent Adjoint Light Tracing for Lighting Design Optimization ACM Trans. Graph. (IF 7.8) Pub Date : 2024-05-22 Lukas Lipp, David Hahn, Pierre Ecormier-Nocca, Florian Rist, Michael Wimmer
Differentiable rendering methods promise the ability to optimize various parameters of three-dimensional (3D) scenes to achieve a desired result. However, lighting design has so far received little attention in this field. In this article, we introduce a method that enables continuous optimization of the arrangement of luminaires in a 3D scene via differentiable light tracing. Our experiments show
-
Plug-and-Play Algorithms for Dynamic Non-line-of-sight Imaging ACM Trans. Graph. (IF 7.8) Pub Date : 2024-05-14 Juntian Ye, Yu Hong, Xiongfei Su, Xin Yuan, Feihu Xu
Non-line-of-sight (NLOS) imaging has the ability to recover 3D images of scenes outside the direct line of sight, which is of growing interest for diverse applications. Despite the remarkable progress, NLOS imaging of dynamic objects is still challenging. It requires a large amount of multibounce photons for the reconstruction of single frame data. To overcome this obstacle, we develop a computational
-
I❤MESH: A DSL for Mesh Processing ACM Trans. Graph. (IF 7.8) Pub Date : 2024-05-01 Yong Li, Shoaib Kamil, Keenan Crane, Alec Jacobson, Yotam Gingold
Mesh processing algorithms are often communicated via concise mathematical notation (e.g., summation over mesh neighborhoods). However, conversion of notation into working code remains a time consuming and error-prone process which requires arcane knowledge of low-level data structures and libraries—impeding rapid exploration of high-level algorithms. We address this problem by introducing a domain-specific
-
Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022 ACM Trans. Graph. (IF 7.8) Pub Date : 2024-04-27 Taras Kucherenko, Pieter Wolfert, Youngwoo Yoon, Carla Viegas, Teodor Nikolov, Mihail Tsakov, Gustav Eje Henter
This paper reports on the second GENEA Challenge to benchmark data-driven automatic co-speech gesture generation. Participating teams used the same speech and motion dataset to build gesture-generation systems. Motion generated by all these systems was rendered to video using a standardised visualisation pipeline and evaluated in several large, crowdsourced user studies. Unlike when comparing different
-
Real-Time Neural Appearance Models ACM Trans. Graph. (IF 7.8) Pub Date : 2024-04-20 Tizian Zeltner, Fabrice Rousselle, Andrea Weidlich, Petrik Clarberg, Jan Novák, Benedikt Bitterli, Alex Evans, Tomáš Davidovič, Simon Kallweit, Aaron Lefohn
We present a complete system for real-time rendering of scenes with complex appearance previously reserved for offline use. This is achieved with a combination of algorithmic and system level innovations. Our appearance model utilizes learned hierarchical textures that are interpreted using neural decoders, which produce reflectance values and importance-sampled directions. To best utilize the modeling
-
ConceptLab: Creative Concept Generation using VLM-Guided Diffusion Prior Constraints ACM Trans. Graph. (IF 7.8) Pub Date : 2024-04-16 Elad Richardson, Kfir Goldberg, Yuval Alaluf, Daniel Cohen-Or
Recent text-to-image generative models have enabled us to transform our words into vibrant, captivating imagery. The surge of personalization techniques that has followed has also allowed us to imagine unique concepts in new scenes. However, an intriguing question remains: How can we generate a new, imaginary concept that has never been seen before? In this paper, we present the task of creative text-to-image
-
Importance Sampling BRDF Derivatives ACM Trans. Graph. (IF 7.8) Pub Date : 2024-04-09 Yash Belhe, Bing Xu, Sai Praveen Bangaru, Ravi Ramamoorthi, Tzu-Mao Li
We propose a set of techniques to efficiently importance sample the derivatives of a wide range of Bidirectional Reflectance Distribution Function (BRDF) models. In differentiable rendering, BRDFs are replaced by their differential BRDF counterparts, which are real-valued and can have negative values. This leads to a new source of variance arising from their change in sign. Real-valued functions cannot
-
Online Neural Path Guiding with Normalized Anisotropic Spherical Gaussians ACM Trans. Graph. (IF 7.8) Pub Date : 2024-04-09 Jiawei Huang, Akito Iizuka, Hajime Tanaka, Taku Komura, Yoshifumi Kitamura
Importance sampling techniques significantly reduce variance in physically based rendering. In this article, we propose a novel online framework to learn the spatial-varying distribution of the full product of the rendering equation, with a single small neural network using stochastic ray samples. The learned distributions can be used to efficiently sample the full product of incident light. To accomplish
-
HQ3DAvatar: High-quality Implicit 3D Head Avatar ACM Trans. Graph. (IF 7.8) Pub Date : 2024-04-09 Kartik Teotia, Mallikarjun B R, Xingang Pan, Hyeongwoo Kim, Pablo Garrido, Mohamed Elgharib, Christian Theobalt
Multi-view volumetric rendering techniques have recently shown great potential in modeling and synthesizing high-quality head avatars. A common approach to capture full head dynamic performances is to track the underlying geometry using a mesh-based template or 3D cube-based graphics primitives. While these model-based approaches achieve promising results, they often fail to learn complex geometric
-
A Dual-Particle Approach for Incompressible SPH Fluids ACM Trans. Graph. (IF 7.8) Pub Date : 2024-04-09 Shusen Liu, Xiaowei He, Yuzhong Guo, Yue Chang, Wencheng Wang
Tensile instability is one of the major obstacles to particle methods in fluid simulation, which would cause particles to clump in pairs under tension and prevent fluid simulation to generate small-scale thin features. To address this issue, previous particle methods either use a background pressure or a finite difference scheme to alleviate the particle clustering artifacts, yet still fail to produce
-
Joint Stroke Tracing and Correspondence for 2D Animation ACM Trans. Graph. (IF 7.8) Pub Date : 2024-04-09 Haoran Mo, Chengying Gao, Ruomei Wang
To alleviate human labor in redrawing keyframes with ordered vector strokes for automatic inbetweening, we for the first time propose a joint stroke tracing and correspondence approach. Given consecutive raster keyframes along with a single vector image of the starting frame as a guidance, the approach generates vector drawings for the remaining keyframes while ensuring one-to-one stroke correspondence
-
DMHomo: Learning Homography with Diffusion Models ACM Trans. Graph. (IF 7.8) Pub Date : 2024-04-09 Haipeng Li, Hai Jiang, Ao Luo, Ping Tan, Haoqiang Fan, Bing Zeng, Shuaicheng Liu
Supervised homography estimation methods face a challenge due to the lack of adequate labeled training data. To address this issue, we propose DMHomo, a diffusion model-based framework for supervised homography learning. This framework generates image pairs with accurate labels, realistic image content, and realistic interval motion, ensuring that they satisfy adequate pairs. We utilize unlabeled image
-
GIPC: Fast and Stable Gauss-Newton Optimization of IPC Barrier Energy ACM Trans. Graph. (IF 7.8) Pub Date : 2024-03-23 Kemeng Huang, Floyd M. Chitalu, Huancheng Lin, Taku Komura
Barrier functions are crucial for maintaining an intersection- and inversion-free simulation trajectory but existing methods, which directly use distance can restrict implementation design and performance. We present an approach to rewriting the barrier function for arriving at an efficient and robust approximation of its Hessian. The key idea is to formulate a simplicial geometric measure of contact
-
Self-supervised High Dynamic Range Imaging: What Can Be Learned from a Single 8-bit Video? ACM Trans. Graph. (IF 7.8) Pub Date : 2024-03-23 Francesco Banterle, Demetris Marnerides, Thomas Bashford-rogers, Kurt Debattista
Recently, Deep Learning-based methods for inverse tone mapping standard dynamic range (SDR) images to obtain high dynamic range (HDR) images have become very popular. These methods manage to fill over-exposed areas convincingly both in terms of details and dynamic range. To be effective, deep learning-based methods need to learn from large datasets and transfer this knowledge to the network weights
-
NeuralVDB: High-resolution Sparse Volume Representation using Hierarchical Neural Networks ACM Trans. Graph. (IF 7.8) Pub Date : 2024-02-28 Doyub Kim, Minjae Lee, Ken Museth
We introduce NeuralVDB, which improves on an existing industry standard for efficient storage of sparse volumetric data, denoted VDB [Museth 2013], by leveraging recent advancements in machine learning. Our novel hybrid data structure can reduce the memory footprints of VDB volumes by orders of magnitude, while maintaining its flexibility and only incurring small (user-controlled) compression errors
-
DeadWood: Including Disturbance and Decay in the Depiction of Digital Nature ACM Trans. Graph. (IF 7.8) Pub Date : 2024-02-14 Adrien Peytavie, James Gain, Eric Guérin, Oscar Argudo, Eric Galin
The creation of truly believable simulated natural environments remains an unsolved problem in Computer Graphics. This is, in part, due to a lack of visual variety. In nature, apart from variation due to abiotic and biotic growth factors, a significant role is played by disturbance events, such as fires, windstorms, disease, and death and decay processes, which give rise to both standing dead trees
-
Spectral Total-variation Processing of Shapes—Theory and Applications ACM Trans. Graph. (IF 7.8) Pub Date : 2024-02-14 Jonathan Brokman, Martin Burger, Guy Gilboa
We present a comprehensive analysis of total variation (TV) on non-Euclidean domains and its eigenfunctions. We specifically address parameterized surfaces, a natural representation of the shapes used in 3D graphics. Our work sheds new light on the celebrated Beltrami and Anisotropic TV flows and explains experimental findings from recent years on shape spectral TV [Fumero et al. 2020] and adaptive
-
Creation of Dihedral Escher-like Tilings Based on As-Rigid-As-Possible Deformation ACM Trans. Graph. (IF 7.8) Pub Date : 2024-01-22 Yuichi Nagata, Shinji Imahori
An Escher-like tiling is a tiling consisting of one or a few artistic shapes of tile. This article proposes a method for generating Escher-like tilings consisting of two distinct shapes (dihedral Escher-like tilings) that are as similar as possible to the two goal shapes specified by the user. This study is an extension of a previous study that successfully generated Escher-like tilings consisting
-
Neural Wavelet-domain Diffusion for 3D Shape Generation, Inversion, and Manipulation ACM Trans. Graph. (IF 7.8) Pub Date : 2024-01-03 Jingyu Hu, Ka-Hei Hui, Zhengzhe Liu, Ruihui Li, Chi-Wing Fu
This paper presents a new approach for 3D shape generation, inversion, and manipulation, through a direct generative modeling on a continuous implicit representation in wavelet domain. Specifically, we propose a compact wavelet representation with a pair of coarse and detail coefficient volumes to implicitly represent 3D shapes via truncated signed distance functions and multi-scale biorthogonal wavelets
-
Digital Three-dimensional Smocking Design ACM Trans. Graph. (IF 7.8) Pub Date : 2024-01-03 Jing Ren, Aviv Segall, Olga Sorkine-Hornung
We develop an optimization-based method to model smocking, a surface embroidery technique that provides decorative geometric texturing while maintaining stretch properties of the fabric. During smocking, multiple pairs of points on the fabric are stitched together, creating non-manifold geometric features and visually pleasing textures. Designing smocking patterns is challenging, because the outcome
-
Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement Learning ACM Trans. Graph. (IF 7.8) Pub Date : 2024-01-03 Jia-Mu Sun, Jie Yang, Kaichun Mo, Yu-Kun Lai, Leonidas Guibas, Lin Gao
3D scene synthesis facilitates and benefits many real-world applications. Most scene generators focus on making indoor scenes plausible via learning from training data and leveraging extra constraints such as adjacency and symmetry. Although the generated 3D scenes are mostly plausible with visually realistic layouts, they can be functionally unsuitable for human users to navigate and interact with
-
Promptable Game Models: Text-guided Game Simulation via Masked Diffusion Models ACM Trans. Graph. (IF 7.8) Pub Date : 2024-01-03 Willi Menapace, Aliaksandr Siarohin, Stéphane Lathuilière, Panos Achlioptas, Vladislav Golyanik, Sergey Tulyakov, Elisa Ricci
Neural video game simulators emerged as powerful tools to generate and edit videos. Their idea is to represent games as the evolution of an environment’s state driven by the actions of its agents. While such a paradigm enables users to play a game action-by-action, its rigidity precludes more semantic forms of control. To overcome this limitation, we augment game models with prompts specified as a
-
A Unified MPM Framework Supporting Phase-field Models and Elastic-viscoplastic Phase Transition ACM Trans. Graph. (IF 7.8) Pub Date : 2024-01-03 Zaili Tu, Chen Li, Zipeng Zhao, Long Liu, Chenhui Wang, Changbo Wang, Hong Qin
Recent years have witnessed the rapid deployment of numerous physics-based modeling and simulation algorithms and techniques for fluids, solids, and their delicate coupling in computer animation. However, it still remains a challenging problem to model the complex elastic-viscoplastic behaviors during fluid–solid phase transitions and facilitate their seamless interactions inside the same framework
-
HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Field ACM Trans. Graph. (IF 7.8) Pub Date : 2023-11-30 Xiaochen Zhao, Lizhen Wang, Jingxiang Sun, Hongwen Zhang, Jinli Suo, Yebin Liu
The problem of modeling an animatable 3D human head avatar under lightweight setups is of significant importance but has not been well solved. Existing 3D representations either perform well in the realism of portrait images synthesis or the accuracy of expression control, but not both. To address the problem, we introduce a novel hybrid explicit-implicit 3D representation, Facial Model Conditioned
-
Disentangling Structure and Appearance in ViT Feature Space ACM Trans. Graph. (IF 7.8) Pub Date : 2023-11-30 Narek Tumanyan, Omer Bar-Tal, Shir Amir, Shai Bagon, Tali Dekel
We present a method for semantically transferring the visual appearance of one natural image to another. Specifically, our goal is to generate an image in which objects in a source structure image are “painted” with the visual appearance of their semantically related objects in a target appearance image. To integrate semantic information into our framework, our key idea is to leverage a pre-trained
-
Intrinsic Image Decomposition via Ordinal Shading ACM Trans. Graph. (IF 7.8) Pub Date : 2023-11-30 Chris Careaga, Yağız Aksoy
Intrinsic decomposition is a fundamental mid-level vision problem that plays a crucial role in various inverse rendering and computational photography pipelines. Generating highly accurate intrinsic decompositions is an inherently under-constrained task that requires precisely estimating continuous-valued shading and albedo. In this work, we achieve high-resolution intrinsic decomposition by breaking
-
Implicit Surface Tension for SPH Fluid Simulation ACM Trans. Graph. (IF 7.8) Pub Date : 2023-11-30 Stefan Rhys Jeske, Lukas Westhofen, Fabian Löschner, José Antonio Fernández-fernández, Jan Bender
The numerical simulation of surface tension is an active area of research in many different fields of application and has been attempted using a wide range of methods. Our contribution is the derivation and implementation of an implicit cohesion force based approach for the simulation of surface tension effects using the Smoothed Particle Hydrodynamics (SPH) method. We define a continuous formulation
-
In the Quest for Scale-Optimal Mappings ACM Trans. Graph. (IF 7.8) Pub Date : 2023-11-17 Vladimir Garanzha, Igor Kaporin, Liudmila Kudryavtseva, Francois Protais, Dmitry Sokolov
Optimal mapping is one of the longest-standing problems in computational mathematics. It is natural to measure the relative curve length error under map to assess its quality. The maximum of such error is called the quasi-isometry constant, and its minimization is a nontrivial max-norm optimization problem. We present a physics-based quasi-isometric stiffening (QIS) algorithm for the max-norm minimization
-
Latent L-systems: Transformer-based Tree Generator ACM Trans. Graph. (IF 7.8) Pub Date : 2023-11-02 Jae Joong Lee, Bosheng Li, Bedrich Benes
We show how a Transformer can encode hierarchical tree-like string structures by introducing a new deep learning-based framework for generating 3D biological tree models represented as Lindenmayer system (L-system) strings. L-systems are string-rewriting procedural systems that encode tree topology and geometry. L-systems are efficient, but creating the production rules is one of the most critical
-
Layout-aware Single-image Document Flattening ACM Trans. Graph. (IF 7.8) Pub Date : 2023-11-02 Pu Li, Weize Quan, Jianwei Guo, Dong-Ming Yan
Single image rectification of document deformation is a challenging task. Although some recent deep learning-based methods have attempted to solve this problem, they cannot achieve satisfactory results when dealing with document images with complex deformations. In this article, we propose a new efficient framework for document flattening. Our main insight is that most layout primitives in a document
-
SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data ACM Trans. Graph. (IF 7.8) Pub Date : 2023-10-31 Jose Luis Ponton, Haoran Yun, Andreas Aristidou, Carlos Andujar, Nuria Pelechano
Accurate and reliable human motion reconstruction is crucial for creating natural interactions of full-body avatars in Virtual Reality (VR) and entertainment applications. As the Metaverse and social applications gain popularity, users are seeking cost-effective solutions to create full-body animations that are comparable in quality to those produced by commercial motion capture systems. In order to
-
Decorrelating ReSTIR Samplers via MCMC Mutations ACM Trans. Graph. (IF 7.8) Pub Date : 2023-10-19 Rohan Sawhney, Daqi Lin, Markus Kettunen, Benedikt Bitterli, Ravi Ramamoorthi, Chris Wyman, Matt Pharr
Monte Carlo rendering algorithms often utilize correlations between pixels to improve efficiency and enhance image quality. For real-time applications in particular, repeated reservoir resampling offers a powerful framework to reuse samples both spatially in an image and temporally across multiple frames. While such techniques achieve equal-error up to 100 × faster for real-time direct lighting [5]
-
Real-Time Reconstruction of Fluid Flow under Unknown Disturbance ACM Trans. Graph. (IF 7.8) Pub Date : 2023-10-17 Kinfung Chu, Jiawei Huang, Hidemasa Takana, Yoshifumi Kitamura
We present a framework that captures sparse Lagrangian flow information from a volume of real liquid and reconstructs its detailed kinematic information in real time. Our framework can perform flow reconstruction even when the liquid is disturbed by an object of unknown movement and shape. Through a large dataset of liquid moving under external disturbance, an agent is trained using reinforcement learning
-
A Function-Based Approach to Interactive High-Precision Volumetric Design and Fabrication ACM Trans. Graph. (IF 7.8) Pub Date : 2023-09-29 Christopher Uchytil, Duane Storti
We present a novel function representation (F-Rep) based geometric modeling kernel tailor-made to support computer aided design (CAD) and fabrication of high resolution volumetric models containing hundreds of billions of voxel grid elements. Our modeling kernel addresses existing limitations associated with evaluating, storing, and accessing volumetric data produced by F-Reps in contexts outside of
-
Light Codes for Fast Two-Way Human-Centric Visual Communication ACM Trans. Graph. (IF 7.8) Pub Date : 2023-09-29 Mohit Gupta, Jian Wang, Karl Bayer, Shree K. Nayar
Visual codes, such as QR codes, are widely used in several applications for conveying information to users. However, user interactions based on spatial codes (e.g., displaying codes on phone screens for exchanging contact information) are often tedious, time consuming, and prone to errors due to image corruptions such as noise, blur, saturation, and perspective distortions. We propose Light Codes (LICO)
-
Joint UV Optimization and Texture Baking ACM Trans. Graph. (IF 7.8) Pub Date : 2023-09-28 Julian Knodt, Zherong Pan, Kui Wu, Xifeng Gao
Level of detail has been widely used in interactive computer graphics. In current industrial 3D modeling pipelines, artists rely on commercial software to generate highly detailed models with UV maps and then bake textures for low-poly counterparts. In these pipelines, each step is performed separately, leading to unsatisfactory visual appearances for low polygon count models. Moreover, existing texture
-
The Method of Moving Frames for Surface Global Parametrization ACM Trans. Graph. (IF 7.8) Pub Date : 2023-09-20 Guillaume Coiffier, Etienne Corman
This article introduces a new representation of surface global parametrization based on Cartan’s method of moving frames. We show that a system of structure equations, characterizing the local coordinates changes with respect to a local frame system, completely characterizes the set of possible cone parametrizations. The discretization of this system provably provides necessary and sufficient conditions
-
The Design Space of Kirchhoff Rods ACM Trans. Graph. (IF 7.8) Pub Date : 2023-09-20 Christian Hafner, Bernd Bickel
The Kirchhoff rod model describes the bending and twisting of slender elastic rods in three dimensions and has been widely studied to enable the prediction of how a rod will deform, given its geometry and boundary conditions. In this work, we study a number of inverse problems with the goal of computing the geometry of a straight rod that will automatically deform to match a curved target shape after
-
Real-Time Reconstruction of Fluid Flow under Unknown Disturbance ACM Trans. Graph. (IF 7.8) Pub Date : 2023-09-16 Kinfung Chu, Jiawei Huang, Hidemasa Takana, Yoshifumi Kitamura
We present a framework that captures sparse Lagrangian flow information from a volume of real liquid and reconstructs its detailed kinematic information in real time. Our framework can perform flow reconstruction even when the liquid is disturbed by an object of unknown movement and shape. Through a large dataset of liquid moving under external disturbance, an agent is trained using reinforcement learning
-
Seamless Parametrization with Cone and Partial Loop Control ACM Trans. Graph. (IF 7.8) Pub Date : 2023-08-30 Zohar Levi
We present a method for constructing seamless parametrization for surfaces of any genus that can handle any feasible cone configuration with any type of cones. The mapping is guaranteed to be locally injective, which is due to careful construction of a simple domain boundary polygon. The polygon’s complexity depends on the cones in the field, and it is independent of mesh geometry. The result is a
-
NeLT: Object-Oriented Neural Light Transfer ACM Trans. Graph. (IF 7.8) Pub Date : 2023-08-29 Chuankun Zheng, Yuchi Huo, Shaohua Mo, Zhihua Zhong, Zhizhen Wu, Wei Hua, Rui Wang, Hujun Bao
This article presents object-oriented neural light transfer (NeLT), a novel neural representation of the dynamic light transportation between an object and the environment. Our method disentangles the global illumination of a scene into individual objects’ light transportation represented via neural networks, then composes them explicitly. It therefore enables flexible rendering with dynamic lighting
-
CLIP-guided StyleGAN Inversion for Text-driven Real Image Editing ACM Trans. Graph. (IF 7.8) Pub Date : 2023-08-29 Ahmet Canberk Baykal, Abdul Basit Anees, Duygu Ceylan, Erkut Erdem, Aykut Erdem, Deniz Yuret
Researchers have recently begun exploring the use of StyleGAN-based models for real image editing. One particularly interesting application is using natural language descriptions to guide the editing process. Existing approaches for editing images using language either resort to instance-level latent code optimization or map predefined text prompts to some editing directions in the latent space. However
-
High-Resolution Volumetric Reconstruction for Clothed Humans ACM Trans. Graph. (IF 7.8) Pub Date : 2023-08-21 Sicong Tang, Guangyuan Wang, Qing Ran, Lingzhi Li, Li Shen, Ping Tan
We present a novel method for reconstructing clothed humans from a sparse set of, e.g., 1–6 RGB images. Despite impressive results from recent works employing deep implicit representation, we revisit the volumetric approach and demonstrate that better performance can be achieved with proper system design. The volumetric representation offers significant advantages in leveraging 3D spatial context through
-
Large-scale Terrain Authoring through Interactive Erosion Simulation ACM Trans. Graph. (IF 7.8) Pub Date : 2023-07-28 Hugo Schott, Axel Paris, Lucie Fournier, Eric Guérin, Eric Galin
Large-scale terrains are essential in the definition of virtual worlds. Given the diversity of landforms and the geomorphological complexity, there is a need for authoring techniques offering hydrological consistency without sacrificing user control. In this article, we bridge the gap between large-scale erosion simulation and authoring into an efficient framework. We set aside modeling in the elevation
-
Learning Physically Realizable Skills for Online Packing of General 3D Shapes ACM Trans. Graph. (IF 7.8) Pub Date : 2023-07-28 Hang Zhao, Zherong Pan, Yang Yu, Kai Xu
We study the problem of learning online packing skills for irregular 3D shapes, which is arguably the most challenging setting of bin packing problems. The goal is to consecutively move a sequence of 3D objects with arbitrary shapes into a designated container with only partial observations of the object sequence. We take physical realizability into account, involving physics dynamics and constraints
-
Fast GPU-based Two-way Continuous Collision Handling ACM Trans. Graph. (IF 7.8) Pub Date : 2023-07-28 Tianyu Wang, Jiong Chen, Dongping Li, Xiaowei Liu, Huamin Wang, Kun Zhou
Step-and-project is a popular method to simulate non-penetrating deformable bodies in physically based animation. The strategy is to first integrate the system in time without considering contacts and then resolve potential intersections, striking a good balance between plausibility and efficiency. However, existing methods can be defective and unsafe when using large time steps, taking risks of failure
-
Procedural Metamaterials: A Unified Procedural Graph for Metamaterial Design ACM Trans. Graph. (IF 7.8) Pub Date : 2023-07-28 Liane Makatura, Bohan Wang, Yi-Lu Chen, Bolei Deng, Chris Wojtan, Bernd Bickel, Wojciech Matusik
We introduce a compact, intuitive procedural graph representation for cellular metamaterials, which are small-scale, tileable structures that can be architected to exhibit many useful material properties. Because the structures’ “architectures” vary widely—with elements such as beams, thin shells, and solid bulks—it is difficult to explore them using existing representations. Generic approaches like
-
A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning ACM Trans. Graph. (IF 7.8) Pub Date : 2023-07-28 Yuxin Zhang, Fan Tang, Weiming Dong, Haibin Huang, Chongyang Ma, Tong-Yee Lee, Changsheng Xu
This work presents Unified Contrastive Arbitrary Style Transfer (UCAST), a novel style representation learning and transfer framework, that can fit in most existing arbitrary image style transfer models, such as CNN-based, ViT-based, and flow-based methods. As the key component in image style transfer tasks, a suitable style representation is essential to achieve satisfactory results. Existing approaches
-
CLIP-Guided StyleGAN Inversion for Text-Driven Real Image Editing ACM Trans. Graph. (IF 7.8) Pub Date : 2023-07-19 Ahmet Canberk Baykal, Abdul Basit Anees, Duygu Ceylan, Erkut Erdem, Aykut Erdem, Deniz Yuret
Researchers have recently begun exploring the use of StyleGAN-based models for real image editing. One particularly interesting application is using natural language descriptions to guide the editing process. Existing approaches for editing images using language either resort to instance-level latent code optimization or map predefined text prompts to some editing directions in the latent space. However
-
High-Resolution Volumetric Reconstruction for Clothed Humans ACM Trans. Graph. (IF 7.8) Pub Date : 2023-07-15 Sicong Tang, Guangyuan Wang, Qing Ran, Lingzhi Li, Li Shen, Ping Tan
We present a novel method for reconstructing clothed humans from a sparse set of, e.g., 1–6 RGB images. Despite impressive results from recent works employing deep implicit representation, we revisit the volumetric approach and demonstrate that better performance can be achieved with proper system design. The volumetric representation offers significant advantages in leveraging 3D spatial context through
-
Procedural Metamaterials: A Unified Procedural Graph for Metamaterial Design ACM Trans. Graph. (IF 7.8) Pub Date : 2023-06-29 Liane Makatura, Bohan Wang, Yi-Lu Chen, Bolei Deng, Chris Wojtan, Bernd Bickel, Wojciech Matusik
We introduce a compact, intuitive procedural graph representation for cellular metamaterials, which are small-scale, tileable structures that can be architected to exhibit many useful material properties. Because the structures’ “architectures” vary widely – with elements such as beams, thin shells, and solid bulks – it is difficult to explore them using existing representations. Generic approaches
-
The Design Space of Kirchhoff Rods ACM Trans. Graph. (IF 7.8) Pub Date : 2023-06-27 Christian Hafner, Bernd Bickel
The Kirchhoff rod model describes the bending and twisting of slender elastic rods in three dimensions, and has been widely studied to enable the prediction of how a rod will deform, given its geometry and boundary conditions. In this work, we study a number of inverse problems with the goal of computing the geometry of a straight rod that will automatically deform to match a curved target shape after
-
A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning ACM Trans. Graph. (IF 7.8) Pub Date : 2023-06-20 Yuxin Zhang, Fan Tang, Weiming Dong, Haibin Huang, Chongyang Ma, Tong-Yee Lee, Changsheng Xu
This work presents Unified Contrastive Arbitrary Style Transfer (UCAST), a novel style representation learning and transfer framework, that can fit in most existing arbitrary image style transfer models, such as CNN-based, ViT-based, and flow-based methods. As the key component in image style transfer tasks, a suitable style representation is essential to achieve satisfactory results. Existing approaches
-
Fast GPU-Based Two-Way Continuous Collision Handling ACM Trans. Graph. (IF 7.8) Pub Date : 2023-06-13 Tianyu Wang, Jiong Chen, Dongping Li, Xiaowei Liu, Huamin Wang, Kun Zhou
Step-and-project is a popular method to simulate non-penetrating deformable bodies in physically-based animation. The strategy is to first integrate the system in time without considering contacts and then resolve potential intersections, striking a good balance between plausibility and efficiency. However, existing methods can be defective and unsafe when using large time steps, taking risks of failure
-
The Method of Moving Frames for Surface Global Parametrization ACM Trans. Graph. (IF 7.8) Pub Date : 2023-06-10 Guillaume Coiffier, Etienne Corman
This article introduces a new representation of surface global parametrization based on Cartan’s method of moving frames. We show that a system of structure equations, characterizing the local coordinates changes with respect to a local frame system, completely characterizes the set of possible cone parametrizations. The discretization of this system provably provides necessary and sufficient conditions
-
NeRFFaceLighting: Implicit and Disentangled Face Lighting Representation Leveraging Generative Prior in Neural Radiance Fields ACM Trans. Graph. (IF 7.8) Pub Date : 2023-06-09 Kaiwen Jiang, Shu-Yu Chen, Hongbo Fu, Lin Gao
3D-aware portrait lighting control is an emerging and promising domain, thanks to the recent advance of generative adversarial networks and neural radiance fields. Existing solutions typically try to decouple the lighting from the geometry and appearance for disentangled control with an explicit lighting representation (e.g., Lambertian or Phong). However, they either are limited to a constrained lighting