Despite recent advances in Monte Carlo path tracing at interactive rates, denoised image sequences generated with few samples per‐pixel often yield temporally unstable results and loss of high‐frequency details. We present a novel adaptive rendering method that increases temporal stability and image fidelity of low sample count path tracing by distributing samples via spatio‐temporal joint optimization of sampling and denoising. Adding temporal optimization to the sample predictor enables it to learn spatio‐temporal sampling strategies such as placing more samples in disoccluded regions, tracking specular highlights, etc; adding temporal feedback to the denoiser boosts the effective input sample count and increases temporal stability. The temporal approach also allows us to remove the initial uniform sampling step typically present in adaptive sampling algorithms. The sample predictor and denoiser are deep neural networks that we co‐train end‐to‐end over multiple consecutive frames. Our approach is scalable, allowing trade‐off between quality and performance, and runs at near real‐time rates while achieving significantly better image quality and temporal stability than previous methods.

中文翻译：

---

神经时间自适应采样和去噪

尽管蒙特卡罗路径追踪在交互速率方面取得了最新进展，但每像素很少样本生成的去噪图像序列通常会产生时间不稳定的结果和高频细节的丢失。我们提出了一种新的自适应渲染方法，通过采样和去噪的时空联合优化来分布样本，从而提高低样本计数路径跟踪的时间稳定性和图像保真度。向样本预测器添加时间优化使其能够学习时空采样策略，例如在遮挡区域放置更多样本、跟踪镜面高光等；向降噪器添加时间反馈可提高有效输入样本数并增加时间稳定性。时间方法还允许我们去除自适应采样算法中通常存在的初始均匀采样步骤。样本预测器和降噪器是我们在多个连续帧上端到端协同训练的深度神经网络。我们的方法是可扩展的，允许在质量和性能之间进行权衡，并以接近实时的速率运行，同时实现比以前的方法明显更好的图像质量和时间稳定性。