Deep networks for image processing typically learn from RGB pixels. This paper proposes instead to learn from program traces, the intermediate values computed during program execution. We study this idea in the context of pixel~shaders -- programs that generate images, typically running in parallel (for each pixel) on GPU hardware. The intermediate values computed at each pixel during program execution form the input to the learned model. In a variety of applications, models learned from program traces outperform baseline models learned from RGB, even when augmented with hand-picked shader-specific features. We also investigate strategies for selecting a subset of trace features for learning; using just a small subset of the trace still outperforms the baselines.

中文翻译：

---

从着色器程序跟踪中学习

用于图像处理的深层网络通常会从RGB像素中学习。本文建议从程序跟踪中学习在程序执行期间计算的中间值。我们在像素着色器的上下文中研究了这个想法-生成图像的程序，通常在GPU硬件上并行运行（针对每个像素）。在程序执行期间在每个像素处计算出的中间值构成了学习模型的输入。在各种应用中，即使通过手工选择的着色器特定功能进行了增强，从程序跟踪中学习的模型也优于从RGB中学习的基线模型。我们还研究了选择跟踪特征子集进行学习的策略；仅使用痕迹的一小部分仍然胜过基线。