This paper presents a light-weight, high-quality texture synthesis algorithm that easily generalizes to other applications such as style transfer and texture mixing. We represent texture features through the deep neural activation vectors within the bottleneck layer of an auto-encoder and frame the texture synthesis problem as optimal transport between the activation values of the image being synthesized and those of an exemplar texture. To find this optimal transport mapping, we utilize an N-dimensional probability density function (PDF) transfer process that iterates over multiple random rotations of the PDF basis and matches the 1D marginal distributions across each dimension. This achieves quality and flexibility on par with expensive back-propagation based neural texture synthesis methods, but with the potential of achieving interactive rates. We demonstrate that first order statistics offer a more robust representation for texture than the second order statistics that are used today. We propose an extension of this algorithm that reduces the dimensionality of the neural feature space. We utilize a multi-scale coarse-to-fine synthesis pyramid to capture and preserve larger image features; unify color and style transfer under one framework; and further augment this system with a novel masking scheme that re-samples and re-weights the feature distribution for user-guided texture painting and targeted style transfer.

中文翻译：

---

最优纹理：通过最优传输实现快速、稳健的纹理合成和风格转换

本文提出了一种轻量级、高质量的纹理合成算法，可以轻松地推广到其他应用程序，例如样式转换和纹理混合。我们通过自动编码器瓶颈层内的深度神经激活向量表示纹理特征，并将纹理合成问题构建为正在合成的图像的激活值与示例纹理的激活值之间的最佳传输。为了找到这种最佳传输映射，我们利用了 N 维概率密度函数 (PDF) 传输过程，该过程在 PDF 基础的多次随机旋转上进行迭代，并匹配每个维度上的一维边缘分布。这实现了与昂贵的基于反向传播的神经纹理合成方法相当的质量和灵活性，但具有实现互动率的潜力。我们证明，一阶统计数据比当今使用的二阶统计数据提供了更稳健的纹理表示。我们提出了该算法的扩展，以降低神经特征空间的维数。我们利用多尺度从粗到细的合成金字塔来捕捉和保留更大的图像特征；在一个框架下统一颜色和风格传递；并通过一种新颖的掩蔽方案进一步增强该系统，该方案为用户引导的纹理绘制和有针对性的风格转移重新采样和重新加权特征分布。我们提出了该算法的扩展，以降低神经特征空间的维数。我们利用多尺度从粗到细的合成金字塔来捕捉和保留更大的图像特征；在一个框架下统一颜色和风格传递；并通过一种新颖的掩蔽方案进一步增强该系统，该方案为用户引导的纹理绘制和有针对性的风格转移重新采样和重新加权特征分布。我们提出了该算法的扩展，以降低神经特征空间的维数。我们利用多尺度从粗到细的合成金字塔来捕捉和保留更大的图像特征；在一个框架下统一颜色和风格传递；并通过一种新颖的掩蔽方案进一步增强该系统，该方案为用户引导的纹理绘制和有针对性的风格转移重新采样和重新加权特征分布。