Automatic cost learning for steganography based on deep neural networks is receiving increasing attention. Steganographic methods under such a framework have been shown to achieve better security performance than methods adopting hand-crafted costs. However, they still exhibit some limitations that prevent a full exploitation of their potentiality, including using a function-approximated neural-network-based embedding simulator and a coarse-grained optimization objective without explicitly using pixel-wise information. In this article, we propose a new embedding cost learning framework called SPAR-RL (Steganographic Pixel-wise Actions and Rewards with Reinforcement Learning) that overcomes the above limitations. In SPAR-RL, an agent utilizes a policy network which decomposes the embedding process into pixel-wise actions and aims at maximizing the total rewards from a simulated steganalytic environment, while the environment employs an environment network for pixel-wise reward assignment. A sampling process is utilized to emulate the message embedding of an optimal embedding simulator. Through the iterative interactions between the agent and the environment, the policy network learns a secure embedding policy which can be converted into pixel-wise embedding costs for practical message embedding. Experimental results demonstrate that the proposed framework achieves state-of-the-art security performance against various modern steganalyzers, and outperforms existing cost learning frameworks with regard to learning stability and efficiency.

中文翻译：

---

使用深度强化学习的图像隐写术自动成本学习框架

基于深度神经网络的隐写术的自动成本学习正受到越来越多的关注。与采用手工成本的方法相比，在这种框架下的隐秘方法已显示出更好的安全性能。但是，它们仍然表现出一些局限性，无法充分利用其潜力，包括使用功能近似的基于神经网络的嵌入模拟器和粗粒度优化目标，而未明确使用像素级信息。在本文中，我们提出了一种新的嵌入成本学习框架，称为SPAR-RL（具有增强学习功能的隐写像素行为和奖励），它克服了上述限制。在SPAR-RL中，代理使用策略网络，该策略网络将嵌入过程分解为像素行为，旨在最大程度地提高模拟隐写分析环境的总回报，而环境则使用环境网络进行像素奖励分配。利用采样过程来模拟最佳嵌入模拟器的消息嵌入。通过代理与环境之间的迭代交互，策略网络学习了安全的嵌入策略，可以将其转换为按像素的嵌入成本，以进行实际的消息嵌入。实验结果表明，提出的框架针对各种现代隐写分析仪实现了最先进的安全性能，并且在学习稳定性和效率方面都优于现有的成本学习框架。