PURPOSE We propose a novel methodology for generating synthetic X-rays from 2D RGB images. This method creates accurate simulations for use in non-diagnostic visualization problems where the only input comes from a generic camera. Traditional methods are restricted to using simulation algorithms on 3D computer models. To solve this problem, we propose a method of synthetic X-ray generation using conditional generative adversarial networks (CGANs). METHODS We create a custom synthetic X-ray dataset generator to generate image triplets for X-ray images, pose images, and RGB images of natural hand poses sampled from the NYU hand pose dataset. This dataset is used to train two general-purpose CGAN networks, pix2pix and CycleGAN, as well as our novel architecture called pix2xray which expands upon the pix2pix architecture to include the hand pose into the network. RESULTS Our results demonstrate that our pix2xray architecture outperforms both pix2pix and CycleGAN in producing higher-quality X-ray images. We measure higher similarity metrics in our approach, with pix2pix coming in second, and CycleGAN producing the worst results. Our network performs better in the difficult cases which involve high occlusion due to occluded poses or large rotations. CONCLUSION Overall our work establishes a baseline that synthetic X-rays can be simulated using 2D RGB input. We establish the need for additional data such as the hand pose to produce clearer results and show that future research must focus on more specialized architectures to improve overall image clarity and structure.

中文翻译：

---

pix2xray：使用生成的对抗网络将RGB图像转换为X射线。

目的我们提出了一种从2D RGB图像生成合成X射线的新颖方法。此方法可创建准确的模拟，用于非诊断性可视化问题，其中唯一的输入来自通用摄像机。传统方法仅限于在3D计算机模型上使用仿真算法。为解决此问题，我们提出了一种使用条件生成对抗网络（CGAN）生成X射线的方法。方法我们创建了一个定制的合成X射线数据集生成器，以为从NYU手姿数据集中采样的自然手姿的X射线图像，姿势图像和RGB图像生成图像三元组。此数据集用于训练两个通用CGAN网络pix2pix和CycleGAN，以及我们称为pix2xray的新颖架构，该架构在pix2pix架构的基础上进行了扩展，将手势包含在网络中。结果我们的结果表明，我们的pix2xray体系结构在生成更高质量的X射线图像方面优于pix2pix和CycleGAN。我们在我们的方法中测量了更​​高的相似性指标，其中pix2pix位居第二，而CycleGAN产生的结果最差。在由于遮挡的姿势或较大的旋转而导致高度遮挡的困难情况下，我们的网络性能会更好。结论总体而言，我们的工作为使用2D RGB输入可以模拟合成X射线建立了基线。我们确定需要其他数据（例如手势）以产生更清晰的结果，并表明未来的研究必须专注于更专业的体系结构以提高整体图像的清晰度和结构。