Objective

Constructing a 3D model from its 2D images, known as 2D-3D reconstruction, is a challenging task. Conventionally, a parametric 3D model such as a statistical shape model (SSM) is deformed by matching the shapes in its 2D images through a series of processes, including calibration, 2D-3D registration, and optimization for nonrigid deformation. To overcome this complicated procedure, a streamlined 2D-3D reconstruction using a single X-ray image is developed in this study.

Methods

We propose 2D-3D reconstruction of a femur by adopting a deep neural network, where the deformation parameters in the SSM determining the 3D shape of the femur are predicted from a single X-ray image using a deep transfer-learning network. For learning the network from distinct features representing the 3D shape information in the X-ray image, a specific proximal part of the femur from a unique X-ray pose that allows accurate prediction of the 3D femur shape is designated and used to train the network. Then, the corresponding proximal/distal 3D femur model is reconstructed from only the single X-ray image acquired at the designated position.

Results

Experiments were conducted using actual X-ray images of a femur phantom and X-ray images of a patient's femur derived from computed tomography to verify the proposed method. The average errors of the reconstructed 3D shape of the proximal and distal femurs from the proposed method were 1.20 mm and 1.08 mm in terms of root mean squared point-to-surface distance, respectively.

Conclusion

The proposed method presents an innovative approach to simplifying the 2D-3D reconstruction using deep neural networks that exhibits performance compatible with the existing methodologies.

中文翻译：

---

基于深度迁移学习网络的单张X射线图像股骨2D-3D重建

客观的

从 2D 图像构建 3D 模型（称为 2D-3D 重建）是一项具有挑战性的任务。传统上，诸如统计形状模型（SSM）之类的参数化3D模型是通过一系列过程（包括校准、2D-3D配准和非刚性变形优化）匹配其2D图像中的形状来变形的。为了克服这一复杂的过程，本研究开发了使用单个 X 射线图像的简化 2D-3D 重建。

方法

我们提出通过采用深度神经网络对股骨进行 2D-3D 重建，其中确定股骨 3D 形状的 SSM 中的变形参数是使用深度迁移学习网络从单个 X 射线图像中预测的。为了根据 X 射线图像中表示 3D 形状信息的不同特征来学习网络，指定了来自独特 X 射线姿势的股骨的特定近端部分，该位置允许准确预测 3D 股骨形状，并用于训练网络。然后，仅根据在指定位置采集的单个 X 射线图像重建相应的近端/远端 3D 股骨模型。

结果

使用股骨模型的实际 X 射线图像和计算机断层扫描得出的患者股骨的 X 射线图像进行实验，以验证所提出的方法。根据所提出的方法重建的近端和远端股骨 3D 形状的平均误差（以均方根点到表面距离计算）分别为 1.20 mm 和 1.08 mm。

结论

所提出的方法提出了一种使用深度神经网络简化 2D-3D 重建的创新方法，该方法表现出与现有方法兼容的性能。