Code generation from graphical user interface images is a promising area of research. Recent progress on machine learning methods made it possible to transform user interface into the code using several methods. The encoder–decoder framework represents one of the possible ways to tackle code generation tasks. Our model implements the encoder–decoder framework with an attention mechanism that helps the decoder to focus on a subset of salient image features when needed. Our attention mechanism also helps the decoder to generate token sequences with higher accuracy. Experimental results show that our model outperforms previously proposed models on the pix2code benchmark dataset.

从图形用户界面图像生成代码是一个有前途的研究领域。机器学习方法的最新进展使得可以使用多种方法将用户界面转换为代码。编码器-解码器框架代表了解决代码生成任务的一种可能方法。我们的模型通过注意力机制实现了编码器-解码器框架，该机制可帮助解码器在需要时专注于显着图像特征的子集。我们的注意力机制还可以帮助解码器以更高的精度生成令牌序列。实验结果表明，我们的模型优于pix2code基准数据集上先前提出的模型。