To train a convolutional neural network (CNN) from scratch is not suitable for medical image tasks with insufficient data. Benefiting from the transfer learning, the pre-trained CNN model can provide a reliable initial solution for model optimization of medical image classification. A key concern in breast cancer histology classification is that the model should cover the multi-scale features including nuclei-scale, nuclei organization, and structure-scale features. Inspired by these conjectures, we proposed a novel fusion convolutional neural network (FCNN) based on pre-trained VGG19. The FCNN fuses the shallow, intermediate abstract, and abstract layers to approximately cover the multi-scale features. In order to improve the sensitivity of carcinoma classes, the prediction priority is introduced to enable the lesions can be detected as early as possible. Experimental results show that the proposed FCNN can approximately cover the nuclei-scale, nuclei organization, and structure-scale features. Accuracies of 85%, 75%, and 80.56% are achieved in Initial, Extended, and Overall test set, respectively. The source code for this research is available at https://github.com/yxchspring/breasthistolgoy.

从头开始训练卷积神经网络（CNN）不适合数据不足的医学图像任务。受益于转移学习，预训练的CNN模型可以为医学图像分类的模型优化提供可靠的初始解决方案。乳腺癌组织学分类中的一个关键问题是该模型应涵盖多尺度特征，包括核尺度，核组织和结构尺度特征。受这些猜想的启发，我们提出了一种基于预训练的VGG19的新型融合卷积神经网络（FCNN）。FCNN融合了浅层，中间抽象层和抽象层，以大致覆盖多尺度特征。为了提高癌症分类的敏感性，引入了预测优先级，以便可以尽早发现病变。实验结果表明，提出的FCNN可以大致覆盖核尺度，核组织和结构尺度特征。初始，扩展和总体测试集的准确度分别达到85％，75％和80.56％。这项研究的源代码可从https://github.com/yxchspring/breasthistolgoy获得。