It has been a challenging problem to identify species of butterflies, especially from images taken in natural environments. Therefore the First international butterfly species recognition competition was organized at the third Data Mining Competition in China in 2018, so as to find good solutions to this challenging problem. The baseline for the competition was based on the Faster R-CNN for it was the latest deep learning algorithm at that time. Nearly all the competition teams chose the Faster R-CNN, or its variations, to solve the problem. But the identification rates were not good enough, and Faster R-CNN is very time consuming. As a result we have been trying to find the most suitable algorithm to solve the butterfly species identification challenge. This paper will present some investigations we have undertaken in this field over the past two years, and show the results we have obtained. We propose a new partition and augmentation technique for the extremely unbalanced ecological butterfly database. We found that RetinaNet is, so far, the best deep learning algorithm to tackle butterfly species identification based on butterfly images taken in natural environments. The best result we obtained was 79.7% in terms of mAP (mean average precision). This is the best result compared to the state-of-the-art studies in this field on the same database so far.