In the development and utilization process of geothermal energy, the thermal conductivity of the rock plays a key role in engineering design. Potentials for further improvement of the geothermal engineering design lie in the improvement of the accuracy of thermal conductivity model. For the prediction of the thermal conductivity of rocks, emerging extreme learning machine (ELM) methods could prove to be highly accurate and efficient new methods. In this paper, the thermal conductivity of various rocks in the Songliao Basin (China) was measured by thermal conductivity scanning (TCS), and 101 sets of data were obtained. The correlation between porosity, moisture content, density, P-wave velocity and the thermal conductivity was analyzed. The results reveal that four parameters are suitable as input variables for predicting the thermal conductivity. Small-sampling prediction models were created using a new ELM-based approach. To demonstrate the model performance, seven prediction models were developed using ELM, support vector regression (SVR) and back propagation neural network (BPNN) algorithms, and theoretical models. The performance of seven models was compared by mean square error (MSE) and coefficient of determination (R²). The results show that the ELM-based model has better operating speed and forecasting accuracy, and good overall generalization performance in predicting rock thermal conductivity, which can provide accurate data in time for engineering application.