Crop growth condition and production play an important role in food management and economic development. Therefore, estimating yield accurately and timely is of vital importance for regional food security. The long short-term memory (LSTM) model represents a deep network structure to incorporating crop growth processes, which has been proven to accommodate different types and representations of data, recognize sequential patterns over long time spans, and capture complex nonlinear relationships. The LSTM model was developed to estimate wheat yield in the Guanzhong Plain by integrating meteorological data and two remotely sensed indices, vegetation temperature condition index (VTCI) and leaf area index (LAI) at the main growth stages. Considering the LSTM model has characteristics of memorizing time series information, we adopted different time steps to estimate wheat yield. The results showed that the accuracy of yield estimation was highest (RMSE = 357.77 kg/ha and R² = 0.83) under two time steps and the input combination (meteorological data and two remotely sensed indices). We evaluated the yield estimation accuracy of the optimal LSTM model performance compared with the back propagation neural network (BPNN) and support vector machine (SVM). As a result, the LSTM model outperformed BPNN (R² = 0.42 and RMSE = 812.83 kg/ha) and SVM (R² = 0.41 and RMSE = 867.70 kg/ha), since its recurrent neural network structure that can incorporate nonlinear relationships between multi-features inputs and yield. To further validate the robustness of the optimal LSTM method, the correlations between estimated yield and measured yield at the irrigation sites and the rain-fed sites from 2008 to 2016 were analyzed, and the results demonstrated that the proposed model can serve as an effective approach for different type sampling sites and has better adaptability to interannual fluctuations of climate. Our findings demonstrated a reliable and promising approach for improving yield estimation.