To explore the fatigue evolution characterizing the surrounding rock of a compressed air energy storage tunnel, discontinuous cyclic loading tests were conducted on sandstone collected from a coal mine in Henan, China. The experimental results show that the presence of low-stress time intervals accelerates the accumulation of residual deformation. The fatigue life of sandstone in the discontinuous fatigue test is much shorter than that in the traditional fatigue test. The time intervals could influence the residual strain of each cycle when the stress is at a very low level (10 MPa), and the effect increases with the duration of the time intervals. The proportion of the residual strain from cycle A to that from cycle B is defined as the accelerant rate and is a negative exponential function of the time interval duration. A mechanical model was developed to reveal the mechanical mechanism of the internal residual stress generation. The mechanical response of the sandstone microstructure in one loading–unloading cycle was simulated using the finite element method. The results show that the partial internal residual stress around the inclusion remains during the unloading process due to the difference in states between the inclusions and the interstitial materials. The sustained damage effect of the internal residual stress during intervals may be the main reason for the reduction in fatigue life and the acceleration of plastic deformation.