Currently, the high viscosity modified asphalt (HVMA) is widely used in the porous asphalt pavement (PA) in China. HVMA provides good raveling resistance and rutting resistance for PA due to the excellent rheological property of it. However, the high construction temperature and the well-developed void structure of PA lead to a severe aging of HVMA. In order to understand the thermal aging behavior and mechanism of HVMA, the changes of rheological property, chemical composition, and phase morphology of HVMA, SBS modified asphalt (SBSMA) and PEN70 asphalt (70#A) during thermal aging process were investigated in this study. Rotational viscosity (RV) test, frequency sweep test, and multiple stress creep recovery (MSCR) test were used to evaluate the rheological property of HVMA. Then, Fourier transform infrared spectroscopy (FTIR) test and Gel permeation chromatography (GPC) test were used to determine the effect of thermal aging on the chemical composition of HVMA. Moreover, the phase morphology of HVMA, and SBSMA were observed by Fluorescence microscopic (FM) test. Test results indicate that the rotational viscosity, softening point, and strain recovery of HVMA show a decreasing trend initially and then increase with the process of thermal aging. The polymer phase structure of HVMA degrades obviously in short-term aging process, which leads to a deteriorated rheological property of HVMA. After long-term aging, the oxidation of base asphalt is the main reason for the changes in rheological property of HVMA. The aging resistance of HVMA is better than 70#A and SBSMA. As the amount of polymers increases, the aging resistance of HVMA is further improved. During thermal aging process, the polymer phase structure of HVMA is more abundant than SBSMA, which contributes to the superior anti-aging property of HVMA.