Sensory impairments, such as visual and hearing impairments, and cognitive decline are prevalent among mid-age and older adults in China. With 4-year longitudinal data from the China Health and Retirement Longitudinal Study, we assessed the association between self-reported sensory impairments and episodic memory. Multivariate linear mixed-effects models were used to estimate the association of baseline sensory impairment in 2011-2012 with cognitive decline at 2- and 4-year follow-up visits. Among the 13,097 participants, longitudinal associations were identified between having hearing loss (β = -0.14, 95% CI: -0.22, -0.05), having both poor hearing and vision (β = -0.14, 95% CI: -0.23, -0.04) and decline in immediate word recall over 4 years, compared to those without self-reported sensory impairment. In addition, these associations were more significant among those aged 60 and older and among women. Further research is needed to investigate these associations in the longer term, providing evidence to support interventions that can prevent or delay sensory impairments and preserve cognitive functions in older adults.