Background—The association of brain white matter lesions with arterial stiffness, as assessed by the cardio-ankle vascular index, is not well characterized in the population. Objective: The aim of this study was to quantify the cross-sectional association of brain white matter lesions with arterial stiffness assessed by the cardio-ankle vascular index among 1176 adults. Methods: We used logistic regression to evaluate the associations of the cardio-ankle vascular index with brain white matter lesions, detected by computed tomography or magnetic resonance imaging, versus normal brain white matter. Results: After adjustment for age, sex, systolic blood pressure, blood lipid level, C reactive protein level, homocysteine level, uric acid level, urea nitrogen level, creatinine level, hypertension, diabetes, coronary artery disease, peripheral artery disease, and stroke, the multivariable analysis showed that age, sex, systolic blood pressure, stroke and the cardio-ankle vascular index were independently associated with the presence of brain white matter lesions. Age range (odds ratio: from 2.48 to 33.64, all p < 0.05) showed the strongest association, followed by stroke (odds ratio: 7.16, 95% confidence interval: 4.59 to 11.16), male sex (odds ratio: 1.80, 95% confidence interval: 1.18 to 2.75), cardio-ankle vascular index (odds ratio: 1.26, 95% confidence interval: 1.06 to 1.50) and systolic blood pressure (odds ratio: 1.01, 95% confidence interval: 1.00 to 1.02). Conclusion: Higher arterial stiffness assessed by the cardio-ankle vascular index was associated with the presence of brain white matter lesions. Longitudinal characterization of the observed associations is warranted to assess whether arterial stiffness predicts brain white matter lesions.