Enzyme-linked immunosorbent assay (ELISA) with high specificity and sensitivity is one of the most popular techniques for detecting carbendazim (CBD), a commonly used benzimidazole fungicide in agriculture. However, the traditional ELISA based on the horseradish peroxidase (HRP)-3,3′,5,5′-tetramethylbenzidine (TMB) system for CBD only displays the yellow color of TMB²⁺ from deep to light, making it difficult for the naked eye to judge whether CBD in fruits and vegetables exceeds the maximum residue limit. In this article, we intend to improve the traditional ELISA method to establish a multicolor signal output ELISA to achieve visual semiquantitative detection of CBD. This method is based on the optical properties of gold nanorods (AuNRs). After introducing AuNRs into TMB²⁺ solution, which was produced by the HRP-TMB system of traditional ELISA, AuNRs were quickly etched by TMB²⁺. Consequently, the longitudinal localized surface plasmon resonance peak of AuNRs shows a clear blue shift and a vivid color change. Different concentrations of CBD generate different amounts of TMB²⁺, which in turn leads to different etching degrees of AuNRs, and ultimately results in a rainbow-like color change. As a result, CBD from 0.08 to 100 ng mL⁻¹ can be easily distinguished by the naked eye, which does not require any large instruments. Moreover, the colors displayed by 0.49 ng mL⁻¹ (purple) and 0 ng mL⁻¹ (pink) are significantly different from each other. It should be noted that 0.49 ng mL⁻¹ is far below the most stringent maximum residue limit of CBD in the world. Additionally, the quantitative determination of CBD spiked in canned citrus, citrus fruits, chives, and cabbage samples showed satisfactory recoveries. The good performance of the AuNR-based ELISA makes it have a wide range of application prospects in food safety and international trade.