Chinese agricultural production has grown rapidly over the past two decades. However, expanding large-scale production faces a sharp conflict with the supply of water resources. Meanwhile, to adapt to changes in consumer preferences and protect the environment, both the livestock and crop industries had to input more to achieve a high output of meat and grains and low graywater footprint emission. Using provincial panel data of China from 2008 to 2019, this study adopted a slacked-based model to measure the country's agricultural water use efficiency (AWUE) that considered graywater from crop and livestock production as an unexpected output. Based on the concept of sustainable development, using the graywater footprint as an undesired output included environmental pollution caused by agricultural production as part of the cost and could more accurately measure the water efficiency of current production. We also applied the Tobit model to identify the main factors influencing AWUE. Our results indicated the following. (1) Wealthy provinces produced more graywater than did poor provinces. If the province's gross domestic product climbed, the volume of graywater of livestock would surpass graywater of crops by a greater amount. (2) The AWUE in Southwest and South China fitted a "U" trend, while the remaining five regions showed an inverted U-shaped tendency. (3) To improve the AWUE, it is necessary to increase the crop area of economic crops and expand the scale of the livestock industry; compared with investment in irrigation infrastructure, investment in research and experimental development produces higher marginal benefits in enhancing the AWUE. (4) Environmental regulations inhibit the improvement of the AWUE, while irrigation infrastructure significantly improves the AWUE.