Tracing the provenances of aeolian sands in the Mu Us Desert is crucial to understanding surficial processes in arid and semiarid regions of China. However, the material source of the aeolian sands in the Mu Us Desert is still under debate. The provenances of modern aeolian sands in different areas and with different particle sizes in the Mu Us Desert remain unclear, as does the link between the sources of different kinds of sediments. Here, the Mu Us Desert is divided into two subzones in the east and the west with hierarchical cluster analysis. According to the characteristics of trace and rare earth elements, we studied the provenances of aeolian sands in different areas and with different particle sizes, and we analyze the provenance linkages among aeolian sands, sandstones, fluvio-lacustrine sediments, loess sediments, and floodplain sediments in the desert. We report the following results: (1) The sources of coarse-grained aeolian sands in the eastern and western Mu Us Desert are different. Aeolian sands from the eastern part are mainly derived from local sandstones and eastern coarse-grained fluvio-lacustrine sediments, whereas those from the western part mainly come from the Qilian orogenic belt in the northeastern Tibetan Plateau. (2) The fine-grained aeolian sands of the whole desert show obvious homogeneity and have geochemical characteristics similar to those of fine-grained fluvio-lacustrine, loess, and floodplain sediments, suggesting that these fine-grained fractions are fully mixed after long-distance transport. (3) The transport and deposition of aeolian sands are affected by aeolian processes and fluvial activity.