Ag₂Se is considered as an attractive candidate for use in room-temperature thermoelectric applications owing to its unique transport properties, such as glass-like thermal conductivity and good electrical conductivity. However, understanding the correlation between composition (Ag/Se ratio), defect structure, and transport properties is an important prerequisite to optimize its figure of merit (ZT). Using in-depth microscopic analysis, this study reveals the coexistence of a metastable and the main orthorhombic crystal structure in stoichiometric Ag₂Se. The formation of the metastable structure was found to be detrimental to the transport properties of bulk Ag₂Se. We were able to successfully inhibit its formation and stabilize the main orthorhombic structure via small anion (Se and S) excess. The compositions Ag₂SeCh_y (y ≤ 0.01; Ch = Se, S) yielded 40–70% rise in carrier mobility with a value of 2510 cm² V⁻¹ s⁻¹ at 300 K and extremely low lattice-thermal-conductivity (0.2–0.1 W m⁻¹ K⁻¹ over 300–375 K). This combination of transport properties yielded a room-temperature power factor of 3.2 mW m⁻¹ K⁻² and a nearly flat ZT value of ∼1.0 over the 300–375 K temperature range. Additionally, a record-high conversion efficiency (η_max) of 3.7% was theoretically obtained for single-leg Ag₂Se for a small temperature gradient of ∼80 K.