Poly(ethylene oxide) (PEO)-based polymer electrolytes are extensively investigated, and they have rapidly developed in all-solid-state batteries (ASSBs) over recent years for their good interface contact with electrodes, easy shaping and decent flexibility. However, their low ionic conductivity remains a serious issue to be solved urgently. Researchers have found that the addition of fillers can improve the ionic conductivity via creating more amorphous regions conducive to ion transport in PEO. Here, we adopted waste eggshell as a precursor to prepare a sub-micron filler, which is mainly CaO, and it was added into the PEO electrolyte to get a composite polymer electrolyte. Close contact via Lewis base-acid interaction between CaO and PEO can be obtained, as CaO is an effective filler with strong alkalinity. To the best of our knowledge, this is the first time it is applied in the PEO electrolyte. The filler significantly decreases the crystallinity of PEO, and hence, boosted ionic conductivity of the composite polymer electrolyte is observed, which is about 4.5 times higher than that of the polymer reference electrolyte. It additionally brings a higher tensile strength along with a more stable electrochemical window (up to 5 V) to the composite electrolyte. Therefore, as expected, the all-solid-state Li/Na battery with this composite electrolyte exhibits long cycling life and excellent rate performances. The corresponding Li/LiFeO₄ battery has a stable capacity of 142.8 mA h g⁻¹ at 1C after 200 cycles, and the corresponding solid-state Na/Na₃V₂(PO₄)₃ battery gives a reversible capacity of 101.2 mA h g⁻¹ at 0.5C after 100 cycles.