The memristor is a kind of non-linear element with memory function, which can be applied to chaotic systems to increase signal randomness and complexity. In this paper, a new four-dimensional hyper-chaotic system is designed based on a flux controlled memristor model, which can generate complex chaotic attractors. The basic dynamic theory analysis and numerical simulations of the system, such as the stability of equilibrium points, the Lyapunov exponents and dimension, Poincare maps, the power spectrum, and the waveform graph prove that it has rich dynamic behaviors. Then, the circuit implementation of this system is established. The consistency of simulation program with integrated circuit emphasis (SPICE) simulation and numerical analysis proves the ability to generate chaos. Finally, a new image encryption scheme is designed by using the memristor-based hyper-chaotic system proposed in this paper. The scheme involves a total of two encryptions. By using different security analysis factors, the proposed algorithm is compared with other image encryption schemes, including correlation analysis, information entropy, etc. The results show that the proposed image encryption scheme has a large key space and presents a better encryption effect.