Due to lacking use of prior information, the anomaly detection results are not always satisfactory. However, with the establishment of the spectral library, it becomes possible to obtain one or more spectra of the background in the image to be detected. If we can make use of such background information that is always ignored or discarded, the detection result is very likely to be improved. Hence, we proposed a hyperspectral anomaly detection method using a background endmember signature. To better separate the anomaly from the background, we first perform spectral unmixing to estimate the abundance matrix for further study instead of the original spectral data. In this process, we introduce a non-negative matrix factorization-based unmixing method and a corresponding initialization method using a background endmember. Then the low-rank property contained in the abundance matrix is exploited. A low-rank decomposition method is used to separate the anomalies. The proposed algorithm is evaluated on both synthetic and real data sets. Experiment results show the effectiveness of the proposed method and the improvement brought by the usage of a known background endmember.