Principal component analysis (PCA) has been successfully employed for face recognition. However, if the training process occurs frequently, owing to the update or downdate of the face images used for training, batch PCA becomes prohibitively expensive to recalculate. To overcome this limitation, incremental principal component analysis (IPCA) and decremental principal component analysis (DPCA) can be utilized as a good alternative to PCA because it reuses their previous results for its updates. Many IPCA or DPCA algorithms have been proposed; however, inaccurate tracking of the mean values of the face image data accumulates decomposition errors, which results in poor performance compared with batch PCA. We proposed faster and more accurate algorithms for IPCA and DPCA that maintain accurate decomposition results. The experimental results reveal that the proposed algorithms produce eigenvectors that are significantly close to the eigenvectors of batch PCA and exhibit faster execution speed for face recognition.