Machine learning-based offline signature verification systems: A systematic review
Graphical abstract
Introduction
The offline signature is a distinctive handwritten representation of person’s names or a mark which is used as a proof of identity on a bank cheque, loan, property and all other legal documents. It is a biometric measure. Biometric refers to the detailed information about someone’s body such as a pattern of colour in the eyes, handwritten recognition and many more. The verification of an offline signature is a crucial task. Traditionally, offline signatures are verified either analysing the fluency of pattern found in the signature or by visual comparison of signature patterns with the previously collected samples. However, manual offline signature verification of a large number of documents is time-consuming and depends on human vigilance, experience and expertise to detect a signature forgery. Based on the image acquisition method, the verification process of offline signatures can be performed either by online or offline. The online method is also referred to as a dynamic signature verification method, whereas, the offline method is referred to as a static signature verification method. The online signatures are captured using various devices like tablets, pen (pressure-sensitive), etc. Where the offline signature’s inherited dynamic information is collected with a sequence of time intervals. This dynamic information contains pen inclination, pressure and the position of the pen, etc. On the other hand, offline signature is acquired by scanning (i.e., converting into a digital image).
Thus, with recent advancements in technology, the machine learning-based signature verification methods have become more sophisticated. Therefore, offline signatures’ manual verification adopted by financial institutions, has become a very challenging task due to the huge increase in forgery cases. Besides, it becomes a more herculean task when there is a huge number of bank cheques are needed to be verified manually. This challenging task leads to the dire need of a robust automated computerized verification system for offline signatures, which is capable of distinguishing among the genuine and forged offline signatures with accuracy and speed automatically. The machine learning-based offline signature verification research is extensively being made for human verification in access control, employment documents matters, identification in finance and security of various applications in recent years.
The basic aim of any machine learning-based offline signature verification system (hereafter called offline signature verification system) to distinguish the genuine offline signature from the forged one automatically. The offline signature verification (OfSV) system is different from the online signature verification (OnSV) system in the sense that it is not using any inherent dynamic information to identify the forged signature. This inherent dynamic information can only be available in OnSV systems. This is the reason, OfSV system is more complicated, and it is difficult to recover offline signatures from the scanned signature images. It is to be mentioned that the OnSV is out of the scope of this study. The forgeries in offline signatures are of three types; simple, random and skilled. In case of simple forgery, the person, who is going to forge the offline signature knows the signer’s real name, but do not know about the genuine signature of the same signer. However, in random forgeries, one of the information; the signer’s name or signer’s genuine signature, is known to the forger but forger uses own signature instead. Both information, i.e. the signer’s name and signer’s genuine signature are in the knowledge of the forger in skilled forgeries and the forger often practice to imitate the signer’s signatures [1], [2].
The OfSV system can be writer-dependent, writer-independent or hybrid. Writer-dependent (WD) approach is the most common approach and achieved better classification accuracies. In this approach, the verification model is trained for each user. WD OfSV systems are secure because offline signature templates are not stored for verification [3]. However, this approach requires a separate classifier for each user, which increase complexity and computational cost, when more users are added [3]. A more practical and user-convenient approach is a writer-independent (WI) approach, which needs only a single global classifier for all users. WI systems can be used by providing a single signature sample, which makes this approach more popular than WD approach [3]. [4] recently explored the effectiveness of deep learning-based WI OfSV systems. Their proposed stamp cleaning process on real-world documents significantly improved the performance of offline signature verification. Another approach can be a hybrid WD–WI OfSV system, which is developed by switching between WD and WI approaches. A typical scheme of an offline signature verification system (OfSV) can be found in Fig. 1.
Since 1989, the first review paper related to the OfSV system published by [5], there have been several comprehensive review articles on OfSV systems published. [6] has analysed various aspects of OfSV and OnSV systems in details and provided detailed insights into the advancements in this dire field. This review has evaluated the most common offline signature datasets with their distinctions, used preprocessing techniques, features and verifiers. This literature review also covered the performance of various OfSV systems with performance metrics along with current research issues in the domains. However, it has missed the number of current research domains issues, like the need for a large public dataset for OfSV system, the role of transfer learning in improving the accuracy of OfSV systems, cursive handwritten scripts and the usefulness of ensemble classifiers for more accurate results. Similarly, [7] critically evaluated the fifteen signature verification systems from the literature and classified them as per machine learning (ML) models and feature extraction techniques. Furthermore, they also summarized the overall advantages and limitations of OfSV systems. However, this literature review does not cover the used public and exclusive datasets and preprocessing techniques used in OfSV systems. Accuracy rates comparison of these OfSV systems is also not described.
The objective of this review is to present an overview of the ML-based OfSV systems using five aspects such as datasets, preprocessing techniques, feature extraction methods, ML-based verification models and performance evaluation metrics. To achieve this goal, we conducted a literature review by collecting the most relevant articles from six well-known journal databases systematically. Moreover, the articles were selected from January 2014 to October 2019. The main contribution of this systematic review is that we consolidate the state-of-the-art OfSV systems performances in 56 studies on five public datasets: CEDAR in Table 26, GPDS in Table 25, MCYT-75 in Table 22, UTSig in Table 23 and BHSig260 in Table 24. It has been observed that most of the work lacks in better performance in terms of accuracy and robustness. This will enable researchers to identify the best combination of OfSV systems. Moreover, statistical and critical analysis of ML-based OfSV systems for the aforementioned five aspects are also discussed in details. The statistical analysis is conducted using charts and comparison tables. For critical analysis, selected articles are grouped in several categories. The advantages and limitations of each category are compared to produce a critical analysis.
Lastly, this systematic literature review (SLR) presents fifteen current research challenges for future researchers who interested to work in offline signature verification. We organized this SLR study in various sections. Section: 2 describes the research methodology presenting the process of conducting the systematic review. The results are listed in Section 3. Section 4 contains detailed discussion about defined research questions. Current research challenges for future researchers are discussed in Section 5. The paper conclusion is available in Section 6.
Section snippets
Research methodology
A rigorous and well-defined systematic literature review methodology proposed by [8] has been adopted in this review study. A systematic review is performed to define the research questions, as well as, to identify a current research gap and different contributions to research issues in the domain. We conduct this systematic review in two-phased; the review planning and the review conduction. The review planning phase describes the need for this review, specification of research questions and
Results
This section explains the SLR results and how each research questions were answered according to the available data in each selected study. Fifty-six studies were selected after careful quality checks. The results are elaborated while answering the research questions in the following sections.
Discussion
This section covers the results presented in the previous sections and discussed some of the ideas and key points which were identified so that we can outline new research directions in the area of ML-based OfSV systems. For that purpose, we reviewed the articles on ML-based OfSV systems published from January 2014 to October 2019 in six academic repositories. This systematic review will enable researchers to make a better decision about the selection of various aspects of ML-based OfSV systems
Current research open issues and challenges to ML-based OfSV systems
This section presents current open research issues and challenges faced by ML-based OfSV systems. It gives eminence to future research directions to develop robust algorithms for OfSV. These issues are listed below after reviewing the selected studies.
- 1.
Limited Number of Signatures Samples:
This study discovered that most of OfSV models performance is suffering from an insufficient number of signature images per user. A sufficient number of image samples per user is required to train the model
Conclusion
This study analysed the major research endeavours in recent offline signature verification systems to aid new researchers in this domain. The quality research articles from 2014–2019 on offline signature verification system were extensively reviewed. A systematic approach has been adopted to select the 56 articles. Based on the selected studies, five major aspects of any of OfSV systems, namely datasets, image preprocessing techniques, feature extraction methods, ML-based verification models,
CRediT authorship contribution statement
M. Muzaffar Hameed: Data collection & analysis, Wrote the paper. Rodina Ahmad: Provided the critical revision, Supervised the research. Miss Laiha Mat Kiah: Provided the critical revision, Supervised the research. Ghulam Murtaza: Analysed the data, Made critical revision, Co-wrote the paper.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (112)
- et al.
Writer independent offline signature verification based on asymmetric pixel relations and unrelated training-testing datasets
Expert Syst. Appl.
(2019) - et al.
Automatic signature verification and writer identification—the state of the art
Pattern Recognit.
(1989) - et al.
Image-based handwritten signature verification using hybrid methods of discrete Radon transform, principal component analysis and probabilistic neural network
Appl. Soft Comput.
(2016) - et al.
Towards the design of an offline signature verifier based on a small number of genuine samples for training
Expert Syst. Appl.
(2018) Synergy of foreground–background images for feature extraction: Offline signature verification using Fisher vector with fused KAZE features
Pattern Recognit.
(2018)- et al.
New off-line handwritten signature verification method based on artificial immune recognition system
Expert Syst. Appl.
(2016) From BoVW to VLAD with KAZE features: Offline signature verification considering cognitive processes of forensic experts
Pattern Recognit. Lett.
(2018)- et al.
Offline signature verification and quality characterization using poset-oriented grid features
Pattern Recognit.
(2016) - et al.
Handwritten signature verification using the quad-tree histogram of templates and a support vector-based artificial immune classification
Image Vis. Comput.
(2017) - et al.
An efficient open system for offline handwritten signature identification based on curvelet transform and one-class principal component analysis
Neurocomputing
(2017)
Multi-representational learning for offline signature verification using multi-loss snapshot ensemble of CNNs
Expert Syst. Appl.
Learning features for offline handwritten signature verification using deep convolutional neural networks
Pattern Recognit.
Neural networks and statistical techniques: A review of applications
Expert Syst. Appl.
Deep learning in neural networks: An overview
Neural Netw.
Offline handwritten signature verification—Literature review
Hybrid writer-independent–writer-dependent offline signature verification system
IET Biom.
A perspective analysis of handwritten signature technology
ACM Comput. Surv.
An appraisal of off-line signature verification techniques
Int. J. Mod. Educ. Comput. Sci.
Guidelines for Performing Systematic Literature Reviews in Software Engineering
An experimental application of the Delphi method to the use of experts
Manag. Sci.
Off-line signature verification using global & local features with neural networks
The effective use of the one-class SVM classifier for handwritten signature verification based on writer-independent parameters
Pattern Recognit.
Offline signature verification using artificial neural network
Verification of static signature pattern based on random subspace, REP tree and bagging
Multimedia Tools Appl.
Off-line handwritten signature recognition by wavelet entropy and neural network
Entropy
Eigen value based features for offline handwritten signature verification using neural network approach
A robust offline handwritten signature verification system using writer independent approach
Handwritten signature verification using haar cascade classifier approach
A novel end-to-end approach for offline signature verification system
Handwritten signature verification via deep sparse coding architecture
Offline signature verification using convolution siamese network
Signature verification approach using fusion of hybrid texture features
Neural Comput. Appl.
Large-scale offline signature recognition via deep neural networks and feature embedding
Neurocomputing
Combining graph edit distance and triplet networks for offline signature verification
Pattern Recognit. Lett.
Writer-independent feature learning for offline signature verification using deep convolutional neural networks
A combined feature extraction model using SIFT and LBP for offline signature verification system
Offline signature verification with VLAD using fused KAZE features from foreground and background signature images
KAZE features via Fisher vector encoding for offline signature verification
A framework for offline signature verification system: Best features selection approach
Pattern Recognit. Lett.
Offline handwritten signature verification using cylindrical shape context
3D Res.
Offline signature verification by combining graph edit distance and triplet networks
Offline signature verification via structural methods: Graph edit distance and inkball models
Off-line signature verification based on multi-scale local structural pattern
Int. J. Pattern Recognit. Artif. Intell.
Static signature authentication based on J48 and random forest
A structural approach to offline signature verification using graph edit distance
Offline signature verification using local features and decision trees
Int. J. Pattern Recognit. Artif. Intell.
Cited by (35)
Synergy of on-surface and in-air trajectories: Exploratory analysis of forensic online signatures implementing lessons learned from biometrics
2023, Forensic Science International: ReportsA multi-task approach for contrastive learning of handwritten signature feature representations
2023, Expert Systems with ApplicationsSubscripto multiplex: A Riemannian symmetric positive definite strategy for offline signature verification
2023, Pattern Recognition LettersA new synthetic feature generation scheme based on artificial immune systems for robust offline signature verification
2023, Expert Systems with ApplicationsCitation Excerpt :Such system can verify the authenticity of questioned signatures without giving any information about the signer identity. Commonly, the writer-dependent verification is preferred since it allows much better performances for a growing number of individuals (Impedovo & Pirlo, 2008; Fierrez-Aguilar et al., 2005; Nemmour & Chibani, 2013; Hu & Chen, 2013; Serdouk et al., 2016; Hafemann et al., 2017b; Hameed et al., 2021). In both scenarios, the offline SVS is composed of the feature generation and the verification steps.
Dynamic and static feature fusion for increased accuracy in signature verification
2022, Signal Processing: Image CommunicationCitation Excerpt :These results can express the consistency of the data set built. Considering that this study used only 4 genuine samples in the training of the classifier and obtained results are similar to some of the results from studies in Table 2, Tables 5–6 and publication [19], it can be said that the proposed approach using only offline (static) signature image data is quite comparable to the offline signature verification studies at the state-of-the-art level. Spectral Flux Onset Strength Envelope and Spectral Centroid images of the genuine signature sounds and the forged signature sounds of the same signatures corresponding to offline signatures (Fig. 26) are given in Fig. 27.
Writer-independent online signature verification based on 2D representation of time series data using triplet supervised network
2022, Measurement: Journal of the International Measurement Confederation