Machine learning-based offline signature verification systems: A systematic review

https://doi.org/10.1016/j.image.2021.116139Get rights and content

Highlights

  • A systematic review of ML-based offline signature verification systems.

  • Concentrates on datasets, preprocessing techniques and feature extraction methods.

  • Reviewed ML-based forgery detection models and performance evaluation metrics.

  • Consolidates the state-of-the-art OfSV systems performances on five public datasets.

  • Fifteen open research issues are also identified.

Abstract

The offline signatures are the most widely adopted biometric authentication techniques in banking systems, administrative and financial applications due to its simplicity and uniqueness. Several automated techniques have been developed to anticipate the genuineness of the offline signature. However, the recapitulate of the existing literature on machine learning-based offline signature verification (OfSV) systems are available in a few review studies only. The objective of this systematic review is to present the state-of-the-art machine learning-based models for OfSV systems using five aspects like datasets, preprocessing techniques, feature extraction methods, machine learning-based verification models and performance evaluation metrics. Thus, five research questions were identified and analysed in this context. This review covers the articles published between January 2014 and October 2019. A systematic approach has been adopted to select the 56 articles. This systematic review revealed that recently, the deep learning-based neural network attained the most promising results for the OfSV systems on public datasets. This review consolidates the state-of-the-art OfSV systems performances in selected studies on five public datasets (CEDAR, GPDS, MCYT-75, UTSig and BHSig260). Finally, fifteen open research issues were identified for future development.

Introduction

The offline signature is a distinctive handwritten representation of person’s names or a mark which is used as a proof of identity on a bank cheque, loan, property and all other legal documents. It is a biometric measure. Biometric refers to the detailed information about someone’s body such as a pattern of colour in the eyes, handwritten recognition and many more. The verification of an offline signature is a crucial task. Traditionally, offline signatures are verified either analysing the fluency of pattern found in the signature or by visual comparison of signature patterns with the previously collected samples. However, manual offline signature verification of a large number of documents is time-consuming and depends on human vigilance, experience and expertise to detect a signature forgery. Based on the image acquisition method, the verification process of offline signatures can be performed either by online or offline. The online method is also referred to as a dynamic signature verification method, whereas, the offline method is referred to as a static signature verification method. The online signatures are captured using various devices like tablets, pen (pressure-sensitive), etc. Where the offline signature’s inherited dynamic information is collected with a sequence of time intervals. This dynamic information contains pen inclination, pressure and the position of the pen, etc. On the other hand, offline signature is acquired by scanning (i.e., converting into a digital image).

Thus, with recent advancements in technology, the machine learning-based signature verification methods have become more sophisticated. Therefore, offline signatures’ manual verification adopted by financial institutions, has become a very challenging task due to the huge increase in forgery cases. Besides, it becomes a more herculean task when there is a huge number of bank cheques are needed to be verified manually. This challenging task leads to the dire need of a robust automated computerized verification system for offline signatures, which is capable of distinguishing among the genuine and forged offline signatures with accuracy and speed automatically. The machine learning-based offline signature verification research is extensively being made for human verification in access control, employment documents matters, identification in finance and security of various applications in recent years.

The basic aim of any machine learning-based offline signature verification system (hereafter called offline signature verification system) to distinguish the genuine offline signature from the forged one automatically. The offline signature verification (OfSV) system is different from the online signature verification (OnSV) system in the sense that it is not using any inherent dynamic information to identify the forged signature. This inherent dynamic information can only be available in OnSV systems. This is the reason, OfSV system is more complicated, and it is difficult to recover offline signatures from the scanned signature images. It is to be mentioned that the OnSV is out of the scope of this study. The forgeries in offline signatures are of three types; simple, random and skilled. In case of simple forgery, the person, who is going to forge the offline signature knows the signer’s real name, but do not know about the genuine signature of the same signer. However, in random forgeries, one of the information; the signer’s name or signer’s genuine signature, is known to the forger but forger uses own signature instead. Both information, i.e. the signer’s name and signer’s genuine signature are in the knowledge of the forger in skilled forgeries and the forger often practice to imitate the signer’s signatures [1], [2].

The OfSV system can be writer-dependent, writer-independent or hybrid. Writer-dependent (WD) approach is the most common approach and achieved better classification accuracies. In this approach, the verification model is trained for each user. WD OfSV systems are secure because offline signature templates are not stored for verification [3]. However, this approach requires a separate classifier for each user, which increase complexity and computational cost, when more users are added [3]. A more practical and user-convenient approach is a writer-independent (WI) approach, which needs only a single global classifier for all users. WI systems can be used by providing a single signature sample, which makes this approach more popular than WD approach [3]. [4] recently explored the effectiveness of deep learning-based WI OfSV systems. Their proposed stamp cleaning process on real-world documents significantly improved the performance of offline signature verification. Another approach can be a hybrid WD–WI OfSV system, which is developed by switching between WD and WI approaches. A typical scheme of an offline signature verification system (OfSV) can be found in Fig. 1.

Since 1989, the first review paper related to the OfSV system published by [5], there have been several comprehensive review articles on OfSV systems published. [6] has analysed various aspects of OfSV and OnSV systems in details and provided detailed insights into the advancements in this dire field. This review has evaluated the most common offline signature datasets with their distinctions, used preprocessing techniques, features and verifiers. This literature review also covered the performance of various OfSV systems with performance metrics along with current research issues in the domains. However, it has missed the number of current research domains issues, like the need for a large public dataset for OfSV system, the role of transfer learning in improving the accuracy of OfSV systems, cursive handwritten scripts and the usefulness of ensemble classifiers for more accurate results. Similarly, [7] critically evaluated the fifteen signature verification systems from the literature and classified them as per machine learning (ML) models and feature extraction techniques. Furthermore, they also summarized the overall advantages and limitations of OfSV systems. However, this literature review does not cover the used public and exclusive datasets and preprocessing techniques used in OfSV systems. Accuracy rates comparison of these OfSV systems is also not described.

The objective of this review is to present an overview of the ML-based OfSV systems using five aspects such as datasets, preprocessing techniques, feature extraction methods, ML-based verification models and performance evaluation metrics. To achieve this goal, we conducted a literature review by collecting the most relevant articles from six well-known journal databases systematically. Moreover, the articles were selected from January 2014 to October 2019. The main contribution of this systematic review is that we consolidate the state-of-the-art OfSV systems performances in 56 studies on five public datasets: CEDAR in Table 26, GPDS in Table 25, MCYT-75 in Table 22, UTSig in Table 23 and BHSig260 in Table 24. It has been observed that most of the work lacks in better performance in terms of accuracy and robustness. This will enable researchers to identify the best combination of OfSV systems. Moreover, statistical and critical analysis of ML-based OfSV systems for the aforementioned five aspects are also discussed in details. The statistical analysis is conducted using charts and comparison tables. For critical analysis, selected articles are grouped in several categories. The advantages and limitations of each category are compared to produce a critical analysis.

Lastly, this systematic literature review (SLR) presents fifteen current research challenges for future researchers who interested to work in offline signature verification. We organized this SLR study in various sections. Section: 2 describes the research methodology presenting the process of conducting the systematic review. The results are listed in Section 3. Section 4 contains detailed discussion about defined research questions. Current research challenges for future researchers are discussed in Section 5. The paper conclusion is available in Section 6.

Section snippets

Research methodology

A rigorous and well-defined systematic literature review methodology proposed by [8] has been adopted in this review study. A systematic review is performed to define the research questions, as well as, to identify a current research gap and different contributions to research issues in the domain. We conduct this systematic review in two-phased; the review planning and the review conduction. The review planning phase describes the need for this review, specification of research questions and

Results

This section explains the SLR results and how each research questions were answered according to the available data in each selected study. Fifty-six studies were selected after careful quality checks. The results are elaborated while answering the research questions in the following sections.

Discussion

This section covers the results presented in the previous sections and discussed some of the ideas and key points which were identified so that we can outline new research directions in the area of ML-based OfSV systems. For that purpose, we reviewed the articles on ML-based OfSV systems published from January 2014 to October 2019 in six academic repositories. This systematic review will enable researchers to make a better decision about the selection of various aspects of ML-based OfSV systems

Current research open issues and challenges to ML-based OfSV systems

This section presents current open research issues and challenges faced by ML-based OfSV systems. It gives eminence to future research directions to develop robust algorithms for OfSV. These issues are listed below after reviewing the selected studies.

  • 1.

    Limited Number of Signatures Samples:

    This study discovered that most of OfSV models performance is suffering from an insufficient number of signature images per user. A sufficient number of image samples per user is required to train the model

Conclusion

This study analysed the major research endeavours in recent offline signature verification systems to aid new researchers in this domain. The quality research articles from 2014–2019 on offline signature verification system were extensively reviewed. A systematic approach has been adopted to select the 56 articles. Based on the selected studies, five major aspects of any of OfSV systems, namely datasets, image preprocessing techniques, feature extraction methods, ML-based verification models,

CRediT authorship contribution statement

M. Muzaffar Hameed: Data collection & analysis, Wrote the paper. Rodina Ahmad: Provided the critical revision, Supervised the research. Miss Laiha Mat Kiah: Provided the critical revision, Supervised the research. Ghulam Murtaza: Analysed the data, Made critical revision, Co-wrote the paper.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (112)

  • MasoudniaS. et al.

    Multi-representational learning for offline signature verification using multi-loss snapshot ensemble of CNNs

    Expert Syst. Appl.

    (2019)
  • HafemannL.G. et al.

    Learning features for offline handwritten signature verification using deep convolutional neural networks

    Pattern Recognit.

    (2017)
  • PaliwalM. et al.

    Neural networks and statistical techniques: A review of applications

    Expert Syst. Appl.

    (2009)
  • SchmidhuberJ.

    Deep learning in neural networks: An overview

    Neural Netw.

    (2015)
  • HafemannL.G. et al.

    Offline handwritten signature verification—Literature review

  • EskanderG.S. et al.

    Hybrid writer-independent–writer-dependent offline signature verification system

    IET Biom.

    (2013)
  • D. Engin, A. Kantarci, S. Arslan, H. Kemel Ekenel, Offline signature verification on real-world documents, in:...
  • DiazM. et al.

    A perspective analysis of handwritten signature technology

    ACM Comput. Surv.

    (2019)
  • ShahA.S. et al.

    An appraisal of off-line signature verification techniques

    Int. J. Mod. Educ. Comput. Sci.

    (2015)
  • KitchenhamB. et al.

    Guidelines for Performing Systematic Literature Reviews in Software Engineering

    (2007)
  • DalkeyN. et al.

    An experimental application of the Delphi method to the use of experts

    Manag. Sci.

    (1963)
  • JaiswalG.S. et al.

    Off-line signature verification using global & local features with neural networks

  • GuerbaiG.S. et al.

    The effective use of the one-class SVM classifier for handwritten signature verification based on writer-independent parameters

    Pattern Recognit.

    (2015)
  • SubhashC. et al.

    Offline signature verification using artificial neural network

  • ChandraS. et al.

    Verification of static signature pattern based on random subspace, REP tree and bagging

    Multimedia Tools Appl.

    (2017)
  • DaqrouqK. et al.

    Off-line handwritten signature recognition by wavelet entropy and neural network

    Entropy

    (2017)
  • JagtapB.A. et al.

    Eigen value based features for offline handwritten signature verification using neural network approach

  • KumarA. et al.

    A robust offline handwritten signature verification system using writer independent approach

  • AshrafA. et al.

    Handwritten signature verification using haar cascade classifier approach

  • E.N. Zois, I. Theodorakopoulos, D. Tsourounis, G. Economou, Parsimonious coding and verification of offline handwritten...
  • ThakareS.B. et al.

    A novel end-to-end approach for offline signature verification system

  • TsourounisD. et al.

    Handwritten signature verification via deep sparse coding architecture

  • XingZ.-J. et al.

    Offline signature verification using convolution siamese network

  • B. Yilmaz, . Mustafa, K. Ozturk, Hybrid user-independent and user-dependent offline signature verification with a...
  • BhuniaA.K. et al.

    Signature verification approach using fusion of hybrid texture features

    Neural Comput. Appl.

    (2019)
  • ÇalikN. et al.

    Large-scale offline signature recognition via deep neural networks and feature embedding

    Neurocomputing

    (2019)
  • MaergnerP. et al.

    Combining graph edit distance and triplet networks for offline signature verification

    Pattern Recognit. Lett.

    (2019)
  • HafemannL.G. et al.

    Writer-independent feature learning for offline signature verification using deep convolutional neural networks

  • ThakareS.B. et al.

    A combined feature extraction model using SIFT and LBP for offline signature verification system

  • OkawaM.

    Offline signature verification with VLAD using fused KAZE features from foreground and background signature images

  • OkawaM.

    KAZE features via Fisher vector encoding for offline signature verification

  • SharifM. et al.

    A framework for offline signature verification system: Best features selection approach

    Pattern Recognit. Lett.

    (2018)
  • NarwadeP.N. et al.

    Offline handwritten signature verification using cylindrical shape context

    3D Res.

    (2018)
  • MaergnerP. et al.

    Offline signature verification by combining graph edit distance and triplet networks

  • MaergnerP. et al.

    Offline signature verification via structural methods: Graph edit distance and inkball models

  • E.N. Zois, I. Theodorakopoulos, G. Economou, Offline handwritten signature modeling and verification based on...
  • WenJ. et al.

    Off-line signature verification based on multi-scale local structural pattern

    Int. J. Pattern Recognit. Artif. Intell.

    (2017)
  • RanjanY. et al.

    Static signature authentication based on J48 and random forest

    (2017)
  • MaergnerP. et al.

    A structural approach to offline signature verification using graph edit distance

  • HuJ. et al.

    Offline signature verification using local features and decision trees

    Int. J. Pattern Recognit. Artif. Intell.

    (2017)
  • Cited by (35)

    • A new synthetic feature generation scheme based on artificial immune systems for robust offline signature verification

      2023, Expert Systems with Applications
      Citation Excerpt :

      Such system can verify the authenticity of questioned signatures without giving any information about the signer identity. Commonly, the writer-dependent verification is preferred since it allows much better performances for a growing number of individuals (Impedovo & Pirlo, 2008; Fierrez-Aguilar et al., 2005; Nemmour & Chibani, 2013; Hu & Chen, 2013; Serdouk et al., 2016; Hafemann et al., 2017b; Hameed et al., 2021). In both scenarios, the offline SVS is composed of the feature generation and the verification steps.

    • Dynamic and static feature fusion for increased accuracy in signature verification

      2022, Signal Processing: Image Communication
      Citation Excerpt :

      These results can express the consistency of the data set built. Considering that this study used only 4 genuine samples in the training of the classifier and obtained results are similar to some of the results from studies in Table 2, Tables 5–6 and publication [19], it can be said that the proposed approach using only offline (static) signature image data is quite comparable to the offline signature verification studies at the state-of-the-art level. Spectral Flux Onset Strength Envelope and Spectral Centroid images of the genuine signature sounds and the forged signature sounds of the same signatures corresponding to offline signatures (Fig. 26) are given in Fig. 27.

    View all citing articles on Scopus
    View full text