The success rate in offline signature verification studies has reached high and limiting levels recently. However, any increase in this performance is and will be highly valuable in terms of fraud detection. This study assesses the impact of the sound arising from the friction of pen and paper on handwritten signature verification. A dataset was built containing static data from the signature image and dynamic data from the signature sound by taking samples from 75 participants according to different combinations of pen, paper types, and mobile phone models for recording the sounds of the signatures with their internal microphones. It was aimed to increase verification success by fusing dynamic and static features. From the static data, the features are extracted by the LBP and SIFT algorithms. For dynamic data, spectral flux onset envelopes and spectral centroids of audio signals are plotted and converted to image files. Thus, the dynamic data of the signature sound signal became static data and as in the static image of the signature, feature extraction was performed with the LBP and SIFT algorithms. Classification is performed with the OC-SVM algorithm. Moreover, instead of LBP and SIFT features, another verification method with the deep features obtained with a CNN-based model was also proposed and comparatively analyzed. Test results indicate that the aforementioned fusion of these two traits leads to increased signature verification success rates (statistical significance test results are provided), without incurring large costs, considering the sensor availability and acquisition times.

离线签名验证研究的成功率最近达到了很高的极限水平。然而，这种性能的任何提高在欺诈检测方面都是非常有价值的。本研究评估了笔和纸摩擦产生的声音对手写签名验证的影响。通过根据笔、纸类型和手机型号的不同组合从 75 名参与者中采集样本，构建一个包含签名图像的静态数据和签名声音的动态数据的数据集，用于使用内置麦克风记录签名的声音。它旨在通过融合动态和静态特征来提高验证成功率。从静态数据中，通过 LBP 和 SIFT 算法提取特征。对于动态数据，音频信号的频谱通量起始包络和频谱质心被绘制并转换为图像文件。因此，签名声音信号的动态数据变为静态数据，并且在签名的静态图像中，使用 LBP 和 SIFT 算法进行特征提取。使用 OC-SVM 算法进行分类。此外，代替 LBP 和 SIFT 特征，还提出了另一种使用基于 CNN 的模型获得的深度特征的验证方法并进行了比较分析。测试结果表明，考虑到传感器的可用性和采集时间，上述两个特征的融合导致签名验证成功率（提供统计显着性测试结果）的增加，而不会产生大量成本。