Abstract
Signatures are one of the most important means to ensure the authenticity of documents and are commonly used in life and work. In identifying imitation handwriting, it is easy to make mistakes that cannot correctly identify and evaluate different writing characteristics. In this paper, from the perspective of dynamic handwriting detection, we propose RF sign, a signature anticounterfeiting real-time monitoring model, which achieves passive recognition of signature behavior using only a single antenna with a single tag. The RF sign identifies different users by extracting fine-grained reflection features from the original RF signal. We introduced a dynamic time regularization and neural network technique for similarity calculation and signature recognition matching to achieve template matching and classification. We compiled a real-time signature handwriting detection system. The system effectively identifies the person’s signature by checking real-time spatial and temporal information. Comprehensive experiments show that the recognition accuracy of my signature can reach over 93% and is robust to input location, environmental changes, and user diversity.
1. Introduction
1.1. Motivation
As one of the important branches of biometric identification technology, handwriting detection is a hot topic in the field of information security technology. The traditional handwriting detection method based on visual images needs to collect a large number of users’ private information, which has great risks. As a dynamic detection scheme, the handwriting detection based on radio frequency (RFID) can solve the problem of encroachment on user privacy to a certain extent. The RFID scheme detects handwriting by collecting information such as the relationship between handwriting strokes and time as well as writing speed, which has higher resolution and lower accuracy than the vision-based solution. In this paper, we aim to design a robust contactless handwriting recognition system to achieve accurate and robust handwriting recognition.
1.2. Prior Approach
The existing vision-based handwriting recognition is easily affected by multiple factors, such as environment, illumination, production process, and noise, and has the disadvantages of small interaction space, insufficient algorithm ability, and low real-time performance. Handwriting recognition methods based on RFID extend the interaction space beyond the limited screen and avoid the problems of screen inclusion and insufficient light, which are divided into two categories: active behavior recognition and passive behavior recognition. Active activity recognition is to attach a label to the target object, although such a method can locate the writing trajectory more clearly, carrying the label affects the user experience. Although the problem of portability is solved by placing tags around the target object to realize passive activity recognition, passive activity recognition is more susceptible to multipath effects and has low recognition accuracy.
The above two methods are limited to the identification of writing trajectories and cannot correctly distinguish the identification of authorship. This paper proposes a dynamic handwriting recognition model RF sign based on radio frequency identification (RFID), which enables dynamic handwriting recognition in the signature process. It can verify the authenticity of handwriting and distinguish different users by collecting the writing track of the writer in real time and has broad application prospects in case detection and judicial identification. Good robustness and high accuracy.
1.3. Challenges
To implement RF-based handwriting detection, we propose an innovative work RF sign. We constructed an I/O system using a single antenna, a single tag, a reader, and a PVC plastic homemade three-dimensional container. At the same time, we use the multipath effect to enhance the RF reflection signal to achieve high recognition accuracy. The basic idea of RF sign is to use the I/O system to identify different writers and authenticate at least five handwriting samples of the writers on this basis. Three major challenges have now been recognized: (i)In order to enhance the universality of the system, we use a commercial RFID reader, single tag, and single antenna deployment experiment scenario. However, in real life, small changes in the position of the receiver or target will lead to changes in the received signal mode. As a result, the signal measurement of the moving target is inaccurate, and the sampling accuracy is greatly reduced. How to solve the problem of the low sampling rate of a commercial reader and difficult to recover effective motion signal is the key to realize real-time signature azimuth monitoring model(ii)In the passive recognition deployment, the writing action is too fine-grained, and the rich interaction makes the backscattered signals inevitably mixed together, obscuring the information written by individuals. How to separate and extract writing action signals from mixed signals is an urgent problem for us to solve(iii)Radiofrequency technology captures moving objects with coarse-grained, so it is impossible to identify and acquire user biometrics with too fine-grained, which is bound to affect the accuracy. Therefore, how to use the extracted user characteristics to accurately identify different users becomes a difficult problem
1.4. Our Solution
In order to solve the above problems, this paper proposes a real-time monitoring model of signature anticounterfeiting based on a single label. The system uses single label and single antenna deployment mode to realize handwriting recognition and uses compressed sensing algorithm to recover effective moving target signals from low sampling rate signals. A three-dimensional baffle made of PVC plastic was used to reflect the signal, which produced a small-scale effect. The interactive multipath effect enhanced the signal and obtained more observable data. The linear difference with strong autocorrelation was used to remove the outliers in the received signal and automatically eliminate the phase shift. A monitoring model combining machine learning and deep learning is built to solve the challenges of fine-grained recognition and detection. The training model constructed only requires a few parameters for training, which makes experimental deployment and training easier and lays a foundation for accurate handwriting identification. Overall, the proposed RF sign achieves an overall accuracy rate of over 92% and is robust to environmental changes and user diversity.
2. Related Work
Existing handwriting monitoring solutions can be divided into two categories: wearable-based devices [1–5] and wireless-based devices [6–9].
2.1. Wearable Devices
Wearable device-based approaches typically require the user to wear a sensor, such as an RFID tag [10] or a smartwatch [8] and track the motion of the sensor to identify the handwriting. These studies usually derive handwritten content by building theoretical models to describe the signal variations received from the sensors. However, device-based monitoring methods can result in uncomfortable user experiences or short life cycles due to high energy consumption. The device-free approach recognizes handwritten fonts from environmental signals through different types of technology without requiring the user to wear any device. As the most popular solution for wireless devices, camera-based solutions [11] apply real-time image recognition, capture targets by camera shots, such as kinect and leap motion, and build writing structures from the video stream for accurate handwriting recognition. However, they usually involve high computation and may raise privacy issues for users. WiFi signal-based handwriting recognition is easy to deploy, low cost, and not limited by light vision, etc. However, a large number of WiFi devices have been put into use, resulting in signal strength and channel state information for feature extraction [12, 13]. CSI is subject to severe multipath interference, making signal feature extraction difficult, adapting to few scenarios, and not suitable for large scale deployment. Shangguan and Jamieson proposed to locate the human body based on COTS RFID technology through a device-free approach [14], which shows the potential of device-free sensing in RFID systems. The radio frequency-based approach uses radio frequency signals for real-time tracking of hand movements to achieve handwriting recognition. The target is located by measuring the effect of the target on the reflected signal (eg., phase shift [15–17], Doppler shift [18], or signal received intensity (RSSI) [19]) metric. Based on the adopted metrics and deployment methods, the previous wireless RF approaches can be further classified as follows.
2.2. Wireless Device Based
2.2.1. Device-Free: Antenna Array
RF-IDraw [10], the first RFID-based system, uses eight antennas to rigidly position and form an array that can accurately track a user’s writing or gestures in the air. Because the distance between the antenna and the tag requires much greater than the spacing between the antennas, RF-IDraw is difficult to detect subtle movement caused by changes in direction. Tagoram [20] uses four reader antennas to track moving tags in real time and uses the similarity between measured phase and theoretical values to estimate the motion trajectory with centimeter-level accuracy. However, Tagoram is also insufficient for handwriting monitoring in a small area when the trajectory is unknown. PolarDraw [8] deploys two antennas to implement the translation of the pen on the whiteboard and uses RSS readings to estimate the direction of movement of the tag, which is then combined with the estimated distance traveled by the phase difference to track the trajectory. While these systems can track the trajectory of tags, we focus on handwriting detection, which is a fine-grained handwriting recognition problem.
2.2.2. Device-Free: Tag Array
RFace uses a array of COTS RFID tags to extract the 3D geometry and internal biomaterial features of a human face. RF finger [21] uses a array of tags on letter-size paper to sense fine finger movements executed in front of the paper, allowing precise tracking of fine-grained finger trajectories and recognition of multitouch gestures. RF-IPad [22] builds a -tag array and converts the tagging plane into a virtual touch screen, allowing users to perform over-the-air handwriting and touch screen operations in a common area and recognizing human writing by detecting pen strokes. However, RF-IPad cannot detect different users.
2.2.3. Device-Free: WiFi
Writing-fi [23] recognizes handwriting movements from channel state information (CSI) and designs a signal generalization scheme to simulate the possible changes in handwriting movements of different people by synthesizing variable data so as to mitigate the effects brought by the diversification of writing speed and scale. WiWrite [24] is an accurate deviceless handwriting recognition system. A CSI segmentation scheme is proposed to deal with noisy original WiFi channel state information (CSI), and a low-noise data is retained for recognition by means of self-fixing dense convolutional networks (SPDCN).
2.3. Deep Learning and Artificial Intelligence
Hong et al. [25] proposed a general MDL framework consisting of two subnetworks, Ex-Net and Fu-Net, which aims at providing a baseline solution for pixel-level RS image classification tasks using multimodal data, but its classification heavily depends on the quality and quantity of samples. Wu [26] proposed a simple and effective “U-Net in U-Net” framework, referred to as UIU-Net, to detect small objects in infrared images and realize multilevel and multiscale representation learning of objects. Moreover, UIU-Net can be trained from scratch, and the learned features can effectively enhance the global and local contrast information.
3. Preliminary and System Overview
The main goal of our work is to detect the signature user and identify the signature content by radiofrequency methods. For this purpose, we designed an RFID-based system, RF sign, which reads the signal on the tag for handwriting recognition. As shown in Figure 1, RF sign has four main modules: two core modules for signal acquisition and signal preprocessing and two functional modules for user identification and handwriting content identification. Specifically, RF sign takes as input the time-series signal si(t) received from the tag i, including RSSI, phase, and Doppler shift information. The signal preprocessing module first solves the problem of low reader sampling rate by compressing the sense reconstruction algorithm and then replaces the misread signal by linear interpolation after acquisition and smooths and calibrates the measured signal using low-pass filtering. Next, we cut and extract key feature values from the smoothed signal by analyzing the fluctuation and strength characteristics of all signals.
After extracting the reflected features from the RF signal, two functional modules are used to implement user authentication and handwriting identification. For user identification, the identification module analyzes user writing habits based on the peak fluctuations of each signal, extracts three main feature vectors from them, and clusters them using the k-means algorithm to obtain an accurate user classification. For handwriting content, the handwriting content recognition module convolves the visual features using the neural network AlexNet to train and automatically classify each handwritten word. Among them, our module takes a Markov transform field to convert the extracted main feature vectors into two-dimensional pictures from the perspective of time series analysis, describes the phase changes of hand movement at different stages of signature writing, and learns neural network models for font classification recognition from the independent MRF phase maps constructed from the writing features.
3.1. Indicator Selection
3.1.1. Signal Reception Strength
Received signal strength indicator (RSSI), which indicates the metric of the tag signal energy received by the reader, the relationships between RSSI and signal energy can be expressed as follows:
For RF sign, we use a commercial reader, and the channel fading factor of the commercial reader is . Usually, the value of in the system is taken as 4. In practice, the value of RSSI has changed, and its value corresponds to 0 dBm and -115 dBm from strong to weak. For the accuracy of the experiment, we usually place the distance from the reader to the tag in the range of 1 m, assuming that the value of 1, brings formula (3) into formula (2), the simplification can be obtained
3.1.2. Phasing
The phase is a specific point on the waveform in the periodic motion. The phase information is the phase difference between the transmitted signal and the received signal. Two electromagnetic waves of the same frequency and phase can be superimposed. Similarly, the superposition of two electromagnetic waves of the same frequency and opposite phase, i.e., those with a phase difference , will cancel. The phase information is constantly transformed in the (0, 2) range and can be expressed as
When the relative distance between the receiving antenna of the reader and the tag, the electromagnetic wave scattering distance changes, and the phase information (phase) of the tag signal also changes. Phase information applies to more subtle and fine-grained sensing. If the communication distance between the tag and the reader antenna is , represents the transmission wavelength, and refers to the system noise due to the hardware such as the tag and the reader
3.1.3. Doppler Shift
Commercial readers can directly measure the signal strength of the tag and the phase value of the tag. The reader obtains the Doppler frequency bias by differencing the phases at different moments. Assume that the tag moves at a rate of concerning the reader antenna, and the angle between the velocity vector and the length vector is , the reader antenna signal frequency changes for the tag . Similarly, the tag signal frequency changes relative to the reader . Therefore, the frequency of the tag signal received by the reader antenna becomes relative to the frequency of the transmitted signal which can be summarized as follows:
3.1.4. Index Selection
To achieve the innovation and universality of the system, we build the system and collect the signals from commercial readers. The signal metrics that can be received by commercial readers mainly contain RSSI, phase, and Doppler. It is found that RSSI signals are sensitive to change and susceptible to interference from the external environment, which is not suitable for the signal characteristics analysis in the multipath environment in this paper. The Doppler shift signal is mainly determined by the displacement value, which changes obviously only when the tags move relative to each other. In this experiment, there is almost no relative displacement between the tag and reader antenna, so it is not suitable for signal analysis of gesture recognition in this system. A phase signal is a measure of change from one state point to another state, and it changes periodically. Therefore, we acquire the phase signal under the deployment condition of a single tag and single antenna.
The RF sign is composed of two main parts: the signal foundation and functional design. From Figure 1, the first part is the core module, which is mainly responsible for signal acquisition and signal preprocessing to solve the problems of insufficient sampling rate and phase shift of the reader. The second part is the function module, which mainly uses the k-means algorithm and classification model to realize two functions of user identification and signature content recognition. In the following sections of this section, we will outline the core architecture and tag deployment of RF sign.
3.2. Tag Signal Exploration and Analysis
As shown in the upper part of Figure 2 for signal variation, we observe that large changes in the surrounding environment can override fine-grained handwriting actions and have a significant impact on the reflected signal from the tag. To eliminate this interference, as shown in Figure 3, the RF sign exploits the effect of multipath effect by adding a PVC plastic transparent container, deploying the tag squarely on the antenna, and analyzing the three views projected from the transparent container while superimposing multiple signal waves of the tag to make it better to perceive the gesture fine-grained feature values.
4. System Design
Commercial RFID readers and standard low-level reader protocols (LLRP) can pick up tag signature signals such as phase and Doppler shift. As pointed out in previous research work [10], the accuracy of the read signals is not high enough to be directly applied to activity recognition due to factors such as low sampling rate, misreadings, and phase shifts. To address these factors, we propose an innovative design solution for RF sign to preprocess the measured reflected signals while increasing the sampling rate of the reader (as shown in Figure 4).
4.1. Data Acquisition
The iterative soft thresholding algorithm (IST) solves the coefficient solutions of the linear system of equations and the sparse solutions of the nonlinear constraint problem and thus recovers the reflected signal. Instead of keeping the signal above the threshold, the IST algorithm performs an appropriate shrinkage. In brief, when performing a thresholding operation using the soft thresholding algorithm, a threshold iteration operation is performed on a defined point until the value of that point stabilizes. The threshold shrinkage-related operations used are as follows: where denotes the signal data in the sparse vs. is the threshold value to be applied to, which will follow the change of data. The advantage of the IST algorithm is twofold.
4.1.1. Breaking the Constraints of Traditional Sampling Theorems
The IST algorithm accurately recovers sparse signals that are compressed or under known change domains by a nonlinear reconstruction algorithm. Among the prerequisites that need to be satisfied for compressed perception are sparsity and irrelevance. First, the signal should have the property of sparse expression in a certain domain of variation, i.e., there are fewer nonzero values in a certain domain of variation. If the rotating signal has the property of periodicity in the time domain, the signal will be sparse in the frequency domain after FFT (as shown in Figure 4), i.e., where is the sparse coefficient of the signal in the frequency domain and the matrix is the Fourier basis matrix. Thus, the sparse expression of the correlated signal in the frequency domain is obtained.
4.1.2. Weak Interrelationship
Based on two major premises, it is clear that the CS theory is mainly based on first selecting the sampled signal with sparsity on an appropriately sparse basis , and then the exact reconstruction can be obtained by linear projection of the signal on another incoherent basis .
RF sign adopts an iterative soft thresholding algorithm, nonlinear reconstruction for accurate recovery of sparse signals, and weak mutual relations for interference suppression. As shown in Figure 4, the RF sign solves the problem of the low sampling rate of existing readers with higher adaptivity, and the accurate original signal is obtained by reconstruction and recovery of the IST algorithm.
4.1.3. Adaptive Adjustment Threshold
The IST algorithm can be traced back to various types of iterative algorithms and general splitting algorithms, where can be set to of the maximum mode value in the sparse domain thus making it easier to achieve an adaptive adjustment threshold. The iterative process we can divide into two phases: the time domain data fidelity phase and the threshold shrinkage phase for sparse domain data, viz, where is the undersampled time-domain data and is the undersampled inverse Fourier change, i.e., the random sampling matrix is combined with the operation of inverse Fourier transform, denotes the Fourier transform, denotes the reconstructed signal, and denotes the reconstructed sparse domain signal.
Through the reconstruction and recovery of the IST algorithm, the reader sampling rate in this paper is increased from 10 frames per second to 200 frames per second, which not only solves the problem of a low reader adoption rate but also ensures the integrity and effectiveness of data sample collection. In Figure 4, the upper Figure 1 shows the unimproved reader sampling frequency spectrum and the lower Figure 1 shows the reader sampling spectrum obtained after reconstruction and recovery using the IST algorithm. It can be seen that the signal sampling rate has been significantly improved.
4.2. Signal Preprocessing
In RFID systems affected by the uneven frequency response of the tag antenna, the RF signal received by the reader has inherent measurement defects such as misread tags and noise. When signing, the hand motion is in a highly dynamic state, which makes the frequency of tag misreadings increase and outliers increase. To address this problem, we improve the reliability of the RF signal by linear interpolation and smoothing of the misread tags. Referring to the fundamental threshold of the phase obtained by sampling, we use 3 as the threshold factor, and the value beyond the threshold factor is the outlier (as shown in Figure 5). Linear interpolation of the misread RF signals from adjacent sampling wheels based on continuous finger movements. Take the phase stream as an example, which is the time series phase value from a tag. If there is a misread phase , we calculate the linear interpolation based on the other phase readings: where and are the two-phase readings adjacent to each other before and after the time. After interpolation, the signal is smoothed using low-pass filtering to further remove high-frequency noise. Figure 6 illustrates the effectiveness of our data calibration by comparing the phase flow before and after the data calibration. The phase stream shown in the figure is a set of data in the tag reflection signal when the user writes the letter B. From the enlarged plot, we can see that the misread outliers are well interpolated and replaced, and in addition, the high-frequency sawtooth waves in the smoothed signal are removed.
4.3. Feature Selection
4.3.1. User Feature Extraction
To study the specific signal pattern of the signature movement, we need to determine the starting and ending points of the signal segment, which corresponds to the user’s pen-fall and pen-retract postures. Therefore, we propose a segmentation method for detecting calibrated signature RF signals, which is used to segment the signals by detecting the pen drop (start action) and the pen retraction (release action) of the hand. In brief, we observe that the signal is stable when one puts down the pen; the phase wave signal of the tag changes significantly when the user performs a signature movement. Based on the first and last peaks of the calibrated reflection signal phase, we can detect the action of the hand-held pen and the pen lift and then use the signal flow between them as the signature signal. The cutting method is shown in Figure 7. The calibration signal is divided into three parts by us: pen drop, signature, and pen receipt, and the signal is segmented by using the action signals of pen drop and pen receipt to cut the signal into three parts: raise hand, write, and release hand, and finally, the signature signal of each user is extracted and obtained. Based on the extracted signature signals, we can see that different users have different writing speeds, strength levels, and starting and closing speeds, and we can use these three features to achieve recognition of different user identities.
4.3.2. Time Domain Characteristics of the Tag Signal
We analyze the three tag characteristic signals (RSSI, phase, and Doppler shift) received by the reader separately. RSSI signals are sensitive to changes and susceptible to external environmental interference and are not suitable for signal characterization in multipath environments. The Doppler shift signal is mainly determined by the displacement value, which only changes significantly when the tags move relative to each other. As shown in Equations (10)–(12), there is almost no relative displacement between the tag and the reader antenna in this experiment, and when the relative movement speed of the tag and the reader antenna is almost 0, the Doppler shift will not produce more obvious changes, so it does not apply to the signal analysis of the gesture recognition of this system. The phase signal is a measure of the change from one state point to another and shows a periodic variation. Because the phase information is a continuous signal belonging to a one-dimensional time series, this paper further analyzes and processes the tag phase signal to abstract the problem as a time-series-based signal processing.
The variation in the emission process of the signal is generated by the following expression:
is the time-varying attenuation of the signal amplitude of different paths, determined by the path loss and shadow fading; indicates the time delay during signal transmission of different transmission paths; transmission delay , indicates the multispectral phase shift of different paths, and the signature action is performed by the user, causing the tag signal to change.
4.3.3. MKF Transformation
Since the phase signal received by the reader belongs to the data set that conforms to the time series and the value at the moment of the phase signal on the time series is not necessarily related to the value at the moment, this property is consistent with the Hidden Markov property. In contrast to the traditional Hidden Markov model in which the transfer matrix is not sensitive to the time on the sequence this feature, to make the final experimental matching results more intuitive, the RF sign uses the independent signals that have been separated for transformation, so that the problem is abstracted from one-dimensional time-series data to a two-dimensional image of the model classification problem, more intuitive to reach the purpose of convenience and speed. The steps are as follows:
Step 1. The data in the time series is divided into data segments.
Step 2. The algorithm for constructing the Markov transfer matrix , matrix range , where is determined by the frequency of the immediately adjacent data in and :
Step 3. Construct the Markov variational field , with matrix dimensions as to where the values of are
After assigning a phase space to each position according to a random distribution, the value in any one position is only related to the neighboring positions and not to the other positions. Using this property, we transform the tag phase information conforming to the time series into a Markov transformation field (MTF), and Figure 8(a) shows the independent phase MRF plot of the calibrated RF signal.
(a) Without batch cutting
(b) With batch cutting
4.4. Classifier Optimization
This section describes the main components of our deep model design. Our deep learning design incorporates the results of data preprocessing into the deep learning architecture, i.e., calibrated signed RF signals and hidden Markov graphs, as shown in Figures 8(a) and 8(b), respectively.
As shown in Figure 9, we improve the convolutional neural network AlexNet network and use the K-means clustering algorithm to implement the recognition of user handwriting and user identity, respectively. K-means clustering can group similar RF signals to achieve coarse-grained classification and identification of different users. In contrast, the AlexNet network can learn dynamic temporal relationships from continuous Markovian variograms to achieve fine-grained recognition of signature handwriting.
RF sign first uses the k-means clustering algorithm to extract the user characteristics in the RF signal from the calibrated signature RF signal: signature speed, strength size, pen starting and closing speed as the initial center of mass, calculate the distance from each sample to each center of mass, divide the samples into clusters corresponding to the nearest center of mass, and then achieve the recognition of different users.
After implementing user recognition, we optimize the AlexNet convolutional neural network to achieve recognition of the user’s handwritten content by recognizing the signature signal bipartite map. The dataset consists of independent Markov images of calibrated signed RF signals in 1D to 2D, mostly regular blocks of pixels at resolution 24. By comparing the traditional machine learning SVM, and neural network models DensNet, GoogleNet, VggNet, and AlexNet, we found that AlexNet has certain advantages over other models in terms of accuracy, etc. The results are shown in Table 1. On this basis, the structure of the AlexNet model is adjusted and optimized, and the AlexNet convolutional neural network model, which consists of 5 convolutional layers and 3 fully connected layers with a total of 8 layers indepth, is simplified to 5 layers, making it more suitable for training and classification of MRF images and more accurate discrimination of image features.
In the optimized model training process, the first step of the input data is a RGB image, which outputs a -dimensional image vector-matrix after an convolution kernel and 96 filters, i.e., hidden layer operations. After performing the relu function activation, the dimensional data are output through a maximum pooling layer with local response normalization. After four convolution operations, the final output -dimensional matrix is fed into the fully connected layer of the dropout function. Finally, we use the Softmax function to activate and output 5 classifications to complete the task of handwriting classification recognition.
4.4.1. Metrics
To validate the performance of RF sign, we selected three main metrics, namely, false acceptance rate (FAR), false rejection rate (FRR), and accuracy (ACC). FAR is a measure of the likelihood of RF sign errors accepting access attempts by unauthorized users (i.e., unauthorized arrays), where it is calculated as
FRR is a measure of the likelihood that an RF sign error will deny access to an authorized user
For each component, our main focus is on detection accuracy. We use three metrics, namely, accuracy rate, false acceptance rate (FAR), and false rejection.
4.5. Function Implementation
To implement the signature recognition monitoring function, we further processed the calibrated signature signal, extracted the feature values, generated the feature images, and finally used the improved neural network model to achieve high-precision signature recognition monitoring. This paper mainly implements the user recognition function and the handwriting recognition function.
4.5.1. User Identification
Based on the extracted signature signals, after repeated experiments, we can roughly infer the user’s writing habits from three aspects: the writing speed, the size of the writing strength, and the time difference of the user’s stroke, to infer the writing user of the text during the writing process. In this paper, the user’s writing speed is judged from the amount of data corresponding to a single font, starting from the following three aspects, respectively, inferring the strength of the user’s writing from the change in the size of the wave crest. From the duration of the troughs, and thus the writing times corresponding to the different changes are obtained. From these three aspects, we can further infer that the writers are symbolized by the different fields. Writing and using these three feature vectors to further infer the user to whom the monitoring handwriting belongs. Feature selection is shown in Figure 10.
4.5.2. Handwriting Recognition
In this work, we increase the signal fluctuation of a single tag by artificially adding interferents and consider the identification of 5 handwritten letters using calibrated RF signature signals. To reclassify the signature signals when it has been determined that each field represents a different user, a dataset of 5 handwritten letters was created. The comparison of samples of the same letter written by different users is shown in Figure 11.
5. Implementation and Evaluation Results
5.1. Implementation
5.1.1. Experimental Setup
We conduct experiments in a typical office environment. Figure 12 shows the default settings. It includes a signal acquisition device based on an H47 tag and antenna, a signature signal sensing device based on an Impinj Speedway R420 reader, and a computer-based backend device for user data storage and processing, with the commercial tag reader and computer connected via a network cable and the antenna and commercial tag reader connected. Meanwhile, we made a 0.15 m×0.15 m PVC plastic transparent container to enhance the multipath effect, and the tag was placed inside the plastic container and formed an airtight environment opposite the antenna, with an opening on one side to facilitate the user to sign. During the experiment, max throughput was chosen as the reader mode to improve the accuracy of the experiment so that the measured values could reach the maximum throughput. And since the reader antenna is placed squarely with the tag, the reader continuously interrogates the tag and collects the signature signal. The signal is transmitted over the network cable to the back-end PC running the RF sign.
5.1.2. Metrics
To validate the performance of the RF sign, we selected three main metrics, namely, false acceptance rate (FAR), false rejection rate (FRR), and accuracy (ACC).
FRR is a measure of the likelihood that an RF sign error will deny access to an authorized user.
For each component, our main focus is on detection accuracy. We use three metrics, namely, accuracy rate, false acceptance rate (FAR), and false rejection rate (FRR), to evaluate the overall performance of the system. Accuracy is defined as the rate at which a tag is correctly matched to its corresponding client. FAR is the rate at which the reader incorrectly accepts tag information that is not of interest, and FRR is the rate at which the reader incorrectly rejects tag information for interaction.
5.2. Performance Evaluation
To evaluate the practicality and robustness of the system, we selected the following five additional aspects to conduct experiments to comprehensively evaluate the system performance in terms of distance factor, tag factor, classifier factor, multiuser experience, and accuracy on the improved network model, respectively.
5.2.1. Distance between Tag and Antenna
In the tag-to-antenna distance experiments, we change the communication range of the tag and antenna of the COTS RFID hardware mainly from 40 cm to 60 cm in increments of 5 cm each time. To comply with common sense and ensure signal accuracy under the condition that the distance between the tag and the hand is constant, the initial value of the range was selected for this experiment, 40 cm. The same experiment was performed several times at each distance, as is evident from Figure 13(a). The highest recognition accuracy was achieved at 40 cm. It follows that as the distance increases, the average error increases correspondingly. This is all since the signal propagation distance in the air becomes larger, which indirectly weakens the recognition accuracy. And to achieve the best recognition performance, we also recommend that the antenna and tag distance maintain at about 40 cm, and then effectively in the device placement conditions to improve the accuracy of frequency detection.
(a) Tag to antenna distance accuracy comparison
(b) Tag to hand distance accuracy comparison
5.2.2. The Distance between the Tag and the Hand
For this experiment, we varied the distance between the tag and the hand from 10 cm to 30 cm in increments of 5 cm each time, with a constant distance between the tag and the antenna. We performed several trials at each distance and selected the best, as can be observed from Figure 13(b). When the distance between the tag and the hand is too large, the reflected signal will be too weak due to the late speed recognition accuracy being greatly reduced. To further enhance the accuracy and sensing range, we can further select a more effective directional antenna, which can greatly enhance the experimental accuracy and sensing range by aggregating the energy compared to the previous one.
5.2.3. Impact of Tag Types
In this experiment, we chose to detect the handwriting of the same user with the tags shown in the figure, which contain two commonly used tags. One is 9662 with a size of mm and the other is T92 with a size of mm. In addition, we also selected the size of mm H47; the comparison chart is shown in Figure 14. These three tag positions were replaced in the experiment, keeping the original environment unchanged, and the accuracy of the two major directions was studied separately. From Figure 15(a), it can be learned that the recognition accuracy of using the H47 tag is higher, after thinking that the reason may be due to the smaller cross-section of other tags, which will make its coupling effect weaker than that of the H47 tag. However, it should still be noted that the coupling effect is affected by the variation in tag reader distance, so overall, different types of tags can achieve more similar accuracy with a significant reduction in distance.
(a) Three kinds of tag recognition accuracy comparison
(b) Comparison of signal strength from different angles
5.2.4. Different Angles
In this experiment, we changed the angle of the tag from a 30° angle to a 90° angle, rotating 30° each time and subsequently transposing the tag to 180°, choosing a total of 4 angles. In addition, the distance between the tag and the antenna is constant at 40 cm, and the distance from the hand is constant at 10 cm. To ensure the accuracy of the test data, we conducted several tests on each angle and selected the best, and the final test results are shown in Figure 15(b). We can observe that when the tag placement angle changes, the phase of the reflected signal will produce a certain angle change, but still can be more accurate recognition of the handwritten content. Therefore, the variation of the tag angle has little effect on the experimental results. To reduce the error and facilitate the experimental sampling, our main experiment fixed the tag angle at 0°, which is directly opposite the antenna and hand.
5.2.5. Confusion Matrix of Handwriting Recognition Accuracy
In this work, we increase the signal fluctuation of a single tag by artificially adding interferents and consider the identification of 5 handwritten letters using calibrated RF signature signals. To reclassify the signature signals when it has been determined that each field represents a different user, a dataset of 5 handwritten letters was created. To achieve high accuracy in classification recognition, we extract the feature values to generate feature images and import the images into our improved AlexNet neural network to further recognize the text content written by the user and finally achieve the handwriting recognition function. The improved AlexNet neural network accuracy confusion matrix is shown in Figure 14.
5.2.6. Different Experimenters
Another 10 volunteers, five men and five women, were selected to demonstrate the usability of the experiment. Among them, 10 volunteers collected 5 groups of data of 5 handwritten characters A, B, C, D, and E, respectively, according to the experimental requirements, and 100 groups of real samples were collected. Data extraction is performed according to the experimental deployment, allowing volunteers to avoid carrying reader antennas parallel to each other as much as possible during the measurement process, thus reducing errors caused by human factors. The data was tested on four volunteers, and the test results are shown in Figure 16. In terms of user recognition, RF sign can distinguish different users with a stable accuracy rate of 86% or more on average for all volunteers tested thus proving that RF sign can perform more accurate user recognition and handwriting recognition.
5.3. Comprehensive Assessment
In addition to the above five groups of experiments, this paper conducted field simulation tests to verify the performance of RF sign in terms of security and real-time performance.
5.3.1. Anticounterfeiting
Randomly designate one volunteer to sign 5 groups of signature samples, and randomly select another volunteer to sign 5 groups of signature samples by imitation of the volunteer, and import all samples into RF sign for identification, so as to study whether RF sign can identify the authenticity of the handwriting under the condition that the user deliberately forges the handwriting. Figure 17 shows the letter A written by different users in the handwriting of volunteer 1. According to the test results of this experiment, it can be seen that RF sign can still detect the authenticity of the handwriting through different users’ writing strength and speed when the user deliberately imitates the handwriting to take out the signature, which has anticounterfeiting performance.
(a)
(b)
(c)
(d)
5.3.2. Real-Time Performance
In the test process, we recorded the time required by the RF sign to collect and identify each group of samples, respectively. The following Table 2 shows that RF sign has short operation time, low operation cost, and strong universality.
6. Conclusion
In this paper, we present the RF sign, a device-free note detection system based on the COTS RFID system. Passive recognition of signature behavior is achieved from the perspective of dynamic handwriting detection. The RF sign achieves recognition of different users by extracting fine-grained reflection features from the original RF signal. To achieve template matching and classification, we introduced a k-means algorithm with neural network technology for similarity calculation and signature recognition matching and compiled a real-time signature handwriting detection system. The system achieves effective identification of my signature by checking spatial and temporal information in real-time. It has a broad application prospect in case detection and forensic identification. The experimental results confirm the effectiveness of RF sign in user recognition and handwriting detection. The proposed conversion mechanism and the improved neural network model can significantly improve the recognition accuracy, with an accuracy rate of over 88% and 93%, respectively. However, RS-sign classification is very sensitive to the quality and quantity of samples. To further improve its performance bottleneck, in future work, we will augment the dataset and train the model.
Data Availability
There is no public dataset, and the data is collected by ourselves. For some special reasons, data is not available.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
The research is supported by the Intelligent Policing Key Laboratory of Sichuan Province (No. ZNJW2022KFZD004), Basic Research Plan of Shanxi Province (No. 202303021211339), Virtual Teaching and Research Office of Cyber Security (BJPC) of Ministry of Education (No. WAXVKF-2202), Anhui Natural Science Foundation (No. 2108085MF207), and Shanxi Provincial Higher Education Teaching Reform and Innovation Project, Teaching Reform Project of Shanxi Police College.