Individuals who are blind adopt multiple procedures to tactually explore images. Automatically recognizing and classifying users’ exploration behaviors is the first step towards the development of an intelligent system that could assist users to explore images more efficiently. In this paper, a computational framework was developed to classify different procedures used by blind users during image exploration. Translation-, rotation- and scale-invariant features were extracted from the trajectories of users’ movements. These features were divided as numerical and logical features and were fed into neural networks. More specifically, we trained spiking neural networks (SNNs) to further encode the numerical features as model strings. The proposed framework employed a distance-based classification scheme to determine the final class/label of the exploratory procedures. Dempster-Shafter Theory (DST) was applied to integrate the distances obtained from all the features. Through the experiments of different dynamics of spiking neurons, the proposed framework achieved a good performance with 95.89% classification accuracy. It is extremely effective in encoding and classifying spatio-temporal data, as compared to Dynamic Time Warping and Hidden Markov Model with 61.30% and 28.70% accuracy. The proposed framework serves as the fundamental block for the development of intelligent interfaces, enhancing the image exploration experience for the blind.

中文翻译：

---

基于尖峰神经网络的盲用户图像探索行为分类

盲人采用多种程序来触觉地浏览图像。自动识别和分类用户的探索行为是开发智能系统的第一步，该系统可以帮助用户更有效地探索图像。在本文中，开发了一种计算框架以对盲用户在图像探索期间使用的不同过程进行分类。从用户运动的轨迹中提取平移，旋转和比例不变的特征。这些特征被分为数字和逻辑特征，并被馈入神经网络。更具体地说，我们训练了尖峰神经网络（SNN），以将数字特征进一步编码为模型字符串。提议的框架采用基于距离的分类方案来确定探索程序的最终类别/标签。应用了Dempster-Shafter理论（DST）来整合从所有特征获得的距离。通过对突跳神经元动力学的不同实验，提出的框架以95.89％的分类精度取得了良好的性能。与动态时间规整和隐马尔可夫模型相比，它在时空数据的编码和分类方面非常有效，准确度为61.30％和28.70％。所提出的框架是开发智能界面的基本模块，可增强盲人的图像探索体验。通过对突跳神经元动力学的不同实验，提出的框架以95.89％的分类精度取得了良好的性能。与动态时间规整和隐马尔可夫模型相比，它在时空数据的编码和分类方面非常有效，准确度为61.30％和28.70％。所提出的框架是开发智能界面的基本模块，可增强盲人的图像探索体验。通过对突跳神经元动力学的不同实验，提出的框架以95.89％的分类精度取得了良好的性能。与动态时间规整和隐马尔可夫模型相比，它在时空数据的编码和分类方面非常有效，准确度为61.30％和28.70％。所提出的框架是开发智能界面的基本模块，可增强盲人的图像探索体验。