当前位置: X-MOL 学术Knowl. Based Syst. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Squashing activation functions in benchmark tests: Towards a more eXplainable Artificial Intelligence using continuous-valued logic
Knowledge-Based Systems ( IF 7.2 ) Pub Date : 2021-02-20 , DOI: 10.1016/j.knosys.2021.106779
Daniel Zeltner , Benedikt Schmid , Gábor Csiszár , Orsolya Csiszár

Over the past few years, deep neural networks have shown excellent results in multiple tasks, however, there is still an increasing need to address the problem of interpretability to improve model transparency, performance, and safety. Logical reasoning is a vital aspect of human intelligence. However, traditional symbolic reasoning methods are mostly based on hard rules, which may only have limited generalization capability. Achieving eXplainable Artificial Intelligence (XAI) by combining neural networks with soft, continuous-valued logic and multi-criteria decision-making tools is one of the most promising ways to approach this problem: by this combination, the black-box nature of neural models can be reduced. The continuous logic-based neural model uses so-called Squashing activation functions, a parametric family of functions that satisfy natural invariance requirements and contain rectified linear units as a particular case. This work demonstrates the first benchmark tests that measure the performance of Squashing functions in neural networks. Three experiments were carried out to examine their usability and a comparison with the most popular activation functions was made for five different network types. The performance was determined by measuring the accuracy, loss, and time per epoch. These experiments and the conducted benchmarks have proven that the use of Squashing functions is possible and similar in performance to conventional activation functions. Moreover, a further experiment was conducted by implementing nilpotent logical gates to demonstrate how simple classification tasks can be solved successfully and with high performance. The results indicate that due to the embedded nilpotent logical operators and the differentiability of the Squashing function, it is possible to solve classification problems, where other commonly used activation functions fail.



中文翻译:

压缩基准测试中的激活功能:使用连续值逻辑实现更可解释的人工智能

在过去的几年中,深度神经网络已经在多个任务中显示出了优异的成绩,但是,仍然越来越需要解决可解释性的问题,以提高模型的透明度,性能和安全性。逻辑推理是人类智能的重要方面。但是,传统的符号推理方法主要基于硬规则,其概括能力可能有限。通过将神经网络与软的,连续值的逻辑和多准则决策工具相结合来实现可扩展的人工智能(XAI)是解决此问题的最有前途的方法之一:通过这种结合,神经模型的黑盒性质可以减少。基于连续逻辑的神经模型使用了所谓的压扁激活函数,满足自然不变性要求并在特定情况下包含校正的线性单位的参数函数系列。这项工作演示了第一个基准测试,用于测量神经网络中压缩函数的性能。进行了三个实验以检查其可用性,并针对五种不同的网络类型与最流行的激活功能进行了比较。通过测量准确性,损失和每个时期的时间来确定性能。这些实验和所进行的基准测试已经证明,可以使用挤压功能,并且在性能上与常规激活功能相似。而且,通过实现幂等逻辑门进行了进一步的实验,以演示如何成功且高性能地解决简单的分类任务。结果表明,由于嵌入的幂等逻辑运算符和压扁函数的可微性,有可能解决其他常用激活函数失败的分类问题。

更新日期:2021-02-26
down
wechat
bug