TopoAct: Visually Exploring the Shape of Activations in Deep Learning,Computer Graphics Forum

当前位置： X-MOL 学术 › Comput. Graph. Forum › 论文详情

Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)

TopoAct: Visually Exploring the Shape of Activations in Deep Learning
Computer Graphics Forum ( IF 2.7 ) Pub Date : 2021-01-13 , DOI: 10.1111/cgf.14195
Archit Rathore ₁ , Nithin Chalapathi ₁ , Sourabh Palande ₁ , Bei Wang ₁

Affiliation

Deep neural networks such as GoogLeNet, ResNet, and BERT have achieved impressive performance in tasks such as image and text classification. To understand how such performance is achieved, we probe a trained deep neural network by studying neuron activations, i.e.combinations of neuron firings, at various layers of the network in response to a particular input. With a large number of inputs, we aim to obtain a global view of what neurons detect by studying their activations. In particular, we develop visualizations that show the shape of the activation space, the organizational principle behind neuron activations, and the relationships of these activations within a layer. Applying tools from topological data analysis, we present TopoAct, a visual exploration system to study topological summaries of activation vectors. We present exploration scenarios using TopoAct that provide valuable insights into learned representations of neural networks. We expect TopoAct to give a topological perspective that enriches the current toolbox of neural network analysis, and to provide a basis for network architecture diagnosis and data anomaly detection.

中文翻译：

TopoAct：在视觉上探索深度学习中的激活形式

诸如GoogLeNet，ResNet和BERT之类的深度神经网络在诸如图像和文本分类之类的任务中取得了令人印象深刻的性能。为了了解如何实现这种性能，我们通过研究响应特定输入的网络各层的神经元激活（即神经元激发的组合）来探究训练有素的深度神经网络。通过大量的输入，我们旨在通过研究神经元的激活来获得神经元检测到的东西的全局视图。特别是，我们开发了可视化，以显示激活空间的形状，神经元激活背后的组织原理以及这些激活在层内的关系。应用来自拓扑数据分析的工具，我们介绍TopoAct，这是一个视觉激活系统，用于研究激活向量的拓扑摘要。我们使用TopoAct展示了探索场景，这些场景提供了对神经网络的学习表示的宝贵见解。我们希望TopoAct能够提供拓扑学的观点，丰富当前神经网络分析的工具箱，并为网络体系结构诊断和数据异常检测提供基础。

更新日期：2021-02-24

点击分享查看原文

点击收藏

公开下载

阅读更多本刊最新论文本刊介绍/投稿指南11