当前位置:
X-MOL 学术
›
arXiv.cs.AI
›
论文详情
Our official English website, www.x-mol.net, welcomes your
feedback! (Note: you will need to create a separate account there.)
MLP-Mixer: An all-MLP Architecture for Vision
arXiv - CS - Artificial Intelligence Pub Date : 2021-05-04 , DOI: arxiv-2105.01601 Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, Alexey Dosovitskiy
arXiv - CS - Artificial Intelligence Pub Date : 2021-05-04 , DOI: arxiv-2105.01601 Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, Alexey Dosovitskiy
Convolutional Neural Networks (CNNs) are the go-to model for computer vision.
Recently, attention-based networks, such as the Vision Transformer, have also
become popular. In this paper we show that while convolutions and attention are
both sufficient for good performance, neither of them are necessary. We present
MLP-Mixer, an architecture based exclusively on multi-layer perceptrons (MLPs).
MLP-Mixer contains two types of layers: one with MLPs applied independently to
image patches (i.e. "mixing" the per-location features), and one with MLPs
applied across patches (i.e. "mixing" spatial information). When trained on
large datasets, or with modern regularization schemes, MLP-Mixer attains
competitive scores on image classification benchmarks, with pre-training and
inference cost comparable to state-of-the-art models. We hope that these
results spark further research beyond the realms of well established CNNs and
Transformers.
中文翻译:
MLP-Mixer:面向视觉的全MLP架构
卷积神经网络(CNN)是计算机视觉的首选模型。最近,基于注意力的网络(例如Vision Transformer)也变得很流行。在本文中,我们表明,尽管卷积和注意力都足以获得良好的性能,但它们都不是必需的。我们介绍了MLP-Mixer,这是一种仅基于多层感知器(MLP)的体系结构。MLP-Mixer包含两种类型的层:一种具有独立应用于图像补丁的MLP(即“混合”每个位置特征),另一种具有跨补丁应用的MLP(即“混合”空间信息)。当在大型数据集上进行训练或采用现代正则化方案进行训练时,MLP-Mixer在图像分类基准上获得竞争性得分,其预训练和推理成本可与最新模型相媲美。
更新日期:2021-05-05
中文翻译:
MLP-Mixer:面向视觉的全MLP架构
卷积神经网络(CNN)是计算机视觉的首选模型。最近,基于注意力的网络(例如Vision Transformer)也变得很流行。在本文中,我们表明,尽管卷积和注意力都足以获得良好的性能,但它们都不是必需的。我们介绍了MLP-Mixer,这是一种仅基于多层感知器(MLP)的体系结构。MLP-Mixer包含两种类型的层:一种具有独立应用于图像补丁的MLP(即“混合”每个位置特征),另一种具有跨补丁应用的MLP(即“混合”空间信息)。当在大型数据集上进行训练或采用现代正则化方案进行训练时,MLP-Mixer在图像分类基准上获得竞争性得分,其预训练和推理成本可与最新模型相媲美。