Named Entity Recognition (NER) from social media posts is a challenging task. User generated content that forms the nature of social media, is noisy and contains grammatical and linguistic errors. This noisy content makes it much harder for tasks such as named entity recognition. We propose two novel deep learning approaches utilizing multimodal deep learning and Transformers. Both of our approaches use image features from short social media posts to provide better results on the NER task. On the first approach, we extract image features using InceptionV3 and use fusion to combine textual and image features. This presents more reliable name entity recognition when the images related to the entities are provided by the user. On the second approach, we use image features combined with text and feed it into a BERT like Transformer. The experimental results, namely, the precision, recall and F1 score metrics show the superiority of our work compared to other state-of-the-art NER solutions.

中文翻译：

---

一种用于社交媒体命名实体识别的多模态深度学习方法

来自社交媒体帖子的命名实体识别 (NER) 是一项具有挑战性的任务。构成社交媒体性质的用户生成内容嘈杂且包含语法和语言错误。这种嘈杂的内容使命名实体识别等任务变得更加困难。我们提出了两种利用多模态深度学习和 Transformers 的新型深度学习方法。我们的两种方法都使用来自社交媒体短帖子的图像特征来在 NER 任务上提供更好的结果。在第一种方法中，我们使用 InceptionV3 提取图像特征并使用融合来组合文本和图像特征。当与实体相关的图像由用户提供时，这提供了更可靠的名称实体识别。在第二种方法中，我们将图像特征与文本结合使用，并将其输入到像 Transformer 这样的 BERT 中。