Multimodal analysis traditionally involves conceptualising abstract frameworks for language, images, and other resources and their intersemiotic relations (e.g. text and image relations) and then demonstrating these frameworks with some examples. This scenario has changed with the recent move towards multimodal approaches to big data analytics which will involve empirically testing and validating multimodal theory and frameworks through the analysis of large data sets. However, large training sets of analysed texts are required to develop computational models based on multimodal theory. Therefore, an alternative approach which involves integrating multimodal frameworks with existing computational models for big data, cloud computing, natural language processing, image processing, video processing, and contextual metadata is proposed. The integration of these disparate fields has the potential to dramatically improve computational tools and techniques, thus placing multimodality at the forefront of research aimed at mapping and understanding multimodal communication. As a step forward in this direction, we explore how existing computational tools and approaches can be integrated into a multimodal analysis platform (MAP) with facilities for searching, storing and analysing text, images and videos in online media, together with dashboards for visualising the results. Preliminary analyses and classifications of text and images about COVID-19 and George Floyd in five online newspapers and Twitter postings show how media patterns can be studied using existing computational tools. The study highlights (a) the benefits and current limitations of big data approach to multimodal discourse analysis and (b) the need to incorporate knowledge about language, images, metadata, and other resources as semiotic systems (rather simply sets of symbols and pixels) to improve computational techniques for big data analytics.

传统上，多模式分析涉及将语言，图像和其他资源及其符号间关系（例如，文本和图像关系）的抽象框架概念化，然后通过一些示例演示这些框架。随着近来向大数据分析采用多模式方法的转变，这种情况已经发生了变化，这将涉及通过对大数据集的分析对多模式理论和框架进行经验测试和验证。但是，需要大量的分析文本训练集才能开发基于多峰理论的计算模型。因此，提出了一种替代方法，该方法涉及将多模式框架与现有的大数据数据，云计算，自然语言处理，图像处理，视频处理和上下文元数据的计算模型集成在一起。这些不同领域的整合具有极大地改善计算工具和技术的潜力，因此将多模式性置于旨在映射和理解多模式通信的研究的最前沿。作为朝着这个方向迈出的一步，我们探索了如何将现有的计算工具和方法集成到多峰分析平台（MAP）中，该平台具有用于在线媒体中搜索，存储和分析文本，图像和视频的工具，以及用于可视化仪表板的仪表板。结果。对五种在线报纸和Twitter帖子中有关COVID-19和George Floyd的文本和图像的初步分析和分类，显示了如何使用现有的计算工具来研究媒体模式。