当前位置: X-MOL 学术Math. Probl. Eng. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Automatic Selection and Parameter Configuration of Big Data Software Core Components Based on Retention Pattern
Mathematical Problems in Engineering Pub Date : 2021-01-22 , DOI: 10.1155/2021/6667275
Ping Xu 1
Affiliation  

This paper conducts an in-depth analysis and research on the automatic selection and parameter configuration of the core components of Big Data software by using the retention model and the automatic selection of Big Data components by establishing a standardized requirement index and using the decision tree model to solve the problem of component selection in Big Data application development. By establishing standardized demand indicators and based on the retention model, a data transmission intermediate platform for bidirectional data detection is proposed based on the three demands of user input: storage, computation, and analysis, as well as the problem of undetectable packet loss in data transmission of existing IoT and Web service platforms. The data communication module of the data transmission intermediate platform enables mutual monitoring and detection of data interaction between IoT smart terminals and cloud platforms. The retention mode is built separately to realize the automatic selection of Big Data components. In this paper, we start from several mainstream distributed storage systems and use Cassandra as an example for experiments and tests. We use the multiple regression fitting method to build a corresponding performance model for hardware parameters, take user requirements as input, and use the performance model to configure system hardware parameters; by studying its system principle, architecture, features, and application scenarios, we build a software parameter configuration knowledge base to guide the software. This solves the difficult problem of selecting, deploying, and configuring parameters for Big Data applications.

中文翻译:

基于保留模式的大数据软件核心组件自动选择和参数配置

本文通过使用保留模型对大数据软件核心组件的自动选择和参数配置进行深入分析和研究,通过建立标准化的需求指数并使用决策树模型对大数据软件的自动选择进行了研究。解决大数据应用开发中组件选择的问题。通过建立标准化的需求指标并基于保留模型,基于用户输入的三个需求:存储,计算和分析,以及数据中不可检测的丢包问题,提出了一种双向数据检测的数据传输中间平台。现有物联网和Web服务平台的传输。数据传输中间平台的数据通信模块可以相互监视和检测IoT智能终端与云平台之间的数据交互。保留模式是单独构建的,以实现大数据组件的自动选择。在本文中,我们从几种主流的分布式存储系统开始,并以Cassandra为例进行实验和测试。我们使用多元回归拟合方法为硬件参数建立相应的性能模型,以用户需求为输入,并使用性能模型配置系统硬件参数。通过研究其系统原理,体系结构,功能和应用场景,我们建立了用于指导软件的软件参数配置知识库。这样解决了选择困难的问题,
更新日期:2021-01-22
down
wechat
bug