当前位置: X-MOL 学术Complexity › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Research on the Key Issues of Big Data Quality Management, Evaluation, and Testing for Automotive Application Scenarios
Complexity ( IF 1.7 ) Pub Date : 2021-05-06 , DOI: 10.1155/2021/9996011
Yingzi Wang 1, 2 , Ce Yu 1 , Jue Hou 2 , Yongjia Zhang 2 , Xiangyi Fang 3 , Shuyue Wu 2
Affiliation  

This paper provides an in-depth analysis and discussion of the key issues of quality management, evaluation, and detection contained in big data for automotive application scenarios. A generalized big data quality management model and programming framework are proposed, and a series of data quality detection and repair interfaces are built to express the processing semantics of various data quality issues. Through this data quality management model and detection and repair interfaces, users can quickly build custom data quality detection and repair tasks for different data quality requirements. To improve the operational efficiency of complex data quality management algorithms in large-scale data scenarios, corresponding parallelization algorithms are studied and implemented for detection and repair algorithms with long computation time, including priority-based multiconditional function-dependent detection and repair algorithms, entity detection, and extraction algorithms based on semantic information and chunking techniques, and plain Bayesian-based missing value filling algorithms, and this paper proposes a data validity evaluation algorithm and enhances the validity of the original data in practical applications by adding temporal weights, and finally it passed the experimental validation. Through the comprehensive detection process of data importance, network busyness, duration of transmission process, and failure situation, the efficiency has been increased by 20%, and an adaptive data integrity detection method based on random algorithm and encryption algorithm is designed. After experimental verification, this method can effectively detect the integrity of the data transmission process and improve the application of data value, and the final effect is increased by 30.5%.

中文翻译:

汽车应用场景大数据质量管理,评估与测试的关键问题研究

本文对汽车应用场景中大数据中包含的质量管理,评估和检测的关键问题进行了深入的分析和讨论。提出了一种通用的大数据质量管理模型和编程框架,构建了一系列数据质量检测和修复接口来表达各种数据质量问题的处理语义。通过这种数据质量管理模型以及检测和修复界面,用户可以快速构建针对不同数据质量要求的自定义数据质量检测和修复任务。为了提高复杂数据质量管理算法在大规模数据场景下的运行效率,研究并实现了相应的并行化算法,用于计算时间长的检测和修复算法,包括基于优先级的多条件函数依赖的检测和修复算法,基于语义信息和分块技术的实体检测和提取算法以及基于贝叶斯的缺失值填充算法,本文提出了一种数据有效性评估算法,并提高了有效性在实际应用中通过添加时间权重来确定原始数据,最后通过了实验验证。通过对数据重要性,网络繁忙,传输过程的持续时间和故障情况的综合检测过程,效率提高了20%,并设计了一种基于随机算法和加密算法的自适应数据完整性检测方法。经过实验验证后,
更新日期:2021-05-06
down
wechat
bug