当前位置: X-MOL 学术arXiv.cs.DB › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
SYNC: A Copula based Framework for Generating Synthetic Data from Aggregated Sources
arXiv - CS - Databases Pub Date : 2020-09-20 , DOI: arxiv-2009.09471
Zheng Li, Yue Zhao, Jialin Fu

A synthetic dataset is a data object that is generated programmatically, and it may be valuable to creating a single dataset from multiple sources when direct collection is difficult or costly. Although it is a fundamental step for many data science tasks, an efficient and standard framework is absent. In this paper, we study a specific synthetic data generation task called downscaling, a procedure to infer high-resolution, harder-to-collect information (e.g., individual level records) from many low-resolution, easy-to-collect sources, and propose a multi-stage framework called SYNC (Synthetic Data Generation via Gaussian Copula). For given low-resolution datasets, the central idea of SYNC is to fit Gaussian copula models to each of the low-resolution datasets in order to correctly capture dependencies and marginal distributions, and then sample from the fitted models to obtain the desired high-resolution subsets. Predictive models are then used to merge sampled subsets into one, and finally, sampled datasets are scaled according to low-resolution marginal constraints. We make four key contributions in this work: 1) propose a novel framework for generating individual level data from aggregated data sources by combining state-of-the-art machine learning and statistical techniques, 2) perform simulation studies to validate SYNC's performance as a synthetic data generation algorithm, 3) demonstrate its value as a feature engineering tool, as well as an alternative to data collection in situations where gathering is difficult through two real-world datasets, 4) release an easy-to-use framework implementation for reproducibility and scalability at the production level that easily incorporates new data.

中文翻译:

SYNC:一个基于 Copula 的框架,用于从聚合源生成合成数据

合成数据集是以编程方式生成的数据对象,当直接收集困难或成本高昂时,从多个来源创建单个数据集可能很有​​价值。尽管它是许多数据科学任务的基本步骤,但缺乏高效且标准的框架。在本文中,我们研究了一种称为降尺度的特定合成数据生成任务,这是一种从许多低分辨率、易于收集的来源推断高分辨率、难以收集的信息(例如,个人级别的记录)的过程,以及提出了一个称为 SYNC(通过高斯 Copula 的合成数据生成)的多阶段框架。对于给定的低分辨率数据集,SYNC 的中心思想是将高斯 copula 模型拟合到每个低分辨率数据集,以便正确捕获依赖关系和边缘分布,然后从拟合模型中采样以获得所需的高分辨率子集。然后使用预测模型将采样子集合并为一个,最后根据低分辨率边际约束对采样数据集进行缩放。我们在这项工作中做出了四个关键贡献:1) 提出了一个新的框架,通过结合最先进的机器学习和统计技术,从聚合数据源生成单个级别的数据,2) 进行模拟研究以验证 SYNC 作为合成数据生成算法,3) 展示其作为特征工程工具的价值,以及在难以通过两个真实世界数据集收集数据的情况下替代数据收集的价值,
更新日期:2020-09-22
down
wechat
bug