Duplicate record is a common problem within data sets especially in huge volume databases. The accuracy of duplicate detection determines the efficiency of duplicate removal process. However, duplicate detection has become more challenging due to the presence of missing values within the records where during the clustering and matching process, missing values can cause records deemed similar to be inserted into the wrong group, hence, leading to undetected duplicates. In this paper, duplicate detection improvement was proposed despite the presence of missing values within a data set through Duplicate Detection within the Incomplete Data set (DDID) method. The missing values were hypothetically added to the key attributes of three data sets under study, using an arbitrary pattern to simulate both complete and incomplete data sets. The results were analyzed, then, the performance of duplicate detection was evaluated by using the Hot Deck method to compensate for the missing values in the key attributes. It was hypothesized that by using Hot Deck, duplicate detection performance would be improved. Furthermore, the DDID performance was compared to an early duplicate detection method namely DuDe, in terms of its accuracy and speed. The findings yielded that even though the data sets were incomplete, DDID was able to offer a better accuracy and faster duplicate detection as compared to DuDe. The results of this study offer insights into constraints of duplicate detection within incomplete data sets.

重复记录是数据集中的常见问题，尤其是在海量数据库中。重复检测的准确性决定了重复删除过程的效率。然而，由于记录中存在缺失值，重复检测变得更具挑战性，在聚类和匹配过程中，缺失值可能导致被认为相似的记录被插入到错误的组中，从而导致未检测到的重复。在本文中，通过不完整数据集内的重复检测（DDID）方法，尽管数据集中存在缺失值，但仍提出了重复检测改进。假设将缺失值添加到正在研究的三个数据集的关键属性中，使用任意模式来模拟完整和不完整的数据集。对结果进行分析，然后利用Hot Deck方法对关键属性中的缺失值进行补偿，评估重复检测的性能。假设通过使用 Hot Deck，重复检测性能将得到提高。此外，在准确性和速度方面，DDID 的性能与早期的重复检测方法 DuDe 进行了比较。结果表明，即使数据集不完整，与 DuDe 相比，DDID 也能够提供更好的准确性和更快的重复检测。这项研究的结果提供了对不完整数据集中重复检测限制的见解。假设通过使用 Hot Deck，可以提高重复检测性能。此外，在准确性和速度方面，DDID 的性能与早期的重复检测方法 DuDe 进行了比较。结果表明，即使数据集不完整，与 DuDe 相比，DDID 也能够提供更好的准确性和更快的重复检测。这项研究的结果提供了对不完整数据集中重复检测限制的见解。假设通过使用 Hot Deck，重复检测性能将得到提高。此外，在准确性和速度方面，DDID 的性能与早期的重复检测方法 DuDe 进行了比较。结果表明，即使数据集不完整，与 DuDe 相比，DDID 也能够提供更好的准确性和更快的重复检测。这项研究的结果提供了对不完整数据集中重复检测限制的见解。与 DuDe 相比，DDID 能够提供更好的准确性和更快的重复检测。这项研究的结果提供了对不完整数据集中重复检测限制的见解。与 DuDe 相比，DDID 能够提供更好的准确性和更快的重复检测。这项研究的结果提供了对不完整数据集中重复检测限制的见解。