|
扩展功能
|
|
| 本文信息 | |
 |
Supporting info |
|
PDF(548KB) |
|
[HTML全文](KB) |
|
参考文献[PDF] |
|
参考文献 |
| 服务与反馈 | |
|
加入引用管理器 |
|
引用本文 |
|
Email Alert |
| 本文关键词相关文章 | |
 |
数据清洗 |
|
数据质量 |
|
重复记录检测 |
|
异常数据检测 |
| 本文作者相关文章 | |
|
王曰芬 |
|
章成志 |
|
张蓓蓓 |
|
吴婷婷 |
| PubMed | |
|
Article by |
|
Article by |
|
Article by |
|
Article by |
数据清洗研究综述
王曰芬1,2 章成志1,2,3 张蓓蓓1,2 吴婷婷1,2
1(南京理工大学经济管理学院 南京 210094)
2(万方数据研究院-南京理工大学“企业创新服务联合实验室” 南京 210094)
3(中国科学技术信息研究所 北京 100038)
对数据清洗问题进行综述。介绍数据清洗问题产生的背景和国内外研究现状。给出数据清洗的定义和对象，说明数据清洗的基本原理、模型，分析相关算法与工具，给出数据清洗评估方法；并对今后数据清洗的研究和应用进行展望。
A Survey of Data Cleaning
Wang Yuefen1,2 Zhang Chengzhi1,2,3 Zhang Beibei1,2 Wu Tingting1,2
1(Department of Information Management, Nanjing University of Science & Technology, Nanjing 210094,China)
2(Laboratory for Enterprise Innovation Service, Nanjing University of Science & Technology, Nanjing 210094,China)
3(Institute of Scientific & Technical Information of China, Beijing 100038,China)
Data cleaning problem is surveyed in this paper. Firstly, the background of data cleaning problem and research status is explained. Then, the definition and objects of data cleaning are given. The basic principle and some models of data cleaning are presented. Related algorithms and tools are analyzed and evaluation methods of data cleaning are proposed. Finally, the future research topics and application related to data cleaning problems are discussed.
G350
［1］ Rahm E, Do H.H. Data Cleaning: Problems and Current Approaches［J］. IEEE Data Engineering Bulletin, 2000, 23(4): 3-13.
［2］ Galhardas H, Florescu D. An Extensible Framework for Data Cleaning［C］. In: Proceedings of the 16th IEEE International Conference on Data Engineering. San Diego, California, 2000: 312-312.
［3］ 查峰. 数据仓库化中数据清洗问题的研究［D］.南京:东南大学, 2002.
［4］ 刘奕群, 张敏, 马少平.面向信息检索需要的网络数据清理研究［J］.中文信息学报, 2007, 20(3):70-77.
［5］ BI Case Study［EB/OL］. ［2007-01-09］.http://www.parsintl.com/pdf/14705-BIJ-Informatica.pdf.
［6］ 郭志懋, 周傲英. 数据质量和数据清洗研究综述［J］. 软件学报, 2002, 13(11): 2076-2082.
［7］ Harte-Hanks Trillium Software［EB/OL］. ［2007-01-09］.http://www.trilliumsoftware.com.
［8］ Bohn K. Converting Data for Warehouses［J］. DBMS, 1997, 10(7): 61-66.
［9］ Helena G. Generative and Transformational Techniques in Software Engineering. In: Helena G eds.Data Cleaning and Transformation Using the AJAX Framework［M］. Springer Berlin/Heidelberg,2006.
［10］ 周奕辛. 数据清洗算法的研究与应用［D］. 青岛: 青岛大学, 2005.
［11］ 唐懿芳, 钟达夫, 严小卫.基于聚类模式的数据清洗技术［J］.计算机应用, 2004, 24(5): 116-119.
［12］ Monge A, Elkan C. The Field Matching Problem: Algorithms and Applications［C］. In: Proceedings of the 2nd International Conference of Knowledge Discovery and Data Mining. Portland, Oregon, 1996.
［13］ Masek W, Paterson M A. Faster Algorithm Computing String Edit Distance［J］. Journal of Computer System Science, 1980(20):18-31.
［14］ 周芝芬. 基于数据仓库的数据清洗方法研究［D］. 上海:东华大学,2004.
［15］ Salon G, Mcgill M J. Introduction to Modern Information Retrieval［M］. NewYork：McGraw-Hill Book Co., 1983.
［16］ Monge A, Elkan C. An Efficient Domain Independent Algorithm for Detecting Approximately Duplicate Database Records［C］. In: Proceedings of the SIGMOD Workshop on Data Mining and Knowledge Discovery. Tucson, Arizona, 1997.
［17］ Hernandez M, Stolfo S. Real World Data is Dirty: Data Cleansing and the Merge/ Purge Problem［J］. Data Mining and Knowledge Discovery, 1998, 2(1): 9-37.
［18］ 梁文斌.数据仓库中数据清洗的研究与设计［D］. 苏州:苏州大学,2005.
［19］ 王咏梅, 陈家琪, 耿玉良.一种可交互的数据清洗系统［J］.计算机工程与设计,2005,26(4): 955-957.
［20］ Yair Wand, Rihard Y Wang. Anchoring Data Quality Dimensions in Ontological Foundations［J］. Communications of the ACM, 1996, 39(11):86-95.
［21］ Richard Y Wang, Veda C Storey, Christopher P Firth. A Framework for Analysis of Data Quality Research［J］. IEEE Transactions on Knowledge and Data Engineering, 1995, 7(4): 623-640.
［22］ Dominik Lueebber, Udo Grimmer. Systematic Development of Data Mining Based Data Quality Tools［C］. In: Proceedings of the 29th VLDB (VLDB 2003). Berlin, Germany, 2003: 548-559.