Distributed data mining: an attribute-oriented key-preserving method

Maybin Muyeba; John A. Keane

doi:10.1117/12.381730

6 April 2000 Distributed data mining: an attribute-oriented key-preserving method

Maybin Muyeba, John A. Keane

Proceedings Volume 4057, Data Mining and Knowledge Discovery: Theory, Tools, and Technology II; (2000) https://doi.org/10.1117/12.381730
Event: AeroSense 2000, 2000, Orlando, FL, United States

Abstract

Data mining algorithms are constantly being challenged by the need to process large data volumes efficiently. Attribute- Oriented Induction (AOI) is an inductive set-oriented technique used to mine large data by reducing its search space through attribute generalization and form summary rules. Most data mining techniques only end at producing rules for user analysis. The Key-Preserving method of AOI (AOI-KP) allows users to query data related to the learning task efficiently by using keys (attributes that index relations) to relations in the database and the generated rules. However, the initial problem is loading the whole data set into memory on a single memory machine. As data input size increases, the preserved keys and the data itself use up memory. Further, to solve the file I/O bottleneck for writing preserved keys, concurrency mechanisms were used on a single cluster of a Windows NT machine and improvements in execution time were obtained. One of the major solutions is to employ parallelism i.e. utilizing a distributed memory machine with explicit message passing. A Network of Workstations offers attractive scalability in terms of computational power and memory availability. We analyze performance of our algorithm on NOW and compare speed-up and scalability, which showed significant improvements.

Citation Download Citation

Maybin Muyeba and John A. Keane "Distributed data mining: an attribute-oriented key-preserving method", Proc. SPIE 4057, Data Mining and Knowledge Discovery: Theory, Tools, and Technology II, (6 April 2000); https://doi.org/10.1117/12.381730

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available