Madhu V. Ahluwalia,
University of Maryland, Baltimore County (UMBC), USA Aryya Gangopadhyay,
University of Maryland, Baltimore County (UMBC), USA
Abstract
This chapter gives a synopsis of the techniques that exist in the area of privacy pre- serving data mining. Privacy preserving data mining is important because there is a
need to develop accurate data mining models without using confidential data items
in individual records. In providing a neat categorization of the current algorithms that preserve privacy for major data mining tasks, the authors hope that students, teachers and researchers can gain an understanding of this vast area and apply
the knowledge gained to find new ways of simultaneously preserving privacy and
Introduction
Data mining, also referred to as knowledge discovery in databases (KDD), has been embraced with much enthusiasm by researchers and market analysts due to its
promise to reveal information useful for scientific and technical research, business
intelligence, and decision-support. A multitude of tools and techniques to facilitate knowledge discovery have therefore been developed and used increasingly. In the post 9/11 era, interest in data mining techniques has escalated due to their usefulness in counter-terrorism activities. However, the revelation of private information that may occur with data mining is unconstitutional. Laws have been enforced to forbid the U.S. Department of Defense to conduct data mining unless deemed necessary for security purposes. It also is mandatory that all operations of U.S. government
agencies involving data mining ensure individual privacy. This conflict has given
birth to a novel research direction known as the privacy preserving data mining. Privacy preserving data mining entails two notions: 1) extracting or mining knowledge from large amounts of data and 2) performing data mining in such a way that data privacy is not compromised. This is a daunting task in an information age where we generate data with every move that we make. One challenge is the ease with
which unauthorized parties can deduce confidential information from released data sources. Also known as the inference problem, this difficulty is discussed at length
by Samarati (2001) and Sweeney (2002). Another major challenge is nailing down the concept of privacy. Whose privacy ought to be protected, an organization’s or an individual’s (Clifton, Kantarcioglu, & Vaidya, 2002)? How do we measure privacy (Pinkas, 2002)? What kind of adversarial models do we deal with (Gangopadhyay
& Ahluwalia, 2006; Pinkas, 2002)? In addition to these issues that complicate the
development of models and algorithms to preserve privacy, there are other legal, commercial, governmental, philosophical, ethical, and personal perspectives that
need to be incorporated into the definition of privacy. However, this makes it even more difficult to address privacy concerns and provide a universally satisfactory resolution to the problem. Finally, all privacy-enhancing technologies influence the outcome of data mining to some extent. Depending on the modifications made to
the data or the accuracy of information obtained from subjects who are unwilling to divulge their personal data due to privacy concerns, knowledge discovery tools might taint the results so that they exhibit lower accuracy or, sometimes even worse, false knowledge. Therefore, it is necessary to strike a balance between the need to privatize data and to retain the meaningfulness of the data mining results.
As mentioned earlier, there is no dearth of tools and techniques to achieve the twin
goals of sufficient privacy of input data and sufficient utility of mining results in
vides an overview of the many different models and algorithms used in privacy preserving data mining (PPDM). Summarizing and classifying the various PPDM techniques presents a clear picture of the work performed in this area. It also al-
lows future work to be classified in the right category and bring order to this new
research area(see Figure 1 for taxonomy of current PPDM techniques). The last
review presented in this context proposed a classification hierarchy to analyze the
research accomplished in PPDM (Verykios et al., 2004a). Since then, no work has been published that provides taxonomy of the existing PPDM research. We extend
the work presented in Verykios et al. (2004a). This work classifies the approaches
taken to conduct PPDM, based on five dimensions: data distribution, data modifi-
cation, data mining algorithm, data or rule hiding and privacy preservation. Data distribution refers to situations where all data is either located at a single site or a logically related database is stored over two or more physically independent sites.
Data modification simply means altering the original values of a database. By data
algorithm, Verykios et al. (2004a) refer to the data mining task, for example, as-
sociation rule, classification or clustering, for which data is modified. Data or rule
hiding presents a choice to either hide raw data or aggregated data. Finally, privacy preservation refers to the technique used to preserve data privacy. Data privacy is
preserved when data are selectively modified, that is, only sensitive or confidential data values are altered. Data should be selectively modified in order to preserve the utility of the modified data. If all data are modified, all meaningful information
is lost. The techniques used to preserve the privacy of data are heuristic-based, cryptography-based and reconstruction-based. Heuristic-based techniques are used because modifying data selectively or hiding aggregated data or rules is a complex or an NP-Hard problem. Cryptography-based techniques have only been applied where data is distributed at multiple sites. Finally, reconstruction-based techniques reconstruct the original distribution of the data from randomly perturbed data. One of the drawbacks of classifying existing work on PPDM in the way proposed by Verykios et al. (2004a) is that no PPDM technique falls clearly under any one par- ticular dimension. Generally, PPDM algorithms can be grouped under more than
one dimension; for instance heuristic-based techniques have been applied both for
data and rule hiding (Gangopadhyay & Ahluwalia, 2006; Saygin, Verykios, & Clif-
ton, 2001) and there are PPDM algorithms that work both for centralized as well as distributed data scenarios (Mukherjee, Chen, & Gangopadhyay, 2006). Moreover, a reconstruction-based technique may well be a heuristic technique (Agrawal &
Srikant, 2000). We provide a true classification hierarchy that clearly categorizes
most existing techniques.
Our contribution is to organize the current PPDM related work into well-defined groups to facilitate conceptualizing this broad field. This chapter provides an outline
The solutions or algorithms that have been developed to mine data securely depend on the data-mining task at hand. We outline PPDM solutions for major data mining
tasks, such as classification, association and clustering to give the reader an idea
of the type of solutions that have been developed to solve each of these problems. We do not attempt to present all currently existing PPDM algorithms. Instead, each algorithm presented introduces a new perspective on simultaneously preserving pri- vacy and conducting useful mining. We believe that our work will provide a broad
and a solid foundation for new researchers in the field of PPDM and give them a
strong foothold to start their careers as PPDM researchers. We start with a look at the meaning of privacy from different perspectives, and then discuss the theoretical background and taxonomy of the current privacy preserving data mining techniques.
Different classes of privacy preserving data mining solutions are outlined briefly.
Contrasts, comparisons and examples are given where necessary. Finally, we present the course of future research in the area of privacy preserving data mining.
An.Overview.of.Privacy
Typically, privacy is viewed in the context of individual data; Alice must not know
that Bob’s annual salary is x amount ofdollars or that he owns a property in down- town Washington D.C. If Alice cannot trace the salary/property attribute values to Bob, Bob’s privacy is protected. Here, privacy is seen as the ability to identify an individual. However, there is another view. This view applies to data mining in general. Here, the aim is to learn from a body of data. The adversary gets to know the overall patterns and trends from the data collection. Known as corporate privacy in contrast to individual privacy, an adversary must be prevented from breaching this privacy. Individual and corporate privacy issues are discussed in (Clifton et al., 2002).
In matters of privacy, the intent, behavior and the knowledge base of an adversary are crucial. Each one of these aspects is discussed below to present an overall idea of how it relates to privacy. Adversarial intent or motive has been considered in the context of secure multi-party computation (SMC) (Pinkas, 2002) and (Lindell & Pinkas, 2002), where data sources are distributed across several sites. Adversarial behavior is also discussed under SMC (Clifton et al., 2002). Adversarial knowledge base has been discussed in the context of association rule mining (Gangopadhyay & Ahluwalia, 2006).
Multi-party computation settings distinguish between a semi-honest and a malicious adversary. Honesty and deception show the intent of parties involved. Semi-honest
adversaries are simply curious. They obey protocol specifications, however are keen
on analyzing the results of mining in order to obtain additional information from the
analysis. Malicious adversaries purposely deviate from the specification in order to
gain additional information. In the data mining setting, malicious adversaries can
define their input to be an empty database. The output obtained is, then, the result of
running an algorithm against the other party’s database alone. This is a gross viola-
tion of the other party’s confidential data. But unfortunately, the research community
that deals with SMC offers no solution to prevent such an attack. It just assumes that real-life adversaries are semi-honest and not malicious. Further, it also assumes
that in real-life, the parties involved in SMC do not collude. Non-collusion reflects
a party’s behavior in that it refrains from combining its data with any other party to gain meaningful information from the combined data. Consequently, there are no solutions to preserve privacy in a multi-party computation environment where adversaries collude and/or are malicious.
The knowledge base of adversaries is targeted in the context of association rule mining (Gangopadhyay & Ahluwalia, 2006). We present adversaries with different levels of prior knowledge and discuss the likelihood with which these adversaries can determine the raw data from the privatized data. We are of the opinion that prior knowledge about the instances (rows) of a database is more damaging than prior knowledge about the dimensions (columns) of a database. We also offer a solution to handle adversaries with and without prior knowledge.
We have given our readers pointers to literature where technical issues related to privacy are a topic. We have mentioned that privacy has many meanings. Especially online privacy issues, which are important in the data-mining realm, are intrinsically complex because they represent an intersection of legal, commercial, governmental, ethical, personal, and global perspectives. Since privacy considerations involve technical and non-technical issues, we agree with Clifton et al. (2002) that solutions to achieving privacy must also be technical and non-technical. From a technical
perspective privacy solutions should be feasible in terms of efficiency, security,
and usability. The SMC discussion above suggests that technical solutions can be formulated without limitations in usability by making appropriate assumptions. Of consequence is whether the non-technical means are used prudently to realize these assumptions in the real world (Clifton et al., 2002).
Privacy.Preserving.Data.Mining.(PPDM):.Background.and.
Techniques
Privacy preserving data mining (PPDM) has been proposed as a solution to the problem of violating privacy while sharing data for knowledge extraction. It is a
relatively new research direction with the objective of developing algorithms to modify the original data or mining techniques in such a way that the process of mining does not reveal private data and private knowledge, yet remains useful in
that the original data distributions and correlations stay sufficiently intact.
For a substantial amount of research in the field of PPDM, the focus has been on the
pool of data. Regardless of whether the data pool is mined or not, a data holder’s
first concern is protecting this data from public scrutiny. The data holder’s objective
may be anything from simply publishing the data to sharing it with fellow research- ers or passing it to a data miner for analysis or computing a secure function of his input jointly with others. As a custodian of the data, the owner of the data must do something to the collection, if it must go public. In general, one assumes that tech-
niques need to be developed because data changes hands. In specific, it is assumed
that the development of techniques only makes sense for data that is distributed, because privacy loses its relevance when data is collected and mined at the same site (Vaidya, Clifton, & Zhu, 2006). While this is true, we consider the domain of privacy preserving data mining to also encompass centralized data scenarios, as data controlled by a single owner might need to change hands for the purpose of mining. Centralization and distribution of data impact privacy preserving data mining solu-
tions. This is because some data mining tasks such as classification and association
Figure 1. Classification of the existing privacy preserving data mining techniques
PPDM Techniques Data Micro- aggregation Data Transform- ation Univariate, Multivariate Discrete Fourier Transform, Discrete Wavelet Transform Sampling, Analytic Approach Data Hiding Data Anonymization Data
Perturbation Distributed Data Mining (DDM)
Hiding Identities of Entities Hiding Values Secure Multi- Party Computation (SMC) K- anonymity Data Swapp- ing Encryption, Secure Sum Projection- Based Perturbation Element- Wise Perturbation Probability Distribution Generalization, Suppression Principal Component Analysis (PCA), Random Projection Additive, Multiplicative, Randomized - Response Rule Hiding Hiding Association Rules Hiding Classification Rules Data Perturbation, Data Blocking, Data Sampling Parsimonious Downgrading
rules will give incorrect results, if the same data pool with different data distribution
strategies were to be mined for good classifiers and robust association rules. Strictly defined, distributed data mining (DDM) is data mining where the data and mining
are spread over many independent sites. Consider some vertically partitioned data, where attributes are dispersed over many sites. If we were to conduct association rule mining on such a data set, the correlations between attributes would not be captured as they would be captured from centralized data, and hence the association rules obtained would be distorted. So, different PPDM solutions have been developed based on whether the data is centralized or distributed. We will begin our journey
across a multitude of techniques and methodologies with a classification of methods
that have been adopted to secure data (see Figure 1).
The main work on privacy preserving data mining can be divided under two major categories: data hiding and rule hiding. While sensitive data is obfuscated in both categories, in the former original data patterns are disclosed whereas in the latter some original data patterns are hidden. The underlying data patterns are preserved in both instances. Data patterns are characteristics of data. These characteristics manifest themselves in different ways depending on the kind mining performed on
the data; for instance in association rule mining, the underlying data patterns are the relationships among data items, in classification and cluster analysis, the properties
of data that categorize data under one group or another are the data patterns. And in outlier analysis, rare events or noise in the data are considered data patterns.
Data.Hiding
The main objective of data hiding is to design new protocols to perturb, anonymize or encrypt raw data so that sensitive data remains sensitive during and after the min- ing operation while the underlying data patterns can still be discovered. Broadly, data hiding is discussed in the context of data perturbation, distributed data mining
(DDM), its sub-field secure multi-party computation (SMC), and data anonymization.
We refer to Vaidya et al. (2006) for a detailed discussion on research in PPDM.