There are recent papers that try to define and tackle the problem of cross-domain classification. The definition of the cross-domain classification, however, varies from paper to paper. It is im- portant to clarify these differences in the definition of domains as used by different researchers. In our case, the distinction between two domains is due to the difference in the underlying writing style. Examples of different domains in our research are news articles, e-mails and Wikipedia articles etc. Our definition of domains closely resembles the concept often described by the word genre.
Dai et al. defined the domains as two similar topics but from the same source of text [24]. Although this definition is significantly different from ours, it is still an interesting research paper due to the fact that they also used documents from one set to classify documents from another set. The two domains created by the authors did not differ in terms of their genre of the text, but they differed in terms of the subject of the text. They used 20-newsgroups, SRAA [39], and Reuter’s datasets. As an example, after splitting the 20-newsgroups dataset into two domains, one set contained documents on subject recreation.hockey and another set contained documents on recreation.baseball. While splitting the Reuters dataset [35] in different domains, they selected the documents in orgs class as one domain and documents in people class as the second domain. As can be observed, these different subjects that are split across different domains, share a common, broader underlying topic.
Another similarity of this paper to our research is that they also compared and showed an im- provement in classification accuracy over more conventional machine learning methods such as naive Bayes and SVMs etc. Their algorithm used co-clustering algorithms presented by Dhillon et al. [40]. The co-clustering algorithm, as described in Dhillon’s paper, works on a word count matrix, where each row represents a document and each column represents the words. Each entry in the matrix, thus, represents the counts of the words in that particular doc- ument. Co-clustering algorithms groups these rows and columns together simultaneously into pre-determined number of clusters. Assuming that the row values are from a random variable X and column values are a random variable Y , Co-clustering algorithm seeks to find the following mapping, that is grouping X values into k clusters and Y values into l clusters:
CX : {x1, x2, ..., xm} → {ˆx1, ˆx2, ..., ˆxk} CY : {y1, y2, ..., yn} → {ˆy1, ˆy2, ..., ˆyl}
Once we define ˆX as CX(X), and ˆY as CY(Y ), the goal of co-clustering algorithm becomes
finding the mapping that minimizes the mutual information loss between the (X, Y ) and ( ˆX, ˆY ).
Dai et al. extended Dhillon’s co-clustering algorithm by adding the minimization from the labeled clusters obtained from the documents in the labeled domain [24]. Following is the cost function used by the authors to perform the co-clustering:
I(D0; W ) − I( ˆD0; ˆW ) + λ · (I(C; W ) − I(C; ˆW )) (2.7)
where I is the mutual information function. D0 is the unlabeled domain documents, W is the
word counts, ˆD0and ˆW are the co-clustering of these documents and words, and C is the labeled
clusters obtained from the labeled domain documents. λ is simply a weighting parameter that determines the weight of cost function part that depends on the labeled set of documents i.e.
I(C; W ) − I(C; ˆW ). In the paper, they equated minimization of the above expression with the
minimization of the KL divergence (D(· || ·)) between the distributions as follows: I(D0; W ) − I( ˆD0; W ) + λ · I(C; W ) − I(C; ˆW ) = Df (D0, W )|| ˆf (D0, W ) + λ · D (g(C, W )||ˆg(C, W )) (2.8)
where ˆf and ˆg are the joint probability distributions after the co-clustering. Their method
showed a significant improvement in cross-domain classification accuracy for both SRAA and 20-newsgroups dataset but did not show a significant accuracy improvement for Reuters dataset. Wang et al. also used the same domain definition as authors in [24], however, the Reuters dataset is not used in their paper [19]. Similar to the paper by Dai et al., the 20-newsgroups data is split into two domains by using some newsgroups as the first domain and some similar newsgroups as the second domain. The methods in their paper augmented the feature vectors using the Wikipedia based features [19, 41]. The empirical results showed an increase in accuracy for all pairs of domains. They did not discuss the number of features that were added to achieve this increase in the accuracy.
Swarup et al. addressed the cross-domain classification problem and motivated this problem from a similar point of view as ours [42]. The premise was that since humans are capable to transferring psychological or neurological concepts from one domain to another, a machine learning algorithm should be able to do the same. However, this paper lacked any formal con-
structs to their approach and any robust empirical results. This paper was motivated by a con- ceptual cross-domain learning framework based on human learning. The structural representa-
tionsused for cross-domain learning are made from multiple neural networks and correspond-
ing genomes to weight the neural networks differently in the presence of different domains. Although this paper did not contain any formal results or proofs, it presented a new direc- tion for tackling the cross-domain learning by developing structures that can mimic so-called
learning-to-learnor accomplish knowledge transfer across domains.