Numerous unsupervised methods exist to make sense of real-valued datasets, most notably methods for dimensionality reduction and clustering. Labels (or more gen- erally description attributes as in this chapter) associated with the data points are then often used to interpret these results, e.g., by measuring enrichment of cer- tain labels within a cluster, or by coloring data points in a scatter plot of a 2-D projection of the data with a color depending on the labels of the points, for subse- quent visual inspection. However, whether such analyses provide explanations or insights is a matter of coincidence: there is no a priori reason that clusters should be enriched, and there is no guarantee that equally colored points are grouped in a scatter plot.
Here, we propose an alternative approach, in directly using the description attributes to guide the search for surprising multivariate relations in the data. Re- sulting subgroups are then automatically explained well by the descriptions. Our approach contrasts with traditional supervised methods in focusing on local pat- terns: properties of the target attributes that apply only to subsets of the data de- fined in terms of conditions on their metadata. Arguably, with increasing amounts and resulting inhomogeneity of datasets, the importance of local patterns is bound to increase.
Our approach generalizes the literature on Subgroup Discovery and Excep- tional Model Mining in being applicable for real-valued target attributes of ar- bitrary dimensionality, and in searching for multivariate local patterns across all these dimensions, including unusual covariance structures of subgroups in the data. Moreover, the interestingness of the patterns of this type is formalized in a rigorous manner, quantifying the amount of information the user gains by observing them. We have demonstrated that the resulting algorithms are effective and efficient, in theory and in practice.
In further work, we plan to remove the dependency on third party tools (Matlab and Cortana) and produce a standalone version of the method for public dissemi-
nation. Furthermore, it would be interesting to study similar pattern syntaxes for binary, categorical, and mixed sets of target attributes. Besides, although we have little hope to improve the search for optimal spread patterns, it may be feasible to devise a branch-and-bound approach to mine optimal location patterns efficiently. Indeed this appears to be the most relevant question to be addressed in the future. Finally, we aim to integrate this method with SIDE [16, 26], our online tool for ex- ploration of numerical data, which currently does not use any labels or description attributes.
Acknowledgements. This work has been supported by the ERC under the EU’s Seventh Framework Programme (FP/2007-2013) / ERC Grant Agreement no. 615517, FWO (project no. G091017N, G0F9816N), the EU’s Horizon 2020 research and innovation programme and the FWO under the Marie Skłodowska-Curie Grant Agreement no. 665501, the Academy of Finland (decision 288814), and Tekes (Revolution of Knowledge Work project).
References
[1] S. D. Bay and M.J. Pazzani. Detecting group differences: Mining contrast sets. DMKD, 5(3):213–246, 2001.
[2] M. Boley, M. Mampaey, B. Kang, P. Tokmakov, and S. Wrobel. One click mining: Interactive local pattern discovery through implicit preference and performance learning. In Proc. of KDD-IDEA Workshop, pages 27–35, 2013. [3] Mario Boley, Bryan R. Goldsmith, Luca M. Ghiringhelli, and Jilles Vreeken. Identifying consistent statements about numerical data with dispersion- corrected subgroup discovery. In Proc. of ECML-PKDD, 2017.
[4] N. Boumal, B. Mishra, P.-A. Absil, and R. Sepulchre. Manopt, a matlab toolbox for optimization on manifolds. JMLR, 15(1):1455–1459, 2014. [5] T.M. Cover and J.A. Thomas. Elements of information theory. Wiley, 2nd
edition, 2005.
[6] Tijl De Bie. An information theoretic framework for data mining. In Pro- ceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 564–572, New York, NY, USA, 2011. ACM.
[7] Tijl De Bie. Subjective interestingness in exploratory data mining. In In- ternational Symposium on Intelligent Data Analysis, pages 19–31, Berlin, Heidelberg, 2013. Springer.
[8] G. Dong and J. Li. Efficient mining of emerging patterns: Discovering trends and differences. In Proc. of KDD, pages 43–52, 1999.
[9] W. Duivesteijn, A. Feelders, and A.J. Knobbe. Exceptional model mining - supervised descriptive local pattern mining with complex target concepts. DMKD, 30(1):47–98, 2016.
[10] S. Dˇzeroski, D. Demˇsar, and J. Grbovi´c. Predicting chemical parameters of river water quality from bioindicator data. Appl. Intell., 13(1):7–17, 2000. [11] E. Galbrun and P. Miettinen. From black and white to full color: extending
redescription mining outside the boolean world. SADM, 5(4):284–303, 2012. [12] Floris Geerts, Bart Goethals, and Taneli Mielik¨ainen. Tiling databases. In
Proc. of DS, pages 278–289, 2004.
[13] H. Heikinheimo, M. Fortelius, J. Eronen, and H. Mannila. Biogeography of european land mammals shows environmentally distinct and spatially coher- ent clusters. J. Biogeogr, 34(6):1053–1064, 2007.
[14] A.M. Jorge, P.J. Azevedo, and F. Pereira. Distribution rules with numeric attributes of interest. In Proc. of PKDD, pages 247–258, 2006.
[15] B. Kang, J. Lijffijt, R. Santos-Rodr´ıguez, and T. De Bie. Subjectively inter- esting component analysis: Data projections that contrast with prior expecta- tions. In Proc. of KDD, pages 1615–1624, 2016.
[16] Bo Kang, Kai Puolam¨aki, Jefrey Lijffijt, and Tijl De Bie. A tool for subjective and interactive visual data exploration. In Proc. of ECML-PKDD - Part III, pages 3–7, 2016.
[17] W. Kl¨osgen. Explora: A multipattern and multistrategy discovery assistant. In AKDDM, pages 249–271, 1996.
[18] K.-N. Kontonasios and T. De Bie. Subjectively interesting alternative clus- terings. MLJ, 98(1):31–56, 2015.
[19] K.-N. Kontonasios, J. Vreeken, and T. De Bie. Maximum entropy modelling for assessing results on real-valued data. In Proc. of ICDM, pages 350–359, 2011.
[20] P. Kralj Novak, N. Lavraˇc, and G.I. Webb. Supervised descriptive rule dis- covery: A unifying survey of contrast set, emerging pattern and subgroup mining. JMLR, 10:377–403, 2009.
[21] D. Leman, A. Feelders, and A.J. Knobbe. Exceptional model mining. In Proc. ECML-PKDD, pages 1–16, 2008.
[22] J. Lijffijt, E. Spyropoulou, B. Kang, and T. De Bie. P-n-rminer: A generic framework for mining interesting structured relational patterns. IJDSA, 1(1): 61–76, 2016.
[23] M. Meeng and A. Knobbe. Flexible enrichment with cortana–software demo. In Proc. of BeneLearn, pages 117–119, 2011.
[24] K. Morik, J.-F. Boulicaut, and A. Siebes, editors. Local Pattern Detection, In- ternational Seminar, Dagstuhl Castle, Germany, April 12-16, 2004, Revised Selected Papers, volume 3539 of LNCS, 2005.
[25] B. Padmanabhan and A. Tuzhilin. A belief-driven method for discovering unexpected patterns. In Proc. of KDD, pages 94–100, 1998.
[26] Kai Puolam¨aki, Bo Kang, Jefrey Lijffijt, and Tijl De Bie. Interactive visual data exploration with subjective feedback. In Proc. of ECML-PKDD - Part II, pages 214–229, 2016.
[27] M. A. Redmond and A. Baveja. A data-driven software tool for enabling cooperative information sharing among police departments. EJOR, 141:660– 678, 2002.
[28] A. Silberschatz and A. Tuzhilin. On subjective measures of interestingness in knowledge discovery. In Proc. of KDD, pages 275–281, 1996.
[29] L. Umek and B. Zupan. Subgroup discovery in data sets with multi- dimensional responses. IDA, 15(4):533–549, 2011.
[30] Matthijs van Leeuwen, Tijl De Bie, Eirini Spyropoulou, and C´edric Mesnage. Subjective interestingness of subgraph patterns. Machine Learning, 105(1): 41–75, Oct 2016. ISSN 1573-0565. doi: 10.1007/s10994-015-5539-3. URL https://doi.org/10.1007/s10994-015-5539-3.
[31] J.-T. Zhang. Approximate and asymptotic distributions of chi-squared–type mixtures with applications. JASA, 100(469):273–285, 2005.
7
Conclusion and Future Work
7.1
Conclusion
Representation learning has gained enormous popularity due to its capability to capture rich information from the data and its easy-to-compute nature. Despite the success of representation learning, it currently has two limitations. First, high- dimensional data often has many aspects, a low-dimensional data representation is typically insufficient to capture all structure in the data and the most salient structure is often already known. It is not obvious how to capture the remaining information in a similarly effective way. Second, the structure in a dataset may exceed the representational power of Euclidean space. Hence, the data might be underrepresented.
To address the two issues, this thesis proposes a framework for learning Sub- jectively Interesting Data Representations (SIDR). The framework delineates how to take a prior about the data and find data representations that complement the prior.
First, by discounting the known salient structure, the SIDR framework en- ables complementary structure to be captured, allowing the remaining information to be captured effectively. Along this line, we developed a linear dimensional- ity reduction method called Subjectively Interesting Component Analysis (SICA) in order to explore the remaining information via linear projections. For com- plex non-linear structure remaining in the data, we further proposed Conditional t-distributed Stochastic Neighbor Embedding (ct-SNE), a conditioned version of
t-distributed Stochastic Neighbor Embedding. SICA and ct-SNE are evaluated in extensive case studies on both synthetic and (large) real-world datasets. The re- sults show both methods effectively discount the prior knowledge and allow the remaining structure in the data to be efficiently explored.
Second, by encoding specific complex structure in the data as prior, the impor- tant information can be represented more accurately in Euclidean representations to capture. Combining the prior and the Euclidean representation, representation learning methods can yield better models of the data. This idea is readily applica- ble to network embeddings, where network structural properties such as (approxi- mate) multipartiteness, certain degree distributions, or assortativity are difficult to express using Euclidean space. By applying SIDR, we derived Conditional Net- work Embeddings (CNE) that optimizes network representations with respect to certain prior knowledge about the network. We evaluated the performance of CNE on standard network analysis tasks such as link prediction and node classifications. Comparing to heuristic methods and state-of-art NE methods on a wide range of networks, CNE shows superior performance. This shows CNE is capable of bet- ter representing network data. Additionally, CNE also demonstrates potential for network visualization.
To enable real users to explore data using subjectively interesting linear pro- jections, this thesis also presented an application of SIDR framework on iterative and interactive visual data analysis, named Subjectively Interesting Data Explo- ration (SIDE). Using SIDE, users can interactively select or label patterns in low- dimensional visualizations during their exploration. SIDE accumulates the learned patterns as prior and presents more informative representations to users. Case studies on both synthetic and real-world data show SIDE is useful for discovering subjectively interesting structure from data in an iterative and interactive manner.
Last but not least, this thesis takes a step along the direction for improving the interpretability of subjectively interesting linear projections. Introduced under the SIDR framework, Subjectively Interesting Subgroup Discovery (SISD) searches subjectively interesting representations that are both informative and descriptive. Case studies on synthetic and real-world datasets show the capability of SISD to provide interesting representations with concise descriptions.