The use of clustering techniques as described in this dissertation produces an hierarchy of clusters each containing a collection of web documents which are similar. These clusters could also be represented graphically through the use of self organizing maps (SOMs). A graphical view of results could aid users in selecting results which are most relevant to their queries.
In conclusion, the importance of web mining and semantic markup techniques like the ones presented in this dissertation will be of paramount importance as the Internet continues to evolve
and new, complex data formats appear on it. Further research will be critical to ensure that the Internet remains a useable, sustainable resource in the future.
Bibliography
[1] D. Stenmark. The relationship between information and knowledge. In Proceedings of IRIS 24, Ulvik, Norway, August 11-14 2001.
[2] J.F. Nunamaker Jr., N.C. Romano Jr., and R.O. Briggs. A Framework for Collaboration and Knowledge Management. In Proceedings of the 34th Hawaii International Confer-ence on System SciConfer-ences. IEEE, 2001.
[3] R.S. Pressman. Software Engineering : a Practitioner’s Approach. McGraw-Hill, fourth edition, 1997.
[4] R.J. McQueen. Four Views of Knowledge and Knowledge Management. In E.D. Hoadley and I. Benbasat, editors, Proceedings of the Fourth Americas Conference on Information Systems, pages 609–611, Atlanta, GA, 1998. Association for Information Systems.
[5] G. Bellinger, D. Castro, and A. Mills. Data, Information, Knowledge, and Wisdom. Avail-able online at: http://www.outsights.com/systems/dikw/dikw.htm, 1997.
[6] H.G. Cragon. Computer Architecture and Implimentation. Cambridge University Press, first edition, 2000.
[7] I. Englander. The Architecture of Computer Hardware and Systems Software : An infor-mation technology approach. John Wiley and Sons, second edition, 2000.
[8] The Unicode Consortium. The Unicode Standard, Version 3.0. Addison-Wesley, first edition, 2000.
[9] W.A. Shay. Understanding Data Communications and Networks. Brooks/Cole Publishing Company, second edition, 1998.
[10] R. Ramakrishnan and J. Gehrke. Database Management Systems. McGraw-Hill, second edition, 2000.
[11] P. Rob and C. Coronel. Database Systems : Design, Implimentation, and Management.
Course Technology, third edition, 1997.
[12] G. Lakemeyer and B. Nebel. Foundations of knowledge representation and reasoning.
In Representation and Reasoning, volume 810 of Lecture Notes on Artificial Intelligence, pages 1–12. Springer-Verlag, 1994.
[13] S.C. Shapiro. Propositional, first-order and higher-order logics: Basic definitions, rules of inference, and examples. In Lucja M. Iwanska and Stuart C. Shapiro, editors, Natu-ral Language Processing and Knowledge Representation: Language for Knowledge and Knowledge for Language, pages 379–395. AAAI Press/The MIT Press, Menlo Park, CA, 2000.
[14] D. Duce and G. Ringland. Background and introduction. In G.A. Ringland and D.A Duce, editors, Approaches to Knowledge Representation - An Introduction. Research Studies Press, 58B Station Road, Letchworth, Herts. SG6 3BE, England, 1988.
[15] N.J. Nilsson. Artificial Intelligence: A New Synthesis. Morgan Kaufmann Publishers, Inc., 1998.
[16] T. Dean, J. Allen, and Y. Aloimonos. Artificial Intelligence Theory and Practice. The Benjamin/Cummings Publishing Company, 1995.
[17] S.C. Shapiro. Knowledge representation. In Lynn Nadel, editor, Encyclopedia of Cog-nitive Science, volume 2, pages 671–680. Macmillan Publishers Ltd., 2003. (PS preprint version).
[18] D. Nardi and R. J. Brachman. An introduction to description logics. In F. Baader, D. Cal-vanese, D.L. McGuinness, D. Nardi, and P.F. Patel-Schneider, editors, The Description Logic Handbook, pages 5–44. Cambridge University Press, 2002.
[19] S.C. Shapiro. Cables, paths, and ‘subconscious’ reasoning in propositional semantic net-works. In J. Sowa, editor, Principles of Semantic Networks, pages 137–156. Morgan Kaufmann Publishers, 1991.
[20] M. Gruninger and J. Lee. Ontology: Applications and design. Communications of the ACM, 45(2):39–41, February 2002.
[21] T.R. Gruber. Towards Principles for the Design of Ontologies Used for Knowledge Shar-ing. In N. Guarino and R. Poli, editors, Formal Ontology in Conceptual Analysis and Knowledge Representation, Deventer, The Netherlands, 1993. Kluwer Academic Publish-ers.
[22] R.R. Stevens, C.A. Goble, and S. Bechhofer. Ontology-based knowledge representation for bioinformatics. To appear in the journal Briefings in bioinformatics, 2000.
[23] M. Uschold and M. Gr¨uninger. Ontologies: principles, methods, and applications. Knowl-edge Engineering Review, 11(2):93–155, 1996.
[24] M. Fernandez, A. Gomez-Perez, and N. Juristino. Methontology: From ontological art toward ontological engineering. In Spring Symposium Series. Stanford University, 1997.
[25] C.W. Holsapple and K.D. Joshi. A collaborative approach to ontology design. Communi-cations of the ACM, 45(2):42–47, February 2002.
[26] S. Bechhofer, J. Broekstra, S. Decker, M. Erdmann, D. Fensel, C. Goble, F. van Harme-len, I. Horrocks, M. Klein, D. McGuinsess, E. Motta, P. Patel-Schneider, S. Staab, and R.Studer. An Informal Description of Standard OIL and Instance OIL. Technical report, The World Wide Web Consortium (W3C), 2000.
[27] S. Pepper. The TAO of Topic Maps: Finding the Way in the Age of Infoglut. Technical report, Ontopia AS, Waldemar Thranes gt. 98, 0175 Oslo, Norway, 2002.
[28] Members of the TopicMaps.Org Authoring Group. XML Topic Maps (XTM) 1.0. Avail-able online at: http://www.topicmaps.org/xtm/1.0/xtm1-20010806.html, 2001.
[29] M. Jackman and C. Pavelin. Conceptual graphs. In G.A. Ringland and D.A Duce, editors, Approaches to Knowledge Representation - An Introduction, pages 161–174. Research Studies Press, 58B Station Road, Letchworth, Herts. SG6 3BE, England, 1988.
[30] D.E. Comer. Computer Networks and Internets with Internet Applications. Prentice Hall, New Jersey, 2001.
[31] The World Wide Web Consortium (W3C). HTML 4.01 Specification. W3C, Desember 1999.
[32] The World Wide Web Consortium (W3C). XHTML 1.0 The Extensible HyperText Markup Language. W3C, second edition, Desember 2002.
[33] The Macromedia Corporation. Macromedia Flash White Paper: Flash Player for Devel-opers and Publishers. Available online at:
http://www.macromedia.com/flash/whitepapers/whitepaper jul03.pdf, 2000.
[34] B.D. Brunk. Overview and Preview Tools for Navigating the World-Wide Web. Technical Report TR-1999-03, SILS, 31 July 1999.
[35] J.E. Conklin. Hypertext: an Introduction and Survey. IEEE Computer, 20:17–41, 1987.
[36] S. Lawrence and C.L. Giles. Accessibility of information on the web. Nature, 400:107 – 109, 8 July 1999.
[37] J. Kleinberg and S. Lawrence. The Structure of the Web. Science, 294:1849 – 1850, November 2001.
[38] H.A. Kautz, B. Selman, and M.A. Shah. The hidden web. AI Magazine, 18(2):27–36, 1997.
[39] A. Broder. A taxonomy of web search. SIGIR Forum, 36(2):3 – 10, Fall 2002.
[40] F. Wilfrid Lancaster. Information Retrieval. John Wiley and Sons, New York, second edition, 1979.
[41] C.J. van Rijnsbergen. Information Retrieval. Butterworths, London, second edition, 1979.
[42] A. Spink, S. Ozmutlu, H.C. Ozmutlu, and B.J. Jansen. U.S. versus European Web Search-ing Trends. SIGIR Forum, 36(2):32 – 37, Fall 2002.
[43] J. Ellman and J. Tait. Internet Challenges for Information Retrieval. In BCS Information Retrieval Specialist Group, 1996.
[44] M.R. Henzinger, R. Motwani, and C. Silverstein. Challenges in Web Search Engines.
SIGIR Forum, 36(2):11 – 22, Fall 2002.
[45] S. Lawrence. Context in Web Search. IEEE Data Engineering Bulletin, 23(3):25–32, 2000.
[46] S.E. Middleton. Interface Agents: A review of the field. Technical Report ECSTR-IAM01-001, University of Southampton, August 2000.
[47] O. Etzioni and D.S. Weld. Intelligent Agents on the Internet: Fact, Fiction, and Forecast.
IEEE Expert, 10(3):44–49, 1995.
[48] C.C. Hayes. Agents in a Nutshell-A Very Brief Introduction. IEEE Transactions on Knowledge and Data Engineering, 11(1):127 – 132, January/Febuary 1999.
[49] J.P. Bigus. Data Mining with Neural Networks - Solving Business Problems - from Appli-cation Development to Decision Support. McGraw-Hill, 1996.
[50] A. Seydim. Intelligent agents: A data mining perspective. Available online at:
http://citeseer.nj.nec.com/article/seydim.html, May 2000.
[51] P. Maes. Agents that reduce work and information overload. Communications of the ACM, 37(7), July 1994.
[52] R. Cooley, J. Srivastava, and B. Mobasher. Web Mining: Information and Pattern Discov-ery on the World Wide Web. In Proceedings of the 9th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’97), November 1997.
[53] G.S. Linoff and M.J.A. Berry. Mining the Web. John Wiley and Sons, first edition, 2001.
[54] O.R. Za¨ıane. Building Virtual Web Views. Data and Knowledge Engineering, 39:143–
163, 2001.
[55] J.M. Kleinberg. Authorative sources in a hyperlinked environment. In Proceedings of ACM-SIAM Symposium on descrete Algorithms, 1998.
[56] S. Brin and L. Page. The anatomy of a Large-Scale Hypertextual Web Search Engine. In 7th International Conference on the WWW, Brisbane, Australia, 1998.
[57] R. Kosala and H. Blockeel. Web mining research: A survey. SIGKDD Explorations, 2(1):1–15, 2000.
[58] O. Etzioni. The World-Wide Web: Quagmire or gold mine? Communications of the ACM, 39(11):65–68, 1996.
[59] J. Cowie and W. Lehnert. Information Extraction. Communications of the ACM, 39(1):80–
91, 1996.
[60] I. Muslea. Extraction patterns for information extraction tasks: A survey. In AAAI-99 Workshop on Machine Learning for Information Extraction, 1999.
[61] D. Lewis and W. Gale. Training Text Classifiers by Uncertainty Sampling. In Proceed-ings of the Seventeenth Annual International ACMSIGIR Conference on Research and Development in Information Retrieval, 1994.
[62] S. Chakrabarti. Data Mining for Hypertext: A Tutorial Survey. SIGKDD Explorations:
Newsletter of the Special Interest Group (SIG) on Knowledge Discovery & Data Mining, 1(2):1–11, 2000.
[63] O. Etzioni and D.A. Weld. A softbot-based interface to the internet. Communications of the ACM, 37(7):72–76, July 1994.
[64] H. Lieberman and D. Maulsby. Instructible agents: Software that just keeps getting better.
IBM Systems Journal, 35(3&4), 1996.
[65] T. Joachims, D. Freitag, and T. Mitchell. Webwatcher: A learning apprentice for the world wide web. In Proceedings of IJCAI97, August 1997.
[66] M. Pazzani, J. Muramatsu, and D. Billsus. Syskill & webert: Identifying interesting web sites. In Proceedings of the National Conference on Artificial Intelligence, 1996.
[67] K.A. Oostendorp, W.F. Punch, and R.W. Wiggins. A tool for individualizing the web. In Proceedings of the 2nd International World Wide Web Conference, 1994.
[68] E. Han, D. Boley, M. Gini, R. Gross, K. Hastings, G. Karypis, V. Kumar, B. Mobasher, and J. Moore. Webace: a web agent for document categorization and exploration. In Proceedings of the Second International Conference on Autonomous Agents, pages 408–
415. ACM Press, 1998.
[69] A. Tan. Text mining: The state of the art and the challenges. In Proceedings of the Pacific Asia Conference on Knowledge Discovery and Data Mining: PAKDD’99 workshop on Knowledge Discovery from Advanced Databases, pages 65–70, 1999.
[70] Y. Lashkari, M. Metral, and P. Maes. Collaborative interface agents. In Proceedings of the Twelfth National Conference on Artificial Intelligence, volume 1. AAAI Press, Seattle, WA, 1994.
[71] O. de Vel and S. Nesbitt. A collaborative filtering agent system for dynamic virtual com-munities on the web. In Working notes of Learning from Text and the Web, Conference on Automated Learning and Discovery CONALD-98, 1998.
[72] D. Goldberg, D. Nichols, B. Oki, and D. Terry. Using collaborative filtering to weave an information tapestry. Communications of the ACM, 35(12):61–69, 1992.
[73] F. Zambonelli, N.R. Jennings, A. Omicini, and M. Wooldridge. Agent-orientated soft-ware engineering for internet applications. In Coordination of Internet Agents: Models, Technologies and Applications, chapter 13. Springer-Verlag, 2000.
[74] Y. Labrou, T. Finin, and Y. Peng. Agent communication languages: The current landscape.
IEEE Intelligent Systems, 14(2):45–52, March/April 1999.
[75] T. Finin, R. Fritzson, D. McKay, and R. McEntire. KQML as an Agent Communication Language. In Proceedings of the Third International Conference on Information and Knowledge Management (CIKM’94). ACM Press, November 1994.
[76] E. Casasola and S. Gauch. Intelligent information agents for the World Wide Web. Tech-nical Report ITTC-FY97-11100-1, Information and Telecommunication Technology Cen-ter, The University of Kansas, 1997.
[77] D. Box, D. Ehnebuske, G. Kakivaya, A. Layman, N. Mendelsohn, H.F. Nielsen, S. Thatte, and D. Winer. Simple object access protocol (soap) 1.1. Note, W3C, May 2000.
[78] Y. Labrou. Standardizing agent communication. Lecture Notes in Computer Science, 2086:74–97, 2001.
[79] I. Sommerville. Software Engineering. Addison-Wesley, sixth edition, 2001.
[80] The Object Management Group. The Common Object Request Broker: Architecture and Specification.
[81] S. Lawrence and C.L. Giles. Inquirus, the NECI meta search engine. In Proceedings of the Seventh International World Wide Web Conference, pages 95–105, Brisbane, Australia, 1998. Elsevier Science.
[82] E. Selberg and O. Etzioni. The MetaCrawler architecture for resource aggregation on the Web. IEEE Expert, pages 11–14, January-February 1997.
[83] E. Glover, S. Lawrence, M.D. Gordon, W. Birmingham, and C.L. Giles. Recommend-ing web documents based on user preferences. In SIGIR 99 Workshop on Recommender Systems, Berkeley, CA, August 1999.
[84] F. Tanudjaja and L. Mui. Persona: A contextualized and personalized web search. In Proceedings of the 35th Hawaii Conference on System Sciences. IEEE, January 7-10 2002.
[85] M. Chau, D. Zeng, H. Chen, M. Huang, and D. Hendrianwan. Design and evaluation of a multi-agent collaborative web mining system. Decision Support Systems, 988, 2002.
[86] H. Chen, H. Fan, M. Chau, and D. Zeng. MetaSpider: Meta-Searching and Categorization on the Web. Journal of the American Society for Information Science and Technology, 52(12):1134–1147, November 2001.
[87] D. Dreilinger and A.E. Howe. Experiences with Selecting Search Engines Using Metasearch. ACM Transactions on Information Systems, 15(3):195–222, 1997.
[88] W. Meng, C.T. Yu, and K. Liu. Building efficient and effective metasearch engines. ACM Computing Surveys, 34(1):48–89, 2002.
[89] S. Gauch, G. Wang, and M. Gomez. ProFusion: Intelligent Fusion from Multiple, Dis-tributed Search Engines. Journal of Universal Computer Science, 2(9):637–649, 1996.
[90] E.M. Ehlers and O.L. Oosthuizen. Difficulties of Knowledge Extraction for Web Content Mining Agents. In C. Heavey, editor, Proceedings of the 32nd International Conference on Computers and Industrial Engineering, volume 1, pages 435–440, August 2003.
[91] D. Mladenic. Text-learning and related Intelligent Agents: a Survey. IEEE Intelligent Systems, 14(4):44–54, July–August 1999.
[92] S.K. Pal, V. Talwar, and P. Mitra. Web Mining in Soft Computing Framework: Rele-vace, State of the Art and Future Directions. IEEE Transactions on Neural Networks, 13(5):1163 – 1177, September 2002 2002.
[93] S. Chakrabarti. Mining the Web: Discovering Knowledge from Hypertext Data. Morgan Kaufmann Publishers, 2003.
[94] N. Eiron and K.S. McCurley. Analysis of Anchor Text for Web Search. In Proceedings of the 26th Annual International ACM SIGIR conference on Research and Development in Informaion Retrieval, pages 459 – 460, Toronto, Canada, 2003.
[95] D. Boley. Principal Direction Divisive Partitioning. Data Mining and Knowledge Discov-ery, 2(4):325–344, 1998.
[96] S. Mukherjea. WTMS: A system for collecting and analysing topic-specific web informa-tion. In Proceedings of the 9th International World Wide Web Conference, Amsterdam, Netherlands, May 15-19 2000.
[97] M. Chau, D. Zeng, and H. Chen. Personalized Spiders for Web Search and Analysis. In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries, pages 79–87, 2001.
[98] Q. Liu, R. Safavi-Naini, and N.P. Sheppard. Digital Rights Management for Content Distribution. Australasian Information Security Workshop (AISW), 2003.
[99] S. St. Laurent. XML: A Primer. M&T Books, New York, 2001.
[100] M. Koster. A Standard for Robot Exclusion. Available online at:
http://www.robotstxt.org/wc/norobots.html, 1994.
[101] M. Kratz, M. Ackerman, T. Hanss, and S. Corbato. NGI and Internet2: Accelerating the Creation of Tomorrow’s Internet. In V. Patel et al., editor, MEDINFO 2001, Amsterdam, 2001. IMIA, IOS Press.