To begin the exploration of how to teach the Invisible Web, we drafted a sur- vey to gain some insights into what educators and librarians know about the Invisible Web and how they use it. During the summer of 2011, we admin- istered the survey to check the pulse, if any, of that concept. We wanted to learn if teachers, academics, and librarians in school, academic, and public
libraries talked about the Invisible Web. Was the Invisible Web being taught and, if yes, under what circumstances? The survey was intended not only to produce data but also to create an opportunity for dialogue with colleagues all over the world in the education and library professions. The survey con- sisted of twelve questions (see appendix) and was administered anonymously via SurveyMonkey. It was posted to electronic discussion lists for librarians and educators and reached people in North America, Europe, Australia, New Zealand, South Africa, and Hong Kong. Over 1,000 responses were received.
The survey consisted of two parts. The first five questions were intended to gather information about familiarity with the Invisible Web. In the second part, questions 6 to 11 targeted issues about teaching the Invisible Web. The final question thanked the participants and solicited additional comments.
Part one:
What do edUcatorS and librarianS knoW aboUt the inviSible Web?
Questions 1 and 2: Do you know about the Invisible Web? If not, do you want to know more?
Of the 1,019 who responded to this question, 64% (652) answered that they knew about the Invisible Web and the rest, 36% (367), answered that they did not (fig. 3.1). Were they interested in knowing more about the Invisible Web? Of the 584 people who answered this question, 91% (531) said that they would like to know more, and 9% (53) said they would not. Those answering no were thanked for their participation and taken to the end of the survey.
Question 3: How would you define the Invisible Web?
With question 3, we wanted to learn what people generally understood the Invisible Web to be. Most of the 568 respondents to this question offered
Source: Invisible Web/Deep Web/Hidden Web survey, question 1 (see appendix).
Figure 3.1 Do you know about the invisible Web? (n = 1,019)
Yes No
64% 36%
the essential elements of the definition as shown in figure 3.2. As respon- dents could include multiple concepts in their definitions, the percentages listed in figure 3.2 will add up to more than 100%.
Here is a summary of the responses to question 3:
• The majority of people defined the Invisible Web as material not found by using search engines. Google provided the standard for search engines, as it was listed more times than any other when a specific tool was mentioned at all.
• The next most common response was that the Invisible Web rep- resented database content (19.5%).
• A small percentage of responses, 7% (39), emphasized that Invis- ible Web materials are not easily found and require advanced search skills. Such a response implies that more teaching about the Invisible Web is needed.
• The responses show a distinction between those who said that Invisible Web materials are not found by search engines and those Source: Invisible Web/Deep Web/Hidden Web survey, question 3 (see appendix).
Figure 3.2 How would you define the invisible Web? (n = 568)
Information not found by search engines Databases Information behind passwords/subscriptions Not sure/Don’t know Sites not indexed by search engines Not easily accessed/Advanced skills needed Must dig deeper to find/Deep within websites Fee-based information Private websites Reliable resources Incorrect answers Must know specific URL Information found due to format No answer given 53% 19.5% 14% 11% 7% 7% 6.3% 4.4% 3.3% 2.8% 2.6% 1.4% 1% .7% 0 50 100 150 200 250 300 350
who wrote that the materials are not indexed by search engines. We interpreted indexing to refer to the way that a search engine identifies sites and is constructed. Indexing was mentioned in more technical answers and grouped with linking issues (search engine spiders follow links from one site to another) as well as no-indexing protocols, which prevent search engines from index- ing a site and making it available as part of search results.
• Other factors that contribute to the Invisible Web and that were mentioned to a lesser degree, such as fees and passwords, are all relevant to the definition but fail to represent a complete picture of the Invisible Web.
• Only sixteen responses specifically defined Invisible Web resources with a value statement. Across these answers were ref- erences to reliability, accuracy, “gems of information,” and “good stuff.” There were no negative statements.
• Of all the 502 people who offered correct definitions of the Invis- ible Web, 304 (60%) offered only one concept in their definition. The other 198 (40%) gave more complex definitions with multiple concepts. Survey forms almost always seem to encourage brevity, and that may explain some of the single-concept answers. How- ever, it may also point to a real need for teaching the complex reasons for material falling into the Invisible Web.
Question 4: Have you used it in your own research?
Over half of the 682 people responding to question 4, or 383 (56%), reported using the Invisible Web in their own research (fig. 3.3). This figure is a smaller percentage than the 64% who had stated that they knew about the Invisible Web. This difference may be due to the fact that relatively few people (19.5%) equated the Invisible Web with databases (question 3, figure 3.2). Anyone
Source: Data from Invisible Web/Deep Web/Hidden Web survey, question 4 (see appendix).
Figure 3.3 Have you used the invisible Web in your own research? (n = 682)
Yes No
56% 44%
doing research usually has to make use of subscription databases offered through a public or academic institution, if for nothing else than a literature review. Nonetheless, of those people who know about the Invisible Web, almost all have used it in their own research—a strong endorsement for teaching about the Invisible Web to anyone who has to conduct research.
Question 5: How did you learn about the Invisible Web?
Question 5 included six possible answers: journal articles, books, presenta- tions/workshops, course work, colleagues, and other. Each of the 496 respondents could choose more than one category, so the percentages listed will equal more than 100%. Their answers are charted in figure 3.4.
It is interesting to note that only 41% of respondents to question 5 men- tioned learning about the Invisible Web in course work. These respondents may represent more recent graduates. Many of the participants, of course, may have gone through their course work before the potential of the web as a research tool was developed or realized. With the systematic introduction of the Invisible Web in schools, this number would naturally increase.
Many respondents (169) included “other” in their answers and wrote additional comments which indicated that they had learned about the Invis- ible Web through various professional development activities such as blogs, on the job, or from their own research experiences. Thirty-two of these responses simply listed the first five categories in question 5 and were thus added to the percentages for those categories. This left 137 responses (29%) in the category “other.” These 137 responses are enumerated in table 3.1. Source: Invisible Web/Deep Web/Hidden Web survey, question 5 (see appendix).
Figure 3.4 How did you learn about the invisible Web? (n = 496)
55% 43% 41% 38% 29% 21% 0 50 100 150 200 250 300 Journal articles Colleagues Course work Presentations/Workshops Other Books
Table 3.1