A second online questionnaire-based study (see Appendix D) was again administered to geospatial data users and producers. This questionnaire comprised six sections, A to F, each of which is described below.
Section A consisted of a small number of questions to gather background information about the respondents and their requirements when selecting geospatial data to use. Here respondents were first asked to identify themselves as one of the following: primarily data users; primarily data producers; or equally data users and producers. The respondents were then asked: to pick one or more statements from a set which best describes their dataset user or producer type; to identify the type of organisation they work for; to indicate how long have they been working with geospatial data; and to approximate the percentage of their time they spend working directly with geospatial data. In this section, the respondents were also asked whether they have a choice of dataset to use, what clearinghouses they use, if any, and whether they find dataset selection a challenging task.
Section B was designed to explore respondents’ levels of understanding of the proposed GEO label facets (informational elements) and the information the facets convey about the datasets they represent. This section first presented a short description of: how quality information can affect dataset selection decisions; what informational aspects may be considered when evaluating a dataset’s quality; at a high level, the proposed GEO label and what it represents; and how the GEO label could be used to support dataset selection. After the introduction, the icons representing the eight GEO label information facets (see Figure 6.1) were presented to the respondents who were asked to describe what geospatial dataset informational aspect they believed each icon represented (see Figure 6.2 for an example; also see Appendix D, Section B).
Figure 6.1: GEO label informational facets presented to study respondents.
The intention was to assess respondents’ unguided initial interpretation of the meaning of each icon and thereby identify the effectiveness and intuitiveness of the graphical representations. After soliciting respondents’ initial and unguided opinions as to what they
(see Figure 6.3 for an example; also see Appendix D, Section B). Armed with this information, the respondents were then asked to rate the intuitiveness of each icon and to provide any comments or suggestions on each icon’s representation and indicate possible improvements.
Figure 6.2: Question B1 (see Appendix D) asking respondents to identify the meaning of the icon.
Figure 6.3: Question B2 (see Appendix D) providing the intended meaning of the icon and asking respondents to grade its intuitiveness relative to that meaning.
Once the intuitiveness of the meaning of the icons themselves had been assessed, the section then evaluated the effectiveness of the variations for each icon in terms of conveying the availability of quality information for a given dataset. As before, the availability variations for each icon were first presented to the respondents without explaining their intended meaning and respondents were asked to interpret their meaning without guidance (see Figure 6.4 for an example; also see Appendix D, Section B). The intended availability meaning was then indicated (see Figure 6.5 for an example; also see Appendix D, Section B), and respondents were asked to consider how intuitive they thought each availability representation was. Respondents were also welcomed to provide any comments and suggestions regarding the icons’ variations and their perceived meaning.
Section C was focused on gauging respondents’ opinions on the effectiveness of the proposed GEO label prototypes – that is, the complete labels rather than the component parts (as studied up until this point) – at conveying availability of dataset quality information. This section presented three GEO label designs, two of which (examples 1 and 2 in Figure 5.17 and Figure 5.18) used collective visualisations of the individual icons examined in section B, and one (example 3 in Figure 5.19) used an alternative visualisation without iconic
representation of the facets (drawing on ideas from the skeleton ST-09-02 proposal). The prototype designs were presented to respondents separately. Each GEO label example was first presented together with a series of both true and false statements about quality information availability for a fictitious dataset that the label was supposed to represent; respondents were asked to indicate those availability statements that they believed were correct (see Figure 6.6, Figure 6.7 and Figure 6.8; also see Appendix D, Section C). This exercise was designed to assess the ease of interpretation or intuitiveness of the proposed visualisations as well as ensure that participants were familiar with each GEO label representation before proceeding to more demanding dataset ranking scenarios (as described later).
Following this, each GEO label example was examined within the context of a dataset selection scenario: respondents were presented with a scenario identifying key dataset qualities that were flagged as important for a given selection decision. They were then provided with 5 different mock-ups (based on the given GEO label design) that conveyed availability of different quality information for 5 fictitious datasets, and were asked to examine the mock-up GEO labels and rank them (as representatives of their underlying datasets) in order of fitness-for-use based on a match between the specified scenario and the information the labels conveyed (see Figure 6.9 for an example; also see Appendix D, Section C). After each ranking scenario, respondents were asked to indicate how effective the GEO label designs were at supporting, and how easy it was to use them for, dataset selection (see Figure 6.10). At the end of the section, respondents were asked to rank the three different GEO label prototypes in order of preference and to provide some justification regarding their indicated preferences (see Figure 6.11).
Figure 6.4: Question B18 (see Appendix D) asking respondents to identify the availability-related meaning of the icon variations.
Figure 6.5: Question B19 (see Appendix D) providing the intended meaning of the icon variations and asking respondents to grade their intuitiveness relative to that meaning.
Figure 6.6: GEO label example 1 presented with a mixed set of true and false availability statements from which the respondents were required to identify the true statements.
Figure 6.7: GEO label example 2 with a mixed set of true and false availability statements from which the respondents were required to identify the true statements.
Figure 6.8: GEO label example 3 with a mixed set of true and false availability statements from which the respondents were required to identify the true statements.
Figure 6.9: GEO label example 1 presented within the context of a dataset selection scenario where respondents were asked to rank the datasets according to best fit for purpose on the basis of the representative GEO labels.
Figure 6.10: Questions C6 and C7 (see Appendix D) in which respondents were asked to reflect on the use of ranking datasets according to the GEO labels for each and to indicate the label effectiveness, respectively.
Figure 6.11: Question C26 (see Appendix D) in which respondents were asked the rank the three prototype GEO labels in order of preference.
Section D consisted of a small number of questions to gather respondents’ opinions on the informational aspects presented in the GEO label examples. Here respondents were asked to identify the informational aspects that they consider important and relevant to the GEO label, to indicate the aspects that they considered to be redundant and should not be included in the GEO label, and to describe any additional aspects that they felt may be relevant to the GEO label function. This was done to (a) further validate findings from the previous phases of this research, and (b) to identify whether, when used in reality within the context of a proposed GEO label, respondents felt differently to the proposed informational elements than had been suggested would be the case in previous phases of this investigation.
Section E consisted of a small number of questions designed to gather respondents’ opinions on the use of ‘GEO’ branding in the GEO label. The main aim of this section was to investigate whether the presence of branding affects perceived credibility and trustworthiness of the label and the information it conveys. As discussed in Section 2.6, the concept of a GEO label was initially proposed by GEO to support the recognition of the GEO activities and promote GEOSS; consequently, ‘GEO’ was an obvious branding to use as part of the
examples (see Figure 6.12), one with ‘GEO’ branding and one without any branding. Using a 5-point Likert scale, ranging from ‘strongly disagree’ (1) to ‘strongly agree (5), respondents were asked to reflect on their level of agreement with a series of statements about the perceived trustworthiness of the two examples (see Figure 6.13 for examples of the statements). Both positive and negative statements were used to try and ensure that respondents provided properly-considered ratings. Respondents were ultimately asked to indicate whether they would prefer a GEO label with ‘GEO’ branding or a label without such branding; they were asked to justify their choice.
Figure 6.12: Examples of a GEO label with and without branding (see Appendix D).
Figure 6.13: Questions E1 and E2 (see Appendix D) asking respondents to reflect on the impact of GEO branding in terms of their interpretation of the label.
Finally, in section F, the respondents were asked to provide any further comments and suggestions on the GEO label and the proposed GEO label designs.
As with the previous GEO label questionnaire, this questionnaire was constructed and administered using the QuestionPro (QuestionPro, 2014) online-survey software. To inform potential respondents about the questionnaire, emails were sent to a number of
professionals, academics and researchers who work in the GIS field, asking them to complete the study. The questionnaire was accessed and completed from a number of countries including Spain, Germany, United States, Netherlands, France, Italy, United Kingdom and China.