2. PROBLEMATIZACIÓN DE LA CORRUPCIÓN
2.4 Diagnóstico
2.4.9 Puntos de contacto entre gobierno e iniciativa privada
The quality assurance guidelines proposed in this section aims to assess the “fitness for purpose” of e-participation tools developed under the context of online citizen engagement in complex decision making issues and policy formulation processes. It is based on research carried out by Guimarães Pereira et al. (2005a) and implements the principles of quality assurance by extended peer review as conceived by Funtowicz and Ravetz (1993).
These guidelines (see Table 5.1) are not a panacea for the quality assessment of all sorts of e-participation tools yet it can be considered as a starting point to assess and assure the quality of tools applied within the context of online citizens’ involvement. It must be pointed that these are still preliminary guidelines and that further work needs to be done in order to validate them.
The implementation of the guidelines is divided in two distinct phases which go beyond software quality assurance and focus also on the usage context, addressing issues related to the deployment of theses type of tools in participatory contexts:
− The first phase of the quality assurance process is dedicated to in-house testing following established procedures in the field of software testing (see for instance, Hetzel, 1993; Kaner et al., 1993; Beizer, 1995; Marick, 1995; Patton, 2001).
However, this phase goes beyond the detection of possible flaws and errors in the architecture of the tool and sought also context dependent issues that can result on ways to improve it and to assure that it is fit for the purpose of concern. If necessary, potential end-users can be involved in the testing through the use of focus groups.
− The second phase consists of a continuous external peer review process, where users are asked to give their comments about the tool based on a set of quality criteria created specifically to assess several aspects of the software. This phase relies heavily on techniques of social research enquiry, such as surveys or focus groups.
Table 5.1 – Quality criteria.
Criteria Issues to consider
Accessibility • Operating system compatibility
• Minimum screen resolution
• Internet connection requirements
• Browser compatibility
• Disability access
• Necessary plug-ins and availability
Reliability • Absence of mistakes and/or inaccuracy
• Absence of incongruities
• Absence of dead ends
• Absence of process failures and code crash
• Capacity to provide the expected service despite of faults having occurred or being occurring
Security • Security protocols implemented
• User authentication system
• Intrusion prevention mechanisms
• Encryption algorithms used
• Document upload system
Updatability • System capacity to manage new material
• Server capacity to store new material
Integrity • Functional relationship between the different components
General Software Issues
Performance • Processing and accessing times
• Latency times
• Multi-tasking techniques implemented
(Continued)
Usability • Clarity and simplicity of architecture
• Clarity and simplicity of interface
• Clear and visible navigation
• Consistency between conventions
• Consistency between objects and their functions
• Aids, tools and help resources available
• Systems status mechanisms
• Informative error messages
Aesthetics • Visual attractiveness
• Professional appearance
• Artistic integration
• Colour harmony
• Text size
Layout • Effective use of the available space
• Effective use of multimedia elements
• Right balance of multimedia elements
• Effective graphical message
Integration • Clear and effective incorporation of elements
• Congruence of the elements
• Unique look and feel
Readability • Clear content
• Clear writing style
• Clear arrangement of text
• Contrast between text and colours
• Text size
Suitability of Design
Interactivity • Level of interest raised
• Level of intuitiveness
(Continued)
Adequacy • Types of contexts and audiences considered
Functionality • Level of functionality accordingly with the context and audience
Relevance • Level of relevance accordingly with the context and audience
Fitness for purpose
Adaptability • Level of adaptability to different contexts and audiences
Inclusiveness • Inclusion of all stakeholders and citizens in the process who feel they have an interest in the topic under discussion
Equality • Equal consideration for the opinion of the participants
• Equal right to be heard and to be part of the process
• Equal access to information
Transparency • Clear aims and objectives
• Clear citizens expectations
• Perception of how participants contribution can influence the outcomes
• Openness to the wider public
Independency • Neutral and balance process
• Independent organisers and moderators
• Participation of stakeholders without affiliation with the sponsoring body of the process
Knowledge Sharing • Environment mutual understanding and respect
• Social interaction
• Visibility of participants
• Common grounds
Fitness for e-participation
Accountability • Explicit relationship between the participants contributions and the process outcomes
• Legitimacy of outcomes
(Continued)
• Response from institutional body
Effectiveness • Outcomes produced
• Relevance of the outcomes
• Impact of outcomes
• Importance of possible alternatives
Participants Evaluation
• Participants satisfaction
• Participants fulfilment
• Participants comments
• Participants contributions and relevance
Integration • Integration with other online techniques and technologies
• Integration with offline methodologies
Source: Adapted from Guimarães Pereira et al., 2005a.
With the aim of tuning b-involved as far its fitness for the purpose of online moderated debate, the described quality assurance guidelines were implemented.
Hence, in a first stage, b-involved was tested in-house with the supervision of the developers for possible system bugs and functionality flaws. During this phase, apart from the several tests performed directly by the developers using a test to pass and test to fail methodology20, two test sessions where organised with possible end-users using a black-box approach21. The main purpose of this phase was to identify critical errors in the platform architecture and to solve them before it was available to the public.
The second step in the quality assurance process (which is still on-going, as we write this report) consisted of inviting all the users of b-involved to evaluate the platform and to make recommendations for possible modifications by responding to an online survey created for this purpose (see Appendix G). The goal is to collect the views and
20 The goal of the test to pass methodology is to assure that the software works as expected. On the other hand, the test to fail methodology consists in designing and running test cases with the sole purpose of
“crashing” the software (see for instance, Patton, 2001).
21 Testing software using a black-box approach means that the person who executes the test does not know exactly how the software works. In alternative, it is possible to use a white-box approach. In this case, the person performing the test has access to software code and can monitor the program as it runs (see for instance, Patton, 2001).
experiences of users so it could be incorporated in future improvements and developments of the platform.
The survey was prearranged into eight groups of questions:
− First impression: Section composed by an open question where it was asked to the user to write his immediate thoughts about b-involved;
− Knowing you better: Group of questions about the user;
− Knowing your system: Set of questions intended to assess the characteristics of the system used by the user;
− Technology integrity: Group of questions addressed to evaluate the behaviour of the platform in terms of accessibility, integrity, performance and suitability of design;
− Context of usage: Questions focused on the user’s opinion about the contexts in which b-involved could be used;
− Moderation: Section addressed only to those users who had a role as moderator in a session. The purpose of this section was to investigate the challenges of online moderation, in particular of the session organised by this particular moderator and gather his suggestions for moderation tools that might lack in the current platform;
− Participation: Section addressed only to users who had a role as participants in a focus group session. Here, the goal was to understand how the platform encourages or hinders participants to contribute to the debate, including the expression of their points of view, and if they could follow and intervene in the debate easily;
− Final Remarks: In this final section of questions, the objective was to know if the users were satisfied with their experience and if they would use b-involved again and/or recommend it to others.