VAST. Evaluation of the execution-visualization functional integration

Texto completo

(1)

Francisco J. Almeida-Martínez

Jaime Urquiza-Fuentes

VAST.

Evaluation

of

the

execution-visualization functional

integration.

Number 2011-07

Serie de Informes Técnicos DLSI1-URJC

ISSN 1988-8074

(2)

Index

1. Introduction ...3

2. Description of the evaluation ...3

2.1. Subjects...3

2.2. Experimental design and tasks...4

2.3. Protocol...4

3. Results…...4

3.1. Instructors observations...5

3.2. Answers to questionnaires...5

4. Conclusions...5

Appendix...6

(3)

VAST. Evaluation of the execution-visualization

functional integration

Francisco J. Almeida-Mart´ınez, Jaime Urquiza-Fuentes

Universidad Rey Juan Carlos

Departamento de Lenguajes y Sistemas Inform´aticos I c/ Tulipn s/n, 28933 Madrid, Spain

francisco.almeida@urjc.es, jaime.urquiza@urjc.es

Abstract. This document describe an observational and usability evalu-ation of VAST in the course 2008-2009. This evaluevalu-ation was performed af-ter implementing the first functional integration calledexecution-visualization. The results of this evaluation show that students like VAST but it is necessary to add new features as the second functional integration,

annotation-generation-compilation.

1

Introduction

From the results of previous evaluations [1], we plan the necessity to adapt the process of visualizations generation to the real process of parser generation. Due to this, we planned two functionality integrations. One of this integration is the

execution-visualization. This functionality was added to VAST, so the user can

edit the input stream and run the parser from the own user interface. Thanks to this, the user can generate new visualizations without effort and observe how the parser behave for different input streams. The objective of this evaluation is observe the usability of this new integration.

2

Description of the evaluation

In this section we describe the evaluation. We refer to the participants, the experiment’s design, the tasks performed during the session, the protocol and the results.

2.1 Subjects

In this evaluation participated5 studentsof the subjectCompilers and

Inter-preters of the Rey Juan Carlos University in the 2008-2009 course. The

(4)

2.2 Experimental design and tasks

This evaluation was designed as an usability experiment plus an observational study. In this evaluation the students were divided in two groups. For this task we used a class exercise as a pretest of knowledge (see appendix 5).

The tasks performed during the evaluation consisted in two exercises (see appendix 5). From the lexical and syntax specifications given, the students had to annotate, generate the parser and executed it, using the new functional in-tegration, execution-visualization of VAST. When the groups were created, the students had to annotate the lexical or the syntax specification. The objective of this task was to take note of the difficulties of each part.

The dependent variables were the students’ opinion about four different as-pects: the ease of use, quality, satisfaction and learning help. The experiment lasted 2 hours (one class). During this time the students had to use VAST’s documentation. At the end of the session we asked the students for answering an opinion questionnaire about the tool, including the new integration.

2.3 Protocol

Two weeks before the evaluation we performed a pretest of knowledge, which was done by only two students. For this reason we did a random division of the participants being impossible to balance both groups. One week before the evaluation we presented VAST and how to work with the tool. This lasted about 30 minutes, during the rest of the session students worked on the class exercises. In the table 1 we show the protocol followed in this evaluation.

Group 1 Group 2

Pretest of knowledge Introduction to VAST

Lexical annotation 1 Syntax annotation 1 Syntax annotation 2 Lexical annotation 2

Answers to questionnaries

Table 1.Protocol followed in this evaluation

3

Results

(5)

3.1 Instructor’s observations

During the evaluation we observed how the students performed some of the tasks of the test. Some students did changes in the visual representation of the syntax tree using the configuration tool of VAST. Others were more interested on the changes of the user’s interface (distribution of views). In general, students preferred the used of the floating view instead of using the horizontal/vertical distribution. It was interesting that students made a high use of the parser’s stack and the automatic animation of the syntax tree building process.

Furthermore, we observed that students required the

annotation-generation-compilation integration.

3.2 Answers to questionnaires

The results of the questionnaires are focus in the values of the dependent vari-ables: ease to use, general quality of the tool, student satisfaction and learning support.

Aspect % of answers between 4/5

Ease to use 100%(5/5) Quality of the tool 100% (5/5) Student satisfaction 80%(4/5)

Learning support 80%(4/5)

Table 2.Summary of questionnaires answers

In table 2 we show a summary of the results obtained in the opinion ques-tionnaires filled after the evaluation by students.

In the questionnaire we also included open questions in order to make possible students to express their opinion easily. As positive aspect students pointed out the visualization of the syntax tree, its building process, the ease of use and the user interface. As posible improvements they mentioned the interaction with the syntax tree, the automatic lexical/syntax annotation and the exportation of animations.

4

Conclusions

The results obtained show that the development of VAST is centered on the students.

(6)

as hard and monotonous. Because of this the general satisfaction ofVASTapiis lower. Finally, the learning improvement also receive a worse mark. One student gave a 3 to the use of the stack.

As result of this evaluation we plan to implement the another integration

annotation-generation-compilation.

5

Acknowledgment

This project is supported by project TIN2008- 04103/TSI of the Spanish Min-istry of Science and Innovation.

A

Exercise used for the pretest

Exercise:

WWW stands forWorld Wide Web, a global net based on the architecture to transfer hypertext files between machines connected to the internet. To make posible the localization of a hypertext file on the internet it has been defined a language which sentences are called URL (Uniform Resource Locator). The URL allows to specify the communication protocol, the machine and the file. For example, the following URLftp://apolo.uma.es/dist/doc/drol.ps shows that we want to obtain the file/dist/doc/drol.psfrom the machineapolo.uma.esusing the protocolftp. For this exercise we only consider that the WWW only admits the following protocolos:http, ftp andgopher. The name of the machine can be written in two ways:

– In symbol notation, as in the previous example, using the machine’s name and (optionally) the domains, separated by the point character, for example:

apoloorapolo.uma.es.

– Using decimal notation, specifying the four numbers of the internet direction, for example: 150.214.58.55.

The file’s name of the URL address is any correct name of a valid file in UNIX including the complete path (using only alphanumerics characters, the point and the slash to separate directories), for example: index.html or/dist/doc/drol.ps. In a URL reference the protocol is optional, so by default it asumes thehttp pro-tocol. Also the file’s name is optional, so it uses by defaultindex.html. For exam-ple, the following address are valid and equivalent:apolo.uma.es, 150.214.58.55,

apolo.uma.es/index.html andhttp://apolo.uma.es.

Using the tools JFlex and Cup, build a parser to recognize the URL address. A possible grammar could be:

S::=U RI S|U RI

U RI::=P ROT OCOLO://N OM BRE M AQU IN A U RI2|N OM BRE M AQU IN A U RI2

(7)

P ROT OCOLO::=F T P|HT T P|GOP HER

DIRECCION IP::=DIGIT O . DIGIT O . DIGIT O . DIGIT O DOM IN IO::=IDEN T|IDEN T . DOM IN IO

DIRECCION F ICHERO::=/IDEN T /N OM BRE F ICHERO|/IDEN T DIRECCION F ICHERO N OM BRE F ICHERO::=IDEN T . IDEN T

B

Exercises for the evaluation

1. Exercise 1: Given the following grammar to declare procedures/functions:

S::=T Y P E ident(LIST P ARAM)SEN T LIST T Y P E::=boolean|integer|f loat

LIST P ARAM::=T Y P E ident|T Y P E LIST P ARAM, ident SEN T LIST::=SEN T|SEN T LIST; SEN T

SEN T::=ident = EXP;

EXP::=EXP + T|T T::=T ∗ F|F F::=ident|(EXP)|cte

(a) Build an animation using VAST. The animation must show the whole declaration of a function with N parameters and at least one assignment instruction.

(b) Modify the input stream to include four assignment instructions. 2. Exercise 2: Given the following grammar to declare SQL queries:

S::=LQU ERY

LQU ERY ::=LQU ERY ;QU ERY|QU ERY

QU ERY ::=select LID in LIST T ABLAS where LIST CON D LID::=LID, ident|ident

LIST T ABLAS::=LIST T ABLAS ident|ident LIST CON D::=F OP LIST CON D|F OP F OP::=and|or|nand|xor

F::=ident|cte

(a) Build an animation using VAST. The animation must show the whole declaration at least one SQL query with at least two conditions. (b) Modify the input stream to include two SQL queries with at least two

conditions each one.

References

Figure

Actualización...

Referencias

Actualización...

Descargar ahora (7 pages)