Introducción al transporte de sedimento

This section revisits the specific objectives set out in the first chapter of this thesis: 29. Investigate a new model for end-user Web automation that would offer an

easy to use approach to automate Web tasks.

30. Design, implement, and evaluate a new simple and effective scripting language intended for the development of Web automation tools.

31. Design, implement, and evaluate a Web-based framework to enable end users to create, run, and share Web automation tools.

32. Design, implement, and evaluate a repository of Web commands as a reusable, efficient, and customisable Web automation tool.

The next four sub-sections address each of these objectives linking them to the work achieved in this thesis.

9.5.1 New Model for End-user Web Automation

This thesis presented a novel model that combined end-user programming concepts with Web standards and enabling technologies. The model presented can be used to enable the automation of Web tasks, activities, and interactions. The thesis defines this area of research as end-user Web automation.

• UNIX shell scripting, see section 2.4;

• SPHINX for creating personal, site-specific Web crawlers, see section 2.3.1; • YubNub simple command line for the Web, see section 2.3.2;

• The software tools philosophy, see section 5.3; • The shift towards the “Web as a platform” vision.

The model approach was to apply the software tools philosophy to the Web, using UNIX shell scripting as a proven successful model. The idea of site-specific and customisable Web crawlers in the SPHINX framework is reflected in the introduced concept of the Web command, at the same time considering the software tools philosophy. The YubNub application, simple as it is, inspired the idea of a simple command line for the Web. In order for this model to work, an important enabling technology was Asynchronous JavaScript and XML (AJAX), which allowed the communication through HTTP streams between the command line interface and Web2Sh backend server.

The Web2Sh framework implemented the introduced model to show its feasibility and usefulness as illustrated in Chapters 4-7. Chapter 8 described the evaluation of the Web2Sh framework that was carried out through an experiment in which end users used both Web2Sh and Yahoo pipes systems. The majority of the participants confirmed that the Web2Sh approach was easy to use for them; only one participant out of the twenty-four participants answered that the system was somewhat difficult to use.

9.5.2 WSh, a Scripting Language for Web Automation

All investigated systems, which aimed at achieving a degree of Web automation, have based their approach on the sole use of GUI-based programming, or have used an existing Web scripting language, most commonly JavaScript. The arguments presented in this thesis in section 2.3 and the evaluation in Chapter 8 demonstrate the desirability and usefulness of implementing a focused scripting language with the target of enabling easy and effective Web automation.

The success and power of the shell-scripting model in UNIX platforms inspired the design of WSh. However, WSh extended this into the dimension of Web automation. Hence, WSh offers a set of unique features, which enable it to provide extensive and sophisticated tools to achieve its goal of Web automation.

One of the aims of WSh design was to enable the creation of Web commands that are ‘smart’ enough to act in a logical way based on the MIME type, context, and content of the resource it process. In order for a smart Web command to provide certain functionality to its users, it must be able to detect the resource MIME type, the context within the pipeline, and attempt to examine the content itself when needed. It then determines what actions to execute. WSh design supports the creation of intelligent Web commands more easily. In addition, it supports the extension of core Web commands without the need to alter the framework.

WSh design takes into account the variety of Web formats, standards, and technologies. It implements a design whereby any Web resource’s MIME type can be processed by the framework. According to this design (named Polymorphic MIME types), each Web command may support different functionalities for resources of different types and varying contents. The context of the Web command may also affect the actions it takes.

Chapter 5 presented the knowledge and experience gained by designing and implementing the WSh scripting language. It also described the key enabling computer science concepts and technologies that facilitated the development of the WSh scripting language.

The design of the WSh language makes the best of the new Web enabling technologies and tools. By providing a Web-based command line interface, Web2Sh users can simply create and use Web commands using a JavaScript-enabled Web browser.

WSh contributes to the research of end-user programming and programming language processors. Although it is not the focus of this research, it was necessary to investigate these research topics in order to gain the knowledge required to design and implement a scripting language that can be useful, effective, simple, and powerful.

9.5.3 Web2Sh, a Web Automation Framework

Web2Sh contributes to the research of interactive end-user Web automation by suggesting a new approach of automating Web tasks, activities, and interactions. This approach is both simple and effective. As the development of Web automation tools in general is a complex task for the majority of end users, Web2Sh can be of great benefit to end users with little or no programming experience (casual programmers) who want to automate their Web interactions simply, effectively, and rapidly.

Web2Sh is also well suited for users with a UNIX shell background, as the presented Web shell language (WSh) was inspired by the UNIX platform. Web2Sh also offers access to those users to some of the very powerful tools of the UNIX platform such as sed and grep. This is achieved by implementing a UNIX tools wrapper that acts as a Web command, passes the input from Web2Sh to the appropriate UNIX tool, and then returns the output back to Web2Sh. This approach brings the power of these tools directly to any Web-enabled device that has a Web browser supporting JavaScript. The evaluation presented in chapter 8 highlighted that out of seven participants, who practised programming using UNIX shell scripting, six preferred the Web2Sh approach in comparison to Yahoo pipes.

An additional target group of potential users is researchers. Web2Sh provides a set of tools that enable researchers to save time and effort by enabling them to automate some of the common research tasks, such as citing online references, processing pdf files, collecting raw data from the Web, and checking for word synonyms.

Web2Sh utilises the user-generated content approach by offering its users the possibility to create and share new commands. This approach allows the Web commands repository to grow further. It also enables all its users to utilise and share their experiences through the creation of new commands, which can bring about many new innovative Web commands that may have not been foreseen when developing the core Web commands.

9.5.4 Repository of Web Commands

This work introduced the Web command as a tool created with a specific goal to provide a solution for automating simple or sophisticated tasks in various domains related to end users’ Web interactions. As discussed in Chapter 6, Web2Sh framework offers a repository of core Web commands with various functionalities as a proof of concept.

The generic approach of the Web2Sh framework, combined with the WSh scripting language, allows for connecting any set of Web commands using pipes. This provides general solutions for a larger set of problems in order to meet the Web users’ needs. The design of the Web2Sh framework also allows for the easy extension of the core Web commands. Thereby, the developers can implement any functionality they require as a Java class and then plug it into the framework.

Participants’ feedback after using both Web2Sh and Yahoo pipes, described in chapter 8, indicates that providing general Web commands, that is commands that can fit into any domain, is essential and enable more flexibility. However, for each specific domain of users a more focused and advanced Web commands need to be provided in order to offer easy and quick retrieval of data that can be of interest to them from specific Web sources. Hence, different version of WSh language and Web commands can be customised as a domain-specific language for each group of users based on their needs.