FACTORES Y CAUSAS DEL DETERIORO URBANO DE LA VEGA CENTRAL

The S000 dataset allowed correlational studies unlike the BAWE corpus which does not give the essay marks as numerical values but with distinction and merits. After conducting correlational studies with team reports from each year, the 2012 and 2013 team reports were collated and the study was repeated. This was possible as the marking rubrics remained the same which eliminates the instrumentation biases. However, although the nature of the assignment was very similar in both years along with the assessment rubrics, different teams were working on different scenarios. This creates a challenge in terms of reliability because changes in mark might be affected by the topic of the scenario and the background information provided to the teams. For instance, the 2013 teams were given an example report from the previous year. Therefore, changes in marks cannot solely be explained by the better quality of writing because of other possible variables such as the topic difficulty and the quality of the material provided to students.

Since the correlational studies yielded weak results, in order to gain a better understanding of how each XIP sentence type affects the essay mark, multiple regression analysis was carried out, but the model had no explanatory power on mark. S000’s multiple regression analysis result aligns well with the BAWE’s Physical Sciences regression model.

Therefore, it can be concluded that XIP does not perform well enough on student essays written for hard disciplines.

Typically, as stated in the literature review, hard knowledge field writing at university level does not actively encourage argumentation. Although S000 was an undergraduate science module, it was a multidisciplinary science module requiring its students to produce critical, argumentative writing. It is therefore expected that argumentation would be picked up by the XIP, which it did indeed do but it was not found to be effective on the essay mark for the S000 sample.

Prior to the statistical analysis, false hits from the XIP output were eliminated. Bibliography sections were discarded from all of the texts as some of the titles were identified as salient by the XIP. Similarly, non-existence of the line breaks between

figure/table headings and the following sentences meant the following sentence was picked up as salient although it was not. The reason for this was that those figure titles had

linguistic features, meta-discourse, that XIP could identify as salient. Therefore, certain kinds of false hits, that would weaken the statistics, were excluded in order to obtain a more precise test of whether XIP’s categories relate with quality. Moreover, additional analysis of the false negatives was conducted, in order to make sure the XIP tool does not miss any rhetoric. There were no false negatives found within the dataset.

Since the XIP output was eliminated from such false hits, one possible reason for not finding any statistical evidence that the XIP categories were not effective on the essay mark could be that the linguistic features used by these students did not align with the linguistic features that XIP looks for. To investigate this further, the accuracy of the XIP results were examined from all the student writing. This showed that the accuracy of these results was not flawed as there was not a single poor identification. All the sentences were accurately picked up as salient under accurate XIP categories. The following examples from the student texts for each XIP category illustrate this:

 EMPHASIS

“It is believed this is a vital study for the success of further long stay missions and that at this important stage of exploration all precautions must be taken to protect both planets biospheres prior to the possible habitation of Mars.”

 BACKGROUND

“Previous NASA research has shown there is enough carbon dioxide on Mars to support microbial life (McKay et al, 2001).”

 CONTRAST

“The 1976 twin Viking landers provided only evidence that Mars was an inhospitable, sterile planet (Smith, 2011) however the missions since then have proven Mars to be a more complex and interesting planet than we previously knew.”

 SUMMARY

“This report sets out several possibilities for power on a mission to Mars; it states the information that will be required to decide on a source before the mission departs.”

 NOVELTY

“This investigation can add further evidence to any results found or if no evidence of life is found then this would be an opportunity to investigate a wider range of possible environments using and improving previous methods.”

 TENDENCY

“Not only will important questions about the planet be answered, but it could also serve as a starting point for future manned missions, and would create a huge increase in potential investigations that could be carried out on the Martian planet.”

 SURPRISE

“Data suggests that most of Mars' crustal rocks are mafic in composition,

specifically basaltic, with evidence of some quartz - containing felsic igneous rocks depending on their molecular structure, varying minerals emit long-wave TIR (thermal infra-red region) radiation differently, so that they have distinct spectral patterns. These rocks were compositionally similar to each other and to terrestrial andesite, but unexpectedly high in silica and potassium, and low in magnesium compared to Martian soils.”

Since the false hits were eliminated and the accuracy of the XIP results was controlled, the S000 study triangulates the BAWE study results, and suggests that the XIP did not work well with undergraduate science writing, especially in hard disciplines. The research question can be answered to say that the automated rhetorical parser XIP cannot be used to identify good indicators of academic writing in undergraduate student writing for hard knowledfe fields and so no further repetition is needed with a different dataset.

To answer the extent of how XIP can be used, the analysis however should be repeated on student writing from other disciplines, a complementary social sciences or humanities discipline (e.g. educational psychology, social sciences etc.). The next sections describe the studies repeated with student writing from these disciplines, in which students’ ability to write critically and argumentatively was measured and valued.

5.5 E000 DATASET

5.5.1 Introduction

The student writing in this dataset came from one of the third-level undergraduate education and arts modules of The Open University, which aligns with the arts and humanities BAWE category. The reason for selecting this dataset was the nature of the assignment, given in learning outcomes and marking rubrics used by the tutors. Students were expected to read academic texts critically, identify and evaluate positions and arguments from those, synthesise that information, and analyse and critically evaluate others’ arguments and evidence in order to reach their own conclusions. The statistical studies were conducted to understand how XIP performs within soft disciplines as this sort of discipline is the one most likely to award marks for rhetoric, unlike hard disciplines.

5.5.2 Dataset

In E000, a course on literature for young people, students study key examples of novels, picture books, poems and creative performance produced for children aged between 3 and 18. Students read a selection of related critical material and consider major themes, issues and debates in the field such as the relationship of children’s literature to the conceptions of childhood, the question of whether children’s literature should instruct or delight, and the tension between popular and prestigious literature for children. Throughout the module, students write six essays, each of 2000 words, about these themes. At the end of the

module, students are expected to produce an individually written, longer essay (3000 words) which is the examinable component of the module; that means it takes the place of an examination. Unlike the previous six essays, the end of module assessment aims to assess the knowledge, understanding and skills students have developed throughout the module as a whole.

5.5.3 E000 assignment

In the assignment, students were required to answer the question: “Does children’s literature always have an overt and/or a hidden agenda? Discuss this question with

reference to the chapters by Kimberley Reynolds and Peter Hunt in Reader 1, Approaches and Territories, together with at least three Set Books from two or more blocks of E000.” The assignment required students to engage in depth with texts and approaches explored within the module. Students were expected to engage in critical thinking and in evaluating the materials they had studied. They were assessed, in part, on their ability to think through the strengths and limitations of the materials they used, and to express this critical thinking clearly in their writing. Students were expected to demonstrate that they could sift material and present points so as to set out a coherent argument, garnering support from the sources

and positions represented across the module. In terms of learning outcomes (see APPENDIX B for full list), students were expected to:

 read academic and other texts critically, identifying and evaluating positions and

arguments

 synthesise information and ideas from different sources, in order to reach their own

conclusions

 analyse and critically evaluate arguments and evidence, from a variety of sources,

relating to particular theoretical and analytical approaches to the study of children’s literature.

5.5.4 Assessment

For this assignment 1307 students each submitted an essay. Each of these essays was marked out of 100 and those that received a mark below 40 failed.

Each essay was marked by the tutors. Each tutor marked around 80 essays and all tutors used the same marking guidelines and rubrics (see APPENDIX C and D) provided by the module team. Based on these guidelines, tutors marked the student writing in six points:

1. relevance to question

2. knowledge and understanding

3. approach to alternative explanations and arguments

4. construction of argument

5. clear expression and use of academic conventions

Unlike S000, the mark distribution for each of these points was not made clear to the tutors. Therefore, what each of these points would mean numerically is not information that was available.