MIGRANTES Y EL CRECIMIENTO URBANO.

After conducting binary logistic regression analyses with student texts from the BAWE corpus based on disciplinary groupings and levels under each group, it was noted that XIP performed differently for each discipline and each level. Table 5.5 summarises the

statistically significant XIP categories for each discipline and level, including the p values and odds ratios.

Table 5.5 Summary of the binary logistic regression analysis results for BAWE corpus

Disciplinary group

Overall

Level 1 Level 2 Level 3

Social Sciences NOT SIGNIFICANT NOT SIGNIFICANT SUMMARY (p=0.007, odds=1.222) CONTRAST (p=0.039, odds=0.844) NOVELTY (p=0.02, odds=1.759) Arts and Humanities SUMMARY (p≤0.001, odds=1.093) EMPHASIS (p=0.033, odds=1.173) SUMMARY (p=0.005, odds=1.148) NOT SIGNIFICANT Life Sciences SUMMARY (p=0.007, odds=1.139) NOT SIGNIFICANT TENDENCY (p≤0.001, odds=14.459) OPEN QUESTION (p=0.002, odds=6.923) Physical Sciences NOVELTY (p=0.03, odds=2.272) NOT SIGNIFICANT NOT SIGNIFICANT NOT SIGNIFICANT

The following table provides some examples of the XIP analysis, demonstrating the salient sentences identified by XIP for significant XIP categories; to show the accuracy of its results. For each statistically significant XIP category, two distinct examples are given.

The linguistic features, meta-discourse, shown in bold demonstrate the reason why XIP assigned this particular sentence category.

Table 5.6 Examples of the salient sentences picked up by the XIP from the BAWE corpus

Level 1 Level 2 Level 3

SS SUMMARY

“The aim of this essay is to expose the

importance of China through a theoretical approach which places China within a larger capitalist world order.” “The final section of this essay will discuss the possible disruption of such a chain with reference to China's domestic politics and the strain its role in the chain puts on domestic politics, thereby exposing the potential importance of China to the global order.”

CONTRAST

“All these overriding assumptions become problematic when we examine contemporary examples of victimization, when women are also agents of violence while men become the victims of sexual abuse.”

“While women may play an assertive role in ethnic conflicts, they may not be military combatants fighting for a worthy cause; rather they could be active perpetrators of inhumane war crimes.”

NOVELTY

“In a similar light, the ethnic conflicts that have engulfed the regions of the former Yugoslavia during the past decade offer us new perspectives of looking at the roles women play.”

“More importantly, its implications for gender studies are paramount; it will provide new insight for feminist accounts of women's agency in conflicts.”

AH EMPHASIS “Dante was an important figure in the development of Renaissance literature.” “Florence had a key role in the development of the Italian Renaissance because its cultural pre - eminence coincided with the largest territorial expansion of the time .” SUMMARY “Patterns of births, marriages, deaths and migrations helped to shape society and economy and were themselves shaped by society and economy, as I have attempted to show in this essay.”

“After all, the authors are aware that methodological

shortcomings exist; this work does not pretend to be concrete evidence but merely a likely

estimation.”

LS TENDENCY

“The use of verbal reports and discourse

OPEN QUESTION

“Many foods contain non-nutritive

analysis has increased in popularity as a research method and has resulted in many contributions to the study of psychology.” “Hopefully as research into discourse analysis increases, journals might adapt their guideline in order to cater more for qualitative research (Coyle, 1995).”

phenolic compounds which provide

protection against chronic diseases through multiple effects are as yet poorly

understood.”

“Very little is known about the absorption and metabolism of anthocyanins and whether they are absorbed in sufficient quantities and in a form in which effects on in vivo measures of oxidative cellular damage.”

PS

The results demonstrated that XIP did not perform well on BAWE essays drawn from the Physical Sciences. The XIP categories do not work well for undergraduate science essays. Writing for hard knowledge disciplines requires different discourse (Kelly, 2007), which could explain this result. On the other hand, XIP’s performance showed promising results on the other disciplines.

The highest odds ratios were found in the Life Sciences, meaning the XIP categories TENDENCY for Level 2 and OPEN QUESTION for Level 3 were highly effective on the given essay mark. Each of these sentence types increased the students’ final essay mark significantly with odds of 14.459, and 6.923 respectively. These two categories did not come up in any other discipline or level. Considering that XIP was originally implemented to analyse the abstracts of journal papers from the life sciences disciplines, this result suggests that the XIP also works for the student writing in this disciplinary area. These two categories require higher-order writing skills; which are expected to be seen in the

discourse moves of the experienced researchers’ writing. As explained in the literature review, all of the XIP categories are created based on the linguistic analysis of the

experienced researchers’ writing; however not all categories require higher-order writing skills such as SUMMARY where the writer should summarise the goals or the results of

the article, or BACKGROUND where the writer needs to describe the previous

knowledge, the literature. On the other hand, TENDENCY describes research trends and emerging research directions and OPEN QUESTIONS describes problems which have not been solved. In order for an undergraduate student to identify and then describe an

unresolved problem requires analytical and critical thinking which is a higher-order writing skill. This is possibly the reason why this category came up in Level 3.

Similarly, such improvement in the ability to use higher-order categories was observed in the Social Sciences. While the category SUMMARY was effective for Level 2,

CONTRAST and NOVELTY were effective on the students’ essay mark for Level 3. ‘Experienced’ students needed to describe tensions, contrasts between ideas, models or research directions and describe new research ideas to get higher marks.

With the exception of the Arts and Humanities discipline, the XIP categories did not influence the essay mark for Level 1 student texts. This result supports the arguments given in the literature review (Sommers & Saltz, 2004; Wingate, 2012) that newcomers to the university struggle with writing, and especially with producing essays rich in

argumentation, but develop this skill later in their studies.

However, the dataset has some drawbacks for making further interpretations. First of all, there is an inconsistency between the numbers of essays under each disciplinary grouping. Arts and Humanities and Social Sciences are relatively good datasets as they include over 200 essays, which allows statistically reliable interpretations; but Life Sciences and especially Physical Sciences are underrepresented disciplines in terms of the sample size. Especially for ensuring the accuracy of the ‘binary’ logistic regression analysis, the sample

size needs to be at least over 100 and preferably 200 because of the small-sample bias problem. Logit coefficients are biased in small samples (under about 200) (King and Zeng, 2001). Although each disciplinary grouping has over 200 essays, when level-based

considerations are made the numbers of texts decrease dramatically for Life Sciences (141) and Physical Sciences (80). The sample size especially for the Physical Sciences is

illustrative, but not definitive as it is too smalland therefore does not permit generalisations, which requires further investigation.

Additionally, the interpretations are based on disciplinary groupings and each include several sub-disciplines. Essays from each of these sub-disciplines also come from various different assignments from various institutions. The texts do not come from a single assignment and therefore do not carry similar features to be interpreted as a whole. Moreover, in all of these analyses, there is no measure to ensure that length of text is not predictive. Although statistically it would not simply mean that higher grade texts are longer, since all of these texts are highly graded either as merit or distinction, it could be the case that longer texts might contain more XIP sentences. Due to the scope of this dataset, this is discounted in the BAWE corpus, but texts are the same length in subsequent databases to overcome this possible issue. Finally, marking rubrics and learning outcomes are not available therefore instrumentation biases cannot be eliminated. Information regarding who marked these essays, using which marking guidelines, is unavailable. In short, in the BAWE corpus there are multiple assignments, genres of writing, institutions all combined, with no access to the grading criteria, or even the whole assignment task itself. Although the selection of assignment types was thoughtfully processed based on the given genre description in the BAWE manual, there is still a possibility that some of the texts written for an assignment were not graded much on argumentative writing.

The BAWE study results showed to what extent XIP can be used to identify good

indicators of academic writing when level and domain based considerations are made. The BAWE study results showed which other datasets, such as soft disciplines (i.e. Arts and Humanities, Social Sciences and Life Sciences), can be included in the study in order to make the work repeatable, reliable, and generalisable. Understanding the extent to which the results are consistent across different disciplines is a way of ensuring reliability, as the results of a study can be reproduced using a similar methodology, which means the

research instrument is considered to be reliable. The drawbacks mentioned above create a challenge in terms of the validity and reliability of the study. Therefore, the extent to which XIP can be used to identify good indicators of academic writing cannot be discussed with confidence on the basis of this dataset. However, these results strongly suggest that XIP performs differently in each discipline as expected and therefore further studies are

required with various other datasets for validity. The following three sub-sections describe these studies.

Based on the examination of the BAWE results, Physical Sciences were chosen for further analysis because it was the discipline least likely to award marks for rhetoric. This would support further investigation of XIP’s performance on hard disciplines, to find out why it did not give significant results in the BAWE study, why XIP did not work for hard knowledge disciplines. Additionally, to validate the BAWE results, soft knowledge disciplines were chosen for further analysis as they were the disciplines most likely to award marks. The studies were designed in such a way that similar datasets were analysed (e.g. two different arts and humanities datasets) so that the findings could be validated.