EXPLIQUE EL CONTEXTO DE LAS SENTENCIAS:

A situational judgement test of 45 questions was administered online in one sitting with five separate but consequent stages (each consisting of 9 questions and survey questions) to undergraduate students in large public university in the U.S. using a system/software called SONA. This system is used by many universities (and other organizations) to collect data from a pool of possible respondents like undergraduate students enrolled in a certain basic course in return for extra credit in the course. The questions were meant to assess business career-related skills; the questions were selected from a group of 174 questions developed and validated by Schmitt, Dimotakis & Quin (2007). 15 questions of three different levels of difficulty were selected based on mean scores. The easy questions had mean scores that ranged from 1.100 to 1.523 on a scale from 0 to 2 (with cumulative difficulty of 18.854) while the questions

designated as moderate had mean scores that ranged from 0.844 to 1.076 (with cumulative difficulty of 14.394) and the questions that were considered difficult had mean scores that ranged from 0.508 to 0.805 (with a cumulative difficulty of 10.936). Only items with more than 0.40 item to total correlations (which are 46 items in total so one item was removed) were selected to ensure quality of the questions (see Appendix A for an example of a question at each level of difficulty and some basic question statistics).

After every stage of the test, which for every question asked the students to identify the best and worst scenario in each situation, feedback was given to the students on their

performance; the feedback was manipulated. The students were told at the beginning of the experiment in instructions that they will receive a score from 0 to 90 with the average score being 45 after every stage of the test. It was emphasized in this note that any score above 45 is considered good and any score below 45 is considered bad. It was also mentioned that they lose points (but cannot get less than a score of zero in any stage of the test) for incorrectly specifying the best course of action or the worst or for identifying the best course of action as the worst and vice versa. Please see tables 1 to 3 in Appendix B for an illustration of what the different

feedback variables and patterns looked like when operationalized in this study; there were more than one combination of feedback pattern that represented a certain mix between valence, trend and inconsistency. So for instance as shown in Table B-2, there are two patterns that represent a

79

positive initial valence, negative trend and inconsistency, one with a positive last feedback and one with a negative last valence. This was done in order to ensure that it was not the valence of the last feedback that drove the results of the study in the case of positive and negative trend as well as inconsistency but the patterns as a whole because these variables and their effects represent the main contributions of this study.

Some of the students (138) in the sample were asked to evaluate the feedback they got (on a scale that ranged from 1 or very positive to 7 or very negative; i.e. scale was reversed) as a manipulation check and the correlation proved that on average the manipulation worked. So there was a negative 0.40 correlation between feedback evaluation and positive initial valence, and a positive 0.44 correlation between feedback evaluation and negative initial valence. Both correlation coefficients were significant at the 0.01 significance level (so p-value<0.01).

After feedback was given, students were asked to answer a set of survey questions that measured all primary outcomes, mediators and the moderators that were not expected to be stable across stages (like effort and task difficulty and difficulty increase perceptions). However, there was a short survey before the first test stage was administered that measured the stable moderators of core self-evaluations and goal orientations as well the control variable of implicit theory of intelligence. The rest of the control variables like age and gender were measured after the fifth and final stage of the test, the fifth feedback and the corresponding survey questions were administered. Performance was not measured or included in the analysis in this study.

Task difficulty was manipulated by the distribution of questions included in every stage of the test so the sample was divided into two groups: one got a test with increasing difficulty and one received the test with stable difficulty. For the respondents with increasing task

difficulty all the questions at the beginning of the experiment were easy, then in the intermediate stages the questions included in the test were selected from the easy and moderate questions, then moderate then moderate with some difficult ones, and finally, all the questions at the final stage were difficult. On the other hand, for the respondents with stable task difficulty, all stages of the experiment contained some easy questions, some moderate ones and some difficult ones (so every stage contained 3 questions of each level of difficulty). Perceptions of task difficulty were also assessed directly and both manipulated and measured forms were used as moderators in the analyses.

Task importance was also manipulated in half of the overall sample by telling the

respondents that the test they were given was a strong indicator of business career success, and a manipulation check was used to evaluate whether this manipulation worked. The note that was

used to manipulate task importance at the beginning of the survey read as follows: “Very

Important Note about the Test: This test has been found across several organizations and industries to predict with a high level of accuracy the performance and progress of the people working in management-related jobs at all levels of the organizational hierarchy”. So the experiment was designed such that half of the sample that was shown the importance note was given the test with increasing difficulty and the other half was given the test with stable difficulty; the same applies to half the sample that was now shown the importance note.

However, perceptions of task importance were also directly measured and both versions of task importance were tested separately as moderators.

In terms of sample statistics, the preliminary sample consisted of 307 respondents who completed some portion of the first part of the test and survey but only 285 completed all stages of the test and survey (with some missing data). For consistency purposes, only the data for the 285 who completed all stages will be used in the analyses. Because the control variables like age and gender were all measured at the end of the survey, it is not possible to compare the

respondents who did the survey with those who did not complete it till the end. However, in terms of initial valence and type of test (test with increasing difficulty versus one with stable difficulty), of the 22 dropout cases in stage 5, in stage 1, 8 had negative initial valence, 5 had neutral initial valence and 9 had positive initial valence. Also, of the 22 dropout cases, 12 had been given the test that increases in difficulty over stages (i.e. had easier questions in the first stage) while the rest were given the stable difficulty test (and so had questions of different levels of difficulty even in stage 1). In terms of the importance note used to manipulate task

importance, 13 cases were shown this note at the beginning of the test while the others were not. Thus, it does not seem like any specific test condition affected the dropout rate.

In terms of the basic characteristics of the final sample included in the analysis, the average age was approximately 24 (or 23.87) years old and the average GPA was 3.28. However, age ranged from 19 to 70 with a standard deviation or 6.78; GPA ranged from no GPA yet (zero or first semester) to 4 with a standard deviation of 0.49. As for years of job experience, the average was 4.89 years but the range was from 0 to 40 so very wide (standard deviation of 5.55

years). In terms of the gender composition of the sample, there were 134 males and 149 females in the sample (and 2 missing data cases). As for racial/ethnic composition, there were 101 white respondents, 95 black of African respondents and 85 of other races (with 4 missing data cases).