Figure 4 shows the SSA scalogram of the 10 items of the TIPI (with negative items reversed), displaying the facet structure for the Big Five traits in four random subsets (A, B, C, and D) with a total of 100 subjects each.
In Figure 4, all four scalograms seem to show a radex pattern like the one in Figure 3, i.e., the combination of a modular patter with the positive items on the inside and the negative ones outside, plus a polar one with the five personality traits. There are, however, some relevant differences between the results for the subsets and the findings for the whole sample, as well as between the subsets themselves.
In scalograms B and D, a radex pattern consistent with five personality traits with positive and negative items can be obtained by simply drawing straight lines and ellipses, whereas scalograms A and D require significantly distorted lines to achieve the same structure.
Furthermore, the specific clockwise order of the items in the polar structure has also varied substantially in the different subsets. Indeed, the orders obtained for each scalogram were:
• Scalogram A: Stability - Agreeableness - Extroversion - Openness Conscientiousness;
• Scalogram B: Stability - Conscientiousness - Openness - Extroversion – Agreeableness;
• Scalogram C: Stability - Conscientiousness - Openness - Extroversion – Agreeableness;
• Scalogram D: Stability - Agreeableness - Conscientiousness - Openness – Extroversion.
Only Scalogram A has exactly the same order of items as the analysis in Figure 3 (the one found for the whole sample).
6. Discussion
The SSA findings and Facet Theory interpretations of the radex structure of the TIPI items for the entire sample of 1679 subjects are completely consistent with the theoretical expectations from Gosling, Rentfrow, and Swann Jr. (2003). This includes the relationship between the five personality traits (Agreeableness being close to Stability and Extroversion, but not so much to Openness and Conscientiousness, etc.).
In the four subsets of data analyzed, radex structures were easily discernible in two of them, but, in the rest, significant allowance had to be given to the definition of the polar facets in order to conform to the predicted structure. The degree of distortion is such that, if an exploratory analysis was done, as opposed to a confirmatory one (i.e., if one was looking for clues to an unknown structure rather than evidence for a known one), it is not difficult at all to imagine the possibility of arriving at different structures for the two pairs of scalograms. Indeed, even in the confirmatory case, in half of the scalograms obtained, one could raise the argument that, rigorously speaking, the structure being sought was not actually found.
Even if one assumes that the polar structure for the personality traits which was identified in the complete sample has effectively been confirmed in all four data subsets, there are still issues with the identification of the relationship between such traits. Only one of the scalograms from the subsamples displayed a clockwise order of personality traits that corresponds exactly to the findings from the whole sample and to the theoretical expectations from the authors of the test. Two of the other three subsets were identical to each other as to the clockwise order of the traits, but they were both different from what was found for the whole sample, and the remaining subset had its own unique distribution of traits. This means that different relationships between personality traits that were found in the present according to the sample/subsample analyzed.
Given all the evidence from the present study, it seems that the results of the application of SSA and Facet Theory to the investigation of the structure of the items of a psychometric test can vary substantially depending on sample size, with relevant implications for the scientific interpretation of the phenomenon underlying the findings.
7. Conclusions
The present paper made the argument that Facet Theory lacks a specific treatment of uncertainty which, from a mathematical, statistical, and scientific point of view, is a serious gap that can lead to erroneous results in research efforts. An investigation was then done to test this notion in the context of the empirical evaluation of the structure of a personality test in a large sample of Brazilian adults and in much smaller subsamples.
The findings obtained showed that, as a consequence of sample size and/or random variations between different subsets of data, relevant differences in the structure of the variables studied were found. Such differences were large enough to alter the relationships identified between the personality traits measured by the test or even the interpretation of what are the traits that the test measures.
It is concluded that the evidence obtained in the present investigation confirms the notion that the absence of an adequate way to deal with uncertainty in SSA and Facet Theory can lead to significant errors when such approaches are used in actual research. Thus, future developments in the field must aim to provide new methods and tools capable of quantifying the reliability and precision of their findings. Without addressing this important point, it will be difficult to view its results without some level of suspicion.
References
Borg, I. & Shye, S. (1995). Facet Theory: Form and Content. Newbury Park, California: Sage.
Canter, D. (1985). How to be a Facet Researcher in Facet Theory: Approaches to Social Research (Ed.) D. Canter. New York: Springer-Verlag, 265-275. Canter, David V. (1996) The Potential of Facet Theory for Applied Social
Psychology. In: Psychology in Action. Dartmouth Benchmark Series . Dartmouth Publishing Company, Hantshire, UK, pp. 39-70. ISBN 1855213656.
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment Research & Evaluation. 10(7). In the Internet at: http://pareonline.net/pdf/v10n7a.pdf.
Dancer, L.S. (2008). Introduction to Facet Theory and its Applications. Applied Psychology. 39(4), 365-377. DOI:10.1111/j.1464-0597.1990. tb01061.x Foster, J.J., Barkus, E., and Yavorsky (2005). Understanding and Using
Advanced Statistics: A Practical Guide for Students. London: Sage. Gosling, S. D., Rentfrow, P. J., & Swann, W. B., Jr. (2003). A Very Brief
Measure of the Big Five Personality Domains. Journal of Research in Personality, 37, 504-528.
Gray, J.M. and Wilson, M.A. (2007). A detailed analysis of the reliability and validity of the sensation seeking scale in a UK sample. Personality and Individual Differences, 42(4), 641-651.
Greenbaum, C.W. (1998). The Past and Present Future of Facet Theory and Related Approaches to Data Analysis in the Social Sciences. In the Internet at: www.facet-theory.org/files/wordocs/1Greenbaum.pdf. Guttman, R. and Greenbaum, C.W. (1998). Facet Theory: Its Developmental
and Current Status. European Psychologist, 3(1), 13-36.
Hackett. P.M.W. (1993). “Consumers’ Environmental Concern Values: Understanding the Structure of Contemporary Green Worldviews”. In: E-European Advances in Consumer Research Volume 1, eds. W. Fred Van Raaij and Gary J. Bamossy, Provo, UT : Association for Consumer Research, Pages: 416-427.
Hair, J.F., Black, W.C., Babib, B.J., and Anderson, R.E. (2009). Multivariate Data Analysis (7th Edition). Prentice Hall. ISBN-10: 0138132631. ISBN-13: 978-0138132637.
Levy, S. (2005). Guttman, Louis. Encyclopedia of Social Measurement. Volume 2. Amsterdam: Elsevier Inc.
Lula, A.M. (2011). Impressions Management in Organizations: Validation of the Bolino and Turnley Scale (1999) for Brazilian Portuguese. Master’s Dissertation, 184 pages. Graduate Program in Business Administration, Federal University of Pernambuco.
Shye, S. (1998). Modern Facet Theory: Content Design and Measurement in Behavioral Research. European Journal of Psychological Assessment. 14, 2, 160-171.
Shye, S., Elizur, D., and Hoffman, M. (1994). Introduction to Facet Theory. Applied Social research methods Serie, Volume 35. London: Sage.