The Manifesto Research Group has conducted the most sustained effort to measure the ideological content of party platforms Budge et al. (2001). The only major differences between what the Manifesto Research Group does and the analysis conducted here is that the Manifesto Research Group focuses on sentence-level policy emphasis and this project examines word-usage. This section first reviews their approach to measuring platform ideology and then reports the results of an effort to replicate their approach.
Because parties so rarely adopt clearly divergent positions in their national platforms, scholars have looked elsewhere for signals of partisan differences. The “salience” the- ory of party competition developed by Robertson 1976 argues that ideological differences are communicated through emphasizing different issues, not through outright counter- argumentation. On this account, liberal parties will emphasize more left-leaning issues
and conservative parties will focus on more right-leaning policies, even if they are de- fending most of the same stances within those issues. Instead of searching for where parties adopt polar-opposite positions, they measure the balance of attention paid to characteristically liberal or conservative issues. This is specifically operationalized as the total percentage of sentences devoted to identified liberal issues minus the percentage of sentences that address conservative concerns. The Manifesto Research Group’s coding procedure is fundamentally a priori. The dimension was selected in advance and the researchers decided whether specific promises reflect a liberal or conservative orientation. The data only enter once the scheme is deployed.
Year Liberalness of Promises −30 −20 −10 0 10 20 1948 1952 1956 1960 1964 1968 1972 1972 1980 1984 1988 1992 1996 2000 2004 Democratic Platform Republican Platform
Figure 3.7: Liberalness of Platform Promises: 1948-2004
Figure 3.7 shows the estimates of ideological location produced by the Manifesto Re- search Group. This approach clearly does capture systematic and consistent differences between the two parties. Moreover, the shift by both parties in a conservative direction over the last 20 years is substantively sensible. The Manifesto Research Group’s approach is not without merit, but their method does not extend well into the murky terrain of word-usage.
The Manifesto Research Group method was replicated step-by-step using word-usage data. First, words were selected that could plausibly be assumed to be either liberal or conservative. These list are not reported, but several attempts were made to select words that have clear ideological overtones. Once these lists were populated, the cumulative attention devoted to each group was calculated for each document. The final measure of ideological location was given by subtracting the total salience of conservative words from the total salience of liberal terms. This technique was applied to party platforms and nomination acceptance speeches from 1940 to 2004.
Year Liberalness of Words −0.015 −0.010 −0.005 0.000 1940 1944 1948 1952 1956 1960 1964 1968 1972 1972 1980 1984 1988 1992 1996 2000 2004 Democratic Platform Republican Platform
Figure 3.8: Liberalness of Platform Language: 1940-2004
First, the results of applying the Manifesto Research Group’s a priori method to the platform word-usage. Figure 3.8 reports the estimates of ideological location for each party’s platform from 1940 to 2004. The effort does not produce completely senseless results. Beginning in the early 1970’s, consistent divisions emerge that only disappear in 2004. Still, these data are much more noisy than is desirable. The two parties cross over each other in several elections and the estimates of the Republican party oscillate significantly from 1940 to 1980. For reasons already discussed, it is clear that we cannot
produce crisp estimates of ideological location by decidinga priori what words are liberal and conservative. Year Liberalness of Words −0.015 −0.010 −0.005 0.000 0.005 1940 1944 1948 1952 1956 1960 1964 1968 1972 1972 1980 1984 1988 1992 1996 2000 2004 Democratic Acceptance Republican Acceptance
Figure 3.9: Liberalness of Acceptance Language: 1940-2004
Applying the Manifesto Research Group method to word-usage in nomination accep- tance speeches also fails to produce satisfying results. Here, there appear to be very little differences in ideological tone between the Democratic and Republican nominees in most election cycles. Once again, we see the two parties moving together over time, but few sustained differences between them. The trend toward more conservative content is present here, as was found in the original Manifesto Research Group data and the pre- ceding analysis of platform word-usage, so there is a consistent pattern that is plausibly ideological.
Conclusion
Existing methods of analyzing language in political science can only take us so far. First, we need to rethink the importance of ideology in how politicians articulate themselves. Language is a dynamically complex phenomenon and we cannot boil it down to a single
dimension without losing a huge amount of what is happening. Secondly, we need to allow the data to tell us where important patterns exist. Particularly when studying language over a long time period, we cannot simply assume that liberty is a conservative word, or that education is a liberal issue, or that putting more cops on the beat is a conservative concern. The ideological or partisan meaning of symbols depends on how they have been used, so we cannot assume that a given word carries the same associations over time. In the dynamic world of political rhetoric, symbols are often appropriated, become broadly salient following events, make for good copy in new media, or disappear altogether. Therefore, it is wise to give the data room to settle the question of what counts as liberal or conservative content.
This is not an argument against studying ideology, or against using theory to design our measures. The conventional wisdom in computer-aided content analysis is to expose the data to different analytical techniques become each method has strengths and limitations. Along the way to settling on the techniques used in the subsequent chapters, a great deal of experimenting took place. The results presented in this chapter are only a small fraction of the failed or partially successful efforts made during this project. The arguments against radically reducing the dimensionality of political language, and against focusing on left-right ideology alone, are born of experience trying to do both. While these remain important goals, we need to be realistic about the prospects of success and what is being left out.
This chapter reviews why the traditional mixture of data and empiricism is ill-suited to word-count data, but does so with an eye for how these two sides of good measurement can interact productively. Chapter 4 chooses a specific dimension but allows the meaning of individual words within that space to be defined by the data. Chapters 6 and 7 impose no dimensional assumptions, but theoretically-informed choices are made about which symbols to analyze. Computer-Aided Content Analysis is still new to our discipline and this chapter records the struggle to address specific theoretical questions while not imposing unreasonable expectations on the world.