T
he Structured Analogies technique applies increased rigor to analogical reasoning by requiring that the issue of concern be compared systematically with multiple analogies rather than with a single analogy.“One of the most widely used tools in intelligence analysis is the analogy. Analogies serve as the basis for constructing many predictive models, are the basis for most hypotheses, and rightly or wrongly, underlie many generalizations about what the other side will do and how they will go about doing it.”
—Jerome K. Clauser and Sandra M. Weir, Intelligence Research Methodology, defense intelligence School (1975).
When to Use It
It seems natural to use analogies when making judgments or forecasts because, by definition, they contain information about what has happened in similar situations in the past. People do this in their daily lives, and analysts do it in their role as intelligence analysts. People recognize similar situations or patterns and then consciously take actions that were successful in a previous experience or avoid actions that previously were unsuccessful. People often turn to analogical reasoning in unfamiliar or uncertain situations where the available information is inadequate for any other approach.
An analogy involves a perception that two things are similar and a judgment that since they are similar in one way they are likely to be similar in other analytically relevant ways. Analysts may observe that a new military aircraft has several features that are similar to an existing aircraft and conclude that the new aircraft has been designed for similar missions. Examples of analogies on a larger scale and with a more tenuous basis include Vietnam as a reason for not undertaking military action in Iraq, and the successful occupation of Germany and Japan after World War II as a reason for believing that military occupation of Iraq will be successful. History records many analogies that have led to bad decisions as well as good decisions.
When one is making any analogy, it is important to think about more than just the similarities. It is also necessary to consider those conditions, qualities, or circumstances that are dissimilar between the two phenomena. This should be standard practice in all reasoning by analogy and especially in those cases when one cannot afford to be wrong.
Many analogies are used loosely and have a broad impact on the thinking of both decision makers and the public at large. One role for analysis is to take analogies that are already being used by others, and that are having an impact, and then subjecting these analogies to rigorous examination.
We recommend that analysts considering the use of this technique read Richard D. Neustadt and Ernest R. May, “Unreasoning from Analogies,” chapter 4, in Thinking in Time: The Uses of History for Decision Makers (New York: Free Press, 1986). Also recommended is Giovanni Gavetti and Jan W. Rivkin, “How Strategists Really Think: Tapping the Power of Analogy,” Harvard Business
Review (April 2005).
Value Added
Reasoning by analogy helps achieve understanding by reducing the unfamiliar to the familiar. In the
absence of data required for a full understanding of the current situation, reasoning by analogy may be the only alternative. If this approach is taken, however, one should be aware of the significant
potential for error, and the analyst should reduce the potential for error to the extent possible through the use of the Structured Analogies technique.
“When resorting to an analogy, [people] tend to seize upon the first that comes to mind. They do not research more widely. Nor do they pause to analyze the case, test its fitness, or even ask in what ways it might be misleading.”
—Ernest R. May, Lessons of the Past:
The Use and Misuse of History in American Foreign Policy (1975).
The benefit of the Structured Analogies technique is that it avoids the tendency to fasten quickly on a single analogy and then focus only on evidence that supports the similarity of that analogy.
Analysts should take into account the time required for this structured approach and may choose to use it only when the cost of being wrong is high.
Structured Analogies is one technique for which there has been an empirical study of its effectiveness. A series of experiments compared Structured Analogies with unaided judgments in predicting the decisions made in eight conflict situations. These were difficult forecasting problems, and the 32 percent accuracy of unaided experts was only slightly better than chance. In contrast, 46 percent of the forecasts made by using the Structured Analogies process described here were
accurate. Among experts who were independently able to think of two or more analogies and who had direct experience with their closest analogy, 60 percent of the forecasts were accurate. (See “Origins of This Technique.”)
Method
Training in this technique is recommended prior to using it. A self-directed training module is available online at no cost at the International Institute of Forecasters Web site
(http://forecastingprinciples.com/practitioners.htm). The Structured Analogies course is listed under Courses for Forecasting.
The following is a step-by-step description of this technique.
* Describe the issue and the judgment or decision that needs to be made.
* Identify a group of experts who are familiar with the problem and who also have a broad background that enables them to identify analogous situations. The more varied the backgrounds the better. There should usually be at least five experts.
* Ask the group of experts to identify as many analogies as possible without focusing too strongly on how similar they are to the current situation. Various universities and international organizations maintain databases to facilitate this type of research. For example, the Massachusetts Institute of Technology (MIT) maintains its Cascon System for Analyzing International Conflict, a database of 85 post–World War II conflicts that are categorized and coded to facilitate their
comparison with current conflicts of interest. The University of Maryland maintains the International
Crisis Behavior Project database covering 452 international crises between 1918 and 2006. Each case is coded for eighty-one descriptive variables.
* Review the list of potential analogies and agree on which ones should be examined further.
* Develop a tentative list of categories for comparing the analogies to determine which analogy is closest to the issue in question. For example, the MIT conflict database codes each case according to the following broad categories as well as finer subcategories: previous or general relations between sides, great power and allied involvement, external relations generally, military-strategic, international organization (UN, legal, public opinion), ethnic (refugees, minorities), economic/resources, internal politics of the sides, communication and information, actions in disputed area.
* Write up an account of each selected analogy, with equal focus on those aspects of the analogy that are similar and those that are different. The task of writing accounts of all the analogies should be divided up among the experts. Each account can be posted on a wiki where each member of the group can read and comment on them.
* Review the tentative list of categories for comparing the analogous situations to make sure they are still appropriate. Then ask each expert to rate the similarity of each analogy to the issue of concern. The experts should do the rating in private using a scale from 0 to 10, where 0 = not at all similar, 5 = somewhat similar, and 10 = very similar.
* After combining the ratings to calculate an average rating for each analogy, discuss the results and make a forecast for the current issue of concern. This will usually be the same as the outcome of the most similar analogy. Alternatively, identify several possible outcomes, or scenarios, based on the diverse outcomes of analogous situations. Then use the analogous cases to identify drivers or policy actions that might influence the outcome of the current situation.
Origins of This Technique
This technique is described in greater detail in Kesten C. Green and J. Scott Armstrong, “Structured Analogies for Forecasting,” in Principles of Forecasting: A Handbook for Researchers and
Practitioners, ed. J. Scott Armstrong (New York: Springer Science+Business Media, 2001), and www.forecastingprinciples.com/paper
pdf/Structured_Analogies.pdf.