Skip to main content

Full text of "DTIC ADA001807: Hindsight: Thinking Backward"

See other formats


AD / A - 001 807 


Baruch Fischhoff 

Oregon Research Institute 

Prepared for: 

Office of Naval Research 
Advanced Research Projects Agency 

4 November 1974 


National Technical Information Service 

’£' •< 'X 

by the Advanced Research Projects Agency of the 
rder No. 2449) end was monitored by ONR under 
Cohttact Mo N00014-73-C-0438 (NR 197-026). 

TSBonV technical 
information service 

U S Department of Commerce 


Baruch Fischhoff 


ONR Technical Report 


Ajt/A- OO) foy 












Oregon Research Institute 
Eugene, Oregon 

Unclassi f led 

Hindsight: Thinking Rackward? 

• • ’ i >•■ > > i 

Technical Report 

Raruch Fischhoff 

November 4 1974 

Nonoi/ ( -7vr-n4i8 

NR (197-026) 

ARPA Order No. 2449 

i f i r ■ i i ’ i I 

Oregon Research Institute Research 
Monograph Vol. 14, No. ] 

' i * " K t » "«!». ■ 1 J /ir.i . \ . T f. .../•• f | , r n„ij I. 

This document has been approved for public release and sale. Its 
distribution is unlimited. 












1 .* i '• J I **U. !.• . I I Al« > K II t'l U 

Office of Naval Research 
Code 455 

ARlington, Virginia 22217 

Is lindsight better than foresight or just different? The formal 
difference between the two tasks is the hindsightful judge's possession 
of outcome knowledge, telling him how things turnei out. It is this 
additional knowledge which is reputed to confer the "wisdom of hindsight". 

Tn the studies reported here, outcome knowledge is found to increase the 
perceived Inevitability of the outcome reported. Judges are, however, 
largely unnwnr of the changes in their perceptions due to outcome knowledge. 
As a result, tuey believe that they and others had in foresight insights 
which thev themselves only had as a result of outcome knowledge. Failure 
to appreciate the effects of outcome knowledge can seriously prejudice the 
evaluation of decisions made in the past and limit what is learned from 
experience . 


di :> ,o " 

Ui 73 

(I a: : 1) 

Si'i mil V C'l.i:.' ill. .itui 

IM / 





Hindsight: Thinking Backward? 

Baruch Fischhoff 
Hebrew University of Jerusalem 
Oregon Research Institute 




Hindsight: Thinking Backward? 

Baruch Fischhoff 
Hebrew University of Jerusalem 
Oregon Research Institute 

In tie spring of 1974, Carl Cletus Bowler, "convicted murderer, 
bank robber and all-around bad actor" (Euge ne Register Guard. June 5, 1974, p. 12A) 
was released on an overnight social pass from the Oregon State Penitentiary 
where he was serving a life sentence. The social pass program had been 
initiated by the Oregon state legislature to help convicts maintain contact 
with the outside world so as to make it easier for them to eventually 
rejoin it. Bowles, who had been a model prisoner, promptly absconded. 

Warden Cupp, who bore ultimate responsibility for all passes issued 

in the penitentiary, came under heavy fire. According to a local newspaper, 

'It was an error in screening. Bowles' record, in the prison and out, as 
we read it now, shows that he could not be trusted." (ibid. ) The state's 
governor informed Warden Cupp that be would have to resign if Bowles 
committed violence while at large. 

Following the Japanese attack on Pearl Harbor, the Naval C. 0 ., 

Admiral Kimmel was removed from his command and lowered in rank. A Congres- 
sional investigatory committee convened in 1946 produced 39 volumes of 
evidence, much of it highly critical of those responsible for Pearl Harbor's 

security. Irving Janis (1972) blames "collective groupthink (pressures 
to conformity) among interlocking groups ... for America's astounding 
unreadiness at Pearl Harbor," after noting the success of American 
intelligence in deciphering the Japanese secret codes known as MAGIC, (p. 99 ). 



In accepting President Ford's pardon, former President Nixon indicated 
that he could see now how he should have understood the meaning of Watergate 
in July of 1972. He regretted his misunderstanding and subsequent failure 
to take suitable action. 

The structure of each of these examples is similar: misfortune occurs, 

a culprit is identified, his decision second-guessed, 
and his folly chastized either bv others or by himself. Wiry are such 
critics in a position to criticize? Because they know how things turned out. 
Their hindsight they feel, provides them with wisdom illuminating the 
follies of others, or even of themselves. Or does it? Would we, could we, 
or should we have known better than Warden Cupp, Admiral Kimmel, President 
Nixon, or would we do better now if placed in their shoes? 

Is hindsight really better than foresight, or just different? How 
aware are we of these differences? Do we, as Nietzsche suggests, begin by 
looking backward and end up thinking backward? Our awareness of these 
differences in large part determines both how much we le c rom experience 
and how we mete out justice. In Admiral Kimmel's case, for example, if he 
saw or could have seen what was happening, but didn't take action accordingly, 
then he was clear lv negligent, incompetent, or worse. Our own lesson from 
his experience is to find better officers and teach him a lesson. If, 
however, the imminence of a Japanese attack is apparent only in retrospect, 
then Kimmel deserves either a fairer shake than he got, or else conviction 
for a rather different crime — failure to be ready for any and all possible 
surprises. In this particular case, with the limited resources at his 
disposal, he may well have done the best he could. Our lesson is, then, to 
spend more on eliminating surprises and/or being readier for them when they 


II l mis ight 


My colleagues and T have recently conducted a series of experiments 
designed to obtain an understanding of the differences between hindsight 
and foresight, and to better understand how to get the most out of each. 
Supposedly, the sharper and surer our hindsight, the better our foresight. 
Our results indicate that this need not be the case. 

The first experiment in the series was designed sinply to see how 
foresight and hindsight differed in a number of aspects. In it, subjects 
read a short, unfamiliar historical passage, such as the following: 

For some years after the arrival of Hastings at, governor-general 
of Lndia, the consolidation of British power involved serious war. The 
first of these wars took place on the northern frontier of Bengal 
where the British were faced by the plundering raids of the Gurkas 
of Nepal. Attempts had been made to stop the raids by an exchange of 
lands, but the Gurkas would not give up their claims to country under 
British control, and Hastings decided to deal with them once and for 
all. The campaign began in November, 1814. It was not glorious. The 
Gurkas were only some 12,000 strong; but they were brave fighters, 
fighting in territory well suited to their raiding tactics. The older 
British commanders were used to war in the plains where the enemy 
ran away from a resolute attack. In the mountains of Nepal it was 
not easy even to find the enemy. The troops and transport animals 
suffered f rom the extremes of heat and cold, and the officers learned 
caution only after sharp reverses. Major-General Sir D. Ochterlony was 
the one commander to escape from these minor defeats. 

The Age of Refor m 
by r.. L. Woodward 
Oxford, 1937, pp. 383-4 

live groups of subjects received each story in varying versions. There 
was one "foresight" group which was not told what had happened subsequently 
(the British won). There were four "hindsight" groups each of which was 
given the same storv with one of several possible outcomes appended to it 
as the "true" outcome. These were, in this case: 

British victory; Gurka 



victory; military stalemate with a peace treaty; military stalemate without 
a peace treaty. Thus, for three of the four hindsight groups, the reported 
outcome was in fact false. Note that even though the event took place a 
century and a half ago, foresight subjects not to’d what had happened are 
in much the same position as contemporary observers of the British-Gurka 

All subjects were asked to: 1) judge the probability that each of the 

four possible outcomes was going to happen; 2) evaluate the relevance of 
each of the facts appearing in the story in determining what happened; 
and 3) give reasons for their answers. 

The most dramatic result was that hindsight subjects consistently 
perceived reported outcomes (whether true or not in fact) as having been 
more likely to occur than did their foresightful counterparts; knowing that 
something had happened roughly doubled the perceived odds that it was going 
to occur. Interestingly, however, subjects almost never assigned 100% 
probability to what was reported to have happened. Evidently, they felt 
that in the light of the facts given in the description other outcomes were 
still possible (e.g., "The Gurkos had a 70% chance of winning, but the British 
still might have pulled it off."). 

^e perceived relevance of the facts in the description and the reasons 
which subjects offered to justify their answers depended, as well, upon 
the outcome reported. Interestingly, though, there were some data 
(e.g.. Hasting's decision to deal with the Ourkas "once and for all") and 
some reasons (e.g., "the experience of Viet-Nnm" or "what's going on in the 
Middle East today") which wore relevant no matter what happened. 


II i mis i ght 

Knowing that something has happened clee-ly increases its perceived 
inevitability, as well as restructures our perceptions of what we know 
about It. How justified are these changes? Tt's hard to say, simply because 
no one knows the objective probabilities associated with unique events like 
the Bri tish-Ourka struggle. Consider another example: Tf we claim that 

there was no chance (or a 77, chance or a 98.67 chance) of a thermonuclear 
war during the I960' s who's to prove us wrong? Indeed, the only wrong 
estimate is that it was 1007 likely. 

Let us call the tendency to see whatever is reported to have happened 
as having been relatively inevitable "creeping determinism" — in contrast 
with philosophical determinism, the conscious belief that whatever happens 
has to happen. These two types of determinism are essentially independent. 

The philosophical determinist may believe that a reported event (e.g., Bowles' 
escape) was inevitable, since whatever happens is, by definition, inevitable. 
He may, however, still be surprised by its occurence. Indeed, he may well 
set for himself the task of researching the situation until its inevitable 
character becomes apparent. He might, thus, insist that Bowles was bound 
to escape, yet deny that anyone was in a position to foresee it. The creeping 
determinist may well be totally unaware of the raging debate over historical 
inevitabili tv and free will. He perceives reported events as having been 
more or less bound to happen, simply as a matter of course. 

Our second experiment was designed to determine how aware judges are 
of the hindsight-foresight differences which we have called "creeping 
determinism". The importance of such awareness for the historical second- 


guesser has boon discussed already; its importance for anyone trying to 
learn from history is discussed below. 

Each group of subjects in Experiment ? received stories like the Rritish- 
(.urka episode, each with one of the four endings appended to it. Thus, subjects 
in each group knew as much as did subjects in the comparable hindsight 
group in Experiment !. They were asked to complete the Experiment 1 
questionnaire as they wouln have "had they noc known what really happened", 
i.e., like foresight subjects. ’People aware of what they have learned from 
knowing what happened should be able to perform this task successfully. 

Our subjects* success was limited at best. They consistently believed 
that without outcome knowledge they would have displayed the creeping determinism 
shown by Experiment 1 subjects with outcome knowledge. They believed that 
had thev been asked, they would have seen the relative inevitability of 
the reported outcomes. Their reconstructed relevance judgments, too, bore 
the imprint of the outcome knowledge they were to ignore. For example, 
foresight subjects in Experiment 1 attached relatively little importance 
to British suffering from "extremes of heat and cold." Experiment 1 hindsight 
subjects told that the Gurkas had won, attached greatest relevance to this fact. 
Experiment 2 subjects told to ignore the r cport of Gurka victory indicated 

that even without that report they would have perceived the importance of 
climatic extremes. 

failure to ignore outcome knowledge is not without its benefits. It is, 
indeed, quite flattering to believe, or lead others to believe, that we 
would have "known all along" what we could only know with outcome knowledge. 



III nds I >* li L 


that is to say, that we possess hindsightful foresight. 

Returning to the introductory examples, perhaps this failure to empathize 
with ourselves in a more ignorant state is paralleled by a failure to empathize 
with outcome-ignorant others. Experiment 3 examined this question. In it, 
five groups of subjects were asked to respond to the questionnaire used 
in Experiments 1 and 2 — as thev thought other subjects, similar to themselves, 
had responded. These other subjects were described as not knowing the 
"true" outcome. One of the Experiment 3 groups was equally ignorant of what 
had happened. The remaining groups each received one of the possible outcomes 
as the "true" outcome. They were essentially asked to respond like foresightful 
others, more ignorant than themselves. 

Subjects who did not know what had happened believed that outcome- 
ignorant others would respond much the same way as foresight subjects in 
Experiment 1. Since this is presumably the way they themselves would respond, 
they seem to have projected their own judgments on others. Experiment 3 
subjects who did know what had happened, however, attributed to outcome-less 
others probabilitv and relevance judgments which clearly bore the imprint 
of the outcome knowledge which they were asked to ignore. In particular, 
thev believed that others in foresight would have seen the relative inevita- 
bility of reported outcomes which they themselves perceived only in hindsight. 

As before, this result was obtained for true and false outcome reports alike. 

In a fourth study, (Fischhoff & Beyth , 1974) subjects were asked oti the 
eve of then-President Nixon’s trips to China and the USSR in 1972 to estimate 
the probability of various possible outcomes of the trips (e.g., meeting 



Chairman Mao, visiting Lenin's tomb, announcing that the trips were successful). 
Two weeks to six months after the trips' completion, these same subjects were 
asked to remember as best they could, their own original predictions. Finally, 
they were asked to indicate for each event whether or not they believed 
that it had actually happened. 

The results showed that subjects remembered having given higher probabi- 
lities than they actually had to events believed to have occurred and lower 
probabilities to events which hadn't. Their original predictions showed that 
they were too often surprised — many highly unlikely or impossible events 
(assigned probability = 0%) did occur. Their remembered probabilities, 
however, indicated that they perceived a past which held too few surprises 
for them; indeed, almost no events which they remembered assigning low proba- 
bilities to were perceived to have occurred. 

Summarizing these results: Finding out that something has happened 

increases its perceived inevitability. We are unaware, however, of this 
effect of outcome knowledge and tend to believe that the inevitability was 
largely apparent in foresight, without the benefit of knowing what happened. 

This undiagnosed creeping determinism biases our impression of what we would 
have known without outcome knowledge (Exp. 2), as well as our impressions 
of what we ourselves (Exp. 4), and others (Exp. 3), actually did know in 
foresight. In retrospect, we tend to believe that we and others had a much 
better idea of what was going to happen than we (or others) did. 

How do we do this? How do we manage to see the relative inevitability 
of whatever is reported to have happened, true or false, yet remain unaware 




of the effect which outcome knowledge has had on our perceptions? The data 
relevance results provide an important cue. In particular they suggest 
that the data in the event description change their meaning or significance 
when different outcomes are reported. Those data which were highly 
relevant whatever happened, for example, must have meant something different 
in each context: just as Bowles' good behavior in prison would have meant 
one thing had he returned from his social pass and actually meant another 
when he did not. In either case, it was a highly relevant datum. 

What kind of meaning adjustment goes on? Two related interpretations 
seem worth considering. The first is that we are biased to view whatever 
happen? as being inevitable, and then juggle or manipulate whatever else 
we know to concur with that feeling of inevitability. For example, upon 
hearing of Bowles' escape we say, "That was bound to happen." and then go 
about figuring out why. Looking at his record, we may highlight details 
pointing to his errant character and reinterpret other details which are 
inherently ambiguous. Critics of deviance labelling such as Lofland (1969), 
Shur (1971), or Rosenhan (1973), have suggested that just such a process 
goes on when the public and professionals rework or reinterpret the 
biographies of deviants to show that their labels are inevitable products of 
their life histories. Doggedly and professionally pursued, such reconstruc- 
tion can find cause for the continued incarceration of even perfectly "normal 
patients who have had themselves committed just to see how hospitals operate 
from the insii'e. 

The alternative explanation proceeds in the opposite direction. It 


suggests that when we receive outcome knowledge, we immediately make sense 
out of it by integrating it into what we already know about the subject. 



Having made this reinterpretation, the reported outcome now seems a more 
nr less inevitable outgrowth of the reinterpreted situation. "Making sense" 
out of what we're told about the past is, in turn, so natural that we may 
well be unaware o c outcome knowledge having had any effect on us. Even 
if we are aware of there having been an effect, we may still be unaware 
of exactly what it was. In trying to reconstruct our foresightful state 
of mind, much evidence suggests that we will remain "anchored" or rooted 
in our hindsight ful perspect ive— leaving the reported outcome too likely 
looking. Both processes may, of course, be operative. 

The negative effects of unperceived creeping determinism are probably 
apparent by now. When we second-guess Warden Cupp's decision, for example, 
our natural tendency seems to be to see Bowles’ escape as having appeared 
more likely at that time than it really did seem. In this light, the warden's 
decision to issue the social pass seems like an act of irresponsibility or 
just plain incompetence— whereas in truth, the probability of escape may 
have justifiably seemed very small to him (judging by the governor's comment, 
even after the escape the probability that he would commit violence while 
at large was still unclear). All the signs in Bowles' "record in the 
prison and out, as we read it now (which) show that he could not be trusted" 
may well have meant something very different before his escape (e.g., his 
good behavior in jail). The fact that he escaped does not in itself mean 
that the decision to release him was a bad one. Good decisions, those which 
maximally utilize all available knowledge, may have bad outcomes. The informa- 
tion about Bowles' true nature which would have led to a better decision 
either may or may not have been available when the warden issued the 

pass . 

pi *■»**-’ 



The point is that from our perspective, it is very herd to tell and therefore 
dangerous to rely on our intuitive (retrospective) impressions. 

Aside from making us unduly harsh in judging decisions made in the 
past, unperceived creeping determinism may also make us insensitive to what 

is to be learned from the past. Tf the description above is accurate, then 
Warden Cupp's lesson from this harrowing experience (Bowles eventually 
kidnapped and probably murdered an Oregon couple) seems to be "don't issue 
passes to prisoners vou can see are going to escape." It's hard to see how 
that kind of advice is going to advance anyone's capabilities. 

As with Pearl Harbor, there is an alternative lesson to be learned. It 
is that the issuing of social passes ;s a risky business, that the predicta- 
bility of temporarily released prisoners' behavior is far from perfect, and 
that the system should be redesigned to accommodate or reduce this uncertainty 
(e.g., denying all passes, statistically identifying risky prisoners, or 
explaining to the public the risks and benefits of the program in order to 
make them understand and assume some responsibility for its successes and 
failures) . 

I f we look at the past and find that it holds few surprises for us, 
we are essentially denying that we have anything to learn from it. Even 
though outcome knowledge changes our perceptions of specific events (by 
making them seem inevitable), without a feeling of surprise we probably 


feel little compunction to reevaluate the "world hypotheses" or rules with 


which we interpret what goes on around us. It is conscious refining of these 
hypotheses which improves our ability to uncerstand our past, present and 

II ! iifls ip, lit 


future. We generally believe that being able to explain or make sense of 
the past increases our ability to predict the future. If what we call 
explaining the past is actually "explaining away" the surprises it holds, 
the very opposite may occur. A surprise-free past may well portend a surprise- 
full future. 

What can we do to make our hindsight more insightful? The most basic 
bit of advice is to accept the existence of uncertainty in historical 
judgments. Even though people don't much like dealing with uncertainty, 
they will usual lv acknowledge its presence in their understanding of the 
present and future. If they don't, events will sooner or later prove their 
fallibility. The past, however, is less able to produce surprises which 
can show up the know-it-all. As a result of its defenselessness, it may 
have deterministic schemata imposed on it that would appear as sheer effrontery 
if imposed on the present or future. The uncertainty is there, however, 
and will remain as long as there is any question about the meaning, reliability, 
or validity of any of the components of our historical knowledge, the 
facts we presumably know about the past, the facts we need to know about 
the past and don't~‘ ,n d must surmise from what we do know — or tne explanatory 
principles (rules) with which we make a coherent whole out of what we know. 

Simply doubting what you believe you kn jw about the past may produce 
some quick benefits. Consider the responses of one subject in Experiment 1 
who was told that the British had been defeated by the Gurkas. She justified 
giving a higher retrospective probability to a Gurka victory by citing the 
"fact" that "in 1812 (the time of the Gurka campaign), the British were also 
defeated by the United States in the War of 1812." The reader may remember that 


r> ■ 



in that war, the British managed to burn Washington, D.C., and conquer 
I ort Detroit by the mere presence of their forces 120 miles away in London, 

Ontario; and that thev only broke off the engagement (i.o., lost the war) 
in 1814 when called away by -vents in Europe. Had she known of the pitfalls 
of ( reeping determinism, she might have adopted a more critical attitude 
toward her ou’u information. 

Similarly, one might make it a habit to ask himself nuestions like, 

"Did ! really give the Donalds' marriage only a one in ten chance of lasting 
when they got married?" or "Was I really certain that Mao and Nixon would 
get together?" Most investigatory committees would also do well by asking 

first not who erred and why, but was there an error at all, i.e. , could anyone else 

on the scene conceivably have known what was going on and acted more 


After acknowledging the existence of uncertainty in the past, a good 
practice might he to try to hunt it down in its original form. For example, 
are there records of the deliberations which preceded the decision to issue 
Bowles the infamous social pass; are there transcripts of the information 
reaching Admiral Kimrnel prior to 7 AM on December 7; is there a notebook 
showing the stocks you considered before settling on Waltham Industries; 
are there diaries capturing Stalin's (or Chamberlain's) view of Hitler in 
1919? Any of these records might show the difficulty of the decisions 
facing these actors. If you have hopes of learning how to forecast better, 
it might well pay to keep a written record of your own predictions and the 
considerations which guided them — a record which can be evaluated in the light 
of reality (what eventually happens). It can't hurt to know when you knew 




and when you didn't know aJ 1 along what would happen and why. 

Such fossilized deliberations are, of course, quite rare. When there 
is no uncertain past to be uncovered, it must be reconstructed. Cne 
simplistic remedy which iray have some value is to effect an across-the-board 
reduction in the perceived likelihood of events reported to have happened. 

That is to say, assume as a rule that you know less than you feel you do. 

To get an idea of how much to discount your hindsight , you might keep close 
tabs on yourself for a couple of weeks to see how prone you are to some of 
the biases noted above. 

Another technique might be to take what you know about past situations 
and see how readily you can generate alternative futures for them. Try 
then to gain a day in psychological court for these counterfactual pasts, 
and see how convincing they are. For example, if you find Thurber's "If 
Grant Had Been Drinking at Appamatox" highly implausible, then you can feel 
safer in the opinion that the Confederacy was doomed on April 14, 1865. If 
it looks reasonable, then you might reconsider. See how much of Admiral 
Kimmel's behavior can be explained by assuming that he believed informed 
reports indicating that the Japanese would not dare attack the US while 
thev still had shipping in the Panama Canal area. 

If you have examined some historical period or event in depth and 
perceived sov.e of its inherent ambiguity, don't lose it. Use expressions 
like, 'The Japanese seem to have attacked Pearl Harbor because they saw 
no other way to break the boycott. This assu-.ies, however that. . . It fails 
to account for the fact that . . . Possibly ... A good second guess would 

be • • . Although ai/kwird, this kind of hedging should make it easier to 



take new facts into consideration and to help you remember j ist how good 
(or poor) your best guess about the past is. One variation on this theme 
would be to concentrate not on producing a best guess about what was 
happening in the past, but on eliminating possible interpretations. "We 
can safelv rule out the possibility that the Japanese hoped to win a protracted 
war in ttie Pacific because. . . " "The warden's personal involvement with 
Bowles' rehabilitation clearly had nothing to do with his decision to approve 
the social pass." 

All of these suggestions involve pitting our mind against itself to 
restore or preserve foresightful perspective. An alternative, and probably 
more expensive, approach is to farm the problem out to genuinely foresightful 
minds. I f we want to know what the warden should have known before he made 
his decision, let's take the case, disguise it to insure anonymity, ship 
it to other wardens, and ask them whether or not they would have issued a 
pass. If tney, too, would have released Bowles, then Warden Cupp's verdict 
bears a more sympathetic review. 

It Should be obvious bv now that this sort of advice is going to make 
the historical judge's work even iiarder by producing a much less tidy 
and coherent picture of the past (than that bestowed by creeping determinism). 
There is clearlv a price to be paid for forfeiting the facile satisfaction 
of cheap hindsight. The profit to be had? A better feeling for what we do 
and do not know, and what we have learned; more systematic hvpothesis-testing 
and learning as we follow events in the world; greater transfer from explana- 
tion to prediction; a better appraisal of the amount of uncertainty inherent 
in the past and future. The decision-maker who knows the limits of his 

Hinds! ght 


knowledge can better plan the Integration of his "executive" and "intelligence" 
functions. The latter can only be expected to take action on the basis 
of what he knows. If he can't know enough, he can best prepare to take 
action if he agrees to "accept the fact of uncertainty and learn to live 
with it," if he realizes that since "no magic will provide certainty, our 
plans must work without it." (Wohlstetter , 1962, p. 401) 

In many jobs mistakes are inevitable. It is self-defeating and unfair 
to change decision-makers who have erred in a fallible system — without doing 
something to improve that system. Consideration of the alternative might 
make this clearer. Ts there a better warden to be had than Warden Cupp? 

Who is to replace Admiral Kimmel? Are their successors likely to be less 
error prone? Or is the main trait which recommends them for the job the fact 
that they have not made the specific mistake made by their predecessor? 

Isn't it just a matter of time before they too will be on the way out, 
tripped up by need to act in the absence of certainty? 

Possibly Kimmel, Nixon, Warden Cupp, et al, did blow it. Maybe they 
should have known better. On the other hand, perhaps the handwriting on 
the wall was in ink visible in hindsight alone. If we're really 
interested in knowledge, and not just revenge, investment in the above 
techniques may save a lot of grief and injustice. The only real loss, beyond 
the effort, will be the illusion that we know it all. 



Kef orenres 

Fischhoff, B. Aspects of historical judgment. Unpublished doctoral 
dissertation, Hebrew University of Jerusalem, 1974a. 

Fischhoff, B. Hindsight + Foresight: Effects of outcome knowledge on 

.judgment under uncertainty. Oregon Research Institute Resc^ch 
Bulletin , 1974b. 

Janis, I. Vi ctims of Groupthink . New York: Houghton Mifflin, 1972. 

Lofland, R. Deviance & Identity . Englewood Cliffs: Prentice-Hall, 1969. 

Rosenhahn, D. On being sane in insane places. Science , 1973, 79 ., 250-252. 

Schur, E. Labelling Deviant Behavior . New York: Harper & Row, 1971. 

Wohlstetter , R. Pearl Harbor: Warning & Decision . Stanford: Stanford 

University Press, 1962. 

Woodward, E. L. Age of Reform . London: Oxford, 193R. (Excerpt reprinted 

by permission of the Oxford University Press, London.) 

Hind eight 


This study was supported by the Advanced Research Projects Agency 
of the Department of Defense (ARPA Order No. 2449) and was monitored by 
ONR under Contract No. N00014-73-C-0438 (r’R 197-026). 

The research reported constituted a portion of a doctoral dissertation 
submitted to the Hebrew University of Jerusalem, May, 1974. More detailed 
and technical presentations are available in Fischhoff (1974a, b). 

The comments of Robyn Dawes, Paul Slovic and Peggy Roecker on earlier 
drafts of this paper were particularly appreciated.