AD / A - 001 807
HINDSIGHT: THINKING BACKWARD
Oregon Research Institute
Office of Naval Research
Advanced Research Projects Agency
4 November 1974
National Technical Information Service
U. S. DEPARTMENT OF COMMERCE
’£' •< 'X
by the Advanced Research Projects Agency of the
rder No. 2449) end was monitored by ONR under
Cohttact Mo N00014-73-C-0438 (NR 197-026).
U S Department of Commerce
HINDSIGHT: THINKING BACKWARD?
ONR Technical Report
Ajt/A- OO) foy
Oregon Research Institute
Unclassi f led
Hindsight: Thinking Rackward?
• • ’ i >•■ > > i
November 4 1974
Nonoi/ ( -7vr-n4i8
ARPA Order No. 2449
i f i r ■ i i ’ i I
Oregon Research Institute Research
Monograph Vol. 14, No. ]
' i * " K t » "«!». ■ 1 J /ir.i . \ . T f. .../•• f | , r n„ij I.
This document has been approved for public release and sale. Its
distribution is unlimited.
1 .* i '• J I **U. !.• . I I Al« > K II t'l U
Office of Naval Research
ARlington, Virginia 22217
Is lindsight better than foresight or just different? The formal
difference between the two tasks is the hindsightful judge's possession
of outcome knowledge, telling him how things turnei out. It is this
additional knowledge which is reputed to confer the "wisdom of hindsight".
Tn the studies reported here, outcome knowledge is found to increase the
perceived Inevitability of the outcome reported. Judges are, however,
largely unnwnr of the changes in their perceptions due to outcome knowledge.
As a result, tuey believe that they and others had in foresight insights
which thev themselves only had as a result of outcome knowledge. Failure
to appreciate the effects of outcome knowledge can seriously prejudice the
evaluation of decisions made in the past and limit what is learned from
di :> ,o "
(I a: : 1)
Si'i mil V C'l.i:.' ill. .itui
Hindsight: Thinking Backward?
Hebrew University of Jerusalem
Oregon Research Institute
Hindsight: Thinking Backward?
Hebrew University of Jerusalem
Oregon Research Institute
In tie spring of 1974, Carl Cletus Bowler, "convicted murderer,
bank robber and all-around bad actor" (Euge ne Register Guard. June 5, 1974, p. 12A)
was released on an overnight social pass from the Oregon State Penitentiary
where he was serving a life sentence. The social pass program had been
initiated by the Oregon state legislature to help convicts maintain contact
with the outside world so as to make it easier for them to eventually
rejoin it. Bowles, who had been a model prisoner, promptly absconded.
Warden Cupp, who bore ultimate responsibility for all passes issued
in the penitentiary, came under heavy fire. According to a local newspaper,
'It was an error in screening. Bowles' record, in the prison and out, as
we read it now, shows that he could not be trusted." (ibid. ) The state's
governor informed Warden Cupp that be would have to resign if Bowles
committed violence while at large.
Following the Japanese attack on Pearl Harbor, the Naval C. 0 .,
Admiral Kimmel was removed from his command and lowered in rank. A Congres-
sional investigatory committee convened in 1946 produced 39 volumes of
evidence, much of it highly critical of those responsible for Pearl Harbor's
security. Irving Janis (1972) blames "collective groupthink (pressures
to conformity) among interlocking groups ... for America's astounding
unreadiness at Pearl Harbor," after noting the success of American
intelligence in deciphering the Japanese secret codes known as MAGIC, (p. 99 ).
In accepting President Ford's pardon, former President Nixon indicated
that he could see now how he should have understood the meaning of Watergate
in July of 1972. He regretted his misunderstanding and subsequent failure
to take suitable action.
The structure of each of these examples is similar: misfortune occurs,
a culprit is identified, his decision second-guessed,
and his folly chastized either bv others or by himself. Wiry are such
critics in a position to criticize? Because they know how things turned out.
Their hindsight they feel, provides them with wisdom illuminating the
follies of others, or even of themselves. Or does it? Would we, could we,
or should we have known better than Warden Cupp, Admiral Kimmel, President
Nixon, or would we do better now if placed in their shoes?
Is hindsight really better than foresight, or just different? How
aware are we of these differences? Do we, as Nietzsche suggests, begin by
looking backward and end up thinking backward? Our awareness of these
differences in large part determines both how much we le c rom experience
and how we mete out justice. In Admiral Kimmel's case, for example, if he
saw or could have seen what was happening, but didn't take action accordingly,
then he was clear lv negligent, incompetent, or worse. Our own lesson from
his experience is to find better officers and teach him a lesson. If,
however, the imminence of a Japanese attack is apparent only in retrospect,
then Kimmel deserves either a fairer shake than he got, or else conviction
for a rather different crime — failure to be ready for any and all possible
surprises. In this particular case, with the limited resources at his
disposal, he may well have done the best he could. Our lesson is, then, to
spend more on eliminating surprises and/or being readier for them when they
II l mis ight
My colleagues and T have recently conducted a series of experiments
designed to obtain an understanding of the differences between hindsight
and foresight, and to better understand how to get the most out of each.
Supposedly, the sharper and surer our hindsight, the better our foresight.
Our results indicate that this need not be the case.
The first experiment in the series was designed sinply to see how
foresight and hindsight differed in a number of aspects. In it, subjects
read a short, unfamiliar historical passage, such as the following:
For some years after the arrival of Hastings at, governor-general
of Lndia, the consolidation of British power involved serious war. The
first of these wars took place on the northern frontier of Bengal
where the British were faced by the plundering raids of the Gurkas
of Nepal. Attempts had been made to stop the raids by an exchange of
lands, but the Gurkas would not give up their claims to country under
British control, and Hastings decided to deal with them once and for
all. The campaign began in November, 1814. It was not glorious. The
Gurkas were only some 12,000 strong; but they were brave fighters,
fighting in territory well suited to their raiding tactics. The older
British commanders were used to war in the plains where the enemy
ran away from a resolute attack. In the mountains of Nepal it was
not easy even to find the enemy. The troops and transport animals
suffered f rom the extremes of heat and cold, and the officers learned
caution only after sharp reverses. Major-General Sir D. Ochterlony was
the one commander to escape from these minor defeats.
The Age of Refor m
by r.. L. Woodward
Oxford, 1937, pp. 383-4
live groups of subjects received each story in varying versions. There
was one "foresight" group which was not told what had happened subsequently
(the British won). There were four "hindsight" groups each of which was
given the same storv with one of several possible outcomes appended to it
as the "true" outcome. These were, in this case:
British victory; Gurka
victory; military stalemate with a peace treaty; military stalemate without
a peace treaty. Thus, for three of the four hindsight groups, the reported
outcome was in fact false. Note that even though the event took place a
century and a half ago, foresight subjects not to’d what had happened are
in much the same position as contemporary observers of the British-Gurka
All subjects were asked to: 1) judge the probability that each of the
four possible outcomes was going to happen; 2) evaluate the relevance of
each of the facts appearing in the story in determining what happened;
and 3) give reasons for their answers.
The most dramatic result was that hindsight subjects consistently
perceived reported outcomes (whether true or not in fact) as having been
more likely to occur than did their foresightful counterparts; knowing that
something had happened roughly doubled the perceived odds that it was going
to occur. Interestingly, however, subjects almost never assigned 100%
probability to what was reported to have happened. Evidently, they felt
that in the light of the facts given in the description other outcomes were
still possible (e.g., "The Gurkos had a 70% chance of winning, but the British
still might have pulled it off.").
^e perceived relevance of the facts in the description and the reasons
which subjects offered to justify their answers depended, as well, upon
the outcome reported. Interestingly, though, there were some data
(e.g.. Hasting's decision to deal with the Ourkas "once and for all") and
some reasons (e.g., "the experience of Viet-Nnm" or "what's going on in the
Middle East today") which wore relevant no matter what happened.
II i mis i ght
Knowing that something has happened clee-ly increases its perceived
inevitability, as well as restructures our perceptions of what we know
about It. How justified are these changes? Tt's hard to say, simply because
no one knows the objective probabilities associated with unique events like
the Bri tish-Ourka struggle. Consider another example: Tf we claim that
there was no chance (or a 77, chance or a 98.67 chance) of a thermonuclear
war during the I960' s who's to prove us wrong? Indeed, the only wrong
estimate is that it was 1007 likely.
Let us call the tendency to see whatever is reported to have happened
as having been relatively inevitable "creeping determinism" — in contrast
with philosophical determinism, the conscious belief that whatever happens
has to happen. These two types of determinism are essentially independent.
The philosophical determinist may believe that a reported event (e.g., Bowles'
escape) was inevitable, since whatever happens is, by definition, inevitable.
He may, however, still be surprised by its occurence. Indeed, he may well
set for himself the task of researching the situation until its inevitable
character becomes apparent. He might, thus, insist that Bowles was bound
to escape, yet deny that anyone was in a position to foresee it. The creeping
determinist may well be totally unaware of the raging debate over historical
inevitabili tv and free will. He perceives reported events as having been
more or less bound to happen, simply as a matter of course.
Our second experiment was designed to determine how aware judges are
of the hindsight-foresight differences which we have called "creeping
determinism". The importance of such awareness for the historical second-
guesser has boon discussed already; its importance for anyone trying to
learn from history is discussed below.
Each group of subjects in Experiment ? received stories like the Rritish-
(.urka episode, each with one of the four endings appended to it. Thus, subjects
in each group knew as much as did subjects in the comparable hindsight
group in Experiment !. They were asked to complete the Experiment 1
questionnaire as they wouln have "had they noc known what really happened",
i.e., like foresight subjects. ’People aware of what they have learned from
knowing what happened should be able to perform this task successfully.
Our subjects* success was limited at best. They consistently believed
that without outcome knowledge they would have displayed the creeping determinism
shown by Experiment 1 subjects with outcome knowledge. They believed that
had thev been asked, they would have seen the relative inevitability of
the reported outcomes. Their reconstructed relevance judgments, too, bore
the imprint of the outcome knowledge they were to ignore. For example,
foresight subjects in Experiment 1 attached relatively little importance
to British suffering from "extremes of heat and cold." Experiment 1 hindsight
subjects told that the Gurkas had won, attached greatest relevance to this fact.
Experiment 2 subjects told to ignore the r cport of Gurka victory indicated
that even without that report they would have perceived the importance of
failure to ignore outcome knowledge is not without its benefits. It is,
indeed, quite flattering to believe, or lead others to believe, that we
would have "known all along" what we could only know with outcome knowledge.
III nds I >* li L
that is to say, that we possess hindsightful foresight.
Returning to the introductory examples, perhaps this failure to empathize
with ourselves in a more ignorant state is paralleled by a failure to empathize
with outcome-ignorant others. Experiment 3 examined this question. In it,
five groups of subjects were asked to respond to the questionnaire used
in Experiments 1 and 2 — as thev thought other subjects, similar to themselves,
had responded. These other subjects were described as not knowing the
"true" outcome. One of the Experiment 3 groups was equally ignorant of what
had happened. The remaining groups each received one of the possible outcomes
as the "true" outcome. They were essentially asked to respond like foresightful
others, more ignorant than themselves.
Subjects who did not know what had happened believed that outcome-
ignorant others would respond much the same way as foresight subjects in
Experiment 1. Since this is presumably the way they themselves would respond,
they seem to have projected their own judgments on others. Experiment 3
subjects who did know what had happened, however, attributed to outcome-less
others probabilitv and relevance judgments which clearly bore the imprint
of the outcome knowledge which they were asked to ignore. In particular,
thev believed that others in foresight would have seen the relative inevita-
bility of reported outcomes which they themselves perceived only in hindsight.
As before, this result was obtained for true and false outcome reports alike.
In a fourth study, (Fischhoff & Beyth , 1974) subjects were asked oti the
eve of then-President Nixon’s trips to China and the USSR in 1972 to estimate
the probability of various possible outcomes of the trips (e.g., meeting
Chairman Mao, visiting Lenin's tomb, announcing that the trips were successful).
Two weeks to six months after the trips' completion, these same subjects were
asked to remember as best they could, their own original predictions. Finally,
they were asked to indicate for each event whether or not they believed
that it had actually happened.
The results showed that subjects remembered having given higher probabi-
lities than they actually had to events believed to have occurred and lower
probabilities to events which hadn't. Their original predictions showed that
they were too often surprised — many highly unlikely or impossible events
(assigned probability = 0%) did occur. Their remembered probabilities,
however, indicated that they perceived a past which held too few surprises
for them; indeed, almost no events which they remembered assigning low proba-
bilities to were perceived to have occurred.
Summarizing these results: Finding out that something has happened
increases its perceived inevitability. We are unaware, however, of this
effect of outcome knowledge and tend to believe that the inevitability was
largely apparent in foresight, without the benefit of knowing what happened.
This undiagnosed creeping determinism biases our impression of what we would
have known without outcome knowledge (Exp. 2), as well as our impressions
of what we ourselves (Exp. 4), and others (Exp. 3), actually did know in
foresight. In retrospect, we tend to believe that we and others had a much
better idea of what was going to happen than we (or others) did.
How do we do this? How do we manage to see the relative inevitability
of whatever is reported to have happened, true or false, yet remain unaware
of the effect which outcome knowledge has had on our perceptions? The data
relevance results provide an important cue. In particular they suggest
that the data in the event description change their meaning or significance
when different outcomes are reported. Those data which were highly
relevant whatever happened, for example, must have meant something different
in each context: just as Bowles' good behavior in prison would have meant
one thing had he returned from his social pass and actually meant another
when he did not. In either case, it was a highly relevant datum.
What kind of meaning adjustment goes on? Two related interpretations
seem worth considering. The first is that we are biased to view whatever
happen? as being inevitable, and then juggle or manipulate whatever else
we know to concur with that feeling of inevitability. For example, upon
hearing of Bowles' escape we say, "That was bound to happen." and then go
about figuring out why. Looking at his record, we may highlight details
pointing to his errant character and reinterpret other details which are
inherently ambiguous. Critics of deviance labelling such as Lofland (1969),
Shur (1971), or Rosenhan (1973), have suggested that just such a process
goes on when the public and professionals rework or reinterpret the
biographies of deviants to show that their labels are inevitable products of
their life histories. Doggedly and professionally pursued, such reconstruc-
tion can find cause for the continued incarceration of even perfectly "normal
patients who have had themselves committed just to see how hospitals operate
from the insii'e.
The alternative explanation proceeds in the opposite direction. It
suggests that when we receive outcome knowledge, we immediately make sense
out of it by integrating it into what we already know about the subject.
Having made this reinterpretation, the reported outcome now seems a more
nr less inevitable outgrowth of the reinterpreted situation. "Making sense"
out of what we're told about the past is, in turn, so natural that we may
well be unaware o c outcome knowledge having had any effect on us. Even
if we are aware of there having been an effect, we may still be unaware
of exactly what it was. In trying to reconstruct our foresightful state
of mind, much evidence suggests that we will remain "anchored" or rooted
in our hindsight ful perspect ive— leaving the reported outcome too likely
looking. Both processes may, of course, be operative.
The negative effects of unperceived creeping determinism are probably
apparent by now. When we second-guess Warden Cupp's decision, for example,
our natural tendency seems to be to see Bowles’ escape as having appeared
more likely at that time than it really did seem. In this light, the warden's
decision to issue the social pass seems like an act of irresponsibility or
just plain incompetence— whereas in truth, the probability of escape may
have justifiably seemed very small to him (judging by the governor's comment,
even after the escape the probability that he would commit violence while
at large was still unclear). All the signs in Bowles' "record in the
prison and out, as we read it now (which) show that he could not be trusted"
may well have meant something very different before his escape (e.g., his
good behavior in jail). The fact that he escaped does not in itself mean
that the decision to release him was a bad one. Good decisions, those which
maximally utilize all available knowledge, may have bad outcomes. The informa-
tion about Bowles' true nature which would have led to a better decision
either may or may not have been available when the warden issued the
The point is that from our perspective, it is very herd to tell and therefore
dangerous to rely on our intuitive (retrospective) impressions.
Aside from making us unduly harsh in judging decisions made in the
past, unperceived creeping determinism may also make us insensitive to what
is to be learned from the past. Tf the description above is accurate, then
Warden Cupp's lesson from this harrowing experience (Bowles eventually
kidnapped and probably murdered an Oregon couple) seems to be "don't issue
passes to prisoners vou can see are going to escape." It's hard to see how
that kind of advice is going to advance anyone's capabilities.
As with Pearl Harbor, there is an alternative lesson to be learned. It
is that the issuing of social passes ;s a risky business, that the predicta-
bility of temporarily released prisoners' behavior is far from perfect, and
that the system should be redesigned to accommodate or reduce this uncertainty
(e.g., denying all passes, statistically identifying risky prisoners, or
explaining to the public the risks and benefits of the program in order to
make them understand and assume some responsibility for its successes and
I f we look at the past and find that it holds few surprises for us,
we are essentially denying that we have anything to learn from it. Even
though outcome knowledge changes our perceptions of specific events (by
making them seem inevitable), without a feeling of surprise we probably
feel little compunction to reevaluate the "world hypotheses" or rules with
which we interpret what goes on around us. It is conscious refining of these
hypotheses which improves our ability to uncerstand our past, present and
II ! iifls ip, lit
future. We generally believe that being able to explain or make sense of
the past increases our ability to predict the future. If what we call
explaining the past is actually "explaining away" the surprises it holds,
the very opposite may occur. A surprise-free past may well portend a surprise-
What can we do to make our hindsight more insightful? The most basic
bit of advice is to accept the existence of uncertainty in historical
judgments. Even though people don't much like dealing with uncertainty,
they will usual lv acknowledge its presence in their understanding of the
present and future. If they don't, events will sooner or later prove their
fallibility. The past, however, is less able to produce surprises which
can show up the know-it-all. As a result of its defenselessness, it may
have deterministic schemata imposed on it that would appear as sheer effrontery
if imposed on the present or future. The uncertainty is there, however,
and will remain as long as there is any question about the meaning, reliability,
or validity of any of the components of our historical knowledge, the
facts we presumably know about the past, the facts we need to know about
the past and don't~‘ ,n d must surmise from what we do know — or tne explanatory
principles (rules) with which we make a coherent whole out of what we know.
Simply doubting what you believe you kn jw about the past may produce
some quick benefits. Consider the responses of one subject in Experiment 1
who was told that the British had been defeated by the Gurkas. She justified
giving a higher retrospective probability to a Gurka victory by citing the
"fact" that "in 1812 (the time of the Gurka campaign), the British were also
defeated by the United States in the War of 1812." The reader may remember that
in that war, the British managed to burn Washington, D.C., and conquer
I ort Detroit by the mere presence of their forces 120 miles away in London,
Ontario; and that thev only broke off the engagement (i.o., lost the war)
in 1814 when called away by -vents in Europe. Had she known of the pitfalls
of ( reeping determinism, she might have adopted a more critical attitude
toward her ou’u information.
Similarly, one might make it a habit to ask himself nuestions like,
"Did ! really give the Donalds' marriage only a one in ten chance of lasting
when they got married?" or "Was I really certain that Mao and Nixon would
get together?" Most investigatory committees would also do well by asking
first not who erred and why, but was there an error at all, i.e. , could anyone else
on the scene conceivably have known what was going on and acted more
After acknowledging the existence of uncertainty in the past, a good
practice might he to try to hunt it down in its original form. For example,
are there records of the deliberations which preceded the decision to issue
Bowles the infamous social pass; are there transcripts of the information
reaching Admiral Kimrnel prior to 7 AM on December 7; is there a notebook
showing the stocks you considered before settling on Waltham Industries;
are there diaries capturing Stalin's (or Chamberlain's) view of Hitler in
1919? Any of these records might show the difficulty of the decisions
facing these actors. If you have hopes of learning how to forecast better,
it might well pay to keep a written record of your own predictions and the
considerations which guided them — a record which can be evaluated in the light
of reality (what eventually happens). It can't hurt to know when you knew
and when you didn't know aJ 1 along what would happen and why.
Such fossilized deliberations are, of course, quite rare. When there
is no uncertain past to be uncovered, it must be reconstructed. Cne
simplistic remedy which iray have some value is to effect an across-the-board
reduction in the perceived likelihood of events reported to have happened.
That is to say, assume as a rule that you know less than you feel you do.
To get an idea of how much to discount your hindsight , you might keep close
tabs on yourself for a couple of weeks to see how prone you are to some of
the biases noted above.
Another technique might be to take what you know about past situations
and see how readily you can generate alternative futures for them. Try
then to gain a day in psychological court for these counterfactual pasts,
and see how convincing they are. For example, if you find Thurber's "If
Grant Had Been Drinking at Appamatox" highly implausible, then you can feel
safer in the opinion that the Confederacy was doomed on April 14, 1865. If
it looks reasonable, then you might reconsider. See how much of Admiral
Kimmel's behavior can be explained by assuming that he believed informed
reports indicating that the Japanese would not dare attack the US while
thev still had shipping in the Panama Canal area.
If you have examined some historical period or event in depth and
perceived sov.e of its inherent ambiguity, don't lose it. Use expressions
like, 'The Japanese seem to have attacked Pearl Harbor because they saw
no other way to break the boycott. This assu-.ies, however that. . . It fails
to account for the fact that . . . Possibly ... A good second guess would
be • • . Although ai/kwird, this kind of hedging should make it easier to
take new facts into consideration and to help you remember j ist how good
(or poor) your best guess about the past is. One variation on this theme
would be to concentrate not on producing a best guess about what was
happening in the past, but on eliminating possible interpretations. "We
can safelv rule out the possibility that the Japanese hoped to win a protracted
war in ttie Pacific because. . . " "The warden's personal involvement with
Bowles' rehabilitation clearly had nothing to do with his decision to approve
the social pass."
All of these suggestions involve pitting our mind against itself to
restore or preserve foresightful perspective. An alternative, and probably
more expensive, approach is to farm the problem out to genuinely foresightful
minds. I f we want to know what the warden should have known before he made
his decision, let's take the case, disguise it to insure anonymity, ship
it to other wardens, and ask them whether or not they would have issued a
pass. If tney, too, would have released Bowles, then Warden Cupp's verdict
bears a more sympathetic review.
It Should be obvious bv now that this sort of advice is going to make
the historical judge's work even iiarder by producing a much less tidy
and coherent picture of the past (than that bestowed by creeping determinism).
There is clearlv a price to be paid for forfeiting the facile satisfaction
of cheap hindsight. The profit to be had? A better feeling for what we do
and do not know, and what we have learned; more systematic hvpothesis-testing
and learning as we follow events in the world; greater transfer from explana-
tion to prediction; a better appraisal of the amount of uncertainty inherent
in the past and future. The decision-maker who knows the limits of his
knowledge can better plan the Integration of his "executive" and "intelligence"
functions. The latter can only be expected to take action on the basis
of what he knows. If he can't know enough, he can best prepare to take
action if he agrees to "accept the fact of uncertainty and learn to live
with it," if he realizes that since "no magic will provide certainty, our
plans must work without it." (Wohlstetter , 1962, p. 401)
In many jobs mistakes are inevitable. It is self-defeating and unfair
to change decision-makers who have erred in a fallible system — without doing
something to improve that system. Consideration of the alternative might
make this clearer. Ts there a better warden to be had than Warden Cupp?
Who is to replace Admiral Kimmel? Are their successors likely to be less
error prone? Or is the main trait which recommends them for the job the fact
that they have not made the specific mistake made by their predecessor?
Isn't it just a matter of time before they too will be on the way out,
tripped up by need to act in the absence of certainty?
Possibly Kimmel, Nixon, Warden Cupp, et al, did blow it. Maybe they
should have known better. On the other hand, perhaps the handwriting on
the wall was writi.cn in ink visible in hindsight alone. If we're really
interested in knowledge, and not just revenge, investment in the above
techniques may save a lot of grief and injustice. The only real loss, beyond
the effort, will be the illusion that we know it all.
Fischhoff, B. Aspects of historical judgment. Unpublished doctoral
dissertation, Hebrew University of Jerusalem, 1974a.
Fischhoff, B. Hindsight + Foresight: Effects of outcome knowledge on
.judgment under uncertainty. Oregon Research Institute Resc^ch
Bulletin , 1974b.
Janis, I. Vi ctims of Groupthink . New York: Houghton Mifflin, 1972.
Lofland, R. Deviance & Identity . Englewood Cliffs: Prentice-Hall, 1969.
Rosenhahn, D. On being sane in insane places. Science , 1973, 79 ., 250-252.
Schur, E. Labelling Deviant Behavior . New York: Harper & Row, 1971.
Wohlstetter , R. Pearl Harbor: Warning & Decision . Stanford: Stanford
University Press, 1962.
Woodward, E. L. Age of Reform . London: Oxford, 193R. (Excerpt reprinted
by permission of the Oxford University Press, London.)
This study was supported by the Advanced Research Projects Agency
of the Department of Defense (ARPA Order No. 2449) and was monitored by
ONR under Contract No. N00014-73-C-0438 (r’R 197-026).
The research reported constituted a portion of a doctoral dissertation
submitted to the Hebrew University of Jerusalem, May, 1974. More detailed
and technical presentations are available in Fischhoff (1974a, b).
The comments of Robyn Dawes, Paul Slovic and Peggy Roecker on earlier
drafts of this paper were particularly appreciated.