Communicating Uncertainty in Wargaming Outcomes

In late 1990, Frank Chadwick, the president of Game Designers Workshop, received a letter from an Army soldier deployed to the Middle East. The letter described how the members of his unit were huge fans of the printed, commercial wargames developed by Chadwick’s company. The letter also described an exigent need for which they were seeking immediate help. They wanted to purchase another copy of a particular, hard-to-get wargame. During game play there in the desert, a sandstorm had blown away the board and all the pieces. It turned out that as tensions continued to rise in in the Persian Gulf, this unit had made tactical wargaming part of their everyday preparation for the imminent combat operations.

The impact of wargaming on the Gulf War was enormous--and mostly positive. Yet, Coalition casualty indications were over 20 times too high. These indications had real political and military consequences. How would these events shape the evolution of wargaming in the US and around the world?^[i]

The H.W. Bush administration used the casualty estimates to shape many of their decisions. Did a single leader stop to ask how much confidence do we have in that estimate? If so, did that same leader ask how could we have so grossly missed the mark? Caffrey asks similar questions, but one suggestion in particular stands out:

The intelligence community routinely includes confidence assessments within the products they provide to decision makers. They may say, “We have high confidence in this report as it is based on multiple highly trusted sources,” or, “we have low confidence in this report as it is based on a single fairly new source.” At the time of this writing, it is still unusual for reports on wargames to include explicit statements on their assessed level of confidence.^[ii]

That leads to a particular line of thought about wargaming: If a game were played one hundred times, would the outcome be the same every time? On the one hand, we do not expect the tactical results of this game to appear identical, but would the strategic outcomes appear consistent across countless repeats of game play? Is there any chance that strategic outcomes would vary? If so, how much? It’s certainly expected that small variations of the strategic outcomes may appear in repeated wargaming, but is it plausible to believe that some percentage of outcomes would suggest a completely opposite strategy or strategic outcome? These questions are what we mean when we ask, “How much confidence do we have in the outcome?” Unfortunately, we are ill-prepared to answer this question but not for dearth of tools and technology to make such assessments. Instead, there is a chance that most of us would not accurately comprehend such a confidence statement. This occurs largely because of a lack of shared understanding of a shared language of confidence and uncertainty. To help us build a vocabulary for answering these questions, I would like to propose three foundational rules. First, we should express wargame outcomes both qualitatively and quantitatively. Second, we should attempt to describe the range of possible outcomes. Finally, we ought to assess the frequency of potential outcomes.

Before further developing such a lexicon, we should do away with the word confidence, because it carries both technical and emotional baggage. I would prefer that we refer to it as “communicating uncertainty.” By leading with the word uncertainty, we approach the subject with the appropriate amount of humility. In a sense, we are saying, “my answer might be wrong, and I know that.” Additionally it acknowledges that wargaming is not a crystal ball, not a prediction about the future, but a model. That attitude disarms the tendency to be defensive or argumentative, and it opens the ears of the listeners, making them more receptive for what follows. It also prevents them from having any preconceived notions about what they might hear, as it strips away the technical baggage associated with the other term.

Since a wargame is in fact a game, we can use the common vernacular of winning and losing as a foundation for establishing an appropriate language for communicating uncertainty. The widespread familiarity with sports allows us to apply analogies to shape our understanding of uncertainty and the need to communicate it clearly. Our first principle for communicating uncertainty illustrates the efficacy of this approach.

In the first example cited above, the outcome of a commercial wargame has a relatively well-defined outcome in terms of winners and losers. The rules, pieces, and constraints of the game, together with a role of the dice, determine the outcome, and from these facts, we can derive rigorous quantitative models of each principle above. Furthermore, the repeated game play by the soldiers would allow us to construct a reasonable statistical estimate with less effort than the exhaustive mathematical derivation. However, that may not be necessary. We can agree that this type of wargame was useful to exercise the thought processes, decision skills, and forethought of the players. We can also agree that the outcomes would not accurately predict the infinitely more nuanced situation in front of these battle-ready units. That last statement is sufficient to communicate the uncertainty about the wargame outcome in that case.

At the strategic level, it makes sense to do a more thoughtful and complete review of the wargame outcome. The Desert Storm wargame report that opened this article included at least two critical pieces of information upon which we will focus: the declaration of a victor in the combat operation, and an estimate of casualties. It is also safe to assume that the estimate of casualties was coincident with US victory.

Each of these three rules and our hindsight about Desert Storm yield a starting point for exploring the language of uncertainty and its application to communicating wargame outcomes. This is only a starting point, and it befits us to recall that “no plan survives first contact with the enemy.” The next step in the journey will be applying these principles to ongoing wargames, putting our ideas to the test, and we can expect that our understanding of these concepts will evolve. Just as Chadwick’s board game did not inexorably determine the outcome of any Desert Storm battles, neither should we expect these three guidelines to provide a perfect silhouette of the future battlespace or strategic environment. Instead, we should use them like painted outlines of the practice pitch, boundary markers on the playing field, of which MacArthur said, “On these fields of friendly strife are sown the seeds that on other days, on other fields will bear the fruits of victory.” So should we use these tenets during wargame play to strengthen and shape our ability to encounter uncertainty in war time.

With these games will come a far greater deluge of information, requiring of leaders a greater skill, a more urgent need to make sense of it all and inform decisions. Since the dawn of man and war, we have seen technology improve our ability to strike targets and wage war, and we should expect the same learning curve in our application of these three principles for communicating uncertainty together with advances in simulation and computation. At the dawn of airpower in World War I, hundreds of bombs fell before single targets were destroyed. Today we hit single targets within hundreds of centimeters. In the next war, we will be required to use information, like the uncertainty implicit in the outcomes of a hundred wargames, to create strategic effects with the same precision. This simple introduction to communicating uncertainty may be analogous to those early days, to a single bomb dropped in the first World War. Hopefully, though, the utility of these ideas is more readily apparent and their potential will be realized more quickly.

Mark Jones, Jr. is an experimental test pilot, USAF Reservist, and writer. He is also a member of the Military Writers Guild. The opinions expressed are his alone, and do not reflect those of the U.S. Air Force Reserve, the Department of Defense, or the U.S. Government.

Notes:

[i] Caffrey, Matthew. “On Wargaming: How Wargames have Shaped History and How They may Shape the Future.” Draft Manuscript, 2017. (Vol 1, pg 103).

[ii] Ibid. (Vol 2, pg 66).

This article appeared originally at Strategy Bridge.