Measurable by Design: Project Meridian

King Croesus asked the Oracle of Delphi whether he should attack Persia. She told him a great empire would be destroyed if he attacked, so he did. Unfortunately, the empire destroyed was his own.

The Oracle wasn’t wrong because her prophecy was never specific enough to be wrong. Croesus misunderstood the prophecy because it wasn’t measurable.

On September 30, the Secretary of War announced six important initiatives. Most attention went to cutting the number of General Officer positions or the establishment of the new four-star Autonomous Warfare Command. For planners, however, the initiative that deserves the closest look is Project Meridian.

This article seeks to answer the question, “how can Project Meridian make its findings verifiable over time and give the Joint Staff and services a usable framework?”

The memo also gives an unofficial warning order to the Joint Staff and services to get ready to integrate Project Meridian’s work with their future plans. That direction makes this a practical, time-sensitive question for every staff producing scenarios, wargames, or estimates: how do you create futuristic scenarios that, unlike the Oracle’s, are specific and measurable?

Some of what follows may already be underway, but think of it as a short checklist on forecasting and scenario planning, with Meridian as the timely test case.

The same problem applies to any organization that commissions long-range scenario work.

The Department is right to bring in outside voices. Any organization that asks and answers its own questions carries built-in biases. But the strongest evidence on improving judgment says expertise works best when paired with a way to check results.

IARPA, the intelligence counterpart to DARPA, hosted a forecasting tournament in 2011, to find best practices to predict the near-term future. Five teams competed for best accuracy results. The Good Judgment Project (led by Phil Tetlock, Barbara Mellers, and Don Moore) beat the other teams and the tournament’s accuracy goals so resoundingly that IARPA eliminated funding for all other teams after 2 years of the planned 4-year study. Tetlock and journalist Dan Gardner told the story in Superforecasting, and Tetlock and Mellers detailed the results in a number of journal articles.

A few of their most important findings include:

1)     The top 2% of forecasters, which they called Superforecasters, were open-minded, approached problems with nuance, and iteratively updated predictions.

2)     Giving 1-hour probabilistic training boosted accuracy by as much as 11%.

3)     Collaborative team forecasting, where individuals challenged the views of others, outperformed individuals working alone by almost 23%.

4)     Superforecasters are also more right, earlier. Their forecasts 300 days in advance were as correct as average forecasters’ predictions 100 days in advance.

Tetlock's earlier work in Expert Political Judgment points the same direction. The best predictor of accuracy was cognitive style, not credentials. Forecasters who knew many small things and updated thinking often beat those who fit the world to one big idea. That is a reason to pair domain experts with calibrated forecasters, not to choose between them.

Including these methods in forecasting leads to significantly improved ability to predict the future, but only out to around 12 months. Yet the Department of War needs years to invest in new technology, build capability, and shape the force with the right people. It needs something to bridge near-term accuracy with longer-term decisionmaking.

Forecasting assigns probabilities to specific, dated, resolvable events like whether the price of a commodity will rise five percent in the next six months. Scenario planning generates multiple plausible futures for long-term problems. Its purpose is not pinpoint accuracy but opening decisionmakers' minds to futures they may not have considered. Shell's planners did not predict the 1973 oil shock, but they opened leader’s minds to risk mitigation strategies that protected Shell when the oil shock came.

Scenario planning seems more in line with Project Meridian’s scope. In his September 30 speech, Secretary Hegseth said Project Meridian’s efforts “will be focused on discovering, developing, and fielding the weapons and systems that our children and our grandchildren will need in their lifetimes without any creative limitations or restrictions.”

This creates a validation problem shared by the whole field. Scenario work is hard to score because accuracy isn't the goal and the horizon is long. It is rarely evaluated against outcomes, and even Shell's well-known case was not rigorously tested. Meridian has an opportunity to change that.

Every well-built scenario has signposts—early indicators that a particular future is becoming more likely. They are what make a scenario operationally useful because they tell a planner what to watch.

Signposts can also be written as forecasting questions. A signpost is already an implicit claim that something specific will be observable at some point. Make it explicit, attach a date, and define in advance who resolves it and how, and an un-scoreable narrative becomes something that can be checked. Tetlock proposed as much: clusters of small, resolvable questions that collectively bear on a large unresolvable one. Here are six specific items Project Meridian’s team can incorporate to increase the measurability, accuracy, and usability of their effort.

In 1975, Zvi Lanir and Daniel Kahneman assessed for the Israeli foreign ministry what would follow if Kissinger's negotiations collapsed. One finding was that a particular scenario would raise the chance of full-scale war with Syria by ten percentage points. A senior official heard the number and was reassured because he didn't think ten percent was a big difference.

Kahneman's conclusion, as Michael Lewis recounts in The Undoing Project, was that people don’t make decisions because of numbers; they need a story. But the more detailed and vivid a scenario, the more probable it feels, which is how a well-written narrative can make an unlikely future seem inevitable. Pairing a scenario with probability estimates helps decisionmakers consider its likelihood alongside the consequences it entails.

When Project Meridian’s report becomes public, it will be read for its specific recommendations. Including a short, unclassified appendix of dated signposts would give the Joint Staff and services a way to track the scenarios as they develop. Reviewing those signposts in 2029 and 2031 would also help the Department judge how much confidence to place in the approach next time.

Alex Willard is a retired Army lieutenant colonel. He spent 20 years as an infantry officer, strategist, and marketing officer, with deployments to Iraq, Afghanistan, and Liberia. He holds a Master of Behavioral and Decision Sciences from the University of Pennsylvania and works at Onebrief.