Claude for Clausewitz?

Earlier this month, OpenAI caused an intellectual earthquake by posting a solution to one of the hardest open problems in mathematics, using an internal generative AI model. Stunning as it is controversial, the announcement was not entirely surprising for those watching how newer AI models have been shaping the mathematical scene for the past six months. In the run-up to the latest announcement, several other long-standing mathematical questions had been answered by AI. For some, it was only a matter of time before one of the seven Millennium Prize Problems – long considered the lodestar of mathematical research – fell to the machines.

At the same time, political and security affairs thinkers can improve their own craft by careful integration of reasoning models. Conversations so far have largely focused on one aspect of this proposition: the use of AI in preparing text, and the concomitant and unfortunately commonplace practice of passing off AI-generated ideas and words as one’s own. But other possibilities deserve a fair hearing, too.

Tools of the trade (Sou Jest/Unsplash)

Since the arrival of ChatGPT in late 2022, policy wonks, in and out of government, have explored how best to rope the bots in as support. A few examples:

Researchers have looked at promises and pitfalls of fusing AI with wargaming). Large-scale social simulations with AI agents is a promising frontier of computational social science, along with AI for strategic assessment and early warning.

Political risk outfits integrate AI-generated assessments with human judgement. Hobbyists routinely vibe-code dashboards that track specific conflicts, often to rather murky effects.

But you don’t have to be a spook or a nerd to put the machines to work. Depending on your interest, several possibilities present themselves.

Capable GenAI models allow easy data analysis, even for those who are not so disposed temperamentally. The more technically capable can also use AI agents – models given a task to accomplish specific goals near-autonomously – to simplify routine work such as preparation of bibliographies by extracting relevant information from files. Newer models are also quite capable of fact-checking, and synthesising a large volume of supplied information coherently.

There are deeper options for the adventurous.

GenAI can be used for scenario creation and analysis. Language models are capable of generating coherent stories given broad guidance. Many private firms already use this feature to identify plausible futures. In fact, AI-based scenario generation is an active area of research. And if fictional scenarios are what you are after, factual hallucinations – a known AI affliction – is hardly a deal breaker.

But the key to DIY lies in very careful and continuous prompting. I recently conducted an experiment to see whether a frontier model could write a very concrete quasi-fictional scenario and then perform an alternative futures analysis in a prescribed format. While the results are encouraging – details here  – a point of caution. Chatbots can sometimes be fluent and inconsistent at once. Scenarios and futures analysis reward creativity but not logical and causal indiscipline. As AI-generated scenarios become involved, builders have to be on guard.

Paradoxically, GenAI models are also extremely good at close reading. Given well-defined instructions, they can flag possible contradictions and jumps in argument. These are useful abilities in stress-testing drafts as well as archival records. In another recent experiment I had a model assess a declassified memorandum against a fixed set of analytical standards. What stood out was its ability to parse arguments granularly, and accurately summarise the broad thrusts of the memo. The bot’s pedantry – it termed the experiment an “audit” – was impressive. At the same time, it made mistakes, which it caught in a self-directed review.

It may well be that at some point in the near future language models become a core part of qualitative research and analysis. But obstacles remain.

The first is economics. Frontier model subscriptions are expensive for many in the developing world. There is a real risk that this leads to an uneven playing field between those with access and without. And this diminishes analytical diversity.

The second issue is technical. Model outputs remain stochastic – the same prompt twice can often return slightly different outputs. Mathematicians already had a (non-AI) technology – in the form of proof checkers– which can certify the validity of an AI-supplied mathematical answer. This is the main reason why GenAI has found ready allies in that community. No such tool is available to other disciplines.

But the most important obstacle may be sociological. For many analysts and scholars, AI use is viewed pejoratively, tainted by association with intellectual theft. Evangelists must – through words and deeds – demonstrate the gains from transparent and ethical use.

Abhijnan Rej is an Indian researcher and writer, the founder of Tarqeq Research LLP and has published extensively on Asian security and geopolitics.

This article appeared originally at Lowy Institute's the interpreter.