How Good Are Ukraine's Air Defense Systems?

We were captivated by reports out of Ukraine the morning of the 16^th of May 2023 that Ukrainian air defenses intercepted a total of eighteen incoming Russian missiles, to include six top-of-the line “hypersonics”[i]. This event appears to be yet another unexpected – and highly embarrassing - shortcoming for the Russian military. While details from the attack itself in the open press are developing as we write, the key detail that captivated our attention is that the Ukrainians achieved ‘raid annihilation.’  In more colloquial terms, their defense of Kyiv was a clean sweep.

Let’s focus on the first case, where the defender shoots down the incoming missile. We could think of each incoming missile as a coin flip, where the coin might be biased – i.e., not 50/50. We are left with a tradeoff between the effectiveness of the Ukrainian defenses and plain old luck. It turns out with a little knowledge of statistics; we can discover quite a bit.

How lucky do we think that the Ukrainians were with their air defenses? This is the type of question that is best turned around and left to the reader. In Figure 1 below, we show the tradeoff between the how ‘lucky’ the Ukrainians were to achieve this result vs. how effective their air defenses were:[ii]

Figure 1: Ukrainian Air Defenses (x-axis) vs. Probability of defeating all eighteen inbound Russian missiles. To have 95% confidence of defeating such a raid, the per-inbound effectiveness of the Ukrainian air defenses would have had to have been greater than 99.7%.

Harrison Schramm

My friend and mentor, the late Professor Wayne Hughes admonished his students – including me – that “when there is a war, study the war.”  And the war in Ukraine - pitting western defense technologies against Russian weapons – is teaching us valuable lessons about the future of conflict. Using the sort of ‘poker hand and coin flip’ arguments, it has always been my assumption that air defense problems favored the attacker over the defender, because it is really hard to consistently be this good. However, seeing the trend in effectiveness of western-designed air defenses against incoming missiles should make all of us – including our adversaries – reevaluate what we consider the ‘art of the possible’ when it comes to air defense.

And, while I hate to do the Russians[iii] homework for them, we might ask ‘if the Ukrainian Air Defenses are that good, how many missiles would they need to fire for a successful raid (50% chance of 1 or more hits)’. I’m not going to say the number – they can figure it out for themselves – but it’s a lot more than eighteen.

Harrison Schramm is a professional statistician working at the intersection of data, AI, and public policy. A retired US Navy Commander, he is the 2018 recipient of the Clayton Thomas Prize for technical contributions to the Profession of Operations Research. He is Founder and CEO of Schramm and Associates, LLC, an applied mathematics consultancy.

[i] https://www.washingtonpost.com/world/2023/05/16/missiles-russia-ukraine-war-kyiv/

[ii]

[iii] To be fair, pre-Soviet Russians were titans of probability theory; much of my education came from studying the works of Andrey Markov and Pafnuty Chebyshev. I wonder what they would think of this current situation?