Was the Top Kill "Chance of Success" a Made-for-TV Number?

I'm always curious about how people use and abuse the tools of probability and statistics. In structured finance, slices of CDOs failed at a rate completely inconsistent with their high-level ratings. Okay, that's bad. But even worse is that when strips of CDOs started being packaged in other CDOs, not only did the level of complexity rise in the new products (the "CDOs squared"), but sensitivity to initial misrating -- and the implied low probability of default -- exploded. (The explanation of why this is so is a bit technical, but is worth checking out at Marginal Revolution's The Dark Magic of Structured Finance.)

Now Top Kill has failed.

Failed to plug the spewing oil pipe a mile below the water's surface in the Gulf of Mexico.

Failed to stop the environmental carnage taking place in the ocean and along Louisiana's shores.

Top Kill came with its own simple, straightforward probability: a 60 to 70 percent chance of success. The number gave us comfort -- BP was doing the right thing, because it had better than even odds of stopping the oil belching into the seawater -- but not too much comfort, because, obviously, the flip side of that rate of success is a 30 to 40 percent chance of failure.

I thought for a while about BP's public stance on the likely effectiveness of Top Kill, and thought some more, and finally concluded: Whether BP planned it this way or not, they came up with the perfect public relations probability of success. If Top Kill's chance of success was exhaustively and scientifically studied then ascertained to be a number anywhere between 10 percent to 90 percent, and BP came to me, and I was a seasoned PR professional, my advice would be:

"Say publicly that Top Kill has a 60 to 70 percent chance of success."

Why? Well, imagine it actually has a 10 percent chance of success. And BP admits that. What's going to happen? The public, media, elected officials -- everyone will turn on BP and excoriate the oil giant for not coming up with a plan that has reasonable odds of working. Then when Top Kill fails, everyone will roll their eyes and say, "Of course it was going to fail. It was a lousy plan with only a 10 percent chance of success."

Now imagine the converse: there is actually a 90 percent chance that Top Kill will achieve its objective. And BP announces that. If the operation then succeeds, there will be a sigh of relief, but also a sort of collective shrug. What did you expect? After all, a 90 percent chance is the equivalent of an uncontested layup in the game of basketball. But if Top Kill failed, reaction would be furious: how could they screw up something with a 90 percent chance of working?

Now where's the sweet spot? It's a percentage that aligns with "cautiously optimistic." It's a percentage a little north of 50 percent -- but not too far north. It's a percentage that, if you fail, you can say, "Well, we knew from the beginning there was a large chance we weren't going to be able to do this," but if you succeed, you can say, "This was far from a sure thing, but we pulled it off, and congratulations to the great team at BP blah blah blah." That sweet spot, quantified: 60 to 70 percent.

Who knows what the actual chances of success were? I'm not an engineer, but once I learned a bit about Top Kill, it sounded fairly dubious. It sounded more like an operation with a 25 percent chance of working -- if that.

What if we wanted to find out what BP's top executives, in their heart of hearts, really thought were the odds of pulling off Top Kill?

Here's one way: the top 50 BP executives could've been forced to stake half of their wealth to a "futures" market on whether or not Top Kill would work. Sort of like betting on a prize fight. They would be allowed to trade in and out of odds that would fluctuate (much as is done for a heavyweight championship fight) depending on which position the money is favoring, and by how much ... and eventually, we'd get something resembling what BP really thought were its chances of plugging the leak. So, for instance, one guy would be betting half his wealth that Top Kill had a 65 percent chance of working (and would collect 35 percent if he won) and another executive at BP would be wagering it had a 35 percent chance of working (and would collect 65 percent if he won).

A little fanciful, and you may wonder -- well, who cares if BP lied to us (note: I have no idea if they did, but wouldn't be surprised) on the odds? Actually we should care because when the odds are 10 percent, not 60 to 70 percent, there's much more pressure to develop a "Plan B" -- and "Plan C" and "Plan D" as well. And I kind of wonder if the odds were at say 65 percent of Top Kill being a success -- and BP executives could take that position, but it would cost half of their wealth to play -- how many would've gamely said, "I'm in on that bet!" My guess: very, very few.

So Why Did the Stock Markets Do a Bungee Jump on May 6?

What I find interesting about this story (and if you think bungee jump is hyperbole, you haven't looked at the intraday Dow chart) is there's a decent analogy: Imagine that murder victims start turning up in some small, idyllic U.S. city. Throats slashed, bodies mangled. Residents grow fearful. They start locking their doors all the time, glancing over their shoulders, going out less. The pressure mounts for police to find the killer, to calm a jittery city.

Now imagine a hit-and-run of sorts in the financial markets. Stocks make a sudden, vertiginous plunge, only to rapidly recover. Billions of dollars of wealth evaporate, then reappear. But how did this happen? Who caused it? Who profited from it? Could they do it again, and what if the next time the markets don't bounce back? And so investors grow fearful. They wonder: Should I continue to put more funds into volatile stock markets? Can they be trusted?

And the search begins, among the financial forensic teams, to find a culprit for the turbulence on May 6.

Except -- here's where our serial killer analogy breaks down -- was it really a lone actor? We'd like to think that; it fits into a more comforting narrative, with justice to be meted out if we find wrongdoing, and if we don't, we at least know where to take measures to fortify the system. Could it be Mr. Fat Finger Trader? The clumsy oaf who pressed "b" for "billion" on his keyboard when he meant "m" for "million"? Or here's the latest suspect, according to cbsnews.com:

Shares of money manager Waddell & Reed Financial Inc. fell Friday as it was identified as the stock trader that sold off a large number of index futures contracts during last Thursday's market collapse ...

Waddell's sale of 75,000 e-mini futures contracts in a 20-minute span on May 6 drew the attention of regulators, Thomson Reuters reported.

These "minis" are tied to the S&P. A futures contract is a way to bet on which way you think a market, stock or commodity is going to move; narrowly defined it's an agreement to buy or sell something at a set price on a set day in the future.

Now that 75,000 number does seem impressive. Well, at first. Wealthtrader.net tells us:

The current average daily implied volume for the E-mini is over $150 billion making it the most liquid trading derivative in the world.

What was the implied volume (I'm assuming "implied" refers to the contract's notional amount) for the 75,000-unit trade? Start with the notional value of an e-mini: $50 times the price of the S&P index. The high for the S&P that day was 2,407.8, so this trade was probably executed at somewhere south of that. Let's say the e-mini "dump" amounted to less than $9 billion of implied volume on May 6.

In other words, less than 6 percent of the typical daily volume came from this sale. Was this a big trade? Sure. Was it a little hard for the market to digest? Probably. Was it the real villain that we're seeking? I doubt it.

It's a media-ready story though. The storyline is simple. The government also surely realizes that finding a lone actor will make its job so much easier.

Unfortunately, a more sophisticated analysis of possibly the real culprit on May 6 has emerged, an analysis that is not so comforting. It suggests that our stock markets have a deep, systemic flaw. Paul Kedrosky at Infectious Greed nicely, and succinctly, laid out the argument in "The Run on the Shadow Liquidity System."

His view: good old-fashioned liquidity in the stock markets has been transformed in the age of supercomputers and high-frequency trading and algorithm-driven strategies:

... traditional providers of liquidity, market-makers and other participants, are not standing so ready to make the other side of the market. There are fewer traders prepared to make a market for the sake of market health. This is ... mostly because of what has happened with high-frequency trading, algorithms, and the like, which increasingly jump into the trading queue in front of and around orders, creating some liquidity, but also peeling pennies for themselves, frustrating market participants and heretofore liquidity providers...

The result:

... all of this changes market microstructure in insidiously destabilizing ways. For the first time we have large providers of this shadow liquidity, algorithms and high-frequency sorts, that individually account for large percentages of daily trading activity, and, at the same time, that can be turned off with a switch, or at an algorithmic whim.

Certainly we need to find what went wrong on May 6. We need to, to reassure the investing public that our stock markets are safe, fair, reliable -- places where you can feel comfortable putting a large chunk of your retirement nest egg. But, in the rush to judgment, we should be on guard against trying to largely pin the events on a lone actor, just to avoid a truth that is both complex and inconvenient.

Update: Actually, the fingerpointing at Waddell & Reed appears even more misguided than I thought ... you can blow up those calculations I made above; they were for a normal trading day. Turns out that during the 20-minute freefall on May 6, 842,514 e-mini contracts swapped hands (says Reuters). 75,000 is less than 9 percent of that number (and is probably only a few percent of the total for the day) ... again, it's sizable, but remove Waddell & Reed, and you still have a heavy, heavy volume.