Was the Top Kill "Chance of Success" a Made-for-TV Number?

I'm always curious about how people use and abuse the tools of probability and statistics. In structured finance, slices of CDOs failed at a rate completely inconsistent with their high-level ratings. Okay, that's bad. But even worse is that when strips of CDOs started being packaged in other CDOs, not only did the level of complexity rise in the new products (the "CDOs squared"), but sensitivity to initial misrating -- and the implied low probability of default -- exploded. (The explanation of why this is so is a bit technical, but is worth checking out at Marginal Revolution's The Dark Magic of Structured Finance.)

Now Top Kill has failed.

Failed to plug the spewing oil pipe a mile below the water's surface in the Gulf of Mexico.

Failed to stop the environmental carnage taking place in the ocean and along Louisiana's shores.

Top Kill came with its own simple, straightforward probability: a 60 to 70 percent chance of success. The number gave us comfort -- BP was doing the right thing, because it had better than even odds of stopping the oil belching into the seawater -- but not too much comfort, because, obviously, the flip side of that rate of success is a 30 to 40 percent chance of failure.

I thought for a while about BP's public stance on the likely effectiveness of Top Kill, and thought some more, and finally concluded: Whether BP planned it this way or not, they came up with the perfect public relations probability of success. If Top Kill's chance of success was exhaustively and scientifically studied then ascertained to be a number anywhere between 10 percent to 90 percent, and BP came to me, and I was a seasoned PR professional, my advice would be:

"Say publicly that Top Kill has a 60 to 70 percent chance of success."

Why? Well, imagine it actually has a 10 percent chance of success. And BP admits that. What's going to happen? The public, media, elected officials -- everyone will turn on BP and excoriate the oil giant for not coming up with a plan that has reasonable odds of working. Then when Top Kill fails, everyone will roll their eyes and say, "Of course it was going to fail. It was a lousy plan with only a 10 percent chance of success."

Now imagine the converse: there is actually a 90 percent chance that Top Kill will achieve its objective. And BP announces that. If the operation then succeeds, there will be a sigh of relief, but also a sort of collective shrug. What did you expect? After all, a 90 percent chance is the equivalent of an uncontested layup in the game of basketball. But if Top Kill failed, reaction would be furious: how could they screw up something with a 90 percent chance of working?

Now where's the sweet spot? It's a percentage that aligns with "cautiously optimistic." It's a percentage a little north of 50 percent -- but not too far north. It's a percentage that, if you fail, you can say, "Well, we knew from the beginning there was a large chance we weren't going to be able to do this," but if you succeed, you can say, "This was far from a sure thing, but we pulled it off, and congratulations to the great team at BP blah blah blah." That sweet spot, quantified: 60 to 70 percent.

Who knows what the actual chances of success were? I'm not an engineer, but once I learned a bit about Top Kill, it sounded fairly dubious. It sounded more like an operation with a 25 percent chance of working -- if that.

What if we wanted to find out what BP's top executives, in their heart of hearts, really thought were the odds of pulling off Top Kill?

Here's one way: the top 50 BP executives could've been forced to stake half of their wealth to a "futures" market on whether or not Top Kill would work. Sort of like betting on a prize fight. They would be allowed to trade in and out of odds that would fluctuate (much as is done for a heavyweight championship fight) depending on which position the money is favoring, and by how much ... and eventually, we'd get something resembling what BP really thought were its chances of plugging the leak. So, for instance, one guy would be betting half his wealth that Top Kill had a 65 percent chance of working (and would collect 35 percent if he won) and another executive at BP would be wagering it had a 35 percent chance of working (and would collect 65 percent if he won).

A little fanciful, and you may wonder -- well, who cares if BP lied to us (note: I have no idea if they did, but wouldn't be surprised) on the odds? Actually we should care because when the odds are 10 percent, not 60 to 70 percent, there's much more pressure to develop a "Plan B" -- and "Plan C" and "Plan D" as well. And I kind of wonder if the odds were at say 65 percent of Top Kill being a success -- and BP executives could take that position, but it would cost half of their wealth to play -- how many would've gamely said, "I'm in on that bet!" My guess: very, very few.