Is Diversification Really a Free Lunch?

Scooting around the Net today, I found myself checking out what Greg Mankiw has been up to (besides relentlessly plugging the half dozen textbooks he's written). I drop by his blog from time to time, even though he doesn't allow comments on his posts, which I find a bit strange. Dropping by feels like the equivalent of paying a visit to someone but only being allowed to peer in the house windows: You can watch what they're doing, and eavesdrop to your heart's content, but no talking please.

So I came across this New York Times column by the good Harvard professor, brimming with advice for the college bound. I nodded enthusiastically at his scold that high schools spend too much time on Euclidean geometry and trigonometry (when was the last time someone stopped you in the street and asked if you knew the cosine of the angle of the shadow being thrown by the lamp post on the corner?) and not enough on probability and statistics.

Amen, brother!

In fact, I found myself pretty much onboard with the whole piece -- until I reached this paragraph and started scratching my noggin a little:

The evidence of financial naïveté shows up every time some company goes belly up. Whether it is Enron or Lehman Brothers, many company employees are often caught with a large fraction of their wealth in a single stock. They fail to heed the most basic lesson of finance — that diversification provides a free lunch. It reduces risk without lowering expected return.

Okay, let's think about this. Note that, first of all, the good professor has provided a lopsided universe of examples. Yes, Enron and Lehman Brothers flamed out, spectacularly, and punched a saucer-plate sized hole through plenty of their employees' 401(k)'s. Very true. But how many Microsoft and Google millionaires have we also heard about, ordinary secretaries or maybe even guys who changed the water in the fish tank on weekends and scrubbed the toilets, who got a glory ride to early retirement on their company stock? Had they properly diversified, they might have made only enough to buy a used Vespa.

So am I anti-diversification? A heretic in the investing community? Diversification, after all, is one of the ten commandments of smart investing.

Nope. Not at all. I believe in a diversified portfolio (I personally own a mix of U.S. stocks, bonds, emerging market equities, Japanese shares -- ugh, cash-like instruments and cash itself).

The problem is, I think Mankiw's only half-right in his last two sentences.

I agree with: Diversification reduces risk without lowering (or increasing -- he neglects the corollary on the upside) expected return.

Simplification: You work at an S&P 500 company. Let's say the average yearly gain on the S&P is 8 percent. You invest in only your company's stock. For any given year, it may rise 8 percent -- or it may surge 22 percent, or conversely, fall 15 percent. Lesson: an individual stock can be quite volatile. But say you buy shares in 30 S&P companies instead. Some may go up, negating the declines of others, and at the end of the day, you'll probably have a smoother ride -- less volatility.

I strongly disagree with: Diversification provides a free lunch.

What you just witnessed in the example above isn't a free lunch. It's a smoother ride (to mix metaphors). To wit: there's a greater chance, in any year, your company's stock will soar 40 percent or plunge 40 percent than a basket of 30 stocks will do the same. By diversifying, you tamp down volatility. But for the basket, while the losses may be bounded at say 25 percent, the gains won't be as high either (let's say 25 percent to keep it simple).

So by diversifying, you lose your chance to become a Google millionaire, but you also won't have to eat cat food and sleep under the freeway overpass during your golden years.

Ah, but if only "diversification provides a free lunch" were only wrong ... no, it's worse than that. It's very dangerous, as the financial crisis attests to.

Consider that a collateralized debt obligation, or CDO, is the poster child of diversification. It's stuffed with mortgages from all across the country, and once you get into the strange beast known as the CDO squared, it's more bewilderingly complex (and more diversified). Investors obviously thought that, through diversification, they were getting a free lunch by buying up CDO tranches.

Why do I say that? Somehow they came to accept that a CDO could be worth more than the sum of its parts. They must have by definition, because the assembler of the CDO must be paid to put together and market the thing (and extract a profit). So if the mortgages contained within it collectively yield say 6.4 percent on average, the CDO genie will do his magic and transform them into tranches that collectively pay 6.1 percent on average (or whatever the typical spread is for these products).

So where did the 0.3 percent go, which was being paid for assuming a certain amount of extra risk? That's your diversification "free lunch" (arrived at by manipulating correlation numbers foolishly, as it turned out, through the Gaussian copula). Actually it gets even better: investors thought they were getting free dessert too! Because, remember, through the copula wizardy, the slices of the CDO earned high ratings while paying more than similar-rated debt! (A free eclair with that sandwich, sir?)

The "free lunch" was later revealed to be a "fraud lunch" when CDO prices cratered and the ratings turned out to be ridiculously inflated. However, here's what the illusion of the free lunch did: it spawned a long line of hungry investors, who couldn't get enough of these amazing CDOs, which created further demand for dicey mortgages, which got sausaged into more CDOs, until finally the whole sham famously exploded.

So beware of savants, even Harvard professors, touting "free lunches" in the investment world.