Math gone wild
Here’s the cautionary tale of one of those “math guys” or “quants,” and his brilliant formula—gone wrong.
It’s a sobering reminder of how spiffy new computer models and mathematical formulas related to real-world events can end up as just the old “garbage in, garbage out,” because we don’t know all the variables to include. In the case of something called “Li’s cupola function,” a beautiful mind came up with a beautiful mathematical formula that was applied by investors to mortgage risk, and it ended up spelling economic doom for most of us. Oops!:
It was a brilliant simplification of an intractable problem. And Li didn’t just radically dumb down the difficulty of working out correlations; he decided not to even bother trying to map and calculate all the nearly infinite relationships between the various loans that made up a pool. What happens when the number of pool members increases or when you mix negative correlations with positive ones? Never mind all that, he said. The only thing that matters is the final correlation number””one clean, simple, all-sufficient figure that sums up everything.
A few people warned that the map was not the territory, but their cautions were ignored, and why? Too many people were making too much money, that’s why. At least for a while:
Banks dismissed [warnings], partly because the managers empowered to apply the brakes didn’t understand the arguments between various arms of the quant universe. Besides, they were making too much money to stop.
Here’s the mind-boggling part, to my way of thinking. The following should have been a red flag the size of Texas:
[B]ecause the copula function used CDS prices to calculate correlation, it was forced to confine itself to looking at the period of time when those credit default swaps had been in existence: less than a decade, a period when house prices soared. Naturally, default correlations were very low in those years. But when the mortgage boom ended abruptly and home values started falling across the country, correlations soared.
Don’t blame Li—he just made the model, he didn’t apply it. Those who did were unaware of its limitations, partly because they didn’t get the math, partly because they decided to ignore history, and partly because there was gold in them thar hills.
I had the “opportunity” to take a few analysis courses. In every case we were presented with elegant formulae, elegantly explained. All one had to do to get a perfect projection of future reality was to plug in the correct assumptions, and data.
The young fellow teaching one course was an obvious devotee of the McNamara “Whiz Kid” school of conflict management. He was teaching a class of veteran Naval Officers. He tended to become very frustrated with the level of skepticism he encountered.
I am not opposed to the method. I only worry, as Neo cites, about the simple mindedness and naivete of some who use the method. Not to mention the occasional dishonesty.
Fortunately, climatology models don’t have such problems, and cannot, because we understand climatology so much better than the financial world around us.
Many persons ignored those oh so tricky “known unknowns”.
Many problems, it seems to my semi understanding mind, fall at the feet of the Boards of Directors of companies and banks.
Employees and CEOs face temptation to forgo ethical behavior in favor of grabbing fast money and then running to employment opportunity with another company. Boards of Directors ostensibly do not have opportunity to take fast money, and therefore ostensibly are not tempted, but rather exist specifically to watch out for the long term health of the company or the bank. Yet, in many many cases, Boards of Directors appear derelict in their duty.
I kept reading, knowing I was going to see Nassim Taleb’s name come up eventually, giving an equally elegant explanation why this was always crap and people shouldn’t have fallen for it.
Yep, the author of The Black Swan shows up near the end with exactly that.
People tend to forget that a model is basically an analogy, which necessarily and by definition must deviate from reality. The question is whether, how, and when it deviates from reality in aspects that are meangingful to the question being considered.
Hooke’s Law (the restoring force generated by a spring/rubber band is proportional to the displacement) works fine for small displacements. Larger displacements deviate from Hooke’s Law, until with a large enough displacement the spring/rubber band breaks, and so does the Hooke’s Law model.
Bottom line: a model is just a conceptual framework in which to think about a phenomenon. It’s not a law of nature. One has to use judgment in deciding if and when use of the model is appropriate, being ever mindful of its limitations.
GIGO: Garbage In, Garbage Out.
Good Will Hunting????
Occam’s I really have to disagree with you on the climatology thing. Neo quotes
“B]ecause the copula function used CDS prices to calculate correlation, it was forced to confine itself to looking at the period of time when those credit default swaps had been in existence: less than a decade, a period when house prices soared.”
If you look at that decade in comparison to how long the stock market has been around and compare that to the length of the time we have been keeping records (never mind accurate ones) on the climate and it’s effects it would be like a millionth of a nanosecond in the history of the world. So I don’t understand how you can say we can construct a more accurate mode?
Also there is the GIGO factor
“Sunday, 22 February 2009
A glitch in satellite sensors caused scientists to underestimate the extent of Arctic sea ice by 500,000 square kilometers (193,000 square miles), a California- size area, the U.S. National Snow and Ice Data Center said. “
Sorry, dane, that was trenchant sarcasm on my part.
For my money, climatologists might as well cast horoscopes, or runes, or read sheep entrails, to predict the climate. As pointed out previously, Jurassic Park was made with computer models too.
The amazing thing to me is that people aren’t more sceptical of hysterical bleatings from the left that we have to save the planet…by implementing the exact same policies that the left has been trying to get us to implement for 40 years. You’d think that that happy coincidence would send up a red flare or two, but apparently not.
Occam…love the Hooke’s Law analogy!
Writing 50 years ago, C S Lewis says the following about his sociologist hero in the novel That Hideous Strength:
“..his education had had the curious effect of making things that he read and wrote more real to him than the things he saw. Statistics about agricultural laboureres were the substance: any real ditcher, ploughman, or farmer’s boy, was the shadow…he had a great reluctance, in his work, to ever use such words as “man” or “woman.” He preferred to write about “vocational groups,” “elements,” “classes,” and “populations”: for, in his own way, he believed as firmly as any mystic in the superior reality of the things that are not seen.”
See my posts “the dictatorship of theory” and “management mentalities” (which cites some examples from Peter Drucker) at the link here.
most of my days are spent doing math for other people.
what i have learned about these situations is that the person who understands the math, the limits, the usages, and things it wont tell you. are not in a position to make sure those things are all considered.
i worked for fortuen 10 companies at one point… interesting… but one day, i did some extra work on the mainframe… i was bored. so i did this thing that checked out numbers and stuff with products and drew all kinds of conclusion values… number of x in y, etc.
never in my career did i EVER do that again. why? because i was pummled for a week because management wanted to know what book i got the math from. during that time, computer scientists were degreed scientists and were corporate, we were hired for our math skills (they have since clerkified it a lot since those days)…
so basically they wanted to know what book an appllied mathemetician would use to get answers from data.
well, same with quants.
teh ability for the rational scientist to contrl or mediate the actions of others who use their work is famously nil. (think nuclear bombs, nerve toxins, and a long list follows).
the real problem is the meeting of these things. the quants who make a VERY good tool… the investors and management who want to use that tool… and the longer they use it the more they think it can work everywhere, and at any time.
when things start to go screwy, in order for the quants to actually do something and prevent it, they would have to convince the management, shareholders and those who control their lives to a large degree to quit while they are ahead and stop for a while.
after all, before the market plunge its was at its highest and these magic tools gave great answers.
just as you cant stop people from buying a house that they could never afford if the complete deciisoin is up to them… do not think for a minute that a quant could do something about it.
he would be fire,d and they would go forward. if he was a share holder, he would be removed saying that he is trying to hurt the company… if he shut off the software to save them, his life would be ruined and he couldnst show the disaster…
EVERYONE so far keeps imagining levels of control that people in situaitions dont have… and imagining more ability and control from politicians which they too dont have.
the creator of the nuclear bomb could not control what others would do with it.
but who do we blame for it? we blame them for creating it… not the people for saying yeah yeah thats the ticket, lets do that with it. (like the suggestion to build a new panama canal in one month using them).
the market spends 90% of its time in its bull or bear cycles sans crisis…
the problem is that the personality types that generally take up the reins of power are seldome capable of the level of self control necessary to do nothing.
in fact, its a fault of neural nets. unless trained to, they always spit out some answer.
just walk up to someone and ask them what is the sound of one hand clapping… you will literally go through huge numbers of people before someone will say “enlightenment” is the one answer being looked for.
whats funnier is that the lefts tend to give great complex answers and reasons… the right do do, but they tend to temper it with informing that they dont know (more often), but they keep guessing or asserting..
its how we work… its always easy for someone not there or not the key in to say that something stupid should be done… like keep using the software… .
whats hard is to reason with idiots till they understand why they should not.
the outcome is the cassandra syndrome… where you get screwed worse… and the quants know this the way i do since they are adults with high smarts and get blamed a lot for what they notice…
Fortunately, climatology models don’t have such problems, and cannot, because we understand climatology so much better than the financial world around us.
oh and occam.. you know that computers do math to the infinite decimal place too… so errors dont accumulate the more the results are fed in as inputs. so they dont decline in accuracy the longer they are run…
(oh, how i blow people away by showing them a paper on rounding. just plain old rounding. my son who is on math scholarship, laughed at that one too. then i gave him the huge paper on the subject and he was like… wow, i never even had a hint that that one tiny thing was so fraught with things!)
Doesn’t bode well for the 2010 census now does it!
Later
Artfldgr….you may already know this, but rounding errors contributed to at least one tragedy, during Gulf War I, when a Patriot missile battery lost the track on a SCUD because of an accumulation of rounding errors. Apparently the problem had not previously occurred because the system hadn’t been operated continuously without rebooting for so long before. The Israelis had just detected the problem and told the Americans…a patch was on its way when the incident occurred.
If this article is true, and it rings that way, it largely corroborates something I’ve said on this blog before: a failure of due diligence was a major contributor to the economic problems we’re having right now. In this case, the failure occurred when people who didn’t understand the math, to include its severe limitations, used it as the basis for major investment decisions. These were the bankers, the investment managers, and the executives, who didn’t fully grasp what the quants’ models could–and couldn’t–do, but who didn’t ask the right questions, insist on getting the right answers, or even want to know. Why ruin the party? In hindsight, maybe a few embarrassing questions would have been appropriate. A few more fund managers saying, “I don’t know how your model gets from A to Z. Walk me through the steps, and tell me the variables you plug in at each point. And tell me how you decided to use those variables.” They may still have screwed it up. But at least they would have known how the model did what it did.
Artfldgr, I well remember from my days as an academic seeing calculations to 12 decimal places involving numbers known to maybe two significant digits. The passing of the slide rule era was a dark day indeed.
The point made by waltj and others that decision-makers clearly did not understand the math would seem to suggest a potential opportunity, except that, paradoxically, one could broke waiting for the herd to realize they’d made a mistake.
Artfldgr:
I remember seeing a TV show, probably Nova, years ago that had climate models giving wildly different predictions depending on the number of decimal places in the temperatures that were input. Now that I think about it, the show was probably about chaos theory.
This was long before the global warming cult got started, but it seems germane.
You have performed a service by providing access to this information. I wrote some software once for a bond broker. all it had to do was be able to figure out, at any given time, the value of a share of a fund based on the time the share was bought and which bonds at which maturity dates at what interest rates were in it. That was 2 + 2 compared to what Li was trying to do.
But, really, if his universe didn’t even go back to the dot-com bubble it seems somebody should have squeaked.
Well, somebody did. But who knew? Literally. It seems like the financial press here was about as diligent as they were with Obama.
Same thing with Madoff. Several people said he couldn’t be doing what he said he was. Those people were told to shut up and get out of the way.
If someone answers an question with “shut up” or “everybody knows,’ suggests that the matter is too trifling to warrant discussion, implies you’re a moron, or some such similar answer, then you know you’ve got ’em.
People will bore you to death with excruciating detail when they have the goods. When they don’t have the goods, they act as though the objection is beneath consideration, bluffing you and hoping to God you’ll let the matter drop.
Don’t. It’s much more entertaining to smile sweetly, calmly agree that they’re so bright that such trifles are beneath them, but as a lesser light you’re asknig their assistance…and then bore in with the tenacity of a pit bull. Ignore patronizing comments, but refuse to be put off, make clear that you’ve got all the time in the worlld, and insist on a complete, factual response that is hermetic in its logic.
In a public situation, the utterly characteristic look of stark terror is unmistakable.
Damn, I miss oral examinations.
“Don’t blame Li–he just made the model, he didn’t apply it. Those who did were unaware of its limitations, partly because they didn’t get the math, partly because they decided to ignore history, and partly because there was gold in them thar hills.”
You done broke the code, Neo.
In the CNBC special, “House of Cards,” the various players all excused their actions by the fact that it all looked good/plausible and there was so much money being made who wanted to question success. Or that other old chestnut, “If I didn’t do it someone else would.” It was not unlike many other bubble manias.
Easy money is very intoxicating and we’re now experiencing the hangover. We will swear off that brand of intoxication, but eventually someone will find a new vein to mine and another, different kind of bubble will arise.
In the meantime we are still waiting for an explanation of how the Treasury and FED are going to help the banks unwind these opaque secutities and unfreeze the credit markets. Dow up today, but………….????
“Math gone wild”
2% of the wealthiest American’s are going to pay for the several trillion dollar spendulus program, everybody else gets a free ride and a tax break. Carbon caps aren’t going to result in severely inflated prices for automobile and energy prices…
Oops guess this belongs in the commentary on Obama’s baloney session tonite…
GIGO: Garbage In, Garbage Out.
Way back when I wrote computer programms for microwave antenna design We used to joke that GIGO means garbage in, gospel out.
People would believe nonsense as long as it was on a computer printout.
DAvid, it sounds vaguely familiar, but to be honest i dont remember that! so… thank you so very much for adding to my voluminous repetoire! you will find that this and other things (like length conversion dooming a space mission) are not uncommon… just dont think about it when your on a plane and realize the same people write that software too. 🙂
Occam, and rickl, thanks so much for your comments. its a desert here.. 🙂
and nolanimrod… i feel your pain… frustration… and horror knowing that all over there is similar.
i lost a job from a firm that did 75 million in PROFIT per year with less than 20 employees. i use it as my example that being smart does not equate success… they could make 75 millino a year, but they fired me because i couldnt number to 750,000,000 (ten mass mailings for a illegal contest), in 8 numeric places ONLY using numbers.
yes, marketing convince the boss that i was just being difficult and wouldnt use my intellgence to help them.. [people hate the intellgent, and leftists especially hate them. Why do so many people hate this girl simply for being clever? http://www.dailymail.co.uk/news/article-1152008/Your-starter-Why-hate-girl-simply-clever.html
they wanted to scan $100 dollar bills and put them in their mailing to grab attention… they decided that saving them from the secret service freezing their business, computer and assets was not good either.
Same thing with Madoff. Several people said he couldn’t be doing what he said he was. Those people were told to shut up and get out of the way.
look at the article link i put (i am trying not to post to be shorter), you will see the effect yourself. the people who saw thigns no one else did were intelligent (not cargo cult intelligentsiatic posers)..
but since the less intelligent couldnt understand them, and they had so much self esteem and equality training, they discount them. (i watch this at the hospital where only medical stuff that is dumb enough gets worked on since one cant get permission from the appointees to do the work since they dont understand things!!!)
if you look at the article, and read the commentary from those online who commented negatively on this, you will see the parallel to Lee attacking me, and others… they sound like one voice…
well, just look at the prior thread where lee dug into me because i didnt use spell check. carco cult concept of genius… and twisted other things.
its THESE people.. elevated above their ability wno are the problem. theya ctually cant compete at the level they have been put into. they think that by being put into it, they have been vetted by others as to being equal. but one only has to see that they cant perform. so they spend all their time smacking and intimidating the truly smart with all kinds of little games… (meanwhile, there is no benifit to them for doing this).
no boss thats appointed feels safe with employees who are dumber than them. switching from merit and accepting reality means that we appointed people to higher positions than they can do, and so they are threatened if anyone is percieved to have ability to take that away (whether they do or not, false premises are just as valid to a believer as real ones are). so they get rid of them. i dont work in academia, so i usually dnot work long unless i am somplace where the ruler appointee realizes i dont want to move upwards! (then i am a darling for a while till somone else breaks us up to prevent our collusion from trumping them – sigh)
it took me years to do an end aroudn this system.. now they are about to license and patent new tech i have because theya re hard up for the cash… (i hope neo will smile big when she reads my real name in the paper next year… 🙂 )
the most frustrating part is that the smart guys are over helpful and usually nice… they wull let others “borrow their minds”: if harvard type ed hasnt gotten to them… yet so many would rather alienate them in favor of more positive heroes… like paris hilton, or maybe che, or richard speck.
thanks again for the comments!!!
oh. forgot to mention.. madoff is a different beast…
theer were one of those scrubbed news things.. .two of his big clients were russian mob and drug cartels. if they run business like the italian mob runs restaurants when people team up with them for profit, there would be a reason why madoff is always smiling like a man who has been reprieved from hell and will only get a breaking on the wheel instead.
[no one notices the strategic importance of what maddoff was doing… when isreal was created, who funded a lot of it? when there were wars, who funded weapons? if there was a war, or an altercation that was big enough, who would have taken their assets of large amount, and poured them into the cause on the side of israel? perhaps the many many many targeted jewish wealthy? its not a good analysis… its stretching things way far… but there is no reason that this cant be a part of it.. he got in too deep, they then worked him… he then was trapped till it all fell apart not by his actions… now he is free… or his family is free… that would fit well… but is complete supposition with no basis other than an association that was mentioned…
This is a typical, the most common error of misuse of statistics. These Gaussian approximations are so convinient to use, that is hard to resist temtation to do so even if underlying empirical data do not suggest anything Gaussian about them. That how most applied math works: simplify original formulation of the problem at hand until it becomes tractable (original formulations usually are not). Than pray that the results are at least qualitatively resemble those of original setting, because there is no way to know it for sure. Only well-trained mathematicians can fully appreciate the futility of their discipline when applied not to ideal objects created by their imagination, but to real-world problems (technical and engineering problems sometimes also belong to “real-world” category, while mostly not; we at least attempt to make artificial constructions obey well-known and simple laws). But really complex natural systems, like economics, society, climate, which are products of self-organization and historical development, often do not allow this simplistic engineering approach.
When those who build models and those who use them belong to different subcultures and types of mentality, misunderstanding is inevitable. Limitations of models can be obvious only to builders, but users can be completely unaware of them. This is the case with climate models, at least: built as laboratory tools only to study climate sensitivity to different parameters change, one parameter only changed in each experiment, they were used as prognostic devices – but they have no such power at all! They do not include interplay of similtaneously changing all hand-introduced parameters: this is a problem for another parameter space, of higher dimensionatity, and it is not tractable even algorithmically, to set apart calculateability issues.
Several times I witnessed the same comical misunderstanding: the lecturer, say, Dr. Shmuck, gives a public lecture about how Universe come into being. After two hours of barely understandable formulas, diagramms and profesional jargon the audience is full of awe, and nobody, it seems, is aware that the subject was not Universe creation, but Dr. Shmuck’s model of it, and that there exist dozens other cosmogonical models as well, with completely different assumptions and methologies (and conclusions).
To your excellent points, sergey, let me add the classic: nitwits who airily characterize general relativity as “everything’s relative” and think they’ve said something profound/ insightful/ intelligent / not completely moronic.
Fortunately, climatology models don’t have such problems, and cannot, because we understand climatology so much better than the financial world around us.
And that is why we can be sure that global warming will not be much much worse than the models predict.