"Why Economic Models Are Always Wrong"

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Post Reply
User avatar
Topic Author
Dan Moroboshi
Posts: 866
Joined: Sat Jul 07, 2007 7:10 pm

"Why Economic Models Are Always Wrong"

Post by Dan Moroboshi » Thu Oct 27, 2011 9:44 am

When it comes to assigning blame for the current economic doldrums, the quants who build the complicated mathematic financial risk models, and the traders who rely on them, deserve their share of the blame. [See “A Formula For Economic Calamity” in the November 2011 issue]. But what if there were a way to come up with simpler models that perfectly reflected reality? And what if we had perfect financial data to plug into them?

Incredibly, even under those utterly unrealizable conditions, we'd still get bad predictions from models.

The reason is that current methods used to “calibrate” models often render them inaccurate.
.
.
.
Almost all models have parameters that have to be adjusted to make a model applicable to the specific conditions to which it's being applied--the spring constant in Hooke's law, for example, or the resistance in an electrical circuit. Calibrating a complex model for which parameters can't be directly measured usually involves taking historical data, and, enlisting various computational techniques, adjusting the parameters so that the model would have "predicted" that historical data. At that point the model is considered calibrated, and should predict in theory what will happen going forward.

Carter had initially used arbitrary parameters in his perfect model to generate perfect data, but now, in order to assess his model in a realistic way, he threw those parameters out and used standard calibration techniques to match his perfect model to his perfect data. It was supposed to be a formality--he assumed, reasonably, that the process would simply produce the same parameters that had been used to produce the data in the first place. But it didn't. It turned out that there were many different sets of parameters that seemed to fit the historical data. And that made sense, he realized--given a mathematical expression with many terms and parameters in it, and thus many different ways to add up to the same single result, you'd expect there to be different ways to tweak the parameters so that they can produce similar sets of data over some limited time period.
http://www.scientificamerican.com/artic ... ways-wrong

I'm sure this is no surprise to the finance professionals and academics who frequent this forum, but as a layman I thought this was interesting.

I had been advised, and had accepted, that expectations of ex-ante returns of a portfolio which is optimized based upon backtesting is bullplop. I guess I hadn't questioned why, exactly, that was so. This article lit up the lightbulb above my head, so I thought I'd share.

FafnerMorell
Posts: 686
Joined: Mon Sep 15, 2008 10:27 am

Re: "Why Economic Models Are Always Wrong"

Post by FafnerMorell » Thu Oct 27, 2011 10:08 am

Yeah, it's an interesting article - although a bit more "substance" to back it up would have been nice. Reminds me of having read through Taleb's "Black Swan", my chief takeaway was the rather underwhelming "Gauss curves aren't the answer to everything" - which, while undeniably true, then leads to the "So, what do we have that's better?". It seems like the answer is "We don't really know yet - some sort of chaos, unpredictable thingy, maybe" - which, knowing that you don't know what you thought you knew is something, but doesn't exactly satisfy. But I suppose the good news is we know not to believe those folks claiming they've got it all figured out.

User avatar
Taylor Larimore
Advisory Board
Posts: 28965
Joined: Tue Feb 27, 2007 8:09 pm
Location: Miami FL

Re: "Why Economic Models Are Always Wrong"

Post by Taylor Larimore » Thu Oct 27, 2011 10:15 am

Hi Dan:

Thank you for sharing the article which contained this observation:
That financial models are plagued by calibration problems is no surprise to Wilmott--he notes that it has become routine for modelers in finance to simply keep recalibrating their models over and over again as the models continue to turn out bad predictions. "When you have to keep recalibrating a model, something is wrong with it," he says. "If you had to readjust the constant in Newton's law of gravity every time you got out of bed in the morning in order for it to agree with your scale, it wouldn't be much of a law But in finance they just keep on recalibrating and pretending that the models work."
In my opinion, it is futile for most of us to try to "beat the market." More often than not, the result will be under-performance in risk, return, or both.

Bogleheads structure a personal low-cost, tax-efficient, simple, diversified, asset-allocation plan--then stay-the-course.
"Simplicity is the master key to financial success." -- Jack Bogle

User avatar
bobcat2
Posts: 5370
Joined: Tue Feb 20, 2007 3:27 pm
Location: just barely Outside the Beltway

Risk Management Rather than Forecast-and-Plan

Post by bobcat2 » Thu Oct 27, 2011 10:50 am

Finance professor John Cochrane is someone I don't always agree with, but in this case he is spot on in the difference between how economists use economic forecasts and how others use the same forecasts.
Financial forecasting is next to useless....

If anyone could tell you with any sort of certainty that “the market will go up tomorrow,” you could use that information to buy today and make a fortune. So could everyone else. As we all try to buy, the market would go up today, right to the point that nobody can tell whether tomorrow’s value will be higher or lower.

An “efficient” market should be unpredictable. If markets went steadily up and delivered return without risk, then markets would not be working as they should....

The answer is to change the question, to focus on risk management, as Gardner and Tetlock suggest. There is a set of events that could happen tomorrow—Chicago could have an earthquake, there could be a run on Greek debt, etc. Attached to each event, there is some probability that it could happen.

Now “forecasting” as Gardner and Tetlock characterize it, is an attempt to figure out which event really will happen, whether the coin will land on heads or tails, and then make a plan based on that knowledge. It’s a fool’s game.

Once we recognize that uncertainty will always remain, risk management rather than forecasting is much wiser. Just the step of naming the events that could happen is useful. Then, ask yourself, “if this event happens, let’s make sure we have a contingency plan so we’re not really screwed.” Suppose you’re counting on diesel generators to keep cooling water flowing through a reactor. What if someone forgets to fill the tank?

The good use of “forecasting” is to get a better handle on probabilities, so we focus our risk management resources on the most important events. But we must still pay attention to events, and buy insurance against them, based as much on the painfulness of the event as on its probability. (Note to economics techies: what matters is the risk-neutral probability, probability weighted by marginal utility.)

So it’s not really the forecast that’s wrong, it’s what people do with it. If we all understood the essential unpredictability of the world, especially of rare and very costly events, if we got rid of the habit of mind that asks for a forecast and then makes “plans” as if that were the only state of the world that could occur; if we instead focused on laying out all the bad things that could happen and made sure we had insurance or contingency plans, both personal and public policies might be a lot better.
Link to article. http://www.cato-unbound.org/2011/07/15/ ... hedgehogs/

BobK
In finance risk is defined as uncertainty that is consequential (nontrivial). | The two main methods of dealing with financial risk are the matching of assets to goals & diversifying.

User avatar
baw703916
Posts: 6681
Joined: Sun Apr 01, 2007 1:10 pm
Location: Seattle

Re: "Why Economic Models Are Always Wrong"

Post by baw703916 » Thu Oct 27, 2011 11:09 am

Simple ≠ reality

You can approximate reality with varying degrees of simplicity, depending on which aspects of it you are trying to capture.

But one must remember it's always an approximation.

EMH is a good approximation to reality, and a good basis for portfolio construction. But it has to be wrong, logically, if you push it hard enough.
Most of my posts assume no behavioral errors.

Fallible
Posts: 7082
Joined: Fri Nov 27, 2009 4:44 pm
Contact:

Re: "Why Economic Models Are Always Wrong"

Post by Fallible » Thu Oct 27, 2011 11:23 am

This is an interesting article. Nearly every book I've read on the '08 crisis (and also on the '98 LTCM fiasco) mentions the inadequacy of models - of the models themselves, the people who created them, and the people who seem to have deliberately misused them or continued to believe in them even when it was becoming obvious that they must be misleading or flat wrong. But the "Scientific American" article seems to shed new and needed light on the technical aspects of the models and the failure of "current methods." Maybe, just maybe, better understanding of models and better methods to create them will help avoid another crisis. But I wonder whether even a perfect model is possible since it will be created and used by less-than-perfect humans. :?:
John Bogle on his often bumpy road to low-cost indexing: "When a door closes, if you look long enough and hard enough, if you're strong enough, you'll find a window that opens."

User avatar
nisiprius
Advisory Board
Posts: 39790
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: "Why Economic Models Are Always Wrong"

Post by nisiprius » Thu Oct 27, 2011 11:31 am

The reductio ad absurdam of parameter fitting is this wonderful, wonderful posting Vanguard Tracking Portfolio II (aka The 8% Solution) in which user SirHorace displays a backfitted portfolio of Vanguard funds that closely tracks an ideal compound-interest 8%/year growth. I don't think I'm being snarky or ironic here, because I don't think he really intended it seriously. I really wish he'd kept posting. I'm too lazy to do it myself but I'm curious to know what that portfolio has done since August of 2007.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.

User avatar
House Blend
Posts: 4657
Joined: Fri May 04, 2007 1:02 pm

Re: "Why Economic Models Are Always Wrong"

Post by House Blend » Thu Oct 27, 2011 11:43 am

I think the article needs to be retitled.

"Why economics is not science"

User avatar
baw703916
Posts: 6681
Joined: Sun Apr 01, 2007 1:10 pm
Location: Seattle

Re: "Why Economic Models Are Always Wrong"

Post by baw703916 » Thu Oct 27, 2011 2:21 pm

Actually, models in physical science are "always wrong," too. You have to make some simplifications to make it mathematically possible to get an answer. The key is in realizing the assumptions inherent in the model, their limits of applicability, and in not trusting the model predictions beyond that point without independent verification. Models aren't dangerous, but treating them as black boxes can be.
Most of my posts assume no behavioral errors.

User avatar
mlebuf
Posts: 1915
Joined: Tue Feb 20, 2007 8:27 pm
Location: Paradise Valley, Arizona

Re: "Why Economic Models Are Always Wrong"

Post by mlebuf » Thu Oct 27, 2011 2:49 pm

Sophisticated Economic Modeling = Physics Envy :wink:
Best wishes, | Michael | | Invest your time actively and your money passively.

User avatar
Topic Author
Dan Moroboshi
Posts: 866
Joined: Sat Jul 07, 2007 7:10 pm

Re: "Why Economic Models Are Always Wrong"

Post by Dan Moroboshi » Thu Oct 27, 2011 3:05 pm

mlebuf wrote:Sophisticated Economic Modeling = Physics Envy :wink:
Image

So where are the economists? Probably some weird hybrid of psychologists, historians, and mathematicians.

richard
Posts: 7961
Joined: Tue Feb 20, 2007 3:38 pm
Contact:

Re: Risk Management Rather than Forecast-and-Plan

Post by richard » Thu Oct 27, 2011 3:22 pm

bobcat2 wrote:Finance professor John Cochrane is someone I don't always agree with, but in this case he is spot on in the difference between how economists use economic forecasts and how others use the same forecasts.
Financial forecasting is next to useless
Forecasting financial markets is likely close to useless, for efficient market reasons.

Forecasting the macro economy does not suffer from the same problems. (It has its own problems.)

On calibrating macro-models, see http://krugman.blogs.nytimes.com/2011/1 ... t-wonkish/ and the linked articles.

Fallible
Posts: 7082
Joined: Fri Nov 27, 2009 4:44 pm
Contact:

Re: "Why Economic Models Are Always Wrong"

Post by Fallible » Thu Oct 27, 2011 3:24 pm

baw703916 wrote:Actually, models in physical science are "always wrong," too. You have to make some simplifications to make it mathematically possible to get an answer. The key is in realizing the assumptions inherent in the model, their limits of applicability, and in not trusting the model predictions beyond that point without independent verification. Models aren't dangerous, but treating them as black boxes can be.
As I recall, this is similar to what happened in the LTCM case, where the models had become outdated but continued to be relied upon. And I think there were other examples as the '08 crisis began to unfold where the models no longer reflected new financial realities, e.g., the fancy new financial products like CDOS.
John Bogle on his often bumpy road to low-cost indexing: "When a door closes, if you look long enough and hard enough, if you're strong enough, you'll find a window that opens."

User avatar
bottlecap
Posts: 6255
Joined: Tue Mar 06, 2007 11:21 pm
Location: Tennessee

Re: "Why Economic Models Are Always Wrong"

Post by bottlecap » Thu Oct 27, 2011 5:45 pm

Do scientists have science envy? Every time the subject of economics comes up, people argue about whether it's a science. What does it matter? Does something being a "science" lend it more credibility? Not by any generally accepted definition of "science." Does it make it any less useful (or useless)? I'm sure few people consider criminal behavior a science (at least not any more than economics), but the police sure find it useful.

Perhaps its not that economics is not a science, but that its not a "hard" science or an "exact" science. Does that reduce the envy?

JT

User avatar
nisiprius
Advisory Board
Posts: 39790
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: "Why Economic Models Are Always Wrong"

Post by nisiprius » Thu Oct 27, 2011 6:23 pm

bottlecap wrote:Do scientists have science envy? Every time the subject of economics comes up, people argue about whether it's a science. What does it matter? Does something being a "science" lend it more credibility? Not by any generally accepted definition of "science." Does it make it any less useful (or useless)? I'm sure few people consider criminal behavior a science (at least not any more than economics), but the police sure find it useful.

Perhaps its not that economics is not a science, but that its not a "hard" science or an "exact" science. Does that reduce the envy?

JT
Plenty of sciences manage to be useful without pretending to be quantitative. Psychology, for one. Archaeology, for another. Most of ecology, for another. Criminologists do not fill up page after page with equations describing precisely how criminals would behave if they behaved in some idealized manner that is different from the way they do behave.

The thing that is annoying about economics, and leads to accusations of "physics envy," is that it pretends to be a century or two more advanced than it really is. It is as if Dr. Maturin in Patrick O'Brian's novels were to use a table of logarithms to balance the four humors to three decimal places.

In Jack London's novel, The Sea-Wolf, there are references to "Dr. [David Starr] Jordan's final test of truth." His test of truth is "Can we make it work? Can we trust our lives to it?" (David Starr Jordan was president of Stanford University and a famous person at the time).

Airliners and suspension bridges can in fact be made to work, and I trust my life to them routinely. I may not feel quite the same degree of confidence being prepped for surgery, but surgery, generally, can be made to work and I've trusted my life to it a couple of times and haven't been disappointed so far.

Would you trust your life to an economist's equation? Personally I wouldn't even trust my money to it.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.

User avatar
bottlecap
Posts: 6255
Joined: Tue Mar 06, 2007 11:21 pm
Location: Tennessee

Re: "Why Economic Models Are Always Wrong"

Post by bottlecap » Thu Oct 27, 2011 8:43 pm

nisiprius wrote:Plenty of sciences manage to be useful without pretending to be quantitative. Psychology, for one. Archaeology, for another. Most of ecology, for another. Criminologists do not fill up page after page with equations describing precisely how criminals would behave if they behaved in some idealized manner that is different from the way they do behave.
I agree that macroeconomic models are of limited economic use and should only be used to test sound economic theory, not the other way around. There are economists that don't believe so strongly in the utility of the models but we just don't listen to them. We want to believe rather than acknowledge that there is no clear way to predict all of the variables.

JT

Valuethinker
Posts: 39278
Joined: Fri May 11, 2007 11:07 am

Re: "Why Economic Models Are Always Wrong"

Post by Valuethinker » Fri Oct 28, 2011 4:43 am

Fallible wrote:This is an interesting article. Nearly every book I've read on the '08 crisis (and also on the '98 LTCM fiasco) mentions the inadequacy of models - of the models themselves, the people who created them, and the people who seem to have deliberately misused them or continued to believe in them even when it was becoming obvious that they must be misleading or flat wrong. But the "Scientific American" article seems to shed new and needed light on the technical aspects of the models and the failure of "current methods." Maybe, just maybe, better understanding of models and better methods to create them will help avoid another crisis. But I wonder whether even a perfect model is possible since it will be created and used by less-than-perfect humans. :?:
Compounding this your annual bonus depends upon using a model to achieve a result.

Thus models are hopelessly biased. The downside may not be felt for years, so there is every tendency towards over optimism in the models.

Valuethinker
Posts: 39278
Joined: Fri May 11, 2007 11:07 am

Re: "Why Economic Models Are Always Wrong"

Post by Valuethinker » Fri Oct 28, 2011 4:56 am

nisiprius wrote:
bottlecap wrote:Do scientists have science envy? Every time the subject of economics comes up, people argue about whether it's a science. What does it matter? Does something being a "science" lend it more credibility? Not by any generally accepted definition of "science." Does it make it any less useful (or useless)? I'm sure few people consider criminal behavior a science (at least not any more than economics), but the police sure find it useful.

Perhaps its not that economics is not a science, but that its not a "hard" science or an "exact" science. Does that reduce the envy?

JT
Plenty of sciences manage to be useful without pretending to be quantitative. Psychology, for one. Archaeology, for another. Most of ecology, for another. Criminologists do not fill up page after page with equations describing precisely how criminals would behave if they behaved in some idealized manner that is different from the way they do behave.

The thing that is annoying about economics, and leads to accusations of "physics envy," is that it pretends to be a century or two more advanced than it really is. It is as if Dr. Maturin in Patrick O'Brian's novels were to use a table of logarithms to balance the four humors to three decimal places.

In Jack London's novel, The Sea-Wolf, there are references to "Dr. [David Starr] Jordan's final test of truth." His test of truth is "Can we make it work? Can we trust our lives to it?" (David Starr Jordan was president of Stanford University and a famous person at the time).

Airliners and suspension bridges can in fact be made to work, and I trust my life to them routinely. I may not feel quite the same degree of confidence being prepped for surgery, but surgery, generally, can be made to work and I've trusted my life to it a couple of times and haven't been disappointed so far.

Would you trust your life to an economist's equation? Personally I wouldn't even trust my money to it.
Distinguish between the forecasting ability of models and their understanding ability.

The reason models are necessary in economic science is otherwise you just have opinions. By rigorously modelling the outcomes of your assumptions, you get a logically consistent story-- forces you to think through the tradeoffs.

All social sciences use models of human behaviour, economics just uses explicitly mathematical ones. the best example I can think of is the John Nash scene in 'A Beautiful Mind' where he works out, from the behaviour of his colleagues seeking to pick up pretty co-eds in a 1950s Princeton college bar, 'Nash Equilibrium'-- a conclusion of Game Theory that your best move must be made in recognition of every one else making their own best move.

(in the movie, that translates as they should not all try to chat up the prettiest girl. That will leave her friends p'd off and everyone will go home alone. Rather your Nashian best move is to target one of her less pretty friends with a much higher probability of going home with you, because there is less competition).

That insight has shaped huge amounts of competition law and policy, corporate strategy, international diplomacy etc.

Over to models as forecasts for example macro models. Adding complexity and mathematical rigor does not, empirically, produce greater accuracy.

You are therefore generally better off with simpler models that have proven utility (such as the Keynesian IS-LM model) rather than elegantly rigorous and complex ones.

peter71
Posts: 3768
Joined: Tue Jul 24, 2007 8:28 pm

Re: "Why Economic Models Are Always Wrong"

Post by peter71 » Fri Oct 28, 2011 7:29 am

At least the short version of the article seems pretty muddled (I didn't read the long version). Is it just me or does the Paul Wilmott quote emerge out of nowhere, without him ever being introduced and after primarily discussing geophysical models?

Finance is trickier than other branches of social science because arbitrage opportunities disappear once known.

Other branches of economics and other quantitative social sciences are pretty much like epidemiology. These disciplines tend to produce findings of the sort, "controlling for all known confounders, it looks like smoking is highly associated with lung cancer, but of course we can't be absolutely sure we haven't failed to control for some unknown confounder."

Many smart (and not-so-smart) people think that sort of research isn't worth doing, but the fundamental difficulty with it has little do with reality constantly shifting, "model calibration," etc., etc.

Best,
Pete

User avatar
Hexdump
Posts: 1619
Joined: Fri Oct 10, 2008 7:28 am
Location: Houston, Texas

Re: "Why Economic Models Are Always Wrong"

Post by Hexdump » Fri Oct 28, 2011 8:06 am

Thanks for the article.
It's interesting and imho, obvious.
Whenever my wife talked me into going to see a financial planner (with advice I ignored), without exception the analyst would beat away on a keyboard, inputting our data into their analysis program which no doubt had been tweaked as observed in the article. And probably with specific weighing to favor whatever products the company favored.
More and better analysis is on this site.
Thanks Bogleheads.

User avatar
bottlecap
Posts: 6255
Joined: Tue Mar 06, 2007 11:21 pm
Location: Tennessee

Re: "Why Economic Models Are Always Wrong"

Post by bottlecap » Fri Oct 28, 2011 10:20 am

Valuethinker wrote:The reason models are necessary in economic science is otherwise you just have opinions. By rigorously modelling the outcomes of your assumptions, you get a logically consistent story-- forces you to think through the tradeoffs
This really isn't true. Although many economists today have "opinions," it is theory that reigned in economics prior to models and theory that forces you to think through tradeoffs. And behavioral economic theory can be tested without mathematical models - models can help confirm theory, but they certainly do not give you a logically consistent "story" (by which I think you mean "theory"). The Phillips Curve is a good example of a model that was used to produce a theory that to a large extent did not hold.


Valuethinker wrote:You are therefore generally better off with simpler models that have proven utility (such as the Keynesian IS-LM model) rather than elegantly rigorous and complex ones.
Simple doesn't mean it's any better, though. The Phillips Curve is another fine example of a very simple model - so much so that it was exposed as too simple.

I guess my thought is that models are often used as a substitute for theory in economics - come up with a rough model and then tweak so it seems to work, and then come up with a theory as to why the model works. Come to think of it, modern economics does seem to stand the scientific method on its head, which is think out a theory and test it in reality, rather than the other way around.

JT

User avatar
Epsilon Delta
Posts: 8090
Joined: Thu Apr 28, 2011 7:00 pm

Re: "Why Economic Models Are Always Wrong"

Post by Epsilon Delta » Fri Oct 28, 2011 11:51 am

bottlecap wrote:
Valuethinker wrote:The reason models are necessary in economic science is otherwise you just have opinions. By rigorously modelling the outcomes of your assumptions, you get a logically consistent story-- forces you to think through the tradeoffs
This really isn't true. Although many economists today have "opinions," it is theory that reigned in economics prior to models and theory that forces you to think through tradeoffs. And behavioral economic theory can be tested without mathematical models - models can help confirm theory, but they certainly do not give you a logically consistent "story" (by which I think you mean "theory"). The Phillips Curve is a good example of a model that was used to produce a theory that to a large extent did not hold.
I think you are using "mathematical model" to mean what I would call a "numerical model", i.e. a big matrix calculation. This use of the term "mathematical model" in a much more restricted than mine. I would even include things like supply and demand curves as mathematical models. By my definition pretty much all of economics theory is mathematical models.

I don't want to say that I'm right and your wrong but to rather to highlight a possible misunderstanding.

Sulvar
Posts: 293
Joined: Wed May 26, 2010 2:27 pm

Re: "Why Economic Models Are Always Wrong"

Post by Sulvar » Fri Oct 28, 2011 12:06 pm

Almost all models have parameters that have to be adjusted to make a model applicable to the specific conditions to which it's being applied--the spring constant in Hooke's law, for example, or the resistance in an electrical circuit.
I think that his examples (spring constant and resistance in an electrical circuit) are wrong or at least misused and confusing.

In Hooke's law, for example, F = -KX, where K is the spring constant, F is the force (in newtons), and X is the displacement (in meters). Thus -F/X = K. K is just a relationship between force and displacement and K is defined in units of Newton/meters. I don't see any model parameters that have been introduced to make this formula "applicable to the specific conditions to which it is being applied".

The same thing applies to resistance. V = IR (ohms law) where V is voltage, I is current and R is resistance. R is just a simple relationships between voltage and current. Again no model parameters have been introduced that "need to be adjusted to make the model applicable to the specific conditions to which it is applied".

plnelson
Posts: 707
Joined: Wed Feb 24, 2010 6:49 pm

Re: "Why Economic Models Are Always Wrong"

Post by plnelson » Fri Oct 28, 2011 12:26 pm

FafnerMorell wrote: Reminds me of having read through Taleb's "Black Swan", my chief takeaway was the rather underwhelming "Gauss curves aren't the answer to everything" - which, while undeniably true, then leads to the "So, what do we have that's better?"
Taleb doesn't claim that something is better. Taleb's position is that:

1. We've designed our financial systems and industries in such a way that they are uniquely sensitive to shocks. The intricacies and inter-dependencies of modern economies mean that a revolution here or a tsunami there or a sovereign debt default over there can play havoc with supply pipelines, oil prices, exchange rates and aggregate demand and create crises over here. Yet these shocks are inherently unpredictible.

2. Everyone - individuals, companies and governments - needs to focus a lot more than we do now on robustness. Rainy day funds, multiple sources of supply, low fixed expenses, diversification, insurance, and other ways of minimizing exposure to shocks or increasing the ability to withstand them should all receive more priority than simply maximizing return or profit. I heard Taleb recently on an hour-long interview and robustness was the centerpiece of his theme.

Fallible
Posts: 7082
Joined: Fri Nov 27, 2009 4:44 pm
Contact:

Re: "Why Economic Models Are Always Wrong"

Post by Fallible » Fri Oct 28, 2011 12:31 pm

Hexdump wrote:Thanks for the article.
It's interesting and imho, obvious.
Whenever my wife talked me into going to see a financial planner (with advice I ignored), without exception the analyst would beat away on a keyboard, inputting our data into their analysis program which no doubt had been tweaked as observed in the article. And probably with specific weighing to favor whatever products the company favored.
More and better analysis is on this site.
Thanks Bogleheads.
You may have seen a previous forum thread on planners and plans (including Vanguard's) with a ref to computer models at the end, but in case you didn't, here's the link:

http://www.bogleheads.org/forum/viewtop ... =74410&sid
John Bogle on his often bumpy road to low-cost indexing: "When a door closes, if you look long enough and hard enough, if you're strong enough, you'll find a window that opens."

gulliver
Posts: 94
Joined: Wed Aug 05, 2009 12:54 pm

Re: "Why Economic Models Are Always Wrong"

Post by gulliver » Fri Oct 28, 2011 2:57 pm

"Why Economic Models Are Always Wrong"

Lemonade stands, AAPL, Vanguard, dairy farms, the US currency etc. are all based on economic models. Anyone can see that the title of the article is simply untrue.

"When it comes to assigning blame for the current economic doldrums, the quants who build the complicated mathematic financial risk models, and the traders who rely on them, deserve their share of the blame."

In the first place, the "quants" that seem to be the target of the article do not get hired to "build complicated mathematic financial risk models." There are many different kinds of quants and the ones that really matter (and who receive the lion's share of compensation) are the ones who develop trading strategies that actually get executed. Other, lesser (frankly!), quants work on risk analysis and hedging strategies. The quants who build models are even lower on the totem pole and generally work under fierce time and financial constraints and basically resent the fact that by the time they get their models working, the target anomaly has disappeared and the firm is working on something else! Obviously, I am grossly oversimplifying the process but anyone who thinks that Wall Street traders rely on "financial risk models" to make their daily (trading) books, doesn't have sufficient knowledge of the subject to write an article about it. IMO. Now, get off my lawn!

I would also point out that the US (and the rest of the world) has done an excellent job of instigating financial panics (and the often ensuing) "economic doldrums" since long, long before the word "quant" existed!

User avatar
LH
Posts: 5490
Joined: Wed Mar 14, 2007 2:54 am

Re: "Why Economic Models Are Always Wrong"

Post by LH » Fri Oct 28, 2011 3:31 pm

FafnerMorell wrote:Yeah, it's an interesting article - although a bit more "substance" to back it up would have been nice. Reminds me of having read through Taleb's "Black Swan", my chief takeaway was the rather underwhelming "Gauss curves aren't the answer to everything" - which, while undeniably true, then leads to the "So, what do we have that's better?". It seems like the answer is "We don't really know yet - some sort of chaos, unpredictable thingy, maybe" - which, knowing that you don't know what you thought you knew is something, but doesn't exactly satisfy. But I suppose the good news is we know not to believe those folks claiming they've got it all figured out.

The Misbehavior of Markets; Benoit Mandelbrot

You may enjoy that book. It presents a not quite there, may never be there, attempt to address that "thingy" to which you refer. So instead of saying, thingy, you can think "maybe a power law or something" but still be essentially just as clueless, maybe just be more grounded in cluelessness : )

User avatar
LH
Posts: 5490
Joined: Wed Mar 14, 2007 2:54 am

Re: "Why Economic Models Are Always Wrong"

Post by LH » Fri Oct 28, 2011 3:36 pm

Dan Moroboshi wrote:
mlebuf wrote:Sophisticated Economic Modeling = Physics Envy :wink:
Image

So where are the economists? Probably some weird hybrid of psychologists, historians, and mathematicians.
between the biologist and psychologist. Maybe adjacent to and slightly left of the biologist.

User avatar
og15F1
Posts: 347
Joined: Wed Dec 08, 2010 12:08 am

Re: "Why Economic Models Are Always Wrong"

Post by og15F1 » Fri Oct 28, 2011 9:56 pm

AKA why all calibrated models are always wrong / not 100% accurate

Newsflash

User avatar
steve roy
Posts: 1735
Joined: Thu May 13, 2010 5:16 pm

Re: "Why Economic Models Are Always Wrong"

Post by steve roy » Fri Oct 28, 2011 11:36 pm

One of my best friends is an economist with a PhD from Cornell.

He's been teaching and practicing economics (he now has his own company) for thirty-five years. He says this about economics.

"Economics is ALWAYS inexact, has to be. Because NOBODY can know with certainty what 400 million people are going to do with their money on any given day."

Erwin
Posts: 1929
Joined: Fri Apr 27, 2007 11:16 pm

Re: "Why Economic Models Are Always Wrong"

Post by Erwin » Fri Oct 28, 2011 11:45 pm

Read my article "What is the Current State of Economic Science?" in
http://mises.org/daily/4886

Bottomline, economy is about human behavior, and we are far from being able to develop models to simulate it.
Erwin

Valuethinker
Posts: 39278
Joined: Fri May 11, 2007 11:07 am

Re: "Why Economic Models Are Always Wrong"

Post by Valuethinker » Sat Oct 29, 2011 2:03 am

steve roy wrote:One of my best friends is an economist with a PhD from Cornell.

He's been teaching and practicing economics (he now has his own company) for thirty-five years. He says this about economics.

"Economics is ALWAYS inexact, has to be. Because NOBODY can know with certainty what 400 million people are going to do with their money on any given day."
Unless it's an unbiased estimator, Gaussian distribution (mean 0, dispersion 1) :wink: :wink:

djw
Posts: 1456
Joined: Tue Apr 08, 2008 9:34 pm

Re: "Why Economic Models Are Always Wrong"

Post by djw » Sat Oct 29, 2011 3:46 am

Here's my simple, layman's analysis of the difference between a "model" of ANY kind and the real world that we all live in:

A map (such as a highway road map, a globe, etc.) is a model of the surface of the Earth, but no one should mistake it for being an actual, complete, dynamic, scale model of the ACTUAL Earth... which is ever so slightly more complex than the map itself...
Love many, trust few, and always paddle your own canoe

User avatar
grayfox
Posts: 5280
Joined: Sat Sep 15, 2007 4:30 am

Re: "Why Economic Models Are Always Wrong"

Post by grayfox » Sat Oct 29, 2011 5:51 am

The title "Why Economic Models Are Always Wrong" carries the implication that the article well provide an explanation for an assertion that "economic models are always wrong".

Unfortunately, the assertion "economic models are always wrong" is false. The statement is too broad. Another way to phrase it is, no economic model ever has, nor ever will, give the correct answer. Stated this way it is obviously false. There are plenty of economic models that are right some of the time. In fact, there are plenty of simple economic models that give a correct point forecast with 99.999999% accuracy. Or even more complex models that give a correct range of solutions nearly all the time.

Now if the assertion were that many economic models are often wrong, that could be a true assertion. But as stated, the assertion that the author explaining is false. It makes little sense to provide an explanation for a false assertion. It's like when the scientist wrote a scientific paper titled "Why bumblebees can not fly".

Post Reply