The Illusion of "The Long-Term Average" in Investing

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
User avatar
Topic Author
SimpleGift
Posts: 3396
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

The Illusion of "The Long-Term Average" in Investing

Post by SimpleGift » Mon Mar 05, 2018 3:38 pm

There are no fixtures in nature.
The universe is fluid and volatile.
Permanence is but a word of degrees.
~ Ralph Waldo Emerson (1803-1882)

To perceive something as fixed when it is actually transient is usually considered a foolish mistake. Yet, we do it all the time in investing when we treat historical data series as stationary — i.e., in statistical terms, as if their means and variances are constant over time. In reality, nearly all of the historical data sets we encounter as investors are non-stationary — their means and their variances in one time period do not apply in another. Thus the illusion of The Long-Term Average.

FIRST VIEW: As one example, charted below is a time series showing the estimated, real "risk-free" interest rate over eight centuries, from 1314-2017. Drawn across the chart is a dashed line (in red) indicating the 704-year average, at 4.78%. Consciously or unconsciously, this line registers in most investors' minds as the "normal" historical real interest rate. Using this chart, it's easy to get the impression that interest rates are a stationary series with a fixed mean — leading one to naturally wonder, "When will today's rates start reverting back toward their long-term average?"
SECOND VIEW: Statistical tests of this same data would show that, in fact, interest rates are NOT a stationary time series. As a simple proof, when the average interest rate for each of the eight centuries is calculated and charted (in light blue, below), we see that the mean value changes, sometimes dramatically, from one century to the next. This is the very definition of a non-stationary data series — and it makes the idea of "reversion to the mean" for interest rates nonsensical. What mean? Which time period?
CONCLUSION: Non-stationary historical data series are by far the norm in finance and investing. So the next time a pundit shows you a chart like the one below, suggesting that "U.S. stocks are dangerously overvalued today, compared with their historical average," I'd encourage you to kindly ignore their comments and remember the illusion of The Long-Term Average.
Any thoughts on all this?
Last edited by SimpleGift on Tue Apr 17, 2018 5:52 pm, edited 3 times in total.

User avatar
Topic Author
SimpleGift
Posts: 3396
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: The Illusion of "The Long-Term Average" in Investing

Post by SimpleGift » Mon Mar 05, 2018 3:39 pm

Just to add: In no way am I an expert in statistics, and will readily admit to suffering for years from the illusion of The Long-Term Average in historical financial data — that is, until I read this short treatise:

All Change! The Implications of Non-stationarity for Empirical Modeling, Hendry & Pretis, University of Oxford, 2016.

If interested, it's a concise discussion of stationary and non-stationary time series, written for the layperson.

dbr
Posts: 30802
Joined: Sun Mar 04, 2007 9:50 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by dbr » Mon Mar 05, 2018 3:53 pm

SimpleGift wrote:
Mon Mar 05, 2018 3:38 pm

Any thoughts on all this?
Yes, in principle the assumption of stationarity is unjustified until it can be proven to exist as a practical approximation. A different issue is how far off base one gets from assuming stationarity. All models are wrong but some are more useful than others, so the question is whether the model is still useful. I think a stationary model with assumption of normal distributions is still a useful heuristic but should not be taken too far.

The referenced paper looks very interesting.

lack_ey
Posts: 6701
Joined: Wed Nov 19, 2014 11:55 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by lack_ey » Mon Mar 05, 2018 4:18 pm

I think this should all go without saying and be readily apparent, except it most definitely is not. Part of the problem maybe is that a lot of people looking at these things maybe have some background in introductory statistics but haven't learned about random processes.

One of the consequences as you allude to is that if you compute statistics on data sampled over time (e.g. the mean), this does not generally produce good estimates about the underlying statistics for the distribution at any given point in time, such as today.

User avatar
siamond
Posts: 5030
Joined: Mon May 28, 2012 5:50 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by siamond » Mon Mar 05, 2018 5:14 pm

SimpleGift wrote:
Mon Mar 05, 2018 3:38 pm
CONCLUSION: Non-stationary historical data series are by far the norm in finance and investing. So the next time a pundit shows you a chart like the one below, suggesting that "U.S. stocks are dangerously overvalued today, compared with their historical average," I'd encourage you to kindly ignore their comments and remember the illusion of The Long-Term Average.
I think this conclusion is way too black & white. Just because one data series has no meaningful average (and visually, it is trivial to see that on your first chart, come on!) doesn't mean that all historical data series are like that. Just because a historical data series is irregular doesn't mean that there is zero knowledge to be extracted from it. As much as I have severe doubts myself about a CAPE of ~16 being a 'constant of the universe', I would find extremely hard to accept that today's values of 30+ are normal. And it would never cross my mind to ignore the foolish speculative nature of human beings, and its ebbs and flows.

Not a good idea to throw the baby out with the bathwater, in other words. :wink:

PS. I'll read the paper though, as I certainly agree that those things need to be looked at with a very critical eye (incl. seeking a good common sense explanation of hypotheses one might make - besides the empirical observations). Thanks for sharing.

User avatar
Topic Author
SimpleGift
Posts: 3396
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: The Illusion of "The Long-Term Average" in Investing

Post by SimpleGift » Mon Mar 05, 2018 6:01 pm

lack_ey wrote:
Mon Mar 05, 2018 4:18 pm
I think this should all go without saying and be readily apparent, except it most definitely is not.
Agreed. The illusion of The Long-Term Average seems both subtle and pervasive in the collective unconscious.

Even professional forecasters are not immune from assuming stationarity in financial data. For example, in the chart below, the expert long-run forecasts of nominal 10-year rates (the ends of the lines extending to the right) lie within a fairly tight range of 4.4%-6.0% — despite the actual nominal 10-year rate swinging from nearly 8% down to 2%.
The expert forecasters appear to be "anchored" on the historical average of the 10-year nominal rate (about 5%) — despite historical interest rates being non-stationary and having no fixed mean. Thankful, though, that I don't have their job!

airelleofmusic
Posts: 96
Joined: Mon May 01, 2017 8:54 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by airelleofmusic » Mon Mar 05, 2018 6:10 pm

Very interesting topic indeed.

Yes I recently read a post stating that nobody can maintain that a 16/17 median CAPE 10 will be the norm "forever".

Even during the biggest crisis of the last century (2008) - where anchoring I am convinced played a huge role - valuations bottomed at 15 "only" meaning that it will not be extraordinary if the median CAPE 10 increases in years and decades to come.

Or maybe I am trying to convince myself something wrong because I don't want a 3% real returns on equities over the next years :mrgreen:
LBYM and enjoy life ! Thanks BH !

User avatar
nisiprius
Advisory Board
Posts: 39461
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: The Illusion of "The Long-Term Average" in Investing

Post by nisiprius » Mon Mar 05, 2018 6:38 pm

Yes, excellent post.

I think the general idea is common sense once you get past wishful thinking. The search for certainty in long-term data is a fool's errand.

It is a no-win situation because it should be obvious to almost anyone that, e.g. 10-year averages are not long enough to mean much. Now I have mixed feelings about a chart by Jim Otar that I'm about to show. It clearly illustrates one thing--historical financial time series are episodic, in which characteristics persist for periods of time on the rough order of a decade or so, to be fairly suddenly succeeded by a period with different characteristics.

The dangerous thing is that by drawing those straight lines and corners, Otar is highlighting the animal shapes we're supposed to see in the clouds. It's tempting to imagine that you can guess where the next corner is. It is also not clear whether this structure is just the spontaneous outcome of chaotic processes... or whether it is being exogenously driven by (chaotic) historic events.

Nevertheless, it is clear that an average over a century of data is not an average of 100 independent points, it is an average over no more than about eight different episodes, regimes, or "secular markets."

Image

Meanwhile, I do not think it is really possible to believe that the stock market of the early 1900s... when it was the playground of a small number of wealthy individuals, before there was an SEC, when manipulation and inside information were taken for granted, and when bulls and bears did not mean people who were expecting the market to rise or fall but people who were trying to make the market rise and fall... is quantitatively the same as the stock market of 2018.

This shows up very obviously when we look at international data. On the one hand, if we take long-term data at face value, we have to conclude that international stocks have significantly underperformed US stocks; total real return 1900-2015 6.4% U.S., 4.3% ex-US. On the other hand, if you look at what happened, it is obvious that most of the difference had to do with the devastating economic effects of World War II in many countries (versus the stimulating effect it had on the US). So unless we can make meaningful statements about the expected number and intensity of future wars, we have your non-stationarity.

Similarly, what can we make of this (long bond yields), what I call the two-and-a-half-humped camel? How many data points do we have here? Even if we assume that the next sixty years will consist of another thirty-year rise and thirty-year fall, what do we expect for its height? An average of two data points representing the height of the last two humps? Or is it better to say that the market just will not hold still for long enough for us to take a sharp picture of it?
Image
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.

User avatar
Topic Author
SimpleGift
Posts: 3396
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: The Illusion of "The Long-Term Average" in Investing

Post by SimpleGift » Mon Mar 05, 2018 7:05 pm

siamond wrote:
Mon Mar 05, 2018 5:14 pm
Just because a historical data series is irregular doesn't mean that there is zero knowledge to be extracted from it. As much as I have severe doubts myself about a CAPE of ~16 being a 'constant of the universe', I would find extremely hard to accept that today's values of 30+ are normal.
My sense is that introducing the concept of "normal" into a non-stationary data series is a treacherous exercise (chart below). Weren't pundits in 1990 opining that stocks were "dangerously overvalued" when CAPE was at 20? Was it conceivable then that the average CAPE over the next two decades would be 24? That this would become the new "normal"?
  • Image
    Note: The full data series is divided into four 35-year periods, with their means in red.
    Source: Shiller Data
My point is that there is no preordained mean value at which stocks are destined to trade. The amount that investors are willing to pay for a dollar of earnings is fluid and non-stationary across various time periods and different interest rate regimes. The idea of "a normal mean" to which CAPE 10 stock values are destined to revert back to seems fanciful.
Last edited by SimpleGift on Tue Apr 17, 2018 5:56 pm, edited 2 times in total.

1nv35t
Posts: 113
Joined: Wed Dec 13, 2017 3:37 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by 1nv35t » Mon Mar 05, 2018 7:17 pm

Isaac Newton pegged the Pound to gold whilst he was Master of the Royal Mint. Subsequently across the 18th and 19th centuries there was broadly zero inflation, but considerable interim volatility around that (inflation and deflation spikes in volatility tended to cluster). Investors could secure 2% to 4% real gains from simply lending to the state. The 20th century progressive decoupling away from the gold standard however induced lower rewards from treasury bonds, particularly after taxation and inflation, such that investors had to take on greater risk to secure real gains (stocks). Progressively however even stock real gains have tended to progressively decline.

I guess as wealth is distributed across a wider base the competition for real gains bids down the real yields. As is somewhat apparent in the OP's 2nd chart. What if that goes the way of bonds and 0% real becomes the 'norm' i.e. sufficient wealth around that many can accept 0% and still have enough to see them out.

Only a long term illusion if you look at the data in isolation and not account for paradigm shifts.
This shows up very obviously when we look at international data. On the one hand, if we take long-term data at face value, we have to conclude that international stocks have significantly underperformed US stocks; total real return 1900-2015 6.4% U.S., 4.3% ex-US.
The US was a developing market, transitioning from a Emerging Market to a fully developed market. The UK market pretty much fell in with the global average, perhaps because of many global businesses opting to list in that financial hub (something like 80% of the current 100 largest UK listed stocks earnings are sourced from foreign sources of sales/earnings). Around that average some do better (US, Australia), others worse.

User avatar
FreeAtLast
Posts: 676
Joined: Tue Nov 04, 2014 9:08 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by FreeAtLast » Mon Mar 05, 2018 7:37 pm

SimpleGift wrote:
Mon Mar 05, 2018 3:39 pm
Just to add: In no way am I an expert in statistics, and will readily admit to suffering for years from the illusion of The Long-Term Average in historical financial data — that is, until I read this short treatise:

All Change! The Implications of Non-stationarity for Empirical Modeling, Hendry & Pretis, University of Oxford, 2016.

If interested, it's a concise discussion of stationary and non-stationary time series, written for the layperson.
This thread is one of the exemplars which explains why I check out this Forum every day. Thanks for link to the fascinating paper, SimpleGift. I am going to discuss it with my nephew, who is a CFA.
Illegitimi non carborundum.

User avatar
siamond
Posts: 5030
Joined: Mon May 28, 2012 5:50 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by siamond » Mon Mar 05, 2018 8:22 pm

SimpleGift wrote:
Mon Mar 05, 2018 7:05 pm
My point is that there is no preordained mean value at which stocks are destined to trade. The amount that investors are willing to pay for a dollar of earnings is fluid and non-stationary in various time periods and different interest rate regimes. The idea of "a normal mean" to which CAPE 10 stock values are destined to revert back to seems fanciful.
I actually don't disagree with those sentences. I didn't mean 'normal' as in 'return to a pre-ordained mean'. I do not believe that CAPE has a pre-ordained mean to return to. I have yet to hear a logical explanation to the existence of such a constant mean. And yes, the world changes. Slowly and erratically, but it does. Where I do take exception to your reasoning is the fact of saying 'this isn't perfectly stationary, therefore this is useless data'.

The history of stock returns in every known country has shown again & again those valuations peaks followed by crises. The logical explanation is extremely simple, the speculative nature of human beings. You know, human behavior, this simple fact of life that proponents of strict EMH royally missed, against all historical evidence and... basic common sense. I have to hope that nobody believes this silliness any more, except a few academics... So no, the fact that valuation data isn't stationery doesn't make it useless, and it is clear to me that a CAPE of 33x does convey a 'red' signal of bad times to come. (and then I could ramble on 1/CAPE being a much more useful metric than CAPE, but this will be for another day)

Now, if you were to focus on your primary point (interest rates on the long run), then I would wholeheartedly agree with you. That is a trajectory where it seems particularly challenging (understatement) to extract information about the future from past information. And I would tend to agree with you that there must a good number of data series in investment/financials of a similar nature. Just please don't generalize too fast.

Northern Flicker
Posts: 4925
Joined: Fri Apr 10, 2015 12:29 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by Northern Flicker » Mon Mar 05, 2018 8:39 pm

The US was a developing market, transitioning from a Emerging Market to a fully developed market. The UK market pretty much fell in with the global average
No equity markets in the world met the modern standard of a developed market before the passage of three laws in the US:
1. Securities Act of 1933
2. Securities Exchange Act of 1934
3. Investment Company Act of 1940.
Index fund investor since 1987.

User avatar
Epsilon Delta
Posts: 8090
Joined: Thu Apr 28, 2011 7:00 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by Epsilon Delta » Mon Mar 05, 2018 9:43 pm

1nv35t wrote:
Mon Mar 05, 2018 7:17 pm
This shows up very obviously when we look at international data. On the one hand, if we take long-term data at face value, we have to conclude that international stocks have significantly underperformed US stocks; total real return 1900-2015 6.4% U.S., 4.3% ex-US.
The US was a developing market, transitioning from a Emerging Market to a fully developed market.
The US was not a developing market at any time during the 20th century. By 1900 the US is in the top three developed country by pretty much any measurer that does not involve engaging in land wars in Europe.

User avatar
Topic Author
SimpleGift
Posts: 3396
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: The Illusion of "The Long-Term Average" in Investing

Post by SimpleGift » Mon Mar 05, 2018 9:47 pm

siamond wrote:
Mon Mar 05, 2018 8:22 pm
Where I do take exception to your reasoning is the fact of saying 'this isn't perfectly stationary, therefore this is useless data'.
...(snip)...
Now, if you were to focus on your primary point (interest rates on the long run), then I would wholeheartedly agree with you. That is a trajectory where it seems particularly challenging (understatement) to extract information about the future from past information. And I would tend to agree with you that there must a good number of data series in investment/financials of a similar nature. Just please don't generalize too fast.
I see your point that most historical financial data series likely occur somewhere on a continuum between non-stationary and stationary. But how to distinguish them? Perhaps dbr offered the best approach upthread: "In principle, the assumption of stationarity is unjustified until it can be proven to exist as a practical approximation."

My concern is that many Bogleheads (myself included) start out with the naive belief that the historical data series we encounter are stationary and thus the "long-term historical average" is actually a useful piece of information, relevant to portfolio construction and financial planning. But I'll try to avoid the excesses of the reformed sinner! :wink:

lack_ey
Posts: 6701
Joined: Wed Nov 19, 2014 11:55 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by lack_ey » Mon Mar 05, 2018 10:08 pm

SimpleGift wrote:
Mon Mar 05, 2018 9:47 pm
siamond wrote:
Mon Mar 05, 2018 8:22 pm
Where I do take exception to your reasoning is the fact of saying 'this isn't perfectly stationary, therefore this is useless data'.
...(snip)...
Now, if you were to focus on your primary point (interest rates on the long run), then I would wholeheartedly agree with you. That is a trajectory where it seems particularly challenging (understatement) to extract information about the future from past information. And I would tend to agree with you that there must a good number of data series in investment/financials of a similar nature. Just please don't generalize too fast.
I see your point that most historical financial data series likely occur somewhere on a continuum between non-stationary and stationary. But how to distinguish them? Perhaps dbr offered the best approach upthread: "In principle, the assumption of stationarity is unjustified until it can be proven to exist as a practical approximation."

My concern is that many Bogleheads (myself included) start out with the naive belief that the historical data series we encounter are stationary and thus the "long-term historical average" is actually a useful piece of information, relevant to portfolio construction and financial planning. But I'll try to avoid the excesses of the reformed sinner! :wink:
Non-stationary means non-stationary. The question is just about how much the unobservable underlying is changing.

To be honest, for a lot of financial data we don't even really have enough samples to make particularly tight or good parameter estimates even if they were from known, stationary distributions.

Of course there is still plenty to be learned from past data, some things that seem more reliable than others. As always, we supplement data with theory. But really, the intractability of certain things, lack of data, potentially bad assumptions, etc. leave us in a state where in practice we have to rely a decent amount on our priors, which is not necessarily a bad thing at all. But naturally we come up with different priors and thus have disagreements about the way we think certain things work.

User avatar
siamond
Posts: 5030
Joined: Mon May 28, 2012 5:50 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by siamond » Mon Mar 05, 2018 10:32 pm

SimpleGift wrote:
Mon Mar 05, 2018 9:47 pm
My concern is that many Bogleheads (myself included) start out with the naive belief that the historical data series we encounter are stationary and thus the "long-term historical average" is actually a useful piece of information, relevant to portfolio construction and financial planning. But I'll try to avoid the excesses of the reformed sinner! :wink:
Fair enough. I do agree that the 'return to the mean' line of thinking has been overused and abused... And, father, I have sinned too... :wink:

MJW
Posts: 724
Joined: Sun Jul 03, 2016 7:40 pm
Location: Pacific Northwest

Re: The Illusion of "The Long-Term Average" in Investing

Post by MJW » Mon Mar 05, 2018 10:47 pm

siamond wrote:
Mon Mar 05, 2018 10:32 pm
SimpleGift wrote:
Mon Mar 05, 2018 9:47 pm
My concern is that many Bogleheads (myself included) start out with the naive belief that the historical data series we encounter are stationary and thus the "long-term historical average" is actually a useful piece of information, relevant to portfolio construction and financial planning. But I'll try to avoid the excesses of the reformed sinner! :wink:
I do agree that the 'return to the mean' line of thinking has been overused and abused...
Yes, and cited as justification for many an investing decision. I am glad this is being pointed out. Whenever I see someone on this forum say, "I believe such-and-such (e.g. international will outperform U.S.) will happen because of reversion to the mean, it begs the question of "what mean?" and "over what time period?" Is the expression simply being used as slang to imply that tides are bound to shift eventually?

It's almost as nonsensical a concept as "the wisdom of the market." :wink:

AlohaJoe
Posts: 4935
Joined: Mon Nov 26, 2007 2:00 pm
Location: Saigon, Vietnam

Re: The Illusion of "The Long-Term Average" in Investing

Post by AlohaJoe » Mon Mar 05, 2018 10:50 pm

SimpleGift wrote:
Mon Mar 05, 2018 3:38 pm
Any thoughts on all this?
Several years ago there was some briefly well known academic (maybe he published a book or was an early TED talk success? I forget.)

Anyway, the subject was wrote/talked about was that there are 3 kinds of "facts" in the world:

- things that change quickly, over the course of days, weeks, months or even a year or two. These things change so quickly that our human brains can comprehend that they change. No one would ever argue that weather is "stationary", right?
- things that change very slowly, over the course of 50+ years. These change so slowly that, for the purposes of a human life span, they are static. They are "stationary". Something like, I dunno, "the Supreme Court has 7 justices" has been true for so long that it is essentially "forever" from the perspective of a living human.

But there's a third category -- and he had some special name that I can't remember -- things that take 10-25 years to change. These are so slow that our human brains rarely recognise them as changing. But they are so fast that we can't just ignore them; they'll affect us during our lives.

I reckon that a lot of investing "truths" are in that third category.

With investing it hard because there is real money at stake. (Not just something inconsequential like "the Cubs always suck" :D .) But the trends change so slowly that you probably won't lose massive amounts of money by never changing and you probably won't make massive amounts of money by changing. And either way, the results would take many years to even show up.

lack_ey
Posts: 6701
Joined: Wed Nov 19, 2014 11:55 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by lack_ey » Mon Mar 05, 2018 10:54 pm

MJW wrote:
Mon Mar 05, 2018 10:47 pm
Yes, and cited as justification for many an investing decision. I am glad this is being pointed out. Whenever I see someone on this forum say, "I believe such-and-such (e.g. international will outperform U.S.) will happen because of reversion to the mean, it begs the question of "what mean?" and "over what time period?" Is the expression simply being used as slang to imply that tides are bound to shift eventually?
In that case I think the premise is largely, probably right (in a statistical sense, obviously forward results can do whatever and have a propensity to surprise a lot of the time), but the rationale given is more dubious.

What we do see is that stock market returns over the span of several years actually have been fairly convincingly negatively correlated with stock market returns over a future period. That is, there's been evidence of negative autocorrelation on the scale of several years or let's say a decade. More importantly and relatedly, though I haven't done the analysis myself and am not as sure, I think that relative US and ex-US performance is likely cyclical beyond what you would expect from chance.

So I wouldn't say "mean reversion" but I think that seems about right, keeping in mind that the effect size is not huge and returns have a whole lot of unexplained (unforecastable) randomness.

MJW
Posts: 724
Joined: Sun Jul 03, 2016 7:40 pm
Location: Pacific Northwest

Re: The Illusion of "The Long-Term Average" in Investing

Post by MJW » Mon Mar 05, 2018 11:11 pm

lack_ey wrote:
Mon Mar 05, 2018 10:54 pm

In that case I think the premise is largely, probably right (in a statistical sense, obviously forward results can do whatever and have a propensity to surprise a lot of the time), but the rationale given is more dubious.

What we do see is that stock market returns over the span of several years actually have been fairly convincingly negatively correlated with stock market returns over a future period. That is, there's been evidence of negative autocorrelation on the scale of several years or let's say a decade. More importantly and relatedly, though I haven't done the analysis myself and am not as sure, I think that relative US and ex-US performance is likely cyclical beyond what you would expect from chance.

So I wouldn't say "mean reversion" but I think that seems about right, keeping in mind that the effect size is not huge and returns have a whole lot of unexplained (unforecastable) randomness.
Thank you for your response. I don't disagree with your explanation and will acknowledge that people may have the "gist" of it down, but it's shaky reasoning when it comes to making actual decisions. I also think that even the use of the term "outperformance" tells an incomplete story. If Investment A returns 12% and Investment B returns 7%, it may be true that I.A outperformed, but it does not automatically follow that I.B was a poor choice.

In another scenario we could imagine that Investment X has returned 10% over a given period and Y has returned -2%. A "reversion to the mean" here could mean that I.X will later return 5% and I.Y returns returns 2%. So yes, this means that I.X won't do as well, but it does not necessarily follow that I.Y will thus be a better choice. I know this is probably a gross oversimplification of the issue, but I take a similar thought process when I am unwilling to commit to the idea that international stocks will outperform U.S., even though it is rational to believe it will.

hoops777
Posts: 2716
Joined: Sun Apr 10, 2011 12:23 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by hoops777 » Mon Mar 05, 2018 11:49 pm

This thread is another great example of a lot of really smart people proving they have no idea what is going to happen in the markets going forward :D
K.I.S.S........so easy to say so difficult to do.

User avatar
Topic Author
SimpleGift
Posts: 3396
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: The Illusion of "The Long-Term Average" in Investing

Post by SimpleGift » Tue Mar 06, 2018 12:24 am

AlohaJoe wrote:
Mon Mar 05, 2018 10:50 pm
But there's a third category -- and he had some special name that I can't remember -- things that take 10-25 years to change. These are so slow that our human brains rarely recognize them as changing. But they are so fast that we can't just ignore them; they'll affect us during our lives.

I reckon that a lot of investing "truths" are in that third category.
In a similar vein, when researching this topic I came across the idea that some financial time-series are considered "trend stationary" — that is, they revert to means that evolve deterministically over time, rather than being constants. This sounds like an in-between third category, where the stationarity of the data series trends upwards or downwards over long periods as a result of somewhat predictable, secular economic forces.

Probably some of the historical data series we encounter in investing are in this in-between category — but I'd certainly have to learn much more about statistical analysis to be able to properly recognize them.

dbr
Posts: 30802
Joined: Sun Mar 04, 2007 9:50 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by dbr » Tue Mar 06, 2018 9:27 am

SimpleGift wrote:
Tue Mar 06, 2018 12:24 am
AlohaJoe wrote:
Mon Mar 05, 2018 10:50 pm
But there's a third category -- and he had some special name that I can't remember -- things that take 10-25 years to change. These are so slow that our human brains rarely recognize them as changing. But they are so fast that we can't just ignore them; they'll affect us during our lives.

I reckon that a lot of investing "truths" are in that third category.
In a similar vein, when researching this topic I came across the idea that some financial time-series are considered "trend stationary" — that is, they revert to means that evolve deterministically over time, rather than being constants. This sounds like an in-between third category, where the stationarity of the data series trends upwards or downwards over long periods as a result of somewhat predictable, secular economic forces.

Probably some of the historical data series we encounter in investing are in this in-between category — but I'd certainly have to learn much more about statistical analysis to be able to properly recognize them.
The paper quoted above talks about how derivatives of non-stationary processes may be stationary. I guess an example would be that growing investment has an upward trend, but return, the derivative of the growth, might be stationary if the expected return is constant in time.

User avatar
siamond
Posts: 5030
Joined: Mon May 28, 2012 5:50 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by siamond » Tue Mar 06, 2018 1:33 pm

SimpleGift wrote:
Tue Mar 06, 2018 12:24 am
In a similar vein, when researching this topic I came across the idea that some financial time-series are considered "trend stationary" — that is, they revert to means that evolve deterministically over time, rather than being constants. This sounds like an in-between third category, where the stationarity of the data series trends upwards or downwards over long periods as a result of somewhat predictable, secular economic forces.

Probably some of the historical data series we encounter in investing are in this in-between category — but I'd certainly have to learn much more about statistical analysis to be able to properly recognize them.
Well, this is also what I meant when I pushed back on your preliminary conclusion. The authors of the study focus on linear patterns:
''A time series is (weakly) stationary when its first two moments, namely the mean and variance, are finite and constant over time. "

Ok, but... there are many possible patterns which are non-linear (or that can't be made linear by a simple transformation, e.g. logarithmic). That doesn't mean that they do not convey useful information about the future. Sometimes, I feel that some posters on this forum (not you, Simplegift!) try extremely hard to find excuses to ignore lessons & learnings from the past. I don't find that terribly wise. Even if I do agree that the other extreme (find patterns everywhere and assume they will repeat themselves) isn't terribly wise either - and probably worse. As in everything, there is a happy medium between extremes, and common sense plus pragmatism should guide us...

WanderingDoc
Posts: 1341
Joined: Sat Aug 05, 2017 8:21 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by WanderingDoc » Tue Mar 06, 2018 1:41 pm

SimpleGift wrote:
Mon Mar 05, 2018 3:38 pm
There are no fixtures in nature.
The universe is fluid and volatile.
Permanence is but a word of degrees.
~ Ralph Waldo Emerson (1803-1882)

To perceive something as fixed when it is actually transient is usually considered a foolish mistake. Yet, we do it all the time in investing when we treat historical data series as stationary — i.e., in statistical terms, as if their means and variances are constant over time. In reality, nearly all of the historical data sets we encounter as investors are non-stationary — their means and their variances in one time period do not apply in another. Thus the illusion of The Long-Term Average.

FIRST VIEW: As one example, charted below is a time series showing the estimated, real "risk-free" interest rate over eight centuries, from 1314-2017. Drawn across the chart is a dashed line (in red) indicating the 704-year average, at 4.78%. Consciously or unconsciously, this line registers in most investors' minds as the "normal" historical real interest rate. Using this chart, it's easy to get the impression that interest rates are a stationary series with a fixed mean — leading one to naturally wonder, "When will today's rates start reverting back toward their long-term average?"
SECOND VIEW: Statistical tests of this same data would show that, in fact, interest rates are NOT a stationary time series. As a simple proof, when the average interest rate for each of the eight centuries is calculated and charted (in light blue, below), we see that the mean value changes, sometimes dramatically, from one century to the next. This is the very definition of a non-stationary data series — and it makes the idea of "reversion to the mean" for interest rates nonsensical. What mean? Which time period?
CONCLUSION: Non-stationary historical data series are by far the norm in finance and investing. So the next time a pundit shows you a chart like the one below, suggesting that "U.S. stocks are dangerously overvalued today, compared with their historical average," I'd encourage you to kindly ignore their comments and remember the illusion of The Long-Term Average.
Any thoughts on all this?
Very interesting. So the trend for the U.S., Canada, and other developed nations is to get into more and more debt, and create more and more money out of thin air. The data shows that the trend is UP decade after decade. So the hypothesis is that this behavior is the norm, and that borrowing and fake money creation is expected to go up, therefore we shouldn't worry? I am not being facetious, I am genuinely asking in light of what was proposed in the OP.
I'm not looking to get rich quick (stocks), I'm not looking to get rich slow (indexing), I'm looking to get rich, for sure (real estate) | Don't wait to buy real estate. Buy real estate.. and wait.

wrongfunds
Posts: 2025
Joined: Tue Dec 21, 2010 3:55 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by wrongfunds » Tue Mar 06, 2018 2:57 pm

Something like, I dunno, "the Supreme Court has 7 justices" has been true for so long that it is essentially "forever" from the perspective of a living human.
You ARE kidding, right???

Angst
Posts: 2068
Joined: Sat Jun 09, 2007 11:31 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by Angst » Tue Mar 06, 2018 3:24 pm

wrongfunds wrote:
Tue Mar 06, 2018 2:57 pm
AlohaJoe wrote:
Mon Mar 05, 2018 10:50 pm
Something like, I dunno, "the Supreme Court has 7 justices" has been true for so long that it is essentially "forever" from the perspective of a living human.
You ARE kidding, right???
I'm gonna assume that AlohaJoe was just putting himself into the shoes of someone living in 1837, just before the number of US Supreme Court justices was raised from 7 to 9. :wink:

wrongfunds
Posts: 2025
Joined: Tue Dec 21, 2010 3:55 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by wrongfunds » Tue Mar 06, 2018 6:12 pm

It is perfect illustration of the point he was trying to drive through! Very well played!!

There haven’t always been nine justices on the court.

The U.S. Constitution established the Supreme Court but left it to Congress to decide how many justices should make up the court. The Judiciary Act of 1789 set the number at six: a chief justice and five associate justices. In 1807, Congress increased the number of justices to seven; in 1837, the number was bumped up to nine; and in 1863, it rose to 10. In 1866, Congress passed the Judicial Circuits Act, which shrank the number of justices back down to seven and prevented President Andrew Johnson from appointing anyone new to the court. Three years later, in 1869, Congress raised the number of justices to nine, where it has stood ever since. In 1937, in an effort to create a court more friendly to his New Deal programs, President Franklin Roosevelt attempted to convince Congress to pass legislation that would allow a new justice to be added to the court—for a total of up to 15 members—for every justice over 70 who opted not to retire. Congress didn’t go for FDR’s plan.

User avatar
Topic Author
SimpleGift
Posts: 3396
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: The Illusion of "The Long-Term Average" in Investing

Post by SimpleGift » Tue Mar 06, 2018 6:17 pm

^^^ Excellent summary. I think it's safe to conclude that the historical time-series of "number of Supreme Court justices" is non-stationary. :wink:

dbr
Posts: 30802
Joined: Sun Mar 04, 2007 9:50 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by dbr » Tue Mar 06, 2018 6:30 pm

SimpleGift wrote:
Tue Mar 06, 2018 6:17 pm
^^^ Excellent summary. I think it's safe to conclude that the historical time-series of "number of Supreme Court justices" is non-stationary. :wink:
The paper actually deals with ways to account for step change non-stationarity which hides what is actually stationary excepting one event. One of the problems in the area is thinking things are not stationary when except for a disruption they really are. Naturally you don't use 7 for the number when 9 applies, but you also don't decide that the number of justices is variable in all the practical cases where it isn't variable. A more subtle question might be when one or more justices don't participate in or vote on a case.

Northern Flicker
Posts: 4925
Joined: Fri Apr 10, 2015 12:29 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by Northern Flicker » Tue Mar 06, 2018 7:55 pm

The discussion is conflating two separate concepts: stationarity of a random process, and under what circumstances sample means and sample variances are reliable estimators for the mean and variance of a distribution that is sampled. The former concerns whether the distribution itself changes over time. The latter just whether a particular sample is good enough for reliable estimation.

The evidence is overwhelming that backtesting historical samples to estimate the mean or variance of future return suffers from severe sample bias: sample means and sample variances are so sensitive to historical time period that even getting reproducibility from sample to sample is challenging. This may be due to non-stationarity, or just due to high variance and poor sampling methodology/opportunity. It is not obvious how to meet the prerequisites of sampling theory with asset return samples by collecting a sufficiently large random sample of independent data points.

Whether or not the distribution changes over time (non-stationarity) is a difficult question to answer, particularly given the instability of sample statistics that otherwise might be indicative of the distribution at some point in time.
Index fund investor since 1987.

User avatar
willthrill81
Posts: 13959
Joined: Thu Jan 26, 2017 3:17 pm
Location: USA

Re: The Illusion of "The Long-Term Average" in Investing

Post by willthrill81 » Tue Mar 06, 2018 7:58 pm

hoops777 wrote:
Mon Mar 05, 2018 11:49 pm
This thread is another great example of a lot of really smart people proving they have no idea what is going to happen in the markets going forward :D
Image
“It's a dangerous business, Frodo, going out your door. You step onto the road, and if you don't keep your feet, there's no knowing where you might be swept off to.” J.R.R. Tolkien,The Lord of the Rings

AlohaJoe
Posts: 4935
Joined: Mon Nov 26, 2007 2:00 pm
Location: Saigon, Vietnam

Re: The Illusion of "The Long-Term Average" in Investing

Post by AlohaJoe » Tue Mar 06, 2018 8:10 pm

wrongfunds wrote:
Tue Mar 06, 2018 2:57 pm
Something like, I dunno, "the Supreme Court has 7 justices" has been true for so long that it is essentially "forever" from the perspective of a living human.
You ARE kidding, right???
I don't live in America and was trying to think of a reference you Americans could relate to. Clearly I failed hahahaha :sharebeer

User avatar
Wildebeest
Posts: 1203
Joined: Fri Dec 27, 2013 2:36 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by Wildebeest » Wed Mar 07, 2018 9:13 am

SimpleGift wrote:
Mon Mar 05, 2018 3:38 pm


Any thoughts on all this?
SimpleGift,

I always look forward to your posts and charts.
This is another gem.

I have thoughts. I recall when reading Jack Bogle ' lecture "The tell tale heart", where he refers to the reversion of the mean, I was befuddled by it as in: The charts are nice and all, but where is the evidence that there is such a thing and what are the rules of when to revert and what is the "mean".

For the last 10 years I have been looking for evidence for the reversion to the mean and my "layperson" conclusion is that it is based on bias and the mean is what ever you want it to be. We try to make sense of a world, which keeps on changing and we look for security. Kahneman's "Think fast and slow" helped me see and understand why I make stupid decisions, while I should know better.

If we are going to use a "reversion to the mean" as a heuristic to help you stay the course, I would think that another Jack Bogle expression takes precedence: "nobody knows nothing".

I recently read Benoit Mandelbrot's "the (mis) behavior of markets" and it provides a cogent criticism of the " reversion to the mean".
Last edited by Wildebeest on Wed Mar 07, 2018 9:50 am, edited 1 time in total.
The Golden Rule: One should treat others as one would like others to treat oneself.

wrongfunds
Posts: 2025
Joined: Tue Dec 21, 2010 3:55 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by wrongfunds » Wed Mar 07, 2018 9:36 am

AlohaJoe wrote:
Tue Mar 06, 2018 8:10 pm
wrongfunds wrote:
Tue Mar 06, 2018 2:57 pm
Something like, I dunno, "the Supreme Court has 7 justices" has been true for so long that it is essentially "forever" from the perspective of a living human.
You ARE kidding, right???
I don't live in America and was trying to think of a reference you Americans could relate to. Clearly I failed hahahaha :sharebeer
Not only you did NOT fail, you succeeded proving your point beyond your wildest imagination! Your point was cemented with your so called "incorrect'' reference.

Getting back to the topic, I have simple method if the back testing is correct or not.

If the back testing works for the entire interval, does it also work for all quarters if you split the section in to quarters? If not, then obviously, the back testing is not valid. This should be well understood even if you have zero statistical background. I don't believe most of the financial data munging ever consider this seeming simple concept in their study.

User avatar
grayfox
Posts: 5201
Joined: Sat Sep 15, 2007 4:30 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by grayfox » Wed Mar 07, 2018 10:09 am

To try and understand what is being discussed, I looked up Stationary process on Wikipedia.
a stochastic process whose unconditional joint probability distribution does not change when shifted in time. Consequently, parameters such as mean and variance, if they are present, also do not change over time.
But then it goes on to say that you can transform non-stationary data to become stationary.
The most common cause of violation of stationarity is a trend in the mean, which can be due either to the presence of a unit root or of a deterministic trend. In the former case of a unit root, stochastic shocks have permanent effects and the process is not mean-reverting. In the latter case of a deterministic trend, the process is called a trend stationary process, and stochastic shocks have only transitory effects after which the variable tends toward a deterministically evolving (non-constant) mean
.

According to wiki, you can remove the underlying trend which is a function of time, and get a stationary process.

The SECOND VIEW chart, shows a trend that is linear over time.
Somehow we can remove the trend and end up with a stationary process. How can we do this? Is it as simple as subtracting out the trend line from the original data?

User avatar
siamond
Posts: 5030
Joined: Mon May 28, 2012 5:50 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by siamond » Wed Mar 07, 2018 10:24 am

wrongfunds wrote:
Wed Mar 07, 2018 9:36 am
If the back testing works for the entire interval, does it also work for all quarters if you split the section in to quarters? If not, then obviously, the back testing is not valid. This should be well understood even if you have zero statistical background. I don't believe most of the financial data munging ever consider this seeming simple concept in their study.
That is a fair point. When diving in monthly data series for various studies of mine, I did notice that some metrics suddenly said something else than when applied on annual data series. Standard deviation and correlation are primary examples. The case that struck me the most was gold and its much touted negative correlation with US stocks. Well, that is true with annual numbers. Not so with monthly numbers, where the correlation was basically zero (i.e. no correlation). Trouble is it is difficult to run monthly numbers, this makes the math more complicated, and tends to bring Excel to its knees. I would like to explore programming in 'R', and make it an open-source project akin to Simba (which is indeed limited to annual numbers). Seems like a good project for next winter! Need to avoid getting those brain cells frozen! :wink:

Anyhoo, personally, I think that the primary thing is that empirical hypotheses are worth nothing if not accompanied by a common sense explanation of the hypothesis. The explanation may or may not be entirely correct, but if you can't find one that seems pretty convincing, then the hypothesis relies on a foundation of sand... And then seek out-of-sample data for sure, but that's hard.

User avatar
grayfox
Posts: 5201
Joined: Sat Sep 15, 2007 4:30 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by grayfox » Wed Mar 07, 2018 10:47 am

siamond wrote:
Wed Mar 07, 2018 10:24 am

That is a fair point. When diving in monthly data series for various studies of mine, I did notice that some metrics suddenly said something else than when applied on annual data series. Standard deviation and correlation are primary examples. The case that struck me the most was gold and its much touted negative correlation with US stocks. Well, that is true with annual numbers. Not so with monthly numbers, where the correlation was basically zero (i.e. no correlation). Trouble is it is difficult to run monthly numbers, this makes the math more complicated, and tends to bring Excel to its knees. I would like to explore programming in 'R', and make it an open-source project akin to Simba (which is indeed limited to annual numbers). Seems like a good project for next winter! Need to avoid getting those brain cells frozen! :wink:
R is the definitely the way to go for heavy-duty computations. R is to Excel as Excel is to a Calculator as a Calculator is to pencil and paper.

A few years ago, I took a Professor Eric Zivot's online course Introduction to Computational Finance and Financial Econometrics. All computations were done in R.

It looks like he also authored a textbook: Modeling Financial Time Series with R, to be published by Springer-Verlag
Last edited by grayfox on Wed Mar 07, 2018 11:06 am, edited 2 times in total.

bgf
Posts: 1101
Joined: Fri Nov 10, 2017 9:35 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by bgf » Wed Mar 07, 2018 10:53 am

Wildebeest wrote:
Wed Mar 07, 2018 9:13 am
I recently read Benoit Mandelbrot's "the (mis) behavior of markets" and it provides a cogent criticism of the " reversion to the mean".
that is next on my list. have you read "fractals and scaling in finance" by him?
“TE OCCIDERE POSSUNT SED TE EDERE NON POSSUNT NEFAS EST"

protagonist
Posts: 6045
Joined: Sun Dec 26, 2010 12:47 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by protagonist » Wed Mar 07, 2018 11:00 am

https://s9.postimg.org/lyfvruly7/Moving_Chart.jpg

An aside, but was the "risk-free interest rate" around the WW1 period really dramatically lower than at any other time since 1314?

In what country (or part of the world) is the data from? And do you have any idea how it was compiled?

User avatar
Wildebeest
Posts: 1203
Joined: Fri Dec 27, 2013 2:36 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by Wildebeest » Wed Mar 07, 2018 12:11 pm

bgf wrote:
Wed Mar 07, 2018 10:53 am
Wildebeest wrote:
Wed Mar 07, 2018 9:13 am
I recently read Benoit Mandelbrot's "the (mis) behavior of markets" and it provides a cogent criticism of the " reversion to the mean".
that is next on my list. have you read "fractals and scaling in finance" by him?
I have not. It was written in 1997 and I thought that The (Mis) Behavior of the Markets ( 2004) would have covered the main points. Is it good?

I am currently reading Nate Silver"s " The Signal and the Noise: Why So Many Predictions Fail--but Some Don't" and I really liked the first chapter as to his recap as to what brought on the Great Recession and in my mind very pertinent to what can go and has gone wrong with "The Illusion of "The Long term Average" in Investing".

I have bought "Skin in the Game: Hidden Asymmetries in Daily Life": Nassim Nicholas Taleb and I will read it next.
The Golden Rule: One should treat others as one would like others to treat oneself.

User avatar
Topic Author
SimpleGift
Posts: 3396
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: The Illusion of "The Long-Term Average" in Investing

Post by SimpleGift » Wed Mar 07, 2018 12:21 pm

protagonist wrote:
Wed Mar 07, 2018 11:00 am
In what country (or part of the world) is the data from? And do you have any idea how it was compiled?
Their data set begins with private debt instruments in Venice, Italy, which started trading freely on secondary markets in the 1270s. It then picks up interest rates from the bond markets in Genoa, Spain and Amsterdam to 1700, then the U.K. bond market from 1700 to 1900, and finally the U.S. and German bond markets to 2017. This chart shows the data sources and the nominal, "risk-free" interest rate compiled over the full period:
Note that these are the nominal interest rates, which the researchers then adjusted for inflation in the various time periods, to produce the real interest rates charts in the OP. The original paper goes into more detail on how the data series was compiled.

protagonist
Posts: 6045
Joined: Sun Dec 26, 2010 12:47 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by protagonist » Wed Mar 07, 2018 3:14 pm

What I take from the graph of real interest rates is the relatively consistent patterns, eliminating the noise. From about 1450 through about 1700 the rates were steadliy decreasing. In more recent years they would increase for a few decades, then decline for a few decades, then increase again, etc.https://s9.postimg.org/fkqsojz1r/Stationary_Chart.jpg

1700-1750 decline
1750-1800 unstable
1800-1825 increase
1825-1850 decrease
1850-1880 increase
1880-1920 decrease
1920-1935 increase
1935-1950 decrease
1950-1985 increase
1985-present decrease (dates are estimates based on graph)

It there is anything to this pattern, we should be in store for a reversal soon and a few decades of generally rising real interest rates.

The long bond yield chart that nisiprius posted above shows a similar sort of pattern https://i.imgur.com/d1XG2Va.png:
1870-1900 decline
1900-1920 increase
1920-1940 decline
1940-1980 massive increase
1980-present massive decline (estimated dates)

Again, if the pattern means anything, we should be in store for a reversal with steadily increasing bond yields for decades to come.

There may be some good basis for predicting movement of interest rates and bond prices over intermediate periods of a few decades. It seems to me that, in our times, this makes some sense, since interest rates are largely manipulated as a response to the patterns of heating or cooling of the economy, which takes time to happen and more time to recognize and respond. Stock market movements are far more complex.
Last edited by protagonist on Wed Mar 07, 2018 10:11 pm, edited 1 time in total.

DoTheMath
Posts: 319
Joined: Sat Jul 04, 2015 1:11 pm
Location: The Plains

Re: The Illusion of "The Long-Term Average" in Investing

Post by DoTheMath » Wed Mar 07, 2018 7:56 pm

Thanks for the thought provoking post and discussion! It helped kick away some mental crutches I was only half aware I was using.
“I am losing precious days. I am degenerating into a machine for making money. I am learning nothing in this trivial world of men. I must break away and get out into the mountains...” -- John Muir

User avatar
Epsilon Delta
Posts: 8090
Joined: Thu Apr 28, 2011 7:00 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by Epsilon Delta » Wed Mar 07, 2018 11:16 pm

grayfox wrote:
Wed Mar 07, 2018 10:09 am
Somehow we can remove the trend and end up with a stationary process. How can we do this? Is it as simple as subtracting out the trend line from the original data?
Yes. It pretty much is that simple. Instead of working with the series "price(year)" you work with the series "price(year) - k * year" where k is the slope. Or work with "price(year) - price(year -1)". If the trend is not linear you could work with price(year)year, price(year)/log(year) or any other function to try and remove the trend. If you have reason to believe there is something that does not change you beat the data into submission anyway you can.

This doesn't deal with every non-stationary process. Many non-stationary processes do not have a trend. However many non-stationary processes without a trend appear to have a trend.

There are many possible trends, so implicitly you end up with a rather large number of free parameters. As von Neumann is reported to have said "With four parameters I can fit an elephant, with five I can make him wiggle his trunk." So just because you can come up with a trend that makes the data appear stationary doesn't mean you have a model with any predictive or explanatory power.

dbr
Posts: 30802
Joined: Sun Mar 04, 2007 9:50 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by dbr » Thu Mar 08, 2018 9:28 am

grayfox wrote:
Wed Mar 07, 2018 10:09 am


Somehow we can remove the trend and end up with a stationary process. How can we do this? Is it as simple as subtracting out the trend line from the original data?
You can take the derivative. For example a growing value can have a constant rate of growth as in watching an investment grow at a constant rate of return. The problem in investing occurs when we aren't confident the return distribution is stationary and the problem is not just that the return is following a trend over time.

User avatar
grayfox
Posts: 5201
Joined: Sat Sep 15, 2007 4:30 am

Re: The Illusion of "The Long-Term Average" in Investing

Post by grayfox » Thu Mar 08, 2018 10:24 am

Epsilon Delta wrote:
Wed Mar 07, 2018 11:16 pm
grayfox wrote:
Wed Mar 07, 2018 10:09 am
Somehow we can remove the trend and end up with a stationary process. How can we do this? Is it as simple as subtracting out the trend line from the original data?
Yes. It pretty much is that simple. Instead of working with the series "price(year)" you work with the series "price(year) - k * year" where k is the slope. Or work with "price(year) - price(year -1)". If the trend is not linear you could work with price(year)year, price(year)/log(year) or any other function to try and remove the trend. If you have reason to believe there is something that does not change you beat the data into submission anyway you can.

This doesn't deal with every non-stationary process. Many non-stationary processes do not have a trend. However many non-stationary processes without a trend appear to have a trend.

There are many possible trends, so implicitly you end up with a rather large number of free parameters. As von Neumann is reported to have said "With four parameters I can fit an elephant, with five I can make him wiggle his trunk." So just because you can come up with a trend that makes the data appear stationary doesn't mean you have a model with any predictive or explanatory power.
If we can determine the trend, then it can be removed. So the problem is finding the trend. I will get on it.

Later: I got out LibreOffice. It is simple to add mean value lines and trend lines to a data series. Just graph it and click on the series in the chart. You can add Linear, Logarithmic, Exponential, Power, Polynomial, or Moving Average

Zero-Order Model: just use the mean value of all the data. OK, that was the FIST VIEW chart showing 4.78% mean value of risk-free rate from 1300 to 2000. That was determined to be not so great.

First-Order Model: use linear trend line. That's the SECOND VIEW. That doesn't look so great either because if you follow the linear trend line down we'll be at -5% interest rate in about 100 years. That can't be right either.

If risk-free interest rates are approaching zero, the trend might be some kind of exponential decay. That's possible. Maybe the risk-free rate becomes zero in the long-term.

Then I look at this chart from the paper

It looks to me like their are regimes that change. E.g. The UK rates were in a narrow range with no clear trend. The Dutch stepped down. The U.S. had that crazy rise to a peak in 1980 and then fall off over the next 30 years. What's up with that?

I don' see fitting a single trend to the overall chart. Maybe some kind of regime switching where the trend changes, either abruptly or gradually.

azurekep
Posts: 1179
Joined: Tue Jun 16, 2015 7:16 pm

Re: The Illusion of "The Long-Term Average" in Investing

Post by azurekep » Thu Mar 08, 2018 12:50 pm

I kept hearing that interest rates were the lowest since Hammurabi. I finally looked it up.

Image

Source

The chart makes a distinction between short-term and long-term and I can't vouch for the reliability of any of this data. But it does seem to confirm that interest rates do have extreme spikes from time to time.

This whole discussion reminds me of the fact that a lot of non-scientific people think equities go through long-term cycles -- 17-year, 34-year and so on. They may be wrong-headed about using cycles for prediction, but the notion of long-term secular cycles is not necessarily a crazy one.

Nisisiprus posted a chart that showed secular cycles and I think that's pretty much common wisdom/knowledge. Though it's not really actionable since the "shape" of the cycle can differ (e.g., some bull markets are steeply up, others are more horizontal, plus some include both a bull and bear phase within one cycle (that's one way to force a model to fit ;) ). Plus, you have to wait till the current cycle is over to really make any conclusions about it.

User avatar
Topic Author
SimpleGift
Posts: 3396
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: The Illusion of "The Long-Term Average" in Investing

Post by SimpleGift » Thu Mar 08, 2018 12:57 pm

grayfox wrote:
Thu Mar 08, 2018 10:24 am
I don' see fitting a single trend to the overall chart. Maybe some kind of regime switching where the trend changes, either abruptly or gradually.
azurekep wrote:
Thu Mar 08, 2018 12:50 pm
The chart makes a distinction between short-term and long-term and I can't vouch for the reliability of any of this data. But it does seem to confirm that interest rates do have extreme spikes from time to time.
In both cases, I'd suggest you folks would be better off using real (inflation-adjusted) interest rates, from the charts in the OP and the linked Bank of England paper. In both of your quoted comments above, you are mostly seeing the impacts of inflation on the historical time series, not real interest rates. Just my two cents.

Post Reply