What is evidence based investing?

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Post Reply
qwertyjazz
Posts: 1026
Joined: Tue Feb 23, 2016 4:24 am

What is evidence based investing?

Post by qwertyjazz » Thu May 26, 2016 9:33 pm

Reading a recent thread made me wonder what should I consider as good evidence of something in personal finance. Is it a Monte Carlo simulation of past data? A theoretical economic model? Historical analogy of the past few thousand years as Dr Bernstein would seem to imply? I cannot imagine randomized trials as being all that helpful. So, what makes people decide that something is evidence based investing?

User avatar
Taylor Larimore
Advisory Board
Posts: 26115
Joined: Tue Feb 27, 2007 8:09 pm
Location: Miami FL

"Predicting the Past"

Post by Taylor Larimore » Thu May 26, 2016 9:43 pm

qwertyjazz:

Your post about Evidence Based Investing reminds me of this post made long ago by a very wise Boglehead with the username of "Ozark":
--------------------------------------PREDICTING THE PAST--------------------------------

If you feel you can improve your portfolio's asset allocation by running the portfolio through various computer programs, measuring and grading various risk/reward relationships, feel free. It's okay with me. Honest. For myself, I'm not interested.

I'm also not interested in running reams of data through a computer program in order to discover how much I can withdraw yearly from my portfolio and never go broke.

Without having studied it, I'm willing to assume the Risk Grades deal is similar to the well known Efficient Frontier concept: Invest in a mix of assets that will give the best return for the least risk.

Wonderful. The problem in execution is this; both these approaches would seem to be limited to looking at PAST risk/return relationships, in order to predict FUTURE such relationships.

This approach hasn't worked very well and it never will.

There's lots of stuff we can learn by studying the past. One thing we can't learn, though, is how much the future will resemble the past.

There really is an Efficient Frontier. There really is a withdrawal rate that will allow my wife and I to spend all our money during our life times, but never go broke.

But these things are unknown and unknowable, going forward. Such things are only knowable looking backward.

Given that such things are only knowable looking backward, academics with more letters after their names than I have money in the bank, have spent unconscionable amounts of time goobering through the past. They thus invented Modern Portfolio Theory---Beta, Alpha, R-Squared, and the crowning achievement, Sharpe Ratio. These accomplishments were celebrated and awards were given. Yes.

And then...a funny thing happened on the way to the bank. These numbers turned out to have little or no predictive value, regarding returns. And since they couldn't predict returns, they also failed to predict risk/return ratios.

Joining in the fun, M* invented their first Star Rating system, a system that graded...yep...risk- adjusted, past performance.

I wish I had 10 bucks for every post I've read where the poster said, essentially, "I have a balanced portfolio, made up entirely of 4 and 5 star funds." Too late, these jokers discovered what M* eventually discovered; past risk-adjusted performance doesn't predict future risk-adjusted performance.

I don't want to discover the Sharpe Ratio of my portfolio. I don't want to discover its Beta. I don't want to discover its Risk Grade. I have absolutely no confidence that adjusting the portfolio so that these numbers become more favorable will improve future risk/reward.

If others do want to do that, that's okay with me. I seriously doubt, though, that many successful mutual fund managers select securities in that manner. If any do, or if any money managers set their asset allocations in that manner, I'd be interested in their long-term results---results over periods of, say, 10 years, or more.

In short, computers are wonderous tools, but that's all they are. Every computer on Earth, all linked up and working 24/7, from now on, won't tell me my survivable withdrawal rate. Neither will they tell me what asset allocation would give me the best risk/reward ratio.

In my opinion, these things can't be calculated. We have to forge ahead without knowing these things. Deal with it.

Best wishes.
Taylor
"Simplicity is the master key to financial success." -- Jack Bogle

dad2000
Posts: 628
Joined: Fri Feb 03, 2012 7:04 pm

Re: What is evidence based investing?

Post by dad2000 » Thu May 26, 2016 9:52 pm

Sorry for being facetious and/or circular... I think the best evidence we have shows that straight indexing is just as or more effective than evidence based investing (EBI).

I think of EBI as attempting to build an optimal investment strategy using the latest and best academic research. As we've seen many times, these strategies don't seem to pan out in the real world.

alex_686
Posts: 2544
Joined: Mon Feb 09, 2015 2:39 pm

Re: What is evidence based investing?

Post by alex_686 » Thu May 26, 2016 10:00 pm

I have only heard the phrase "evidence based" in regards to medicine. I have heard that medicine is the art of applied biology. To extend the analogy, investing must be the art of applied economics. So I would say yes to investment theory, for that is the science. However, always be aware of the assumptions and simplifications that theory makes. I treat history lightly and back-testing even lighter. The samples sets are small and the underlying factors are in flux. That being said, history certainty informs my wisdom.

As for Monte Carlo - it is a simulation of reality. Do you want to input historical data? Sure - you can do that. Or maybe the various possibilities of a imagined future? sure. You can specify whatever type of reality you want. FYI, I am not being totally glib. I have a very favorable opinion of Monte Carlo simulations. However, once again, it is a applied art, not a science.

qwertyjazz
Posts: 1026
Joined: Tue Feb 23, 2016 4:24 am

Re: What is evidence based investing?

Post by qwertyjazz » Thu May 26, 2016 10:05 pm

Another thread was about a book where the author used the phrase evidence based investing frequently. I was not a fan of the first 3 chapters which were free, but I was fascinated by the phrase as I likewise had heard it starting in medicine. It made me wonder how I should approach the concept of evidence in personal finance.

AlohaJoe
Posts: 2550
Joined: Mon Nov 26, 2007 2:00 pm
Location: Saigon, Vietnam

Re: What is evidence based investing?

Post by AlohaJoe » Thu May 26, 2016 10:17 pm

What the person in the other thread means when they said "evidence based investing". The problem he is trying to address is:

Unsubstantiated claims are perhaps hardest to deal with. Outside academic studies, conclusions and advice are frequently handed out with minimal supporting data. When evidence is provided, even in academic studies, it’s typically based either on one set of United States historical data, or computer simulations lacking a resemblance to true markets. This isn’t enough.


To counter that the author refers to the Wikipedia article on evidence-based research and then adds:

The above definitions, along with a skeptical mindset, capture the approach and philosophy of this book. The only thing missing is a clear statement of the specific norms for proper evidence within the domain of investing during retirement.

The following four criteria define the norms to comply with for all recommendations in this book— if for some reason a recommendation cannot fully comply with these norms then it’s clearly stated.

1. Recommendations must be verified with real market data (i.e., market simulations alone are not sufficient... we don’t know how to fully model markets).

2. Recommendations must be based on methods supporting independent confirmation by other researchers (i.e., results that cannot be replicated are not sufficient).

3. All recommendations must be verified using at least one independent data source but preferably several; testing with independent data is the only way to insure correct results, preventing a data-mining bias (which is thoroughly defined later).

4. All recommendations must hold up to robust testing, designed to seek out counterevidence or identify weaknesses. Typically this requires a diversity of tests, cross-verifying conclusions across multiple contexts and datasets. Only by consciously seeking to identify flaws in favored strategies can the natural biases in research be counterbalanced.

dbr
Posts: 23687
Joined: Sun Mar 04, 2007 9:50 am

Re: What is evidence based investing?

Post by dbr » Thu May 26, 2016 10:25 pm

I would suggest it means absolutely nothing and instead look at what is actually presented instead of going by a label.

qwertyjazz
Posts: 1026
Joined: Tue Feb 23, 2016 4:24 am

Re: What is evidence based investing?

Post by qwertyjazz » Thu May 26, 2016 10:46 pm

Taylor,
I should probably add wisdom to my list of evidence.
Thank you for being a wise teacher,
QJ

User avatar
White Coat Investor
Posts: 12850
Joined: Fri Mar 02, 2007 9:11 pm
Location: Greatest Snow On Earth

Re: What is evidence based investing?

Post by White Coat Investor » Thu May 26, 2016 11:44 pm

Nonsense. Too many responses from physics/engineering types. Evidence based medicine means making your decisions based on the best data and theory available at the time. It doesn't say anything about guarantees.

Investing without knowing what the markets have done in the past or without considering the best academic theories out there is just being foolish. The evidence suggests you are better off keeping your costs low. The evidence suggests you won't successfully pick a portfolio of market beating active funds. The evidence suggests picking individual stocks invites uncompensated risks. The evidence suggests hedge funds don't beat index funds. The evidence suggests long-term stock performance will likely beat long-term bond performance. The evidence suggests a safe withdrawal rate for a 30 year period is in the neighborhood of 4%. Ignore the evidence at your own peril when designing and maintaining your investing plan.

I use the phrase evidence based investing all the time in my writing and presentations. The above paragraph explains why. Doctors in particular understand what I mean when I use the phrase.
1) Invest you must 2) Time is your friend 3) Impulse is your enemy | 4) Basic arithmetic works 5) Stick to simplicity 6) Stay the course

User avatar
JoMoney
Posts: 4396
Joined: Tue Jul 23, 2013 5:31 am

Re: What is evidence based investing?

Post by JoMoney » Fri May 27, 2016 2:10 am

There's a difference between using the "evidence" to avoid major pitfalls we know exist, vs. using "evidence" to chase hoped for performance.

There is strong evidence that being levered to the hilt in stocks will have higher returns than no leverage.
There is evidence that sometimes that's not true.
Mark Twain wrote:Beautiful credit! The foundation of modern society. Who shall say that this is not the golden age of mutual trust, of unlimited reliance upon human promises? That is a peculiar condition of society which enables a whole nation to instantly recognize point and meaning in the familiar newspaper anecdote, which puts into the mouth of a distinguished speculator in lands and mines this remark: ‘I wasn’t worth a cent two years ago, and now I owe two millions of dollars.’
Warren Buffett wrote:When leverage works, it magnifies your gains. Your spouse thinks you’re clever, and your neighbors get envious. But leverage is addictive. Once having profited from its wonders, very few people retreat to more conservative practices. And as we all learned in third grade — and some relearned in 2008 — any series of positive numbers, however impressive the numbers may be, evaporates when multiplied by a single zero. History tells us that leverage all too often produces zeroes, even when it is employed by very smart people.


Medical professionals trying to help someone with a difficult problem have to weigh consequences of using various treatments vs nothing at all... but I'd wager in just about every case they would concede "An ounce of prevention is worth a pound of cure"
"To achieve satisfactory investment results is easier than most people realize; to achieve superior results is harder than it looks." - Benjamin Graham

protagonist
Posts: 4911
Joined: Sun Dec 26, 2010 12:47 pm

Re: "Predicting the Past"

Post by protagonist » Fri May 27, 2016 4:18 am

Taylor Larimore wrote:qwertyjazz:

Your post about Evidence Based Investing reminds me of this post made long ago by a very wise Boglehead with the username of "Ozark":
--------------------------------------PREDICTING THE PAST--------------------------------

If you feel you can improve your portfolio's asset allocation by running the portfolio through various computer programs, measuring and grading various risk/reward relationships, feel free. It's okay with me. Honest. For myself, I'm not interested.

I'm also not interested in running reams of data through a computer program in order to discover how much I can withdraw yearly from my portfolio and never go broke.

Without having studied it, I'm willing to assume the Risk Grades deal is similar to the well known Efficient Frontier concept: Invest in a mix of assets that will give the best return for the least risk.

Wonderful. The problem in execution is this; both these approaches would seem to be limited to looking at PAST risk/return relationships, in order to predict FUTURE such relationships.

This approach hasn't worked very well and it never will.

There's lots of stuff we can learn by studying the past. One thing we can't learn, though, is how much the future will resemble the past.

There really is an Efficient Frontier. There really is a withdrawal rate that will allow my wife and I to spend all our money during our life times, but never go broke.

But these things are unknown and unknowable, going forward. Such things are only knowable looking backward.

Given that such things are only knowable looking backward, academics with more letters after their names than I have money in the bank, have spent unconscionable amounts of time goobering through the past. They thus invented Modern Portfolio Theory---Beta, Alpha, R-Squared, and the crowning achievement, Sharpe Ratio. These accomplishments were celebrated and awards were given. Yes.

And then...a funny thing happened on the way to the bank. These numbers turned out to have little or no predictive value, regarding returns. And since they couldn't predict returns, they also failed to predict risk/return ratios.

Joining in the fun, M* invented their first Star Rating system, a system that graded...yep...risk- adjusted, past performance.

I wish I had 10 bucks for every post I've read where the poster said, essentially, "I have a balanced portfolio, made up entirely of 4 and 5 star funds." Too late, these jokers discovered what M* eventually discovered; past risk-adjusted performance doesn't predict future risk-adjusted performance.

I don't want to discover the Sharpe Ratio of my portfolio. I don't want to discover its Beta. I don't want to discover its Risk Grade. I have absolutely no confidence that adjusting the portfolio so that these numbers become more favorable will improve future risk/reward.

If others do want to do that, that's okay with me. I seriously doubt, though, that many successful mutual fund managers select securities in that manner. If any do, or if any money managers set their asset allocations in that manner, I'd be interested in their long-term results---results over periods of, say, 10 years, or more.

In short, computers are wonderous tools, but that's all they are. Every computer on Earth, all linked up and working 24/7, from now on, won't tell me my survivable withdrawal rate. Neither will they tell me what asset allocation would give me the best risk/reward ratio.

In my opinion, these things can't be calculated. We have to forge ahead without knowing these things. Deal with it.

Best wishes.
Taylor


+1000

protagonist
Posts: 4911
Joined: Sun Dec 26, 2010 12:47 pm

Re: What is evidence based investing?

Post by protagonist » Fri May 27, 2016 4:43 am

Using the term "evidence-based" as it is being used in investing is little more than woo.

"Evidence-based" medicine is based on testable hypotheses going forward. Hypothesis: Drug X lowers blood pressure. You give hundreds, maybe thousands, of people with hypertension Drug X. You give an equal number a placebo. You don't tell either group which they are getting, and the experimenters do not know which drug they are administering ("double-blind"). You control all other possible variables that you can (age, gender, pre-existing conditions, other medications, lifestyle, etc.). At given points going forward in the experiment, you measure all subjects' blood pressure. You collect the data and analyze it using valid statistical methods. You then conclude that, within a previously chosen confidence limit, Drug X was or was not likely effective. That is "evidence-based". The experiment is then critiqued and repeated several times by other investigators with the objective of better controlling variables and providing more evidence as to whether the conclusions were valid.

In finance no experiments are being performed. Past data is being mined, with no testable hypotheses, no ability to control or manipulate variables, no knowledge as to what variables even exist or are relevant in the future (political situation? business climate? market manipulation? etc) and no knowledge as to whether future conditions will even vaguely mirror the past. You then conclude that what worked in the recent past (20 years? 100 years?) will be repeated in the future. "Evidence-based" as applied in this case is just a fancy word meant to sound convincing, just as "quantum" is as applied to everything under the sun these days besides quantum mechanics.
Last edited by protagonist on Fri May 27, 2016 11:58 am, edited 1 time in total.

goingup
Posts: 2773
Joined: Tue Jan 26, 2010 1:02 pm

Re: What is evidence based investing?

Post by goingup » Fri May 27, 2016 9:36 am

White Coat Investor wrote:Investing without knowing what the markets have done in the past or without considering the best academic theories out there is just being foolish. The evidence suggests you are better off keeping your costs low. The evidence suggests you won't successfully pick a portfolio of market beating active funds. The evidence suggests picking individual stocks invites uncompensated risks. The evidence suggests hedge funds don't beat index funds. The evidence suggests long-term stock performance will likely beat long-term bond performance. The evidence suggests a safe withdrawal rate for a 30 year period is in the neighborhood of 4%. Ignore the evidence at your own peril when designing and maintaining your investing plan.

OP-
This summarizes the broad truths of investing. Fantastic post!

User avatar
White Coat Investor
Posts: 12850
Joined: Fri Mar 02, 2007 9:11 pm
Location: Greatest Snow On Earth

Re: What is evidence based investing?

Post by White Coat Investor » Fri May 27, 2016 2:49 pm

protagonist wrote:Using the term "evidence-based" as it is being used in investing is little more than woo.

"Evidence-based" medicine is based on testable hypotheses going forward. Hypothesis: Drug X lowers blood pressure. You give hundreds, maybe thousands, of people with hypertension Drug X. You give an equal number a placebo. You don't tell either group which they are getting, and the experimenters do not know which drug they are administering ("double-blind"). You control all other possible variables that you can (age, gender, pre-existing conditions, other medications, lifestyle, etc.). At given points going forward in the experiment, you measure all subjects' blood pressure. You collect the data and analyze it using valid statistical methods. You then conclude that, within a previously chosen confidence limit, Drug X was or was not likely effective. That is "evidence-based". The experiment is then critiqued and repeated several times by other investigators with the objective of better controlling variables and providing more evidence as to whether the conclusions were valid.

In finance no experiments are being performed. Past data is being mined, with no testable hypotheses, no ability to control or manipulate variables, no knowledge as to what variables even exist or are relevant in the future (political situation? business climate? market manipulation? etc) and no knowledge as to whether future conditions will even vaguely mirror the past. You then conclude that what worked in the recent past (20 years? 100 years?) will be repeated in the future. "Evidence-based" as applied in this case is just a fancy word meant to sound convincing, just as "quantum" is as applied to everything under the sun these days besides quantum mechanics.


In medicine, most of the questions haven't been answered by anything even resembling a double blind placebo controlled trial. A big part of evidence based medicine is weighing the strength of the evidence. You should do the same in finance, especially with retrospective studies.
1) Invest you must 2) Time is your friend 3) Impulse is your enemy | 4) Basic arithmetic works 5) Stick to simplicity 6) Stay the course

User avatar
tadamsmar
Posts: 7436
Joined: Mon May 07, 2007 12:33 pm

Re: What is evidence based investing?

Post by tadamsmar » Fri May 27, 2016 3:10 pm

White Coat Investor wrote:In medicine, most of the questions haven't been answered by anything even resembling a double blind placebo controlled trial. A big part of evidence based medicine is weighing the strength of the evidence. You should do the same in finance, especially with retrospective studies.


The evidence in finance is typically not as strong as a double blind placebo controlled trial.

This paper "Physics Envy May be Hazardous to your Wealth" come to mind:

http://mitsloan.mit.edu/media/Lo_PhysicsEnvy.pdf

qwertyjazz
Posts: 1026
Joined: Tue Feb 23, 2016 4:24 am

Re: What is evidence based investing?

Post by qwertyjazz » Fri May 27, 2016 5:27 pm

Interesting paper. I am inclined by training to want to agree with the authors of the importance of finance PhDs for level 4 complexity. But I am wondering the importance of philosophers and historians for level 5. I also wonder for level 5 complexity how close the statements are of first do no harm and winning the loser's game. Less is more as the rules change. But learning history, philosophy and art to find the right analogies might matter in a world without randomized control trials. I probably need to retread Knight. Thank you

User avatar
JPH
Posts: 659
Joined: Mon Jun 27, 2011 8:56 pm

Re: What is evidence based investing?

Post by JPH » Fri May 27, 2016 6:15 pm

White Coat Investor wrote:
protagonist wrote:Using the term "evidence-based" as it is being used in investing is little more than woo.

"Evidence-based" medicine is based on testable hypotheses going forward. Hypothesis: Drug X lowers blood pressure. You give hundreds, maybe thousands, of people with hypertension Drug X. You give an equal number a placebo. You don't tell either group which they are getting, and the experimenters do not know which drug they are administering ("double-blind"). You control all other possible variables that you can (age, gender, pre-existing conditions, other medications, lifestyle, etc.). At given points going forward in the experiment, you measure all subjects' blood pressure. You collect the data and analyze it using valid statistical methods. You then conclude that, within a previously chosen confidence limit, Drug X was or was not likely effective. That is "evidence-based". The experiment is then critiqued and repeated several times by other investigators with the objective of better controlling variables and providing more evidence as to whether the conclusions were valid.

In finance no experiments are being performed. Past data is being mined, with no testable hypotheses, no ability to control or manipulate variables, no knowledge as to what variables even exist or are relevant in the future (political situation? business climate? market manipulation? etc) and no knowledge as to whether future conditions will even vaguely mirror the past. You then conclude that what worked in the recent past (20 years? 100 years?) will be repeated in the future. "Evidence-based" as applied in this case is just a fancy word meant to sound convincing, just as "quantum" is as applied to everything under the sun these days besides quantum mechanics.


In medicine, most of the questions haven't been answered by anything even resembling a double blind placebo controlled trial. A big part of evidence based medicine is weighing the strength of the evidence. You should do the same in finance, especially with retrospective studies.


I agree with White Coat. The point of evidence-based medicine is to use the strongest evidence that can be found, with full awareness of the limitations, to make good decisions. We also integrate the evidence with clinical wisdom and patients' values. If evidence is available, why would investment decisions be better by ignoring it? To oversimplify the decision making process to the point of holding it to a crystal ball standard is naïve.
While the moments do summersaults into eternity | Cling to their coattails and beg them to stay - Townes Van Zandt

protagonist
Posts: 4911
Joined: Sun Dec 26, 2010 12:47 pm

Re: What is evidence based investing?

Post by protagonist » Sat May 28, 2016 7:59 am

JPH wrote:
White Coat Investor wrote:
protagonist wrote:Using the term "evidence-based" as it is being used in investing is little more than woo.

"Evidence-based" medicine is based on testable hypotheses going forward. Hypothesis: Drug X lowers blood pressure. You give hundreds, maybe thousands, of people with hypertension Drug X. You give an equal number a placebo. You don't tell either group which they are getting, and the experimenters do not know which drug they are administering ("double-blind"). You control all other possible variables that you can (age, gender, pre-existing conditions, other medications, lifestyle, etc.). At given points going forward in the experiment, you measure all subjects' blood pressure. You collect the data and analyze it using valid statistical methods. You then conclude that, within a previously chosen confidence limit, Drug X was or was not likely effective. That is "evidence-based". The experiment is then critiqued and repeated several times by other investigators with the objective of better controlling variables and providing more evidence as to whether the conclusions were valid.

In finance no experiments are being performed. Past data is being mined, with no testable hypotheses, no ability to control or manipulate variables, no knowledge as to what variables even exist or are relevant in the future (political situation? business climate? market manipulation? etc) and no knowledge as to whether future conditions will even vaguely mirror the past. You then conclude that what worked in the recent past (20 years? 100 years?) will be repeated in the future. "Evidence-based" as applied in this case is just a fancy word meant to sound convincing, just as "quantum" is as applied to everything under the sun these days besides quantum mechanics.


In medicine, most of the questions haven't been answered by anything even resembling a double blind placebo controlled trial. A big part of evidence based medicine is weighing the strength of the evidence. You should do the same in finance, especially with retrospective studies.


I agree with White Coat. The point of evidence-based medicine is to use the strongest evidence that can be found, with full awareness of the limitations, to make good decisions. We also integrate the evidence with clinical wisdom and patients' values. If evidence is available, why would investment decisions be better by ignoring it? To oversimplify the decision making process to the point of holding it to a crystal ball standard is naïve.


You are right.

Taken literally, our assumptions about the market are based on evidence, and thus are literally "evidence-based".

The problem is that for me, and I think to most scientists and physicians, the term "evidence-based" is a loaded one, and implies much more solid "evidence" than what we have. Just as the words "hypothesis" and "theory" mean something different to scientists than to the general public (eg. "the theory of evolution" vs. "I have a theory as to why my baby is crying at night".)

If you knew a friend or relative who had a disease, took a particular treatment, and the disease went away, and then you got the same disease, you may well be inclined to take the same treatment, without knowing why or how it works, or if it was the treatment that had any effect in the first place. You would do that based on evidence. But I don't think most physicians would call that "evidence-based medicine"....not unless you had hundreds of similar examples and some sort of controls.

You could also argue that the chances of that treatment working on you would be relatively high, since humans are nearly identical genetically...two random people probably have much more in common biologically than two random centuries do economically.

But yes, we do make our investment decisions based on evidence. I would argue that it is very weak evidence, but it is the only evidence we have, and thus it is logical that we proceed accordingly, as long as we realize that what we are doing is "educated guessing" (not science), and it could possibly blow up in our faces. But so often I hear people here say, and believe, that if you hold stocks long enough, the risk will progressively decrease.....they will outperform other investments and beat inflation. There is nowhere near enough evidence to support that conclusion.

So as a "scientist" (using the term loosely, if you want to call a physician a scientist), seeing how the term is used, I balk at the use of the term "evidence-based" because I think it is too loaded, and conveys to the layman a level of confidence in our futures that is unjustified.

User avatar
Tyler9000
Posts: 295
Joined: Fri Aug 21, 2015 11:57 am

Re: What is evidence based investing?

Post by Tyler9000 » Sat May 28, 2016 11:11 am

I suppose the opposite of evidence-based investing is faith-based investing, where no amount of evidence can persuade one to question the dogma guiding their decisions. That seems particularly shortsighted to me. But importantly, it's not a binary choice -- A wise investor both believes in the underlying theory of their chosen portfolio and seeks evidence of how it has performed in the real world.

I think the point that trips a lot of people up is a shallow understanding of uncertainty and what the best evidence actually implies about the future. Backtesting is terrible at predicting with precision what the future will be, but it's valuable for gauging with a reasonable amount of certainty the range of possible outcomes. Think of it like selecting which archer has the honor of shooting an arrow at an apple on your head. Few people would argue that because it's impossible to know to the millimeter precisely where the arrow will land in the future that studying the spread of historical shots for different archers is irrelevant. No shot is completely predictable, but not every archer is equally uncertain with their aim.

Investing works in a similar way. Different portfolios have different characteristics -- some perform demonstrably better than others, and not every portfolio is equally uncertain. Studying evidence of past performance can't predict the future with precision, but it can help guide you to a portfolio with a range of outcomes best matched to your needs. The mistake too many people make is that they look at only one average or outcome and don't consider the full range. Usually they pick the archer who nailed the last shot without noticing that he missed 100 before that. It's not that the data lied -- they just saw what they wanted to see and didn't understand how uncertainty works.

dbr
Posts: 23687
Joined: Sun Mar 04, 2007 9:50 am

Re: What is evidence based investing?

Post by dbr » Sat May 28, 2016 11:24 am

Tyler9000 wrote:I suppose the opposite of evidence-based investing is faith-based investing, where no amount of evidence can persuade one to question the dogma guiding their decisions. That seems particularly shortsighted to me. But importantly, it's not a binary choice -- A wise investor both believes in the underlying theory of their chosen portfolio and seeks evidence of how it has performed in the real world.



I think what the author really has in mind for the alternative is that investing might be matter of unique skill and individual genius, somewhat like the difference between the individual giants in any field and the simple rational thinking that anyone can do or benefit from once the analysis presented.

The question is whether individual geniuses ever sell their services. They do not. Only con men pretend to do so. It may even be a question if there are individual geniuses in investing rather than just a small class of lucky survivors.

A different question is whether the evidence on which investing can be based extends significantly beyond obvious ideas such as cost matters, don't time the market, don't follow fads, etc., etc. I think a previous post mentioned this.

Also note the post above that points out that evidence can tell you a lot about the general shape of things while precise predictions are impossible to come by.

User avatar
JPH
Posts: 659
Joined: Mon Jun 27, 2011 8:56 pm

Re: What is evidence based investing?

Post by JPH » Sat May 28, 2016 1:07 pm

In 1999. Isaacs & Fitzgerald published a humorous article in BMJ titled "Seven Alternatives to Evidence Based Medicine." The same alternatives probably apply to investing. The 7 were:
Eminence based
Vehemence based
Eloquence based
Providence based
Diffidence based
Nervousness based
Arrogance based (which they said was restricted to surgeons) :happy
While the moments do summersaults into eternity | Cling to their coattails and beg them to stay - Townes Van Zandt

inbox788
Posts: 4142
Joined: Thu Mar 15, 2012 5:24 pm

Re: What is evidence based investing?

Post by inbox788 » Sat May 28, 2016 2:29 pm

protagonist wrote:Using the term "evidence-based" as it is being used in investing is little more than woo.

"Evidence-based" medicine is based on testable hypotheses going forward. Hypothesis: Drug X lowers blood pressure. You give hundreds, maybe thousands, of people with hypertension Drug X. You give an equal number a placebo. You don't tell either group which they are getting, and the experimenters do not know which drug they are administering ("double-blind"). You control all other possible variables that you can (age, gender, pre-existing conditions, other medications, lifestyle, etc.). At given points going forward in the experiment, you measure all subjects' blood pressure. You collect the data and analyze it using valid statistical methods. You then conclude that, within a previously chosen confidence limit, Drug X was or was not likely effective. That is "evidence-based". The experiment is then critiqued and repeated several times by other investigators with the objective of better controlling variables and providing more evidence as to whether the conclusions were valid.

In finance no experiments are being performed. Past data is being mined, with no testable hypotheses, no ability to control or manipulate variables, no knowledge as to what variables even exist or are relevant in the future (political situation? business climate? market manipulation? etc) and no knowledge as to whether future conditions will even vaguely mirror the past. You then conclude that what worked in the recent past (20 years? 100 years?) will be repeated in the future. "Evidence-based" as applied in this case is just a fancy word meant to sound convincing, just as "quantum" is as applied to everything under the sun these days besides quantum mechanics.


Evidence-based implies scientific study of the subject involving observation and measurement. One important aspect of any scientific endeavor is repeatability. Medicine deals with different patient responses by taking large samples, but with finance, it's a lot harder to repeat some experiments. Things like the great depression and great recession come once in a great while and even one business cycle differs greatly from another. Even in medicine, controlling for differences in future experiments lead to variables that are difficult to control, but background economic conditions are nearly impossible to replicate. In medicine, you can restrict a patient population to an age and weight, say average age of 40 and weight 170 lbs, along with other factors, and still the demographics may be responsible for any differences, and not the drug or intervention. Doing so with economic conditions will be nearly impossible (unemployment level, inflation, product cycle, competitive products, among a myriad of others).

pkcrafter
Posts: 12097
Joined: Sun Mar 04, 2007 12:19 pm
Location: CA
Contact:

Re: What is evidence based investing?

Post by pkcrafter » Sat May 28, 2016 5:51 pm

While reading this thread and looking for additional information on evidence based investing, I came across this comprehensive article. It's worth a look.

http://www.bh-co.com/wealth-management/ ... esting.pdf

I also found active management companies that use recent evidence to guide investing decisions, which results in constant portfolio changes.

Paul
When times are good, investors tend to forget about risk and focus on opportunity. When times are bad, investors tend to forget about opportunity and focus on risk.

User avatar
JPH
Posts: 659
Joined: Mon Jun 27, 2011 8:56 pm

Re: What is evidence based investing?

Post by JPH » Sat May 28, 2016 6:46 pm

pkcrafter wrote:While reading this thread and looking for additional information on evidence based investing, I came across this comprehensive article. It's worth a look.

http://www.bh-co.com/wealth-management/ ... esting.pdf

I also found active management companies that use recent evidence to guide investing decisions, which results in constant portfolio changes.

Paul


Thanks for posting this. The crucial step they spend little time on is explaining how they evaluate the quality (i.e., believability) and applicability of the evidence. Most of what they promote are strategies that receive a lot of attention on this forum, and I suppose we think our beliefs are somewhat evidence based. I don't have a strong enough background in economics or finance to critically evaluate their reference list. I also have noticed that a lot of financial articles, including some written by Bogleheads and posted to this forum, do not seem to be peer reviewed. It isn't hard at all to find evidence or do a study that supports the point you are trying to sell. Real scholarship requires a critical assessment of all the evidence and a thoughtful analysis of its strengths and limitations.
While the moments do summersaults into eternity | Cling to their coattails and beg them to stay - Townes Van Zandt

User avatar
Tyler9000
Posts: 295
Joined: Fri Aug 21, 2015 11:57 am

Re: What is evidence based investing?

Post by Tyler9000 » Sat May 28, 2016 8:46 pm

pkcrafter wrote:While reading this thread and looking for additional information on evidence based investing, I came across this comprehensive article. It's worth a look.

http://www.bh-co.com/wealth-management/ ... esting.pdf

I also found active management companies that use recent evidence to guide investing decisions, which results in constant portfolio changes.

Paul


Great article. Thanks for sharing!

User avatar
Wildebeest
Posts: 968
Joined: Fri Dec 27, 2013 2:36 pm

Re: What is evidence based investing?

Post by Wildebeest » Sat May 28, 2016 9:07 pm

JPH wrote:
pkcrafter wrote:While reading this thread and looking for additional information on evidence based investing, I came across this comprehensive article. It's worth a look.

http://www.bh-co.com/wealth-management/ ... esting.pdf

I also found active management companies that use recent evidence to guide investing decisions, which results in constant portfolio changes.

Paul


Thanks for posting this. The crucial step they spend little time on is explaining how they evaluate the quality (i.e., believability) and applicability of the evidence. Most of what they promote are strategies that receive a lot of attention on this forum, and I suppose we think our beliefs are somewhat evidence based. I don't have a strong enough background in economics or finance to critically evaluate their reference list. I also have noticed that a lot of financial articles, including some written by Bogleheads and posted to this forum, do not seem to be peer reviewed. It isn't hard at all to find evidence or do a study that supports the point you are trying to sell. Real scholarship requires a critical assessment of all the evidence and a thoughtful analysis of its strengths and limitations.


I agree. I believe there is no evidence based investing.

I do believe the Bogleheads Wiki is a as fair approximation as one can expect from data collection in a non scientific field as investing advice. I do enjoy reading Harry Markowitz, French/Fama and about the Efficient Frontier.

I actually enjoy reading Bogle, Ferri and Bernstein best, because I feel they have my best interests at heart.
The Golden Rule: One should treat others as one would like others to treat oneself.

User avatar
Rick Ferri
Posts: 8202
Joined: Mon Feb 26, 2007 11:40 am
Location: Vagabond, USA, Twitter: @Rick_Ferri
Contact:

Re: What is evidence based investing?

Post by Rick Ferri » Sat May 28, 2016 9:36 pm

The phrase "evidence based investing" does have one redeeming quality, it means fewer people will call multi-factor investing "smart beta".

Rick Ferri
The views expressed by Rick Ferri are strictly his own as a private investor and author and do not reflect the views of any entity or other persons.

pkcrafter
Posts: 12097
Joined: Sun Mar 04, 2007 12:19 pm
Location: CA
Contact:

Re: What is evidence based investing?

Post by pkcrafter » Sat May 28, 2016 10:26 pm

Rick Ferri wrote:The phrase "evidence based investing" does have one redeeming quality, it means fewer people will call multi-factor investing "smart beta".

Rick Ferri


:-)
When times are good, investors tend to forget about risk and focus on opportunity. When times are bad, investors tend to forget about opportunity and focus on risk.

Post Reply