John C. Bogle exposed? No way! [Investing paper retracted]

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Post Reply
Topic Author
Allan Roth
Posts: 361
Joined: Thu Sep 11, 2008 12:47 pm

John C. Bogle exposed? No way! [Investing paper retracted]

Post by Allan Roth »

It's been 15 years since the Journal of Financial Planning awarded the winner of their annual call-for-papers competition under the "academic" category for a paper entitled "Passive Investing: The Emperor Exposed." Jack was that emperor and the paper showed all previous studies were wrong and that 2/3 of active funds bested passive S&P 500 index funds by more than two percent annually.

https://www.financialplanningassociatio ... T05JFP.pdf

One call to the data provider, Lipper, confirmed the fatal flaws in this so called academic study but, although I challenged, the Journal's anonymous appeals committee refused to make a five-minute call to Lipper to confirm and upheld the paper. Enough pressure was put on the Journal where they eventually made that call and retracted the paper.

https://www.financialplanningassociatio ... or-exposed

It's still the only paper ever retracted by the journal but other so-called academic papers try to disprove arithmetic. We don't have John C. Bogle any longer but millions of us continue to benefit for the index funds he brought us.
Last edited by Allan Roth on Sun Sep 13, 2020 7:18 pm, edited 1 time in total.
livesoft
Posts: 74457
Joined: Thu Mar 01, 2007 8:00 pm

Re: John C. Bogle exposed? No way!

Post by livesoft »

In many journals, retracted papers found in online archives have a much more prominent watermark than the tiny box used by JFP.

See also: https://retractionwatch.com/
Wiki This signature message sponsored by sscritic: Learn to fish.
000
Posts: 4045
Joined: Thu Jul 23, 2020 12:04 am
Location: Ursa Minor

Re: John C. Bogle exposed? No way!

Post by 000 »

Well, the good news is most investors (including me before reading your post) have never heard of The Journal of Financial Planning.
Topic Author
Allan Roth
Posts: 361
Joined: Thu Sep 11, 2008 12:47 pm

Re: John C. Bogle exposed? No way!

Post by Allan Roth »

000 wrote: Sun Sep 13, 2020 7:27 pm Well, the good news is most investors (including me before reading your post) have never heard of The Journal of Financial Planning.
It's a publication by the Financial Planning Association for members who at typically CFPs. People like respect like David Blanchett at Morningstar publish some great work in this Journal.

I can tell you for sure that Jack read that piece about him. It was Jack's assistant that showed it to me. And Barron's just published a piece referencing a study that active bested passive.
User avatar
nisiprius
Advisory Board
Posts: 42544
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: John C. Bogle exposed? No way!

Post by nisiprius »

I appreciate the yeoman work you did in getting Ric Edelman to sort of kind of not quite retract his crazy claim that Vanguard index funds cost investors 1.4%/year in hidden expenses. Although I think you said he started making that claim again?

It's too much to hope for but I do wish that the SPIVA reports also contained percentages of active funds underperforming, not the costless index itself, but actual mutual funds or ETFs based on that index. It could even be an S&P promotional tool as they could use a point of choosing funds and ETFs tracking S&P indexes.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
User avatar
jeffyscott
Posts: 9324
Joined: Tue Feb 27, 2007 9:12 am
Location: Wisconsin

Re: John C. Bogle exposed? No way!

Post by jeffyscott »

Allan Roth wrote: Sun Sep 13, 2020 6:50 pmOne call to the data provider, Lipper, confirmed the fatal flaws in this so called academic study but, although I challenged, the Journal's anonymous appeals committee refused to make a five-minute call to Lipper to confirm and upheld the paper. Enough pressure was put on the Journal where they eventually made that call and retracted the paper.
What were the fatal flaws? Just that the data did not actually exist? The retraction just says: "conclusions were based on data that did not match the data presented in the paper".
The two greatest enemies of the equity fund investor are expenses and emotions. ― John C. Bogle
Robot Monster
Posts: 1803
Joined: Sun May 05, 2019 11:23 am
Location: New York

Re: John C. Bogle exposed? No way!

Post by Robot Monster »

I thought this thread was going to be salacious, based on the subject title. Mildly disappointed it's not.
“There are no answers, only choices.” ― Stanislav Lem, Solaris
User avatar
neurosphere
Posts: 3906
Joined: Sun Jan 17, 2010 1:55 pm

Re: John C. Bogle exposed? No way!

Post by neurosphere »

jeffyscott wrote: Mon Sep 14, 2020 7:50 am
Allan Roth wrote: Sun Sep 13, 2020 6:50 pmOne call to the data provider, Lipper, confirmed the fatal flaws in this so called academic study but, although I challenged, the Journal's anonymous appeals committee refused to make a five-minute call to Lipper to confirm and upheld the paper. Enough pressure was put on the Journal where they eventually made that call and retracted the paper.
What were the fatal flaws? Just that the data did not actually exist? The retraction just says: "conclusions were based on data that did not match the data presented in the paper".
I had the same question. Was the analysis accurate but the data set flawed? Was the author motivated to use this data set (e.g. intentional because it would lead to the desired conclusion).

I've seen several cases where a retracted paper leads to a NEW publication with the opposite conclusion. For example when all of the methods are solid apart from one issue (a contaminated reagent, a calculation error) identified after publication, which leads to a new one. I guess the author couldn't get a paper published which says "uh I've confirmed yet again that passive investing beats active investing". :)
If you have to ask "Is a Target Date fund right for me?", the answer is "Yes".
hnd
Posts: 289
Joined: Mon Jun 22, 2020 11:43 am

Re: John C. Bogle exposed? No way!

Post by hnd »

They came for him then and now that he's gone they'll come for him even harder. Now that indexing is approximately 50% of the mutual fund market. Their sliver of take is shrinking.
User avatar
bertilak
Posts: 8057
Joined: Tue Aug 02, 2011 5:23 pm
Location: East of the Pecos, West of the Mississippi

Re: John C. Bogle exposed? No way!

Post by bertilak »

jeffyscott wrote: Mon Sep 14, 2020 7:50 am
Allan Roth wrote: Sun Sep 13, 2020 6:50 pmOne call to the data provider, Lipper, confirmed the fatal flaws in this so called academic study but, although I challenged, the Journal's anonymous appeals committee refused to make a five-minute call to Lipper to confirm and upheld the paper. Enough pressure was put on the Journal where they eventually made that call and retracted the paper.
What were the fatal flaws? Just that the data did not actually exist? The retraction just says: "conclusions were based on data that did not match the data presented in the paper".
I also would like to know.
May neither drought nor rain nor blizzard disturb the joy juice in your gizzard. -- Squire Omar Barker (aka S.O.B.), the Cowboy Poet
cheezit
Posts: 377
Joined: Sat Jul 14, 2018 7:28 pm

Re: John C. Bogle exposed? No way!

Post by cheezit »

nisiprius wrote: Mon Sep 14, 2020 6:53 am I appreciate the yeoman work you did in getting Ric Edelman to sort of kind of not quite retract his crazy claim that Vanguard index funds cost investors 1.4%/year in hidden expenses. Although I think you said he started making that claim again?

It's too much to hope for but I do wish that the SPIVA reports also contained percentages of active funds underperforming, not the costless index itself, but actual mutual funds or ETFs based on that index. It could even be an S&P promotional tool as they could use a point of choosing funds and ETFs tracking S&P indexes.
How could he seriously make this claim? The tracking error numbers aren't exactly hidden.
mptness
Posts: 178
Joined: Sun Jun 03, 2012 3:54 pm

Re: John C. Bogle exposed? No way!

Post by mptness »

Thanks. I am a big fan of all your writing and appreciate reading you posts on this forum. Bogle's words: " You get what you don't pay for" are supported by years of SPIVA Reports, but fighting words to your average Financial Planner.
Greenman72
Posts: 514
Joined: Fri Nov 01, 2013 2:17 pm

Re: John C. Bogle exposed? No way!

Post by Greenman72 »

000 wrote: Sun Sep 13, 2020 7:27 pm Well, the good news is most investors (including me before reading your post) have never heard of The Journal of Financial Planning.
It is one of the few academic, peer-reviewed journals in the industry.

I have never read the Journal of Petroleum Engineering, but I still fill my car with gasoline.
kupuna
Posts: 79
Joined: Mon Jan 26, 2015 3:37 am
Location: San Diego/Maui

Re: John C. Bogle exposed? No way!

Post by kupuna »

Allan, you are a soldier of the truth and we are all better off because of it.

TOM
"I can calculate the motions of the heavenly bodies, but not the madness of people." Sir Isaac Newton
aristotelian
Posts: 8575
Joined: Wed Jan 11, 2017 8:05 pm

Re: John C. Bogle exposed? No way!

Post by aristotelian »

Claudia Sahm was on Masters in Business podcast (Barry Ritholz) talking about replication problems in Economics. https://www.bloomberg.com/news/audio/20 ... cy-podcast

There is a growing cottage industry in academia of replication studies that shows pretty much across fields a large portion of studies fail replication and overstate their results.
nzahir
Posts: 122
Joined: Mon Jul 27, 2020 11:44 pm

Re: John C. Bogle exposed? No way!

Post by nzahir »

Not reading this, but are they comparing apples to apples or not (likely not)?

Are they comparing tech, large growth, and small value vs large blend?
Doctor Rhythm
Posts: 584
Joined: Mon Jan 22, 2018 3:55 am

Re: John C. Bogle exposed? No way!

Post by Doctor Rhythm »

aristotelian wrote: Mon Sep 14, 2020 10:38 am Claudia Sahm was on Masters in Business podcast (Barry Ritholz) talking about replication problems in Economics. https://www.bloomberg.com/news/audio/20 ... cy-podcast

There is a growing cottage industry in academia of replication studies that shows pretty much across fields a large portion of studies fail replication and overstate their results.
Replication is mainly an issue with experimental research under controlled conditions. I didn’t listen to the podcast, but curious how this applies to macroeconomics/finance - where it seems to be more about analysis of real world data, without an expectation of reproducibility given lack of controllable conditions. It sounds more like a badly chosen data set or improper analysis. Not my area of specialty, so just wondering.
Topic Author
Allan Roth
Posts: 361
Joined: Thu Sep 11, 2008 12:47 pm

Re: John C. Bogle exposed? No way!

Post by Allan Roth »

jeffyscott wrote: Mon Sep 14, 2020 7:50 am
Allan Roth wrote: Sun Sep 13, 2020 6:50 pmOne call to the data provider, Lipper, confirmed the fatal flaws in this so called academic study but, although I challenged, the Journal's anonymous appeals committee refused to make a five-minute call to Lipper to confirm and upheld the paper. Enough pressure was put on the Journal where they eventually made that call and retracted the paper.
What were the fatal flaws? Just that the data did not actually exist? The retraction just says: "conclusions were based on data that did not match the data presented in the paper".
According to Lipper, the data request was for then current 2004 data and weights at that time. So it excluded 40 years of funds that went out of business and, even more importantly, weighted the funds based on the then current assets. So naturally the active funds that performed the best garnered more assets and were weighted more heavily even though they may have been tiny for most of that period.

So it's a combination of extreme survivor bias and extreme weighting bias.
aristotelian
Posts: 8575
Joined: Wed Jan 11, 2017 8:05 pm

Re: John C. Bogle exposed? No way!

Post by aristotelian »

Doctor Rhythm wrote: Mon Sep 14, 2020 1:31 pm
aristotelian wrote: Mon Sep 14, 2020 10:38 am Claudia Sahm was on Masters in Business podcast (Barry Ritholz) talking about replication problems in Economics. https://www.bloomberg.com/news/audio/20 ... cy-podcast

There is a growing cottage industry in academia of replication studies that shows pretty much across fields a large portion of studies fail replication and overstate their results.
Replication is mainly an issue with experimental research under controlled conditions. I didn’t listen to the podcast, but curious how this applies to macroeconomics/finance - where it seems to be more about analysis of real world data, without an expectation of reproducibility given lack of controllable conditions. It sounds more like a badly chosen data set or improper analysis. Not my area of specialty, so just wondering.
Sorry, Econ is not my field so I was not listening for those details. I believe this is the paper she references in the podcast. According to her, it was desk-rejected from all the top field journals.
https://www.federalreserve.gov/econresd ... 083pap.pdf
Last edited by aristotelian on Mon Sep 14, 2020 2:38 pm, edited 1 time in total.
User avatar
neurosphere
Posts: 3906
Joined: Sun Jan 17, 2010 1:55 pm

Re: John C. Bogle exposed? No way!

Post by neurosphere »

Allan Roth wrote: Mon Sep 14, 2020 1:35 pm
jeffyscott wrote: Mon Sep 14, 2020 7:50 am
Allan Roth wrote: Sun Sep 13, 2020 6:50 pmOne call to the data provider, Lipper, confirmed the fatal flaws in this so called academic study but, although I challenged, the Journal's anonymous appeals committee refused to make a five-minute call to Lipper to confirm and upheld the paper. Enough pressure was put on the Journal where they eventually made that call and retracted the paper.
What were the fatal flaws? Just that the data did not actually exist? The retraction just says: "conclusions were based on data that did not match the data presented in the paper".
According to Lipper, the data request was for then current 2004 data and weights at that time. So it excluded 40 years of funds that went out of business and, even more importantly, weighted the funds based on the then current assets. So naturally the active funds that performed the best garnered more assets and were weighted more heavily even though they may have been tiny for most of that period.

So it's a combination of extreme survivor bias and extreme weighting bias.
Thanks Allan, in the end, the problem seems obvious in retrospect.

I've been trying for a long time to find public or cheap data that lists closed funds by fund family. Do those exist? Or are they proprietary data sets that cost a lot of money to obtain? I have a feeling there are a ton of folks on BH with the education, knowledge, and experience to do fun little bias-free research.
If you have to ask "Is a Target Date fund right for me?", the answer is "Yes".
000
Posts: 4045
Joined: Thu Jul 23, 2020 12:04 am
Location: Ursa Minor

Re: John C. Bogle exposed? No way!

Post by 000 »

Allan Roth wrote: Sun Sep 13, 2020 8:01 pm It's a publication by the Financial Planning Association for members who at typically CFPs. People like respect like David Blanchett at Morningstar publish some great work in this Journal.

I can tell you for sure that Jack read that piece about him. It was Jack's assistant that showed it to me. And Barron's just published a piece referencing a study that active bested passive.
I wonder why the journal is still held in such esteem after the events you describe.
Greenman72 wrote: Mon Sep 14, 2020 10:33 am
000 wrote: Sun Sep 13, 2020 7:27 pm Well, the good news is most investors (including me before reading your post) have never heard of The Journal of Financial Planning.
It is one of the few academic, peer-reviewed journals in the industry.

I have never read the Journal of Petroleum Engineering, but I still fill my car with gasoline.
Well, you've made my point for me.

The vast majority of people on Earth (including me) do not make life decisions based on academic publications.

A fraudulent article in the Journal of Petroleum Engineering claiming premium gas is better for my car engine won't induce me to buy premium.

And I certainly won't pay a GUM fee to a CGP to pick fuels for my engine.
User avatar
peetsperk
Posts: 290
Joined: Wed Feb 04, 2015 4:02 pm

Re: John C. Bogle exposed? No way! [Investing paper retracted]

Post by peetsperk »

I noticed that Wikipedia notes many accomplishments of Christopher Carosa, but makes no reference to the retraction. To bring balance, should this be noted?
hnd
Posts: 289
Joined: Mon Jun 22, 2020 11:43 am

Re: John C. Bogle exposed? No way! [Investing paper retracted]

Post by hnd »

peetsperk wrote: Mon Sep 14, 2020 3:21 pm I noticed that Wikipedia notes many accomplishments of Christopher Carosa, but makes no reference to the retraction. To bring balance, should this be noted?
In 2005, the Journal of Financial Planning named Carosa as the winner of their annual call-for-papers competition under the "academic" category. His paper introduced the "snapshot-in-time" anomaly as it pertains to measuring investment performance.[16][17] In 2006, Allan Roth formally challenged and claimed there was a math error, but the Appeal Committee did not find enough evidence in their review to recommend retracting and stated that it "came down to methodology rather than mathematics."[18] After further inquires made by Journal and Financial Planning Association staff and no follow-up research provided by Carosa, the paper was retracted by the Journal in May 2007 for possible flaws.[19] To date, it is the only paper ever retracted by the Journal of Financial Planning per Carley Schulaka, editor of the Journal.[verification needed]
User avatar
jeffyscott
Posts: 9324
Joined: Tue Feb 27, 2007 9:12 am
Location: Wisconsin

Re: John C. Bogle exposed? No way!

Post by jeffyscott »

Allan Roth wrote: Mon Sep 14, 2020 1:35 pm
jeffyscott wrote: Mon Sep 14, 2020 7:50 am
Allan Roth wrote: Sun Sep 13, 2020 6:50 pmOne call to the data provider, Lipper, confirmed the fatal flaws in this so called academic study but, although I challenged, the Journal's anonymous appeals committee refused to make a five-minute call to Lipper to confirm and upheld the paper. Enough pressure was put on the Journal where they eventually made that call and retracted the paper.
What were the fatal flaws? Just that the data did not actually exist? The retraction just says: "conclusions were based on data that did not match the data presented in the paper".
According to Lipper, the data request was for then current 2004 data and weights at that time. So it excluded 40 years of funds that went out of business and, even more importantly, weighted the funds based on the then current assets. So naturally the active funds that performed the best garnered more assets and were weighted more heavily even though they may have been tiny for most of that period.

So it's a combination of extreme survivor bias and extreme weighting bias.
Thanks for those details. Survivorship bias occurred to me, but the paper (apparently mistakenly) claimed the data included closed and merged funds. Maybe they were "included" but then weighted at 0, since they had $0 as of 2004.

I would guess that asset weighting correctly would still help the managed fund results a bit, since larger funds tend to have lower ERs, if nothing else.
The two greatest enemies of the equity fund investor are expenses and emotions. ― John C. Bogle
AlwaysLearningMore
Posts: 78
Joined: Sun Jul 26, 2020 2:29 pm

Re: John C. Bogle exposed? No way!

Post by AlwaysLearningMore »

aristotelian wrote: Mon Sep 14, 2020 10:38 am Claudia Sahm was on Masters in Business podcast (Barry Ritholz) talking about replication problems in Economics. https://www.bloomberg.com/news/audio/20 ... cy-podcast
Do economists routinely perform double-blind experiments to test hypotheses, and are those double-blind experiments replicated by other researchers (i.e., science)?

For those who feel this is a 'no true Scotsman appeal to purity,' I say: Eaconamairean Albannach (mar an co-obraichean anns a h-uile àite) bu mhath leotha a bhith air am faicinn mar 'fìor luchd-saidheans.' Gu mì-fhortanach, leis nach eil iad a ’dèanamh deuchainnean fo smachd dà-dall tha iad fhathast air an ainmeachadh mar‘ luchd-saidheans sòisealta. ’ Bidh ceimigearan, fiosaig agus bith-eòlaichean luath airson na h-eadar-dhealachaidhean a chomharrachadh.
Bidh ceimigearan, fiosaig agus bith-eòlaichean gu luath a ’comharrachadh na h-eadar-dhealachaidhean eadar an dòigh-obrach aca an aghaidh eaconamaichean.
Tha fìor shaidheans air a chomharrachadh le smachd, mionaideachd, reusantachd, caochladairean fo smachd, agus ro-innseachd. Tha saidheans sòisealta an taobh eile air a choileanadh le caochladairean neo-fhaicsinneach no neo-riaghlaidh, agus bidh e a ’dèiligeadh ri faireachdainnean agus giùlan dhaoine.
Doctor Rhythm
Posts: 584
Joined: Mon Jan 22, 2018 3:55 am

Re: John C. Bogle exposed? No way!

Post by Doctor Rhythm »

AlwaysLearningMore wrote: Mon Sep 14, 2020 7:08 pm
aristotelian wrote: Mon Sep 14, 2020 10:38 am Claudia Sahm was on Masters in Business podcast (Barry Ritholz) talking about replication problems in Economics. https://www.bloomberg.com/news/audio/20 ... cy-podcast
Do economists routinely perform double-blind experiments to test hypotheses, and are those double-blind experiments replicated by other researchers (i.e., science)?
This is essentially what I was asking above. I think the term “replication” problem was being used broadly to refer to any problem with external validity.
Topic Author
Allan Roth
Posts: 361
Joined: Thu Sep 11, 2008 12:47 pm

Re: John C. Bogle exposed? No way! [Investing paper retracted]

Post by Allan Roth »

peetsperk wrote: Mon Sep 14, 2020 3:21 pm I noticed that Wikipedia notes many accomplishments of Christopher Carosa, but makes no reference to the retraction. To bring balance, should this be noted?
When I first noticed the lack of the mention for the retraction several months ago, I made the following edit to mention the retraction which reads:

In 2005, the Journal of Financial Planning named Carosa as the winner of their annual call-for-papers competition under the "academic" category. His paper introduced the "snapshot-in-time" anomaly as it pertains to measuring investment performance.[16][17] In 2006, Allan Roth formally challenged and claimed there was a math error, but the Appeal Committee did not find enough evidence in their review to recommend retracting and stated that it "came down to methodology rather than mathematics."[18] After further inquires made by Journal and Financial Planning Association staff and no follow-up research provided by Carosa, the paper was retracted by the Journal in May 2007 for possible flaws.[19] To date, it is the only paper ever retracted by the Journal of Financial Planning per Carly Schulaka, editor of the Journal.[verification needed]

Carly Schulaka told me to the best for her knowledge, it was the only paper ever to be retracted by the journal
afan
Posts: 5368
Joined: Sun Jul 25, 2010 4:01 pm

Re: John C. Bogle exposed? No way! [Investing paper retracted]

Post by afan »

Just to keep the record straight, Journal of Financial Planning is very far from a top academic journal. Based on what it publishes and the editorial standards, it would be difficult to describe it as an academic journal at all.

List of the top academic economics and finance journals. I sorted by H index but you can choose your own metric.

https://www.scimagojr.com/journalrank.p ... h&ord=desc

It goes from 1 (Journal of Finance) to 282. Journal of Financial Planning is not on the list. That may mean it has some citations, just fewer than the 282nd journal. Or that it has none at all.
We don't know how to beat the market on a risk-adjusted basis, and we don't know anyone that does know either | --Swedroe | We assume that markets are efficient, that prices are right | --Fama
Post Reply