Larry Swedroe: Benchmark Smoke And Mirrors

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Post Reply
Random Walker
Posts: 3171
Joined: Fri Feb 23, 2007 8:21 pm

Larry Swedroe: Benchmark Smoke And Mirrors

Post by Random Walker » Wed Sep 12, 2018 9:25 am

https://www.etf.com/sections/index-inve ... nopaging=1

In this article Larry blows up a single active manager’s claim of beating an index benchmark. In the process he demonstrates that choosing an appropriate benchmark is not as straightforward as one would think. It’s likely better to use a passive fund with real costs as a benchmark than a cost free index. Not surprisingly, active managers may have a tendency to choose benchmarks that are self servingly easy to beat. As always, passive investing is the winner’s game.

Dave

User avatar
nedsaid
Posts: 10486
Joined: Fri Nov 23, 2012 12:33 pm

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by nedsaid » Wed Sep 12, 2018 9:40 am

This is why I like to benchmark with Vanguard Index Funds. Real life funds trying to match real life indexes plus Vanguard is pretty darned good at indexing.
A fool and his money are good for business.

Random Walker
Posts: 3171
Joined: Fri Feb 23, 2007 8:21 pm

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by Random Walker » Wed Sep 12, 2018 9:40 am

Earlier this week I was somewhat lambasted for making the comment that I had been told that DFA funds sort of serve as their own benchmarks and that I had begun to see the wisdom in that statement. Every fund is different. No fund is going to have the exact same factor loadings as another benchmark fund or index. If market, size, value, profitability/quality, CS momentum explain about 95% of a fund’s performance, then isn’t it more important that the fund is passive, formulaic, no market timing, no individual security selection, and that it sticks to its own transparent mandate than it is important to meet or beat a given benchmark? An investor should be equally upset if his fund underperforms or overperforms a truly appropriate benchmark. In either case, the fund has deviated from the factor exposures chosen by the investor. Alternatively, the benchmark just wasn’t as appropriate as the investor had thought.
In many ways, truly passive investing completely obviates any need for benchmarking. You get (almost*) exactly what your factor exposure delivers, minus costs.
This is why I believe benchmarking a beast like QSPIX is almost a silly endeavor. It is it’s own thing. And the investor either appreciates that it is accessing its factors and asset classes in a formulaic, passive, transparent fashion, or he doesn’t.

Dave

* patient trading, buy/hold ranges, securities lending can have positive effects too

UpperNwGuy
Posts: 1110
Joined: Sun Oct 08, 2017 7:16 pm

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by UpperNwGuy » Wed Sep 12, 2018 9:49 am

nedsaid wrote:
Wed Sep 12, 2018 9:40 am
This is why I like to benchmark with Vanguard Index Funds. Real life funds trying to match real life indexes plus Vanguard is pretty darned good at indexing.
+1

User avatar
nedsaid
Posts: 10486
Joined: Fri Nov 23, 2012 12:33 pm

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by nedsaid » Wed Sep 12, 2018 9:54 am

Random Walker wrote:
Wed Sep 12, 2018 9:40 am
Earlier this week I was somewhat lambasted for making the comment that I had been told that DFA funds sort of serve as their own benchmarks and that I had begun to see the wisdom in that statement. Every fund is different. No fund is going to have the exact same factor loadings as another benchmark fund or index. If market, size, value, profitability/quality, CS momentum explain about 95% of a fund’s performance, then isn’t it more important that the fund is passive, formulaic, no market timing, no individual security selection, and that it sticks to its own transparent mandate than it is important to meet or beat a given benchmark? An investor should be equally upset if his fund underperforms or overperforms a truly appropriate benchmark. In either case, the fund has deviated from the factor exposures chosen by the investor. Alternatively, the benchmark just wasn’t as appropriate as the investor had thought.
In many ways, truly passive investing completely obviates any need for benchmarking. You get exactly what your factor exposure delivers, minus costs.
This is why I believe benchmarking a beast like QSPIX is almost a silly endeavor. It is it’s own thing. And the investor either appreciates that it is accessing its factors and asset classes in a formulaic, passive, transparent fashion, or he doesn’t.

Dave
Well, you have to compare a fund to something. You have to have some sort of idea how it is doing relative to other options. Even if all you get is a sense of how the fund is doing.

Nisiprius had a thoughtful post on benchmarking a fund like QSPIX. He identified three items that I agreed with: the Morningstar Category Average for Multi-Alternative funds, Moderate Target Risk US Total Return, and a baseline portfolio of Coffeehouse portfolio with QSPIX and a Coffeehouse portfolio without QSPIX. I added a fourth, the hedge fund index.

So the approach looking at four benchmarks is a little more sophisticated version of eyeballing, not at all perfect, but gives you some idea if the fund is doing what it is supposed to be doing. And of course using Portfolio Visualizer is a great tool too.

So I know exactly what you mean by a fund being its own benchmark. I see what you mean but I don't know, I am kind of funny this way, I do like to do comparisons.

You are getting the wet noodle lashing from me because I understand what you are saying and sort of, kind of, you know, agree with what you are saying. At some point, an evaluation needs to be made.
A fool and his money are good for business.

Random Walker
Posts: 3171
Joined: Fri Feb 23, 2007 8:21 pm

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by Random Walker » Wed Sep 12, 2018 10:14 am

Nedsaid,
I especially like your idea of comparing Coffee House +/- QSPIX. The reason is that what really matters is the portfolio as a whole, and consequently, the effect of any new addition or potential addition to the overall portfolio. How a new investment affects a portfolio depends on expected return, volatility, and correlations to other portfolio components. Of course costs critical too.

Dave

User avatar
packer16
Posts: 1044
Joined: Sat Jan 04, 2014 2:28 pm

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by packer16 » Wed Sep 12, 2018 10:36 am

Interesting article but I have a more fundamental question of using factors as an explanatory benchmark (which IMO it was intended to do) versus an active performance benchmark (that some use it for but IMO is Monday morning quarterbacking). The issue with using it for performance benchmarking is an investor will not know beforehand how to weight factors to mimick the portfolio which is a weighting of factors. The benchmarks are crude approximations of what the manager is trying to capture. For example, this manager states he wants to capture internal compounders. There is no factor for this so other than seeing how past factors preformed relative to this is interesting but probably not a good way to measure performance unless a link between factors & internal compounders can be found. I have no problem trying to explain historical returns but when they are turned into performance benchmarks comparing specific factor performance to a strategy that had correlation with in the past but most likely will not in the future, the comparisons become much less useful. IMO these factors can only become benchmarks if over a large number of period you find high correlations of the factors with the strategy followed.

This goes even further with statements like we found the causes that generated Buffet's returns. What was found was past correlations of factors of stocks he invested not the causation of the returns. You have to be careful that you do not confuse correlation with causation. If these guys truly found the causation then they would have been able to build a portfolio of stocks that could match Buffett's performance with identified factors out of sample but they have been able to do this.

Packer
Last edited by packer16 on Wed Sep 12, 2018 10:49 am, edited 1 time in total.
Buy cheap and something good might happen

User avatar
nisiprius
Advisory Board
Posts: 36852
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by nisiprius » Wed Sep 12, 2018 10:45 am

With those facts in mind, let’s benchmark NEAGX against a more appropriate [index] than the Russell 2000 Index benchmark, which is sometimes chosen by active managers because of the ease of outperforming it.

Due to the transparency of its reconstitution rules, active managers could choose to front-run the Russell 2000 Index. That creates a performance drag for funds seeking to match it. This is why Vanguard long ago abandoned that index as the benchmark for its Small Cap Index Fund (NAESX).
Larry Swedroe has occasionally mentioned the DFA US Micro Cap Portfolio fund, DFSCX, more or less favorably, by name and ticker symbol, e.g. here. It's Dimensional's oldest fund.

What benchmark does Dimensional use for DFSCX?
Image

The Russell 2000 index.

1) Should we suspect Dimensional of choosing it "because of the ease of outperforming it?"

2) Is it appropriate to benchmark a "micro cap" fund to a small-cap index?

3) Would it be inappropriate to benchmark a "micro cap" fund to the Russell Microcap Index?

And if you're curious, it did beat the Russell Microcap Index over the period for which Russell has no-cost downloadable data, and almost certainly for the last ten years (can't match the end dates exactly).

Image
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.

User avatar
vineviz
Posts: 2005
Joined: Tue May 15, 2018 1:55 pm

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by vineviz » Wed Sep 12, 2018 11:36 am

nisiprius wrote:
Wed Sep 12, 2018 10:45 am
What benchmark does Dimensional use for DFSCX?

The Russell 2000 index.

1) Should we suspect Dimensional of choosing it "because of the ease of outperforming it?"

2) Is it appropriate to benchmark a "micro cap" fund to a small-cap index?

3) Would it be inappropriate to benchmark a "micro cap" fund to the Russell Microcap Index?
Given that DFSCX launched more that 15 years before the inception date of the Russell Microcap Index, it's not hard to imagine why they didn't choose that index as their benchmark.

In any event, despite the fact that DFSCX has the words "micro cap" in its name it's holdings are actually somewhat closer to the Russell 2000 Index than to the Russel Microcap Index. As is often the case with investment analysis, it pays to give due attention to the details.
"Far more money has been lost by investors preparing for corrections than has been lost in corrections themselves." ~~ Peter Lynch

HEDGEFUNDIE
Posts: 980
Joined: Sun Oct 22, 2017 2:06 pm

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by HEDGEFUNDIE » Wed Sep 12, 2018 12:10 pm

Aren’t we all overthinking this? Isn’t the true benchmark of a fund what you would have invested in, in its place?

So for a multifactor fund, to me the relevant benchmark is the TSM. For the Micro Cap fund, it would be Small Cap blend. For QSPIX, I dunno, TBM/TSM blend?

The point of slicing and dicing and factor investing is to capture premia over the passive market. The more you restrict the benchmark the more you lose sight of those premia.

heyyou
Posts: 3172
Joined: Tue Feb 20, 2007 4:58 pm

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by heyyou » Wed Sep 12, 2018 12:37 pm

More money is good, but striving for perfection drives both innovation and dissatisfaction.

User avatar
packer16
Posts: 1044
Joined: Sat Jan 04, 2014 2:28 pm

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by packer16 » Wed Sep 12, 2018 3:50 pm

I like the idea of the idea of using the next best alternative & the effect of substituting the subject fund with other assets in your portfolio. You can test weather the fund adds value & the effect on volatility using this approach.

Packer
Buy cheap and something good might happen

User avatar
nisiprius
Advisory Board
Posts: 36852
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by nisiprius » Wed Sep 12, 2018 6:45 pm

vineviz wrote:
Wed Sep 12, 2018 11:36 am
nisiprius wrote:
Wed Sep 12, 2018 10:45 am
What benchmark does Dimensional use for DFSCX?

The Russell 2000 index.

1) Should we suspect Dimensional of choosing it "because of the ease of outperforming it?"

2) Is it appropriate to benchmark a "micro cap" fund to a small-cap index?

3) Would it be inappropriate to benchmark a "micro cap" fund to the Russell Microcap Index?
Given that DFSCX launched more that 15 years before the inception date of the Russell Microcap Index, it's not hard to imagine why they didn't choose that index as their benchmark.

In any event, despite the fact that DFSCX has the words "micro cap" in its name it's holdings are actually somewhat closer to the Russell 2000 Index than to the Russel Microcap Index. As is often the case with investment analysis, it pays to give due attention to the details.
Vanguard sometimes uses "spliced" benchmarks, thus-and-such up to one year and something different afterwards.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.

Dead Man Walking
Posts: 717
Joined: Wed Nov 07, 2007 6:51 pm

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by Dead Man Walking » Wed Sep 12, 2018 7:06 pm

HEDGEFUNDIE wrote:
Wed Sep 12, 2018 12:10 pm
Aren’t we all overthinking this? Isn’t the true benchmark of a fund what you would have invested in, in its place?
This is the system that I use when comparing funds. I often use DFA funds as a benchmark. I always consider the cost of an advisor when comparing DFA funds to funds offered by other fund providers. With the proliferation of ETFs based on nearly every benchmark index, this method seems to be reasonable.

DMW

User avatar
HomerJ
Posts: 11827
Joined: Fri Jun 06, 2008 12:50 pm

Re: Larry Swedroe: Benchmark Smoke And Mirrors

Post by HomerJ » Wed Sep 12, 2018 7:37 pm

Random Walker wrote:
Wed Sep 12, 2018 9:40 am
This is why I believe benchmarking a beast like QSPIX is almost a silly endeavor. It is it’s own thing. And the investor either appreciates that it is accessing its factors and asset classes in a formulaic, passive, transparent fashion, or he doesn’t.
Is it transparent?

Cliff Asness is a total genius... He was worth $3 billion this year which put him 264th on the Forbes 400 2018 list (his estimated net worth has grown to $3.6 billion since then), and he wasn't even on the list in 2016 (when you needed $1.7 billion to make the cut).

That's a lot of moolah to make in 2 years. Those fees for his "transparent" product are really paying off.

Where are the customer's yachts?
The J stands for Jay

Post Reply