DFA Emerging Markets Value fund tracking error?

[dnaumov]

Can anyone explain me what exactly happened in 2011 that made the DFA Emerging Markets Value fund have such a massive tracking error vs it's benchmark?

[larryswedroe]

Why do you think there was tracking error that was large?
Value premium was negative pretty much everywhere and so you would expect the EMV to underperform EM, just as EMS did.

Best wishes
Larry

[dnaumov]

Why do you think there was tracking error that was large?
Value premium was negative pretty much everywhere and so you would expect the EMV to underperform EM, just as EMS did.

Because the benchmark is a Value index and the fund massively underperformed said Value index.

[nisiprius]

Executive summary: just exploring, clicking around, a learning exercise for me... if I found the right numbers and the right funds...

...in 2011 the MSCI Emerging Markets Index and the MSCI Emerging Markets Value Index had returns of -18.17 and -17.57, respectively; that is to say, the value index had an 0.6% higher return.

The DFA funds with corresponding names had returns of -17.42% and -25.61%, respectively, i.e. the value fund had an 8.19% lower return.

There is no explanation I could find in the 4/30/2012 semiannual report, in which the chairman's letter says nothing specific about any funds.

*********
The long story:

By the way, is it appropriate to use the phrase "tracking error" for something that is not an index fund?dnaumov, maybe you could be a little bit more specific... exactly what numbers are you talking about, what index, and links to where you got them?

This is a learning exercise for me, I'm just going clickety-click and reporting what I find as I find it... but the Prospectus shows me this:

But, it's an Emerging Markets Value fund and it's being compared with a just-plain Emerging Markets index. The Emerging Markets fund shows this:

In both cases, the benchmark is the just-plain MSCI Emerging Market Index, and the Emerging Markets fund was a) pretty close and b) beat it by a full percentage point.

It's not surprising that an Emerging Markets Value fund would depart from a total Emerging Markets index. What's puzzling, of course, is why they used that index as the comparison, rather than the MSCI Emerging Markets Value Index, assuming of course that there is one.

Yes, there is. OK, what about that index? Google finds me this:MSCI Emerging Markets Value Index Fact Sheet, with this table:


It says that in 2011, the MSCI Emerging Markets index had a return of -18.17% (which, oddly enough, doesn't jibe with the number in the DFA annual report, which says -18.42%), and that the MSCI Emerging Markets Value Index had a return of -17.57%.

So, yeah, MSCI is saying Emerging Markets Value Index had 0.6% higher returns than Emerging Markets, yet the DFA Emerging Markets Value Portfolio got 8.19% lower return than DFA Emerging Markets Portfolio.

It does seem like something that needs an explanation.

[EDN]

Can anyone explain me what exactly happened in 2011 that made the DFA Emerging Markets Value fund have such a massive tracking error vs it's benchmark?

DFAs Emerging Markets funds don't track the MSCI indexes that closely. Country and sector weightings are different, as are average holdings. DFEVX holds the cheapest 25% of value stocks and updates the portfolio daily, whereas the MSCI EM Value Index holds the cheapest 50% and reconstitutes once per year. Last year was a tough year for "deep" value stocks around the world, a result that wasn't that acute for "relative" value indexes such as MSCI, Russell, and S&P. Looking at DFAs internal Emerging Market Value and Emerging Growth Indexes (upper and lower 25% of value/growth), we find a 10% difference between the two (-16% for Growth, -26% for Value).

But tracking error is to be expected. For example, even though DFEVX trailed the MSCI EM Value Index by about 7% in 2011, the DFA Emerging Markets Small Cap fund beat the MSCI Emerging Markets Small Cap Index by about 5%. In 2009, DFEVX outperformed the MSCI EM Value Index by 13%, but DFA EM Small trailed the MSCI EM SC Index by 14%. 1999 was off the charts, DFEVX beat the MSCI EM Value Index by almost 30%, and DFA EM Small beat the MSCI EM Small Index by 45%.

What matters is the long-term track record, not year over year differences. Since 1999 (first year of MSCI EM Index net. div data), DFA EM Value beat the MSCI EM Value Index by 2.2% per year. DFA EM Small beat the MSCI EM Small Index by 4.1% per year.

Eric

[nisiprius]

Well, I agree, it's puzzling. But it also looks as if the index was moving very quickly and small changes in endpoints would make a big difference, so maybe it's some weird technicality about earnings showing up in one year instead of another. I guess I don't know what the point of having a chairman's letter in a semiannual report is, if it's not to explain things like this, though.

Here's MSCI's chart of the index, and this is one of the worst X axes I've ever seen on any chart--a fifteen month interval between the ticks? hoo hah? and the Y axis isn't so great either, and we could use a few more gridlines... what's wrong with the people who make financial charts? But, if I'm eyeballing the start and end of 2011 correctly,



it looks as if in 2011 the index first soared, then dove about twice as far as it had soared. But the fund seems to have the dive without the soar:

[larryswedroe]

Few other thoughts
DFA doesn't care a wit about tracking error, unlike index funds that do. They sacrifice tracking error for higher returns, hopefully.
They deviate for reasons related to momentum screens, and other screens, block trading, country limits, and have different weightings, and how often the index reconstitutes, just a few examples.
Whenever there have been large deviations and we have asked for explanations they have them ready--sometimes it's been things like different weights by industry, or how momentum acted.
Hope that is helpful
Larry

[letsgobobby]

Few other thoughts
DFA doesn't care a wit about tracking error, unlike index funds that do. They sacrifice tracking error for higher returns, hopefully.
They deviate for reasons related to momentum screens, and other screens, block trading, country limits, and have different weightings, and how often the index reconstitutes, just a few examples.
Whenever there have been large deviations and we have asked for explanations they have them ready--sometimes it's been things like different weights by industry, or how momentum acted.
Hope that is helpful
Larry

given all of those possible manipulations, is it accurate to call these DFA funds actively managed?

[larryswedroe]

let'sgobobby
As to the definition of active vs. passive: If you define anything that is not purely an index fund that tracks the index basically perfectly, not wanting any tracking error, then DFA funds are active
That's not how I view it. There's a trade off of higher expected returns for tracking error risk which should be random.
There is nothing different going on here than with any other of their funds, it's the way prospectuses are written to provide flexibility for events that cannot even be anticipated.
Larry

[letsgobobby]

I always thought of DFA funds as 'managed index' funds but I suppose it is semantics or opinion as to whether something is managed index or actively managed.

[nisiprius]

Someone sent me some data.

Let's suppose that someone were to make the claim for DFA that they have some "tracking error" in the short-term, but that DFA investors are not short-term investors, that DFA consistently beats the index for long-term investors, and DFA's investors are therefore happy to shrug off the short-term departures from the index

I decided to use the data to answer this question: what's the smallest number of years, Y, such that the rolling Y-year return of DFEVX has always beaten the rolling 9-year return of the index. The answer turns out to be 9. So far, no investor holding DFEVX for at least nine years would ever have been worse off than an investor holding the index. That does not include the advisor's fee, of course. The X value is the end of the 9-year period.



The picture is not, however, quite so pretty for 5-year rolling returns. There are a very few places before 2011 where the DFEVX investor's 5-year returns were just a skosh less than the index, but just a skosh. It sure does look, however, as if 2011 did some damage.



Here is a similar chart for rolling 1-year returns:



My informant commented: "DFA funds frequently have large 'tracking errors' of this type; it's noise due mainly to composition differences, and doesn't need explanation."

[Rick Ferri]

DFA should not be referred to as index fund manager because they do not manage index funds. Nor should their products be referred to as index funds because they are not. DFA's investment process is purely quantitative. The portfolio managers select securities that have certain size, fundamental and momentum characteristics. Whatever the return of their quant strategy is any given year, that's what it is. It may be close to a traditional benchmark or way off from it.

DFA tries to capture factor returns as opposed to an index return. If you measure their results against the raw factor returns on Ken French's website, you'd see that DFA is only able to capture some of the size, value and tmomentum premiums. They can't capture them all. For example, an investor cannot capture a micro-cap return when it's not possible to invest in an all micro-cap portfolio.

It would do the company no good to measure their fund returns against raw factor returns because they performance would not look good. Rather, they select benchmarks that should be easy to beat in the long-term. Case in point, DFA changed their micro-cap benchmark from the CRSP 9-10 index to the Russell 2000 when the fund started underperforming. They also measure the return of international value funds against benchmarks that are not value, even though international value benchmarks exist.

I accept these limitations and shortfalls as a user of DFA funds because I understand the friction that occurs in the investment process. I also understand that marketing funds against a benchmark that the fund company cannot beat doesn't help bring in assets, and in the end, DFA is a privately held for-profit company.

The problem is what some other advisers try to make out of all this:

Let's say the return of a particular DFA fund does not beat a comparable style benchmark. Advisers who promote DFA funds will say it's because the company doesn't follow traditional benchmarks so some tracking error should be expected. That's the correct answer. However, when the same fund beats a comparable benchmark, those same advisers will say it's DFA's superior investment strategy that caused the out-performance and hyped the excess return to moon and back.

This double standard has been going on for years. DFA has no desire or intent to stop it for the reason noted above. I don't fault DFA for this because they have no control over how advisers "sell" they funds.

Rick Ferri

[nisiprius]

...they select benchmarks that should be easy to beat in the long-term. Case in point, DFA changed their micro-cap benchmark from the CRSP 9-10 index to the Russell 2000 when the fund started underperforming. They also measure the return of international value funds against benchmarks that are not value, even though international value benchmarks exist.

Are there, then, no regulations regarding benchmark selection? If it wanted to, could Vanguard benchmark the Vanguard Total Bond Market Index Fund against the 3-month LIBOR?

[Rick Ferri]

There are no regulations on benchmarks for funds. It's not an SEC regulated issue. If Vanguard want's to compare their High Yield Corporate Bond fund against T-bills, they can do it. It's not ethical, but it's not illegal either.

I see benchmark abuse all over the place, particularly from advisers who have tried to compare everything to the S&P 500 over the past 13 years because that index is still lower in price than it was in 1999.

Funny this has come up. Just last week I was having a discussion about benchmarks on LinkedIn with a money manager. He confessed, "If my client's let me pick the benchmark, I guarantee I'll beat it every time!" Sadly, there was a lot of honesty in that statement.

Rick Ferri

[Random Walker]

My understanding is that DFA funds are truly passive in that there is no market timing, no individual stock selection. Stocks either qualify for a fund or don't based on objective mechanical criteria: size, value, momentum. I would expect that their inclusion rules result in smaller small and more valuey value compared to popular indexes. I would expect virtually all the apparent tracking error to be due to relative factor loads. In a way they are their own indexes. If they didn't have tracking error relative to popular indexes, why would anyone invest in them? Vanguard is cheaper.

Dave

[Rick Ferri]

Someone has to decide what goes into a value screen, a momentum screen, and size screen. And then someone has to make adjustments to these screens as market conditions change (DFA has changed their size criteria more than once and added more momentum rules over the years). If you wish to call these passive decisions, I think there would be a good case against it.

Rick Ferri

[Random Walker]

Why have they changed screens for size and value? Are they consistent in trying to target certain percentages of the market, but the specific numbers change? Have they been forced to incorporate bigger and less valuey as they have accumulated more assets to manage? Thanks.

Dave

[Rick Ferri]

This isn't beta investing, where managers are trying to capture the return of the cap weighted market. It's anything but that. There is always tweaking going on in quant strategies. Markets change, they make a tweak, they change again, another tweak, change yet again, more tweaks. That's quant!

BTW, investors will never know if a tweak doesn't work because there is no benchmark from us to measure it against, which brings this discussion back to the OPs original question.

Rick Ferri

[nisiprius]

Isn't "patient trading" active-ish or market-timing-eseque, too? I'm not too clear on what it consists of, but I believe it consists of deferring transactions to a supposedly favorable moment, allowing the fund composition to drift off-target?

[larryswedroe]

nisprius
That is not what patient trading is
Patient trading means not hitting bids or taking offers, but using for example market orders or algorithmic programs to trade slowly to avoid or minimize market impact. It may also mean being willing to buy block sizes below market as an active trading wants to sell quickly.
Larry

[EDN]

I don't think DFA funds are "index funds" in the strictest sense, such as a Russell 3000 or Wilshire 5000 Index. But DFAs funds have a lot of index-like properties: they seek to hold a predefined, broadly diversified subset of securities that all share a common characteristic (i.e. "large value stocks"). In fact, DFA maintains "Dimensional indexes" for each one of their funds internally and provide them to advisors to give them a rough approximation of the historical and ongoing risk and return characteristics. But as I mentioned above, these indexes, from their value and size sorts to sector exposures and updating frequencies can have material differences over short periods. Over the long-run, the indexes are designed to produce higher returns than traditional indexes with less focused exposure to value stocks or small stocks, or both.

I think DFAs funds are more aptly described as "asset class" funds. They seek to provide consistent and diversified exposure to parts of the market that we have come to understand have higher expected sources of return, also known as asset classes. In many cases, DFA holds more securities than any competing retail index.

Where DFA deviates from a traditional index is only in the execution of their portfolios, however traditional indexes are starting to migrate this way as well. DFA doesn't blindly buy or sell stocks just because their pre-defined index says to. They observe market pricing (avoiding stocks that are too thinly traded) and try to avoid price pressures that detract from returns (buying during index reconstitution, diving into companies with negative momentum, rushing to sell stocks with positive momentum). More than anything, this is done to lower trading costs and to run the portfolios more efficiently. Almost all index funds today use "hold ranges", which is just an older variation on what DFA does currently.

DFA also excludes certain types of securities from their portfolios that may meet a basic definition of the asset class, but upon further inspection, have not exhibited the underlying traits of the asset class's expected returns. Examples include avoiding regulated utilities in their large value funds (RU have high value ratios but very low stock or market exposure, acting more like a stock and bond hybrid, and therefore don't have the same expected returns as true value stocks), avoiding REITs in their small value funds, and avoiding extreme growth stocks in their small cap portfolios. Some might view this as "active", but then we would also have to consider S&P index funds "active" because their index constituents are chosen by committee and exclude many eligible companies in a particular size range, and would also have to consider Russell index funds "active" because they intentionally avoid some of the smallest stocks in the market due to pricing issues around reconstitution time.

In my mind, this index or asset class or active distinction is less important that a consistent approach, broad diversification, and low costs, which most index funds and DFA have in common.

[EDN]

There is no doubt, the last few years have not produced the amount of outperformance for DFAs Emerging Value fund (relative to the MSCI EM Value index) as had been the case in the prior 8-10 years. Looking at 3 year periods of total return outperformance since 1999 (DFEVX minus MSCI EM Value Index):

99-01: +11.5%
02-04: +39.3%
05-07: +13.3%
08-10: +2.6%
11-12: -7.5%

But this recent result is attributable almost entirely to the huge outperformance of emerging market growth stocks over emerging market value stocks, which (according to the Dimensional Emerging Growth Index and Dimensional Emerging Value Index) has been about 18% since 2011.

When you hold a more focused sort (cheapest 25%) on value stocks, and the value premium is sharply negative, you will probably underperform a traditional value index that holds more growth oriented stocks (MSCI holds cheapest 50%).

[EDN]

Why have they changed screens for size and value? Are they consistent in trying to target certain percentages of the market, but the specific numbers change? Have they been forced to incorporate bigger and less valuey as they have accumulated more assets to manage? Thanks.

Dave

In 2000 or 2001, DFA updated their sort on overall company size to move away from the CRSP definitions, which ranked each NYSE stock equally into one of 20 deciles and then forced all other exchanges to fit into the corresponding size breaks, to a "% of market cap" formula. This former approach was leading to vastly different weighted average market cap and eligible # of securities for purchase from year to year, increasing turnover in the area of the market where you least want to be forced to trade.

For example, small cap was changed from CRSP 6-10 to "smallest 10% of the market". Micro Cap was changed from CRSP 9-10 to "smallest 5% of the market". This was done to create a more consistent universe of stocks from which to build an index from. Since 2000, the DFA US Micro Cap fund has had a 1.0 exposure to the size factor, and the DFA US Small Cap fund has had a 0.8 exposure to the size factor--about what you would expect, and both have had positive 3F alpha adjusted for fees, so it appears as though DFA made the right choice in giving themselves more flexibility and a more consistent set of stocks from which to build a portfolio

There haven't been any material updates to the value sort in 20+ years. The US small value fund is still "cheapest 25%", and Int'l value portfolios are still "cheapest 30%". A few years back, they did expand the US Large Value portfolio to "cheapest 20%" from "cheapest 10%" because the portfolio was becoming excessively concentrated (only about 100 stocks on the buy list at any point in time). Today, the large value fund holds a more sufficient list of about 200 companies -- still focused by better diversified without sacrificing expected returns. The Tax Managed Small Value fund was changed to the TM Targeted Value fund to make it more comparable to the regular Targeted Value fund and to give stocks a bit more room to move around for reduced trading and tax loss harvesting efforts. I was happy with the move as it increased the # of holdings to over 1,500, which is important for me in the small cap space -- I'd much prefer more stocks to less.

There have been gradual updates to the portfolios, as mentioned above: more intentional momentum considerations (not just random hold ranges), caps on sectors so the portfolios don't deviate too substantially from overall market, exclusion of extreme small growth stocks in the small cap blend portfolios, but none of these has had a huge impact on the overall nature of the funds.

I'd probably consider making wholesale index provider changes to be more material, as Vanguard has done moving from S&P to MSCI to CRSP in just the last 10 years. But change need not be a bad think, in all cases (DFA and Vanguard), I'm sure these are net benefits.

[EDN]

...they select benchmarks that should be easy to beat in the long-term. Case in point, DFA changed their micro-cap benchmark from the CRSP 9-10 index to the Russell 2000 when the fund started underperforming. They also measure the return of international value funds against benchmarks that are not value, even though international value benchmarks exist.

Are there, then, no regulations regarding benchmark selection? If it wanted to, could Vanguard benchmark the Vanguard Total Bond Market Index Fund against the 3-month LIBOR?

Nisiprius,

I don't think using a less-than-perfect benchmark in a prospectus is that big of a deal. I agree it is pretty common, for example Vanguard's International Value fund "benchmark's" itself against the MSCI World exUS Index and not the "value" index. I don't think anyone seriously uses prospectuses to do their investment research, I'll admit it isn't something I even considered until I read you mention it. All investors are capable of comparing the DFA US Micro Cap fund to the Russell Micro Cap Index, or the DFA Int'l Value and Vanguard Int'l Value funds to the MSCI EAFE Value Index.

I seriously doubt fund families do this to make their funds appear to have better relative returns. I wouldn't be surprised if it didn't have something to do with industry convention (the big institutional investors use a common subset of imperfect indexes: S&P 500, Russell 2000, MSCI World Index or All Country World Index, Barclays Aggregate Bond Index.

I am not saying it is the right thing to do, just never worried about it enough to even ask why.

[Random Walker]

EDN,
Thanks for the great response! Seems to me DFA is quite faithful to asset class purity.

Dave

[Rick Ferri]

I don't think using a less-than-perfect benchmark in a prospectus is that big of a deal. I agree it is pretty common, for example Vanguard's International Value fund "benchmark's" itself against the MSCI World exUS Index and not the "value" index. I don't think anyone seriously uses prospectuses to do their investment research, I'll admit it isn't something I even considered until I read you mention it. All investors are capable of comparing the DFA US Micro Cap fund to the Russell Micro Cap Index, or the DFA Int'l Value and Vanguard Int'l Value funds to the MSCI EAFE Value Index.

I seriously doubt fund families do this to make their funds appear to have better relative returns.

After spending 25 years in this business, I am much more cynical that you are. There is no doubt in my mind that fund companies will, at times, intentionally use a less than perfect benchmark, and that includes Vanguard. Why do so many foreign equity funds use the EAFE index as a benchmark despite much better alternatives? The EAFE is large-cap only, and does not include Canada or emerging markets. Yet funds have Canada and hold about 10% in emerging markets. EAFE is inappropriate - yet easy to beat - and that is the ONLY reason it is used.

Rick Ferri

[larryswedroe]

FWIW
I'm with Rick on this one as to being at least cynical as to choice of benchmarks. Which is why one should not look at the benchmarks per se but run regressions to understand the exposures/sources of returns
Larry

[Phineas J. Whoopee]

let'sgobobby
As to the definition of active vs. passive: If you define anything that is not purely an index fund that tracks the index basically perfectly, not wanting any tracking error, then DFA funds are active
That's not how I view it. There's a trade off of higher expected returns for tracking error risk which should be random.
There is nothing different going on here than with any other of their funds, it's the way prospectuses are written to provide flexibility for events that cannot even be anticipated.
Larry

[Emphasis added.]

Hi Larry,

To begin with, thanks for all your valuable contributions to the forum.

The bolded part doesn't make sense to me. If tracking error risk is indeed random, wouldn't that mean over a long enough time expected returns of the quant-driven DFA fund should be the same as for a simple index fund covering the same investment universe less the excess expense?

Is using the F/F factors not a strategy in and of itself, being used by DFA to seek "higher expected returns?" And if DFA is successful, shouldn't "tracking error" (if we're to call it that for an active, if quant-driven fund) vary year to year but show an upward bias over long periods of time, enough to overcome the expense and then some?

If I've misunderstood I would love to be corrected.

PJW

[larryswedroe]

PJW
Yes you misunderstand.
You can have random TE with an upward bias. So for example, when you harvest losses to improve AT returns you cannot buy back same stock for 31 days, so you will have TE with the index/benchmark. But should be random TE. Same with use of algorithmic and block trading, You use those you will have TE but should be random.
Now with things like MOM and value and size loadings you should not have random errors. They should be systematic exposure, intentionally taken with expectation of higher returns. Not certainty of course.
Hope that helps
Larry

[Phineas J. Whoopee]

Thanks Larry.

I'm pretty sure I now understand what you mean, which is helpful. To argue further would be to quibble over sentence structure and the definitions of words, so I'll drop it.

PJW

[larryswedroe]

PJW
My apologies if I wasn't clear before.
I do have a full time job and do try to respond to most posts I think I can add value to. To do that often the response is quick and I don't worry too much about sentence structure or being so precise as I would in writing a book. Trading off responsiveness for precision. Hopefully I have the right balance
Larry

[dnaumov]

Thanks for the replies everybody. And yes, by "benchmark", I specifically ment the MSCI Emerging Markets Value index (which would be the appropriate benchmark for the fund in question).

[EDN]

Thanks for the replies everybody. And yes, by "benchmark", I specifically ment the MSCI Emerging Markets Value index (which would be the appropriate benchmark for the fund in question).

Dnaumov,

I was going to send you a PM, but figured I would just post this to the thread. Per my comments above about DFA EM Value not tracking the MSCI EM Value index very closely, for 2012, DFEVX returned +19.4% vs. +15.9% for the MSCI EM Value Index. When we compare MSCI EM Index (growth) to the EM Value Index, we find that the value index trailed by 2.3%, not exactly a year where you'd expect DFAs more focused value fund to outperform the less value-oriented MSCI EM Value index. But there is of course more to it that just that. The lesson, I think, is simply that year over year index comparisons have a lot of noise to them. Over time, you'd expect a fund with a greater value orientation to have a higher return than another that doesn't, and since the 1/1999 inception of the MSCI EM Value Index, it has trailed DFEVX by over 2% (and over 3% compared to the plain MSCI EM Index), so expectations have held in this regard.

Eric