Backtesting Int'l Small Stocks => revisiting data sources

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Post Reply
User avatar
Topic Author
siamond
Posts: 5322
Joined: Mon May 28, 2012 5:50 am

Backtesting Int'l Small Stocks => revisiting data sources

Post by siamond » Tue Dec 27, 2016 10:44 am

In the Simba backtesting spreadsheet, one asset class is documented as having some of the best returns, and this is International Small with a whooping 14% CAGR (1972-2015).

Here are the data sources for Simba v15d:
- IFA website (IFA IS, added back expenses) 1972-1996
- Vanguard Int'l Explorer (VINEX) 1997-2009
- Vanguard FTSE All-World ex-US Small-Cap Idx (VFSVX) 2010+

I tracked down what IFA is actually doing, back to the real data source (no meaningful change to the CAGR), this leads to a better list:
- DFA Int'l Small Cap Index (DFA Matrix) 1970-1996
- Vanguard Int'l Explorer (VINEX) 1997-2009
- Vanguard FTSE All-World ex-US Small-Cap Idx (VFSVX) 2010+

The intent of this thread is to investigate and discuss if this series should be kept as is, or maybe change to other data sources.

User avatar
Topic Author
siamond
Posts: 5322
Joined: Mon May 28, 2012 5:50 am

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by siamond » Tue Dec 27, 2016 10:55 am

Let's first ponder about the Vanguard funds being used.

VINEX (Vanguard International Explorer) is an active fund (quite active in truth!), loosely tracking the S&P EPAC Small index, which is centered on developed countries in Europe and Asia-Pacific. While VFSVX (Vanguard FTSE All-World ex-US Small-Cap) is passive, tracking the FTSE Global Small-Cap ex US Index, which includes Canada and many Emerging countries. This is squarely inconsistent.

A discussion about Total-International made me realize that we could make good use of an MSCI ACWI index for small-caps as well. My guess is that, if VFSVX had been created earlier than 2009, it would have tracked this precise MSCI index - before switching to FTSE. MSCI ACWI currently tracks 23 Developed Markets (DM) and 23 Emerging Markets (EM) countries.

We could then do the following, which would appear more consistent and also more in line with our passive-centric thinking:
- DFA Int'l Small Cap Index (DFA Matrix) 1970-1994
- MSCI ACWI Ex USA Small NR USD 1995-2009
- Vanguard FTSE All-World ex-US Small-Cap Index Fund (VFSVX) 2010+

I think this would be a very good step forward. This doesn't exactly help the CAGR though, as the VINEX active managers did quite good: we lose roughy 1.5% on the 1972-2015 period, down to 12.4%. But well, such is the price of consistency.

User avatar
Topic Author
siamond
Posts: 5322
Joined: Mon May 28, 2012 5:50 am

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by siamond » Tue Dec 27, 2016 11:16 am

Now let's explore the very first data source, the DFA International Small Index. The data comes from the DFA Matrix book (here is the PDF file), published by IFA, which is quite the bible of various historical returns.

Digging in the document, we can find a more precise definition of the Dimensional International Small Cap Index:

January 1994-present: Compiled by Dimensional from Bloomberg securities data. Market-capitalization-weighted index of small company securities in the eligible markets excluding those with the lowest profitability and highest relative price within the small cap universe.

July 1981-1993: Created by Dimensional; includes securities of MSCI EAFE countries in the bottom 10% of market capitalization excluding the bottom 1%. All securities are market capitalization weighted. Each country is capped at 50%; rebalanced semiannually.

January 1970-June 1981: 50% Hoare Govett Small Companies Index, 50% Nomura Small Companies Index.


Let's focus on the early days (1970-1981). I tracked down what the Hoare Govett and Nomura indexes are, and they are essentially country-specific small caps indexes (UK and Japan). Note that the Hoarse Govett index is now known as the Numis Smaller Companies index. This is hardly representative of the entire world besides the US, and I would tend to view it as misleading data.

It would appear that we need to restrict our dataset, and do the following:
- DFA Int'l Small Cap Index (DFA Matrix) 1982-1994
- MSCI ACWI Ex USA Small NR USD 1995-2009
- Vanguard FTSE All-World ex-US Small-Cap Index Fund (VFSVX) 2010+
Last edited by siamond on Tue Dec 27, 2016 11:27 am, edited 1 time in total.

User avatar
Topic Author
siamond
Posts: 5322
Joined: Mon May 28, 2012 5:50 am

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by siamond » Tue Dec 27, 2016 11:24 am

For the curious minds, you can find more about the Hoare Govett / Numis index by following those pointers:

http://nwww1.london.edu/news-and-events ... hed-by-rbs
http://citywire.co.uk/wealth-manager/ne ... ex/a578255
http://quotes.morningstar.com/indexquot ... FOUSA06E7I

And you can find more about the Nomura index here:
http://www.ftse.com/products/downloads/ ... ish.pdf?32
http://qr.nomuraholdings.com/en/fcts/do ... NSCC_e.pdf
http://qr.nomuraholdings.com/en/frcnri/index.html

The last pointer is especially fascinating, as it provides various slices of the Japanese market with monthly returns since 1980. This will provide good material for another post... :wink:

User avatar
Topic Author
siamond
Posts: 5322
Joined: Mon May 28, 2012 5:50 am

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by siamond » Tue Dec 27, 2016 11:47 am

Here is where we land with those various changes:
- DFA Int'l Small Cap Index (DFA Matrix) 1982-1994
- MSCI ACWI Ex USA Small NR USD 1995-2009
- Vanguard FTSE All-World ex-US Small-Cap Index Fund (VFSVX) 2010+

If you're pondering about the CAGR for the 1985-2015 time period, it is now 9.7%. If we were to keep VINEX, it would have been around 12%... And to compare to other series, Total Int'l was 8.4%, while Int'l Value was 9.6%. Oh, and Emerging rocked the boat by then, with more than 13%.

A few more thoughts.

1. The DFA index is derived from MSCI EAFE stocks. We're back to Int'l developed countries again, Europe and Asiapac. This bothers me a bit less than the 1995+ years though, as we have no emerging market (EM) returns before 1988, and EM was barely investible by then. So equating developed to the world may not be entirely wrong for this time period. Plus it's all we have, I tried to track various other Int'l Small-Caps indices (e.g. FTSE, S&P, etc), and couldn't find anything that starts before 1995 and goes beyond the 'developed' countries.

2. The DFA index clearly applies some fancy filters, e.g. eliminating tiny companies or unprofitable companies. The intent is obviously to improve returns, and I suspect a bit of ex post reasoning applies here, as DFA would not have chosen a filter that didn't work well in the past.

3. I compared the real DFA small-cap fund (DFISX), its performance loosely follows the DFA Int'l small-cap index performance, although it's official benchmark is MSCI World ex USA Small Cap (a misnomer, by the way, a 'MSCI World' means developed countries). It tends to do a bit better than the MSCI index, but remember, one has to pay advisor fees to get access to DFA funds.

I just do not know how to quantify the effect of any of those three points. I suspect #2 and #3 make the 1982-1994 returns rosier than they should be, while #1 might be a drag (cf. EM performance by the late 80s)... Who knows, so I guess we'd better off sticking to known numbers. Finally, with all those changes, Int'l Small will definitely look less artificially attractive (still worth it imho!) for the naive Simba user, which was part of my overarching concern to start with.

Feedback welcome - and kudos if you read all my words... :beer

User avatar
Tyler9000
Posts: 495
Joined: Fri Aug 21, 2015 11:57 am

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by Tyler9000 » Tue Dec 27, 2016 1:19 pm

Nice work, Siamond!
siamond wrote: 2. The DFA index clearly applies some fancy filters, e.g. eliminating tiny companies or unprofitable companies. The intent is obviously to improve returns, and I suspect a bit of ex post reasoning applies here, as DFA would not have chosen a filter that didn't work well in the past.
Luckily, the DFA years that best reflect a modern "neutral" small cap index are actually the ones we need most from 1982-1993. That segment does not appear to include the arbitrary "unprofitable" filter. And excluding the bottom 1% micro caps is actually standard practice for a small cap index. So I feel pretty good about the data for those years.

The developed vs. all world issue doesn't really concern me in this case. I'm not sure tracking small companies in emerging countries was all that practical an exercise decades ago. For all we know, many of them weren't investable anyway.

Overall, I agree with your analysis and conclusions. I can't imagine 50/50 UK/Japan are sufficient to model global small caps from 1970-1981. But the DFA data starting in 1982 seems very solid, and MSCI + VFSVX are good after that. I definitely like your idea to ditch the active fund when a solid passive alternative is available.

MachineGhost
Posts: 108
Joined: Sat Feb 14, 2009 2:46 am

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by MachineGhost » Wed Dec 28, 2016 3:58 pm

Fascinating reveal!

I've never liked DFA's approach and them pumping up their early numbers makes them even more suspect.

To be fair, S&P screens for profitability. That's what makes their indexes unique from the others (Russell, Wilshire, MSCI, etc.)

And "emerging" in the 1970's was today's frontier. Everything was shifted down an order of magnitude compared to now.

I would prefer to dump using DFA indexes in favor of alternatives whenever possible.

User avatar
Topic Author
siamond
Posts: 5322
Joined: Mon May 28, 2012 5:50 am

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by siamond » Wed Dec 28, 2016 5:16 pm

MachineGhost wrote:To be fair, S&P screens for profitability. That's what makes their indexes unique from the others (Russell, Wilshire, MSCI, etc.)
I wasn't aware of that. Good to know.
MachineGhost wrote:I would prefer to dump using DFA indexes in favor of alternatives whenever possible.
Same here. We actually succeeded to eliminate anything related to IFA or DFA in the coming Simba update, EXCEPT for International Small. Can't find an alternative, as previously explained. This being said, the way they calculated the 1982+ series seems reasonable, from what I can tell.

User avatar
Topic Author
siamond
Posts: 5322
Joined: Mon May 28, 2012 5:50 am

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by siamond » Fri Apr 27, 2018 7:51 pm

Let's re-open this thread about International Small Stocks. The current approach in the Simba backtesting spreadsheet is to use:
- DFA Int'l Small Cap Index (DFA Matrix) 1982-1994
- MSCI ACWI Ex USA Small NR USD 1995-2009
- Vanguard FTSE All-World ex-US Small-Cap Index Fund (VFSVX) 2010+

Some of us (including myself) weren't too happy about using DFA numbers though (see previous posts). Another possibility recently appeared, to use the Independence International Associates data series, as discussed in this recent thread about IIA. Then we could use the following spliced series:
- IIA Int'l (non US) Small Cap 1975-1994
- MSCI ACWI Ex USA Small NR USD 1995-2009
- Vanguard FTSE All-World ex-US Small-Cap Index Fund (VFSVX) 2010+

Pros:
- eliminate dependency on DFA
- IIA data series consistent with MSCI (as discussed in the IIA thread)
- start earlier (1975+)

Cons:
- IIA is not a well-known data source (although referred to by the Triumph of the Optimists authors)
- IIA Int'l SC is more a mid-cap data series than a small-caps data series

The last point doesn't overly concern me. Case in point, comparing the MSCI ACWI Ex USA Small Cap and the MSCI ACWI Ex USA Mid Cap indices shows trajectories (1994-2017) which are VERY similar. Thoughts, feedback?

EDIT: fixed small error, IIA starts in 1975, and not 1976.
Last edited by siamond on Sat Apr 28, 2018 3:19 pm, edited 1 time in total.

Theoretical
Posts: 1504
Joined: Tue Aug 19, 2014 10:09 pm

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by Theoretical » Fri Apr 27, 2018 10:32 pm

I like the consistency this would bring to the table, especially the fact it is extremely hard to invest in truly small international stocks as a US investor without a lot of midcaps.

User avatar
SimpleGift
Posts: 3559
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by SimpleGift » Sat Apr 28, 2018 1:31 am

Reading about the IIA international data series, one has to be impressed by the breadth and quality of the data covering global markets in the 1975-1996 time period:
  • • 21 international equity markets — Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Great Britain, Hong Kong, Ireland, Italy, Japan, Malaysia, Netherlands, New Zealand, Norway, Singapore, Spain, Sweden, Switzerland.

    • No survivorship bias, as index companies were added and deleted over time to match actual market changes, while retaining the historical return data for each individual month.
But as has been discussed, their definition of "small caps" does cause me concern. Not only are their small caps defined as the bottom 30% of market capitalization (versus the bottom 15% for modern MSCI international indexes, and the bottom 10% for DFA), but this further disclaimer is also mentioned:
Arshanapalli et. al. wrote:At this point, it is important to note that the MSCI data base is somewhat biased toward the larger companies in each country. Thus, since the MSCI universe does not include every stock in each country, the term “small stock” is indeed a relative term.

For example, since only about 500 stocks are included in the U.S. sample, most of the truly small stocks in the U.S. are excluded.

Japan is another example. While MSCI reviews securities for its Japanese universe from the First and Second Sections of the Tokyo and Osaka Stock Exchanges, the selected sample consists almost entirely of securities from the First Section of the Tokyo Stock Exchange. Thus most of the truly small stocks in that market are excluded as well. The reader should keep this in mind with regard to the results presented here for the small stock indexes.
Thus it's hard to escape the sense that the IIA international "small cap" index is likely a mid-cap index in disguise — which, though it may move roughly in sync with international small companies, is not likely to be capturing the distinctive volatility and return premium of a true small cap index. Just my two cents.

User avatar
Topic Author
siamond
Posts: 5322
Joined: Mon May 28, 2012 5:50 am

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by siamond » Sat Apr 28, 2018 4:59 am

SimpleGift wrote:
Sat Apr 28, 2018 1:31 am
Thus it's hard to escape the sense that the IIA international "small cap" index is likely a mid-cap index in disguise — which, though it may move roughly in sync with international small companies, is not likely to be capturing the distinctive volatility and return premium of a true small cap index. Just my two cents.
There is no question that the IIA SC index is more of a MC index by today's standards. It seems to me that this might actually better represent what was feasible (investable) in the 70s and 80s though, but I am speculating here.

To stay more factual, here is a comparison between the MSCI Int'l SC and MSCI In'tl MC for the known history of those two indices. The roller coaster is basically the same.

Image

User avatar
SimpleGift
Posts: 3559
Joined: Tue Feb 08, 2011 3:45 pm
Location: Central Oregon

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by SimpleGift » Sat Apr 28, 2018 8:35 am

siamond wrote:
Sat Apr 28, 2018 4:59 am
To stay more factual, here is a comparison between the MSCI Int'l SC and MSCI In'tl MC for the known history of those two indices. The roller coaster is basically the same.
The comparison chart is fairly persuasive, thank you. Taken as a whole then, I'm not seeing anything particularly disqualifying about using the IIA international "small cap" data series for the Simba spreadsheet.

Excellent historical data sleuthing and analysis on this project, by the way!

User avatar
Topic Author
siamond
Posts: 5322
Joined: Mon May 28, 2012 5:50 am

Re: Backtesting Int'l Small Stocks => revisiting data sources

Post by siamond » Mon Apr 30, 2018 9:20 am

It seems that we're in agreement here. I'll include this IIA Int'l Small data series for 1975-1994 in the next update of the Simba spreadsheet, probably for a mid-year update. I'll mark it as a 'crude mapping' though, if only due to the 'mid-cap-y' skew of the IIA series.

Many thanks for the discussion and feedback.

Post Reply