Stock market indexing
The growing market acceptance of index based investing through the media of traditional mutual funds and exchange traded funds[note 1] has resulted in a proliferation of index providers, attracted to the business prospects of licensing their indexes to investment companies and seeing increasing royalty and licensing fee revenue from the growing asset base.[1] For investors this growth has led to an ever expanding universe of available index funds. However, the different methodologies that index providers utilize in measuring and carving up the stock market leads to a dispersion of returns that can be considerable over short and intermediate term time frames.[2]
Benchmark indexes
The benchmark indexes created by the major index providers share a number of characteristics.
Benchmark index strategy box
These characteristics place benchmark indexes in the Passive security selection/ Capitalization security weighting segment of the Index Strategy Box (see Figure 1.) The index weighting of securities is based on the market capitalization of the companies included in the index.
Figure 1. Index Strategy Box[3]
Benchmark index style box
In general, the stock market is composed of 3 levels of market capitalization and 3 styles, resulting in a 3 x 3 "style" box which includes large cap, mid cap and small cap stocks, divided among value, blend, and growth stocks. This is commonly represented in a style box as illustrated below:
| Value | Blend | Growth | |
| Large Cap | LV | LB | LG |
| Mid Cap | MV | MB | MG |
| Small Cap | SV | SB | SG |
One may also see an expanded breakdown of market cap ranges to include Mega cap, Micro cap, and Nano cap stocks.
| Size Characterization | Capitalization Range |
| Mega capitalization stocks | Over $200 billion dollars |
| Large capitalization stocks | Between $10 billion dollar and $200 billion dollars |
| Mid capitalization stocks | Between $2 billion and $10 billion dollars |
| Small capitalization stocks | Between $300 million and $2 billion dollars |
| Micro capitalization stocks | Between $30 million and $375 million dollars |
| Nano capitalization stocks | Below $50 million dollars |
In addition to index provider companies, the Center for Research in Security Prices CRSP[4] has since 1960 provided the academic community with market return data extending from 1926 to the present. CRSP offers Historical Cap-Based Portfolio data tracking micro, small, mid and large-cap stock. For the cap-based portfolios, CRSP ranks all NYSE companies by market capitalization and then divides them into ten equally populated portfolios. Amex and NASDAQ stocks are then placed into the deciles determined by the NYSE breakpoints, based on their market capitalization. CRSP portfolios 1-2 represent large cap stocks, portfolios 3-5 represent mid-caps, portfolios 6-8 represent small caps, and portfolios 9-10 represent a benchmark of micro-cap stocks.[5]
Note: since 2010, CRSP also provides its own investable capitalization-based indexes, which are now tracked by index funds like Vanguard, and defined in a different way than the historical cap-based portfolios (see Index Methodology).
Major index providers
The market coverage of major index providers is supplied in Figure 2. below.
 |
|
| * Size of colored bars are not to scale.
Notes: Although each provider listed has an index intended to mirror the entire stock market, including the micro-cap segment, investors typically use the index that encompasses only large-, mid-, and small-cap securities to represent the broad U.S. market. Those indexes are shown in the light blue bars for each index provider. | |
Indices and funds
Index types
Usually, an index provider supplies three different index returns for its market segment:[7]
- A Price Index (Price, PR) which does not include dividends or dividend reinvestment.
- A Gross Dividend Index (Total Return, TR) which includes all dividends reinvested.
- A Net Dividend Index (Net Return, NR) which recognizes that the dividends are subject to tax and not received by the investor. Only the remaining portion of the dividend (net of tax) can be reinvested.
For benchmarking international indices, the Net Return index is most commonly used. The Morningstar Index Returns[8] identifies indices by type.[note 2]
US total market
The first index to measure the entirety of the US public market, the Wilshire 5000 Index, was established in 1974 by Wilshire Associates.[9] Russell introduced the Russell 3000 Index, measuring 98% of the US Market, in 1984.[10] S&P created its first broad based index of the US market in 1994 with the S&P Composite 1500 index, comprised of the S&P 500, S&P 400, and S&P 600 indices. The index measures approximately 85% of the market. The first retail index fund based on a total market index became available in 1992.
| Index Provider | Inception | Backdate data | Market Coverage |
| Wilshire | 1974 | 1971 | 100% |
| Russell | 1984 | 1984 | 98% |
| Dow Jones | 1991 | 1987 | 100% |
| Morningstar | 1998 | 1998 | 97% |
| MSCI | 2004 | 2004 | 99.5% |
| S&P | 2006 | 2006 (?) | 100% |
| CRSP | 2001 | 2002 | 100% (?) |
The advent of the twenty first century saw the expansion of index providers in the US market, as well as a steady broadening of provider indexes to encompass the total market. Russell added 1000 micro cap stocks to its Russell 3000 index to create its broadest index, the Russell 3000e (extended), measuring 99% of the US market. In 2006, S&P created its S&P Completion Index, which in combination with the S&P 500 index produced the S&P Total Market Index. MSCI, Dow Jones, and Morningstar also created broad based total market indexes in the first decade of the twenty first century. (See Figure 1. for a chart of index provider market coverage.)
At present, only Russell provides total market style indexes (Russell 3000 Value and Russell 3000 Growth).
The similar breadth of index provider market coverage in total market indexes leads to very low turnover of stocks[11] and in a low range of investment return dispersion, especially over long holding periods (see US total market index returns for data).[note 3] The differential in return is most often attributed to the breadth of micro cap stocks in a total market index. For example, Morningstar (97%), Russell (98%), and MSCI (99.5%) total market indexes hold lower ranges of micro cap market coverage than does the Wilshire 5000 index (99.9%).
|
US Total |
US Total |
|
US large and mid cap
Size indices
| Index Provider | Inception | Backdate data | Market Coverage | Holdings |
| S&P | 1954 | 1926 | 75% | 1-500 |
| Russell | 1984 | 1984 | 92% | 1-1000 |
| MSCI | 1996 | 1992 | 86% | 1-750 |
| Morningstar | 1998 | 1998 | 70% | (?) |
| Dow Jones | 2005 | 1991 | 86% | 1-750 |
| CRSP | 2001 | 2002 | 85% | (?) |
| Index Provider | Inception | Backdate data | Market Coverage | Holdings |
| Russell | 1984 | 1984 | 27% | 200-1000 |
| S&P | 1991 | 1991 | 7% | 501-900 |
| MSCI | 1996 | 1992 | 15% | 301-750 |
| Morningstar | 1998 | 1998 | 20% | (?) |
| Dow Jones | 2005 | 1991 | (?) | 501-1000 |
| CRSP | 2001 | 2002 | 15% | (?) |
US small cap
Russell introduced the first small cap index (the Russell 2000) in 1984. Other index providers created small cap indexes in the last decade of the twentieth century: S&P in 1992; MSCI in 1996; Morningstar 1998. The first retail small cap index fund, based on the Russell 2000 index, was initiated in 1989. More institutional investors' small cap allocation assets are benchmarked to the Russell 2000 index than to any other provider.[12]
Size indices
| Index Provider | Inception | Backdate data | Market Coverage | Holdings |
| Russell | 1984 | 1984 | 8% | 1,001-3,000 |
| S&P | 1994 | 1994 | 3% | 901-1,500 |
| MSCI | 1996 | 1992 | 12% | 751-2,500 |
| Morningstar | 1998 | 1998 | 7% | (?) |
| Dow Jones | 2005 | 1991 | 12% | 751-2,500 |
| CRSP | 2001 | 2002 | 13% | (?) |
Style indices
|
US Size and Style |
US Style |
US sector
|
US Sector |
|
US Sector |
|
International and global
| Index Provider | Large Cap | Mid Cap | Small Cap |
| Dow Jones | Top 85% | Top 80% - 90% (overlaps large and small) | 85% - 100% |
| FTSE | > 72% | between 72% and 92% | < 92% |
| MSCI | Top 70% +/- 5% | between 70% and 85% +/- 5% | between 85% and 99% |
| S&P | Top 70% | Top 70% - 85% | 85% - 100% |
|
International |
International |
Index methodology
Index providers differ in the ways they select and weight securities in the benchmarks’ return calculation. There are three key issues:[15]
- Security selection and weighting
- Calculation of returns
- Ongoing index maintenance
In their attempts to make their indexes more easily investable for index funds, index providers have adopted free-float[16] weighting and buffer zones to their methodologies. Free float weighting eliminates non-trading shares in a company's capital base. These can include cross-ownership of shares by other companies, government owned shares, privately held shares, and other restricted shares. A company's free-float weighting will reflect the actual amount of shares available for public investment.
All equity-index providers periodically adjust membership to reflect market changes as well as those made necessary by corporate actions such as mergers or spin-offs.[17] Buffer zones were introduced to reduce the amount of turnover incurred as stocks migrate across market capitalization and style boundaries. Reduced turnover helps reduce transaction costs for funds tracking an index.
Each major index provider has a different methodology for determining buffer zones. For example:
- CRSP: CRSP U.S. Equity Indexes Methodology Guide
- Dow Jones: Dow Jones Size Segment and Style Indexes Methodology
- FTSE: FTSE UK Index Series Ground rules
- MSCI: MSCI US Equity Indices Methodology (Nov 11)
- Russell: Russell US Indexes Construction and Methodology
- Wilshire: Wilshire U.S. Style Indexes: A Methodology Overview
A summary of methodologies is shown in the table below.
| Standard & Poor's | Dow Jones | Russell | MSCI | CRSP | FTSE | |
| Weighting methodology | Float-adjusted market capitalization. | Full market capitalization (float-adjusted market capitalization also available). | Full market capitalization, adjusted for free float only after inclusion in index. | Full market capitalization, adjusted for free float before inclusion in index. | Float-adjusted market capitalization. | Float-adjusted market capitalization. |
| Construction methodology | Based on criteria established by S&P committee. | Rules-based. | Rules-based. | Rules-based. | Rules-based. | Rules-based. |
| Rebalancing methodology | Changes made on an as-needed basis. No annual or semiannual reconstitution. | Quarterly, to coincide with review of Dow Jones U.S. Total Stock Market Index. | Annual rebalance in June, subject to capitalization banding (+/- 2.5%) as of 2007. | Quarterly review (two full reviews, two partial reviews), with security migration subject to "buffer zones." | Quarterly rebalancing, with security migration subject to “packeting.”[note 4] | Semi-annual review in June and December, subject to capitalization buffers around large, mid-, and
small-cap indexes. |
| Number of securities | S&P Total Market Index includes all listed securities. | DJ U.S. Total Stock Market Index includes all U.S. equity securities with readily available prices. | Russell 3000E Index represents approximately 99% of U.S. equity market. | MSCI US Broad Market Index targets 99.5% of cumulative full market capitalization of U.S. equity universe. | CRSP US Total Market Index targets 99.5% of cumulative full-market capitalization of U.S. equity
universe. |
FTSE US Total Market Index captures more than 99.5% of U.S. stock market. |
| Subcomponents of Total Market Index include fixed number of stocks (e.g., S&P 500 contains 500 stocks). | Subcomponents of U.S. Total Stock Market Index include fixed number of stocks (e.g., DJ Large-Cap Index contains 750 stocks). | Subcomponents of 3000E Index include fixed number of stocks (e.g., Russell 3000 Index includes 3,000 stocks). | Subcomponents of US Broad Market Index include fixed number of stocks (e.g., MSCI US Prime Market 750 Index contains 750 stocks). | Subcomponents of US Total Market Index based on market-capitalization targets: Largest 70% constitute mega cap index; next 15%, mid-cap index; final 13.5%, small-cap index. | Subcomponents of US Total Market Index based on market capitalization targets: largest 70% constitute large-cap index; next 20%, mid-cap index; final 9.5%, small-cap index. | |
| Security makeup | Securities that reflect the U.S. equity markets and, through the markets, the U.S. economy. | Equity issues, common stocks, and REITs. | Publicly traded companies in the United States with market cap greater than $30 million. | U.S. common stocks, REITs, and select non-U.S.- domiciled stocks trading in the United States. | U.S. equities traded on NYSE, AMEX, NASDAQ, or NYSE Arca. | Publically traded companies in United States with an investable market cap exceeding $20 million including REITs, Master Limited Partnerships, Royalty Trusts, American Depository Receipts, and securities listed on OTC Bulletin Boards. |
| Liquidity requirements | Ratio of annual dollar value traded to float-adjusted market cap should be 1.00 or greater. Company should trade a minimum of 250,000 shares in each of the six months leading up to the evaluation date. | Stock must meet minimum threshold for trading volume. | Securities must trade above $1 per share on May 31. | Measured along two dimensions: the level of stock price and a relative criterion known as Annualized Traded Value Ratio (ATVR). | Total market capitalization must be greater than $10 million, float shares must be greater than
10% of total shares, there are trading volume requirements, and a security must not have ten sequential days without trading volume. |
Securities must maintain at least $10 million in market capitalization as of review dates. |
| Growth criteria | • Trailing three-year earnings per share (EPS) growth. • Three-year sales growth. • Momentum—12-month percentage price change. |
• Forward P/E ratio. • Projected earnings growth. • Price/book ratio. • Dividend yield. • Trailing revenue growth. • Trailing earnings growth. |
• Price/book ratio. • Long-term earnings growth mean forecast by Institutional Brokers Estimate System (IBES). |
• Long- and short-term projected EPS growth rates. • Current internal growth rate. • Long-term historical EPS growth trend. • Long-term historical sales/share growth trend. |
• Consensus long-term growth forecast from I/B/E/S. • Consensus one-year forecast from I/B/E/S. • Three-year growth in earnings per share. • Three-year growth in sales per share. • Current investment-to-assets ratio. • Return on assets. |
• Trailing three-year sales growth. • Trailing three-year EPS growth. • Two-year forward sales growth. • Two-year forward EPS growth. • Return on equity times retention ratio. |
| Value criteria | • Price/book ratio. • Price/earnings ratio. • Price/sales ratio. |
• Forward P/E ratio. • Projected earnings growth. • Price/book ratio. • Dividend yield. • Trailing revenue growth. • Trailing earnings growth. |
• Price/book ratio. • Long-term earnings growth mean forecast by Institutional Brokers Estimate System (IBES). |
• Price/book ratio. • 12-month forward EPS. • Dividend yield. |
• Book/price ratio. • Earnings per share consensus forecast one year ahead divided by stock price. • Most recent EPS divided by stock price. • Dividend/price ratio. • Sales/price ratio. |
• Price/book ratio. • Sales/price ratio. • Dividend yield. • Cash flow/price ratio. |
| Growth and value crossover | Style indexes- limited overlap between growth and value indexes. Pure style indexes- stocks placed in either growth or value index. | 50% of stocks by float-adjusted market cap are in growth index and 50% are in value index. | Securities may be classified proportionately in both growth and value indexes. | Securities may be classified proportionately in both growth and value indexes. | Securities may be classified proportionately in both growth and value indexes. | Securities may be classified proportionately in both growth and value indexes. |
Alternative methodologies
Although most indexes use market capitalization as a basis for fund selection, there are other types of indexes.
Notes
- ↑ As of year-end 2008, 368 index funds managed total assets of $604 billion. By the end of 2008, the total number of index-based and actively managed ETFs had grown to 728, and total net assets were $531 billion Of these, 12 were actively managed ETFs with less than $250 million in total net assets-- 2009 ICI Factbook p.33; p.40.
- ↑ Note the MSCI international index designator of "NR", such as MSCI EAFE NR USD. MSCI also supplies indices in local currency (LCL) and US Dollars (USD). MSCI Equity Indices Overview
- ↑ The data spreadsheet on this page includes data for the Russell 3000 Growth and Russell 3000 Value Indexes
- ↑ CRSP’s methodology introduces the concept of “packeting,” which cushions movement between adjacent indexes and allows holdings to be shared between two indexes of the same family. At reconstitution, if a company has moved significantly through a breakpoint into the core of an adjacent index, a “packet” of 50% of the total holdings of the company is moved between indexes. The objective is to maximize style purity and minimize turnover.
See also
- Alternative indices
- Bond market indexing
- Exchange-traded funds
- Vanguard ETF info
- Vanguard statistical data spreadsheets: Vanguard equity index funds
References
- ↑ Ferri, Richard A., All About Index Funds: The Easy Way to Get Started, 2nd ed., Mcgraw-Hill, 2008, p.88. ISBN 0-07-138705-6
- ↑ Variance Among Indexes, Craig Israelsen, JOI, May /June 2007.
- ↑ Indexing in the 21st Century: Portfolio Solutions Inc.
- ↑ Center for Research in Security Prices (CRSP)
- ↑ Historical CRSP Cap-Based Portfolios
- ↑ Determining the appropriate benchmark: A review of major market indexes, Vanguard research, p.4. (Reference is for figure only.)
- ↑ 401k Trgt Ret Fund – Pass. to Active Managed, Fee Increase
- ↑ Morningstar Index Returns
- ↑ Wikipedia Wilshire 5000
- ↑ Wikipedia Russell Indexes
- ↑ Portfolio Turnover
- ↑ U.S. Equity Indexes Market Share Survey
- ↑ S&P: A Tale of Two Benchmarks, 2009
- ↑ Evaluating Global Benchmarks Vanguard Research (August 15, 2010)
- ↑ Determining the appropriate benchmark: A review of major market indexes, Vanguard research, page 3.
- ↑ Float, from Investopedia
- ↑ Determining the appropriate benchmark: A review of major market indexes, Vanguard research, page 6.
- ↑ Determining the appropriate benchmark: A review of major market indexes, Vanguard research, p. 14.
External links
- Vanguard - Benchmarks overview - Background, methodology, and latest performance information of the major index providers
- Determining the appropriate benchmark: A review of major market indexes, Christopher B. Philips, Francis M. Kinniry Jr.,Vanguard Research (October 2, 2012). Background of how the major indexes are constructed with additional insight for investors.
- The Index Is Dead. Long Live The Index., Lubos Pastor, John Heaton and Aaron Foss, IndexUniverse.com, Journal of Indexes (July/August 2013). Tutorial background on how the CRSP indexes are constructed.
- The Case for Indexing, Christopher B. Philips and Frank J. Ambrosio, Vanguard Investment Counseling & Research (February, 2011). The theory behind indexing with rationale to support the advantages of indexing. Common myths of indexing strategy are also discussed.
Index research
- Yahoo finance indices - Every major US and world index with real-time charts, list of components, and historical prices
- MSCI Index Definitions - Includes market holdidays (worldwide), MSCI Base Dates, MSCI Earliest Index History Dates
- MSCI Resource Center - MSCI fact sheets, methodologies, publications
- Russell Index Performance Calculator | FTSE Russell, this tool allows you to view total returns for any of Russell's US equity indexes over customizable date ranges. Data starting from January 1, 1995.