MacroHistory database data sources - Q&A and Feedback

For investors outside the US. Personal investments, personal finance, investing news and theory.
Sister forums: Canada, Spain (en español)
---------------
Post Reply
User avatar
Topic Author
siamond
Posts: 4754
Joined: Mon May 28, 2012 5:50 am

MacroHistory database data sources - Q&A and Feedback

Post by siamond » Fri Jun 14, 2019 4:50 pm

A groundbreaking study ('The Rate of Return on Everything') was recently updated, documenting historical annual returns and rates for stocks, housing, bonds and bills worldwide for more than a century, for 16 countries. The corresponding findings are discussed on a thread initiated by SimpleGift.

The corresponding database is made public here. The corresponding documentation can be found on the same Web page: download the PDF document here.

The objective of this thread is to foster a constructive discussion about the data sources and resulting data series. Hopefully, the researchers who are behind this tremendous effort will contribute, answering clarifying questions and reacting to feedback & suggestions provided by posters.

User avatar
Topic Author
siamond
Posts: 4754
Joined: Mon May 28, 2012 5:50 am

Re: MacroHistory database data sources - Q&A and Feedback

Post by siamond » Fri Jun 14, 2019 5:01 pm

Let me start by asking a clarifying question. In the dataset, the 'cpi' variable is described as a consumer price index with no further qualification. In the documentation, I couldn't find any more information about it, in general or in the specific case of the US.

In the main article (section II.C. Calculating returns), the formula being used to derive real returns from nominal returns appears to imply that the CPI index for a given year is the index value at the end of the year. Applying the same formula to the nominal returns (e.g. eq_rt) from the dataset did result in real returns that match the main article (e.g. I checked the arithmetic average of such real returns for a few countries, this matches the article).

On the other hand, I cannot make sense of the actual CPI values (more precisely the annual rates derived from such values) when comparing to the US Bureau of Labor Statistics (BLS) numbers we have in the Simba spreadsheet (which come straight from BLS, computing Dec to Dec).

Could the MacroHistory authors clarify the exact semantics of the 'cpi' variable and also explain where the corresponding numbers are coming from for the US?

longinvest
Posts: 3489
Joined: Sat Aug 11, 2012 8:44 am

Re: MacroHistory database data sources - Q&A and Feedback

Post by longinvest » Mon Jun 17, 2019 3:47 pm

OK. I checked the Canadian data. It seems, at first sight, to be using Consumer Price Index (CPI), annual average, not seasonally adjusted, with 1990=100. On Statistics Canada's website (StatCan), we can get the historical annual average CPI going back to 1914 with 2002=100 here. By translating StatCan's data to 1990=100 and rounding display to a single decimal, I get:

Code: Select all

MDH: MacroHisory Database
StatCan: Statistics Canada
                   StatCan CPI  
                   Translated   
 Year   MHD CPI    To 1990=100    Difference  
 1914     7.7          7.7           0.0      
 1915     7.8          7.8           0.1      
 1916     8.5          8.5           -0.1     
 1917     10.1        10.1           0.0      
 1918     11.4        11.4           0.0      
 1919     12.5        12.5           0.0      
 1920     14.5        14.5           -0.1     
 1921     12.7        12.8           0.0      
 1922     11.7        11.7           -0.1     
 1923     11.7        11.7           0.0      
 1924     11.5        11.5           0.0      
 1925     11.6        11.6           0.0      
 1926     11.7        11.7           0.0      
 1927     11.5        11.6           -0.1     
 1928     11.6        11.6           0.0      
 1929     11.7        11.7           0.0      
 1930     11.6        11.6           0.0      
 1931     10.5        10.5           0.0      
 1932     9.5          9.6           0.0      
 1933     9.1          9.1           0.0      
 1934     9.2          9.2           0.0      
 1935     9.3          9.3           0.0      
 1936     9.4          9.4           0.0      
 1937     9.7          9.8           -0.1     
 1938     9.9          9.8           0.0      
 1939     9.8          9.8           0.0      
 1940     10.2        10.2           0.0      
 1941     10.8        10.8           -0.1     
 1942     11.3        11.2           0.0      
 1943     11.5        11.5           0.0      
 1944     11.5        11.6           -0.1     
 1945     11.6        11.7           -0.1     
 1946     12.0        12.0           0.0      
 1947     13.1        13.1           0.0      
 1948     15.0        15.1           0.0      
 1949     15.5        15.6           -0.1     
 1950     15.9        15.9           0.0      
 1951     17.6        17.6           0.0      
 1952     18.0        18.1           -0.1     
 1953     17.9        17.9           0.0      
 1954     18.0        18.0           0.0      
 1955     18.0        18.0           0.0      
 1956     18.3        18.2           0.0      
 1957     18.9        18.9           0.0      
 1958     19.4        19.4           0.0      
 1959     19.6        19.5           0.1      
 1960     19.8        19.8           0.1      
 1961     20.0        20.0           0.0      
 1962     20.2        20.3           0.0      
 1963     20.6        20.5           0.1      
 1964     21.0        20.9           0.0      
 1965     21.5        21.4           0.1      
 1966     22.3        22.3           0.0      
 1967     23.1        23.1           0.0      
 1968     24.0        24.0           0.0      
 1969     25.1        25.1           0.0      
 1970     25.9        25.9           0.1      
 1971     26.7        26.7           0.0      
 1972     28.0        27.9           0.0      
 1973     30.1        30.1           0.0      
 1974     33.4        33.4           -0.1     
 1975     37.0        37.0           0.0      
 1976     39.7        39.7           0.1      
 1977     42.9        42.9           0.1      
 1978     46.7        46.7           0.1      
 1979     51.0        51.0           0.0      
 1980     56.2        56.1           0.1      
 1981     63.2        63.1           0.1      
 1982     70.0        70.0           0.0      
 1983     74.1        74.1           0.0      
 1984     77.3        77.3           0.0      
 1985     80.4        80.4           0.0      
 1986     83.7        83.7           0.1      
 1987     87.4        87.4           0.0      
 1988     90.9        90.8           0.1      
 1989     95.5        95.4           0.0      
 1990    100.0        100.0          0.0      
 1991    105.6        105.6          0.0      
 1992    107.2        107.1          0.1      
 1993    109.2        109.2          0.0      
 1994    109.4        109.3          0.1      
 1995    111.8        111.7          0.0      
 1996    113.5        113.4          0.1      
 1997    114.6        115.3          -0.7     
 1998    115.9        116.5          -0.5     
 1999    118.7        118.5          0.2      
 2000    122.4        121.7          0.7      
 2001    123.7        124.7          -1.0     
 2002    128.4        127.6          0.9      
 2003    130.6        131.1          -0.5     
 2004    133.6        133.5          0.1      
 2005    136.7        136.5          0.2      
 2006    138.6        139.2          -0.6     
 2007    142.0        142.2          -0.2     
 2008    145.6        145.5          0.1      
 2009    146.1        145.9          0.2      
 2010    148.7        148.6          0.1      
 2011    153.0        152.9          0.0      
 2012    155.3        155.2          0.1      
 2013    156.7        156.6          0.1      
 2014    159.7        159.7          0.1      
 2015    161.6        161.5          0.1      
 2016    163.8        163.8          0.1      
The only noticeable differences appear between 1997 and 2009. I'm pretty sure that authors used annual averages, as inflation calculated from single CPI readings 12-months apart is quite volatile. Taking the average of 12 readings, annually, gives a better assessment of inflation without as much volatility.

Maybe they did the same with US data?
Bogleheads investment philosophy | single-ETF balanced portfolio | VBAL

User avatar
Topic Author
siamond
Posts: 4754
Joined: Mon May 28, 2012 5:50 am

Re: MacroHistory database data sources - Q&A and Feedback

Post by siamond » Mon Jun 17, 2019 8:07 pm

longinvest wrote:
Mon Jun 17, 2019 3:47 pm
I'm pretty sure that authors used annual averages, as inflation calculated from single CPI readings 12-months apart is quite volatile. Taking the average of 12 readings, annually, gives a better assessment of inflation without as much volatility.

Maybe they did the same with US data?
Yes, it seems that you're correct, as one of the authors told me by e-mail. I still can't reproduce their math with the US BLS numbers though. I asked them to provide their full answer on this thread, let's wait until tomorrow.

It doesn't strike me as being a great idea to mix up time frames like that though. Annual (nominal) returns are clearly anchored on calendar boundaries, it seems clear to me that to compute real returns, we'd want to use the same calendar boundaries. True, inflation as perceived by consumers might be less volatile than Dec-to-Dec math might indicate (although I suspect there is a good deal of recency bias in such view, as clearly illustrated by the Ben Roth diary of the Great Depression with prices going all over the place in a matter of weeks). Still, I don't see that it really matters, while mixing time frames might lead to all sorts of inconsistent numbers when doing real (inflation-adjusted) math, notably when playing with time-sensitive concepts like correlation.

My feedback is that it would be very useful to provide BOTH metrics in the next update of the database. An average per annum value AND an end-of-year value. And real returns computation and analysis should definitely be performed with the latter metric.

Post Reply