Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
MachineGhost
Posts: 107
Joined: Sat Feb 14, 2009 2:46 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by MachineGhost » Thu Dec 08, 2016 10:08 pm

AlohaJoe wrote:How does having CMT yield prior to 1977 help with the problem of returns? Ibbotson uses the yields you are talking about to compute the returns.

What you call "sitting on turds" here is the same approach you suggested we follow above. It is the same approach Damodaran takes, too.
From what I read in the other thread, Ibbotson doesn't use actual yields but some model cooking them up before 1977. In other words, they haven't bothered to update their flawed data with either the actual 20-year yields back to 1942 or develop a better model than what was used in the 1960's (?). Just as S&P and CRSP still uses crap data back to 1871 that Shiller and Siegel uses despite the fix being published ages ago (AND THEY KNOW ABOUT IT BUT HAVEN'T DONE DIDDLEY SQUAT). So yeah, they're all sitting on turds but I think this thread is about recognition that their data is turdish with the clear aims to fix it.

As for pre-1977, if you want to do this emulation of a bond multiple inventory fund, I can't help at this time. It's a new problem to me (and believe me, just dealing with the "one bond" conversion problem had my head spinning.)

User avatar
LadyGeek
Site Admin
Posts: 50181
Joined: Sat Dec 20, 2008 5:34 pm
Location: Philadelphia
Contact:

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by LadyGeek » Thu Dec 08, 2016 10:16 pm

MachineGhost wrote:...so eventually I just used a function in R. That models it as a "one bond" portfolio which is good enough for me since what is important is the duration for determining risk or exposure.
Were you aware of the R bond market simulator? It's in the wiki: Using open source software for portfolio analysis (Bond market simulation)

I don't know if these functions will help here, but I wanted to mention it.
Wiki To some, the glass is half full. To others, the glass is half empty. To an engineer, it's twice the size it needs to be.

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Fri Dec 09, 2016 12:36 pm

From another thread, a key input to the LT modeling topic:
MachineGhost wrote:
siamond wrote:
MachineGhost wrote:FRED has 20-year CMT back to 1942, but it is in two separate data sets.
Could you please clarify where exactly is this second data series going back to 1942? All I can find is the GS20 series, which starts in April 1953.
It's under M13058USM156NNBR.
Aaaah... Here is a direct link. This is located in the 'Academic Data' section of the FRED Web site, and comes from the National Bureau of Economic Research (NBER) and their Macro history (chapter XIII. Interest Rates).

Great find! This should allow us to extend the LT series to 1942, hence the near beginning of the WW-II bond crisis. Very cool.

PS. Interestingly enough, NBER is located just a few miles from my home. I guess a visit might be in order! :wink:

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Fri Dec 09, 2016 2:24 pm

Oh... and I just found another one, starting in 1919, till 1944, bonds at large this time:

Yield on Long-Term United States Bonds for United States (M1333AUSM156NNBR)
https://fred.stlouisfed.org/series/M1333AUSM156NNBR

Would be good to check the years of overlap between those two 'academic' series, and with the GS20 FRED series (the one starting in 1954). Now why would FRED *not* include some of those older numbers in their GS20 data series? This is a bit puzzling.

MachineGhost
Posts: 107
Joined: Sat Feb 14, 2009 2:46 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by MachineGhost » Fri Dec 09, 2016 3:44 pm

siamond wrote:Oh... and I just found another one, starting in 1919, till 1944, bonds at large this time:

Yield on Long-Term United States Bonds for United States (M1333AUSM156NNBR)
https://fred.stlouisfed.org/series/M1333AUSM156NNBR

Would be good to check the years of overlap between those two 'academic' series, and with the GS20 FRED series (the one starting in 1954). Now why would FRED *not* include some of those older numbers in their GS20 data series? This is a bit puzzling.
Unless you have definite proof otherwise, you have to assume anything that is called a "bond' is 10 years during the first half of the 20th century. It's relatively recent that a 10-year got changed to being called a note.

I can't find anything that says when 20-year Treasuries were first issued.

Maybe a visit to the NBER is a very good idea. I doubt they get many inquiries like that. :wink:

AlohaJoe
Posts: 4038
Joined: Mon Nov 26, 2007 2:00 pm
Location: Saigon, Vietnam

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by AlohaJoe » Fri Dec 09, 2016 8:11 pm

MachineGhost wrote:
siamond wrote:Oh... and I just found another one, starting in 1919, till 1944, bonds at large this time:

Yield on Long-Term United States Bonds for United States (M1333AUSM156NNBR)
https://fred.stlouisfed.org/series/M1333AUSM156NNBR

Would be good to check the years of overlap between those two 'academic' series, and with the GS20 FRED series (the one starting in 1954). Now why would FRED *not* include some of those older numbers in their GS20 data series? This is a bit puzzling.
Unless you have definite proof otherwise, you have to assume anything that is called a "bond' is 10 years during the first half of the 20th century. It's relatively recent that a 10-year got changed to being called a note.

I can't find anything that says when 20-year Treasuries were first issued.

Maybe a visit to the NBER is a very good idea. I doubt they get many inquiries like that. :wink:
It is a good warning -- I personally hate the bill/bond/note terminology; just call it a 5-year bond or a 7-year bond or a 15-year bond!. But my guess would be that the "yield on long term bonds" is for 20-year bonds. Why do I say that? Because of the footnote about them being callable after 12-years.

Of course, that raises another issue: what to do with the impact of callable bonds? One reason Ibbotson modeled returns the way he did was he excluded all callable bonds from his data, which left him with fewer data points to work with.

AlohaJoe
Posts: 4038
Joined: Mon Nov 26, 2007 2:00 pm
Location: Saigon, Vietnam

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by AlohaJoe » Fri Dec 09, 2016 8:21 pm

siamond wrote:Oh... and I just found another one, starting in 1919, till 1944, bonds at large this time:

Yield on Long-Term United States Bonds for United States (M1333AUSM156NNBR)
https://fred.stlouisfed.org/series/M1333AUSM156NNBR

Would be good to check the years of overlap between those two 'academic' series, and with the GS20 FRED series (the one starting in 1954). Now why would FRED *not* include some of those older numbers in their GS20 data series? This is a bit puzzling.
Hmmm....I'm not sure what to make of these series, actually. As an example:

For January 1, 1954:
2.80 (FRED)
2.74 (NBER "Yields on Twenty Year United States Government Bonds")

For January 1, 1943:
2.32 (NBER "Yields on Twenty Year United States Government Bonds")
2.06 (NBER "Yield on Long-Term United States Bonds")

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Fri Dec 09, 2016 11:52 pm

siamond wrote:Would be good to check the years of overlap between those two 'academic' series, and with the GS20 FRED series (the one starting in 1954). Now why would FRED *not* include some of those older numbers in their GS20 data series? This is a bit puzzling.
I just figured out that I could use the FRED Web site itself to perform such test! Cool! First comparison, the two 20-years series. Sounds good enough to me!

Image

Second comparison, adding the long-term bonds series. Meh.

Image

AlohaJoe
Posts: 4038
Joined: Mon Nov 26, 2007 2:00 pm
Location: Saigon, Vietnam

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by AlohaJoe » Sat Dec 10, 2016 11:34 pm

I tried putting the two new NBER data sources into longinvest's bond model (for a 30-11 fund), which already includes FRED data from 1954 onward. The NBER series have data from 1919 and 1942.

The full spreadsheet can be seen here.

Image

The "flat after 10" is a model where we pretend we never have any long term yield data and use the 10 year yields instead, assuming a flat yield curve.

All charts only show after 1919, since the data is exactly the same before that; the new NBER data only begins in 1919.

(Interactive version)

Here's a telltale of the original vs. including just the 1919 NBER data

Image
(Interactive version

Here's one of the original vs. including just the 1954 NBER data.

Image
(Interactive version)

Here's one of the original vs. both NBER spliced together

Image
(Interactive version)

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Sun Dec 11, 2016 4:39 pm

I took the NBER numbers and created spliced series akin to what AlohaJoe did with his own simulator, but directly modifying longinvest 1.12 latest Excel spreadsheet (30-11 fund). I hope I didn't mess it up (I will let the author did the final change!). Then I compared against SBBI LTT, while applying a uniform 0.20 ER to all returns. I didn't use any Vanguard number so far, just the outcome of the model against SBBI (20016 book).

I had to narrow down the axis quite a lot to see the differences (still hard to see as the series often overlap). The disturbing broad pattern of steady decrease against SBBI remains, except in the early days with the 1919 NBER series. Overall, using both NBER series seems (?!) a better choice, from this point of view. I am fully convinced that we should use the 1942 NBER series, given the FRED comparison graph I posted earlier. I would tend to think that there is not much harm in also using the 1919 NBER series while we're at it, although I'd be glad to figure out exactly what this series is about (will try to go visit NBER and ask nosy questions!). In all fairness, we're splitting hairs here, but I'd rather do so while using available 20-yrs yield data than ignoring it. Longinvest, thoughts?

Image

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Sun Dec 11, 2016 5:27 pm

siamond wrote:I took the NBER numbers and created spliced series akin to what AlohaJoe did with his own simulator, but directly modifying longinvest 1.12 latest Excel spreadsheet (30-11 fund). I hope I didn't mess it up (I will let the author did the final change!). Then I compared against SBBI LTT, while applying a uniform 0.20 ER to all returns. I didn't use any Vanguard number so far, just the outcome of the model against SBBI (20016 book).

I had to narrow down the axis quite a lot to see the differences (still hard to see as the series often overlap). The disturbing broad pattern of steady decrease against SBBI remains, except in the early days with the 1919 NBER series. Overall, using both NBER series seems (?!) a better choice, from this point of view. I am fully convinced that we should use the 1942 NBER series, given the FRED comparison graph I posted earlier. I would tend to think that there is not much harm in also using the 1919 NBER series while we're at it, although I'd be glad to figure out exactly what this series is about (will try to go visit NBER and ask nosy questions!). In all fairness, we're splitting hairs here, but I'd rather do so while using available 20-yrs yield data than ignoring it. Longinvest, thoughts?
Siamond,

I'd like to know about copyright policy of NBER before including the data into the spreadsheet. Do you have a link to it, so that I can check if we're allowed to include it?

The 1919 NBER removes some of the jaggedness relative to SBBI. It's an improvement. But doesn't have much impact on cumulative returns, so they still significantly lag behind SBBI.

I would be curious about the behavior of a Bond Fund (20 to 20-year) relative to SBBI, as it would be a closer model to what SBBI is modeling. Have you checked that?
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Sun Dec 11, 2016 5:51 pm

AlohaJoe wrote: But I did make partial progress by (re-)implementing the bond fund simulator that longinvest describes. It is in python, not a spreadsheet. You can see it here on github (as a jupyter notebook)
AlohaJoe,

How does one execute this notebook and try various funds?
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Sun Dec 11, 2016 6:35 pm

longinvest wrote:I'd like to know about copyright policy of NBER before including the data into the spreadsheet. Do you have a link to it, so that I can check if we're allowed to include it?
Yes, for sure. This is also why I wanted to go visit those fellows. Written policies are one thing though, practical reality is another. I also suspect there might be several levels of indirection here (NBER works with a lot of academics). My investigation is just exploratory so far, to get a sense of the usefulness of spending more energy on this.
longinvest wrote:I would be curious about the behavior of a Bond Fund (20 to 20-year) relative to SBBI, as it would be a closer model to what SBBI is modeling. Have you checked that?
Yes, this would be interesting for sure. But the author of the bond fund spreadsheet didn't provide such a model to play with... :twisted:

AlohaJoe, maybe you could give it a quick try? This should be very straightforward with your new software, right?

MachineGhost
Posts: 107
Joined: Sat Feb 14, 2009 2:46 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by MachineGhost » Sun Dec 11, 2016 6:49 pm

AlohaJoe wrote: But I did make partial progress by (re-)implementing the bond fund simulator that longinvest describes. It is in python, not a spreadsheet. You can see it here on github (as a jupyter notebook)
Looks good! But, could you please add inline code to download all the data used and splice them together?

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Sun Dec 11, 2016 7:03 pm

Actually, this discussion made me ponder about two things:

1. are there more useful data series on the 'academic' side of the FRED Web site? pre-1954 10-years or 30-years yields? Answer: unfortunately, none I can find.

2. since FRED has a nifty user interface to compare data series, let's check the primary 10/20/30 years data series (1953+). Then we can ponder if the holes in the 20-years series are fairly reliably filled by interpolation, and also if using 10-years rates for an LT fund in the very early days (by lack of anything else) might make some sense or not. Answers: I would think yes and quite possibly. This all moves remarkably in sync except for the past 15 years (and the mid 70s to a certain extent).

Image

Longinvest, whether we end up using NBER or not, I am starting to *finally* warm up to the use of your 30-11 for LTT before 1954... Don't quote me just yet though... :wink:

AlohaJoe
Posts: 4038
Joined: Mon Nov 26, 2007 2:00 pm
Location: Saigon, Vietnam

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by AlohaJoe » Sun Dec 11, 2016 8:21 pm

siamond wrote:AlohaJoe, maybe you could give it a quick try? This should be very straightforward with your new software, right?
The 20-20 results are here: https://gist.github.com/hoostus/34f0859 ... dc72829cdc

AlohaJoe
Posts: 4038
Joined: Mon Nov 26, 2007 2:00 pm
Location: Saigon, Vietnam

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by AlohaJoe » Sun Dec 11, 2016 8:25 pm

longinvest wrote:
AlohaJoe wrote: But I did make partial progress by (re-)implementing the bond fund simulator that longinvest describes. It is in python, not a spreadsheet. You can see it here on github (as a jupyter notebook)
AlohaJoe,

How does one execute this notebook and try various funds?
You need python plus jupyter notebook. The easiest way to get all of that installed is to use Anaconda, which will install everything you need: https://www.continuum.io/

Then you need a copy of the Bond Fund Simulator notebook and the bond_rates.csv. You can get them from github https://github.com/hoostus/prime-harvesting either by:

* Downloading the two files manually, individually
* Cloning my whole repository if you know git
* Downloading a zip file of my whole repository (the big green button that says "Clone or download")

If you get stuck somewhere along the way, send me a PM and I'll help out.

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Sun Dec 11, 2016 10:27 pm

Here's a comparison (telltale) of the spreadsheet's fund (30-11) to a new fund (20) (in the newly uploaded version 1.13 of the spreadsheet):

Image

Differences were pretty small during the 10 to 30-year flat-curve period (due to missing 20 and 30-year yields). We see a divergence between the two funds developing in 1954, when the FRED 20-year yields become available. FRED's 30-year yields start in 1978.

At least, we now know that a 20 fund does not deliver similar returns to a 30-11 fund when the yield curve is not flat.

I'll let Siamond check if Bond Fund (20) is any better match to SBBI's returns in the post-1954 period.
Last edited by longinvest on Thu Apr 12, 2018 6:59 am, edited 1 time in total.
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Mon Dec 12, 2016 3:16 pm

longinvest wrote:I'll let Siamond check if Bond Fund (20) is any better match to SBBI's returns in the post-1954 period.
Here you are. I also added the Simba VUSTX data series (same ER), which is currently defined as:
- Longinvest's bonds fund spreadsheet (30-11 model) 1954-1972
- Bloomberg Barclays US Treasury Long TR USD 1973-1986
- Vanguard Long Term Treasury Fund (VUSTX) 1987+

Image

And the same chart starting in 1973.

Image

Room for thoughts, isn't it? Note that VUSTX tracks the Barclays index, but a tad loosely (tracking error = 0.61).

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Mon Dec 12, 2016 3:48 pm

siamond wrote:
longinvest wrote:I'd like to know about copyright policy of NBER before including the data into the spreadsheet. Do you have a link to it, so that I can check if we're allowed to include it?
Yes, for sure. This is also why I wanted to go visit those fellows. Written policies are one thing though, practical reality is another. I also suspect there might be several levels of indirection here (NBER works with a lot of academics). My investigation is just exploratory so far, to get a sense of the usefulness of spending more energy on this.
I took a closer look, I couldn't find terms of use on the NBER Web site, so I sent an e-mail asking for permission to publicly reproduce (note that we're actually doing derivative work here) and received this simple answer:

On Mon, Dec 12, 2016 at 12:41 PM, Jean Roth <jroth@nber.org> wrote:
It should be fine to make it freely available.
Regards, Jean


Note that the corresponding FRED Web page comes with a suggested citation.

MachineGhost
Posts: 107
Joined: Sat Feb 14, 2009 2:46 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by MachineGhost » Mon Dec 12, 2016 3:54 pm

I don't see the NBER 19v2???

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Mon Dec 12, 2016 3:58 pm

MachineGhost wrote:I don't see the NBER 19v2???
Oh, that was a small test of mine (using the NBER 20-years rates in the 20-20 fund), but this was cluttering the chart without adding much information, so I deactivated it. Sorry for the confusion.

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Mon Dec 12, 2016 4:00 pm

siamond wrote: And the same chart starting in 1973.
Siamond,

As the 30-year data starts in 1978, do you have a telltale chart which starts in 1978 (or December 31, 1977)?

Thanks.
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Mon Dec 12, 2016 4:50 pm

longinvest wrote:As the 30-year data starts in 1978, do you have a telltale chart which starts in 1978 (or December 31, 1977)?
Here it is. I also extended the chart to 2015, and added the Barclays index (subtracting the same ER as the other series). This is... a tad puzzling. I double-checked my numbers, no apparent copy & paste mistake.

Image

MachineGhost
Posts: 107
Joined: Sat Feb 14, 2009 2:46 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by MachineGhost » Mon Dec 12, 2016 4:57 pm

Maybe you better add the 20-year CMT for perspective.

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Mon Dec 12, 2016 5:00 pm

siamond wrote:
longinvest wrote:As the 30-year data starts in 1978, do you have a telltale chart which starts in 1978 (or December 31, 1977)?
Here it is. I also extended the chart to 2015, and added the Barclays index (subtracting the same ER as the other series). This is... a tad puzzling. I double-checked my numbers, no apparent copy & paste mistake.

Image
Hmmm... VUSTX's inception is in 1986. In the chart, had it been started with fund (30-11)'s value (instead of Barclay's), fund (30-11) would be a relatively good match to it; much better than fund (20-20).

The more I look at those charts, the less I like the single 20-year bond model (SBBI and 20-20). Not that I ever liked such a simplistic model.
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

MachineGhost
Posts: 107
Joined: Sat Feb 14, 2009 2:46 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by MachineGhost » Mon Dec 12, 2016 5:27 pm

I did a quick check with the Python bond simulator outputting 30-30 and compared to 30yr CMT since 1978. The bond simulator time series offers about 2.9% more CAGR but has 33% less maximum drawdown. That may reflect the use of yearly data and interpolation.
Last edited by MachineGhost on Mon Dec 12, 2016 5:33 pm, edited 1 time in total.

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Mon Dec 12, 2016 5:31 pm

longinvest wrote:Hmmm... VUSTX's inception is in 1986. In the chart, had it been started with fund (30-11)'s value (instead of Barclay's), fund (30-11) would be a relatively good match to it; much better than fund (20-20).
Yes, VUSTX annual returns start in 1987. For sure, the 20-20 fund goes quite out of whack, which is rather strange. The 30-11 fund is indeed a better match, but still displays a tracking error of 2.34% against VUSTX, while its Barclays benchmark displays a tracking error of 0.61%.

I did change the working version of Simba recently in this respect, it used to splice the 30-11 fund straight with VUSTX, but I figured that there is solid history with the Barclays index, and gave it the priority over synthetic returns since this is the VUSTX benchmark. I did a similar change for STT. Unfortunately, there is less history for the ITT Barclays index.

Now that we know that the NBER 20-years data can be publicly reproduced, let me suggest to use it for the next update of your model (including for 30-11), and I'll use the outcome since 1871 for the VUSTX/LTT data series in Simba. While remembering that this all sounds rather approximate.

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Mon Dec 12, 2016 5:38 pm

PS. well, the 1942+ NBER data series seems solid, clearly 20-yrs government bonds, and matching well the 1954+ FRED series. I still wonder about the 1919+ NBER data series, as we're not quite sure what those bonds were, exactly. No need to rush for an update, in truth.

Here is what NBER provides as a description for the 1919+ series:

SOURCE: FEDERAL RESERVE BULLETIN, DECEMBER 1938, P. 1045
AND FOLLOWING MONTHLY ISSUES THROUGH MARCH 1944 (CHECKED
WITH U.S. DEPARTMENT OF COMMERCE, SURVEY OF CURRENT
BUSINESS, MARCH 1939, P. 18, AND FOLLOWING ISSUES).

NOTES: SERIES 13033 IS PRESENTED HERE AS TWO
VARIABLES--(1)--ORIGINAL DATA, 1919-1944 (2)--ORIGINAL
DATA, 1941-1967.
DATA WERE COMPUTED BY THE AVERAGE OF DAILY FIGURES.
DATA INCLUDE YIELDS IN THE PARTIALLY TAX EXEMPT
CATAGORY; DATA FOR 1919-1925 ARE CALLABLE AFTER EIGHT YEARS;
DATA FOR 1926-1944, AFTER TWELVE YEARS.
DATA FOR 1919-OCTOBER 15, 1925 ALSO INCLUDE CERTAIN
LIBERTY LOAN ISSUES.

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Mon Dec 12, 2016 6:01 pm

siamond wrote: I did change the working version of Simba recently in this respect, it used to splice the 30-11 fund straight with VUSTX, but I figured that there is solid history with the Barclays index, and gave it the priority over synthetic returns since this is the VUSTX benchmark. I did a similar change for STT. Unfortunately, there is less history for the ITT Barclays index.
In Simba, I would suggest to always go with an index fund, first, an index when available (and an index fund isn't), second, and would resort to synthetic returns when nothing better is available. I prefer diversified synthetic funds to single-bond models.
siamond wrote:Now that we know that the NBER 20-years data can be publicly reproduced, let me suggest to use it for the next update of your model (including for 30-11), and I'll use the outcome since 1871 for the VUSTX/LTT data series in Simba. While remembering that this all sounds rather approximate.
OK. I'll do that soon.
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

MachineGhost
Posts: 107
Joined: Sat Feb 14, 2009 2:46 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by MachineGhost » Mon Dec 12, 2016 6:08 pm

Interesting, according to Siegel, it seems clear that we don't even want to use government bonds alone before 1920:

Image
Image

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Mon Dec 12, 2016 6:21 pm

longinvest wrote:In Simba, I would suggest to always go with an index fund, first, an index when available (and an index fund isn't), second, and would resort to synthetic returns when nothing better is available. I prefer diversified synthetic funds to single-bond models.
Yes, 100% agreed. This is basically the policy I've used in the recent updates. Well, except that it comes to bonds, most real-life funds are not strictly index funds, they are somewhat active, but still stay reasonably close with their benchmark index.

AlohaJoe
Posts: 4038
Joined: Mon Nov 26, 2007 2:00 pm
Location: Saigon, Vietnam

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by AlohaJoe » Fri Dec 23, 2016 2:43 am

A long time ago... I asked about monthly returns. After digesting longinvest's (annual) bond simulator I extended it to do monthly simulations.

Assuming I haven't messed something up, the results suggest it isn't really worth the effort. I don't really recommend moving away from longinvest's spreadsheet.

When comparing against the Barclays index from 1972-2015:

- annual simulation has a correlation of 0.925
- monthly simulation has a correlation of 0.927

So a very, very slightly higher correlation with the actual index returns. Despite the apparently better correlation, the monthly model dramatically overstates the returns of the actual index over that period.

Image

Even the annual model overstates returns but the effect is exacerbated by using a monthly model. Over a 40-year period the different amounts to about a 24% difference.

Image

The calculations are definitely not suited for a spreadsheet: it takes 20-30 seconds on my laptop to perform the simulation. Most of that are all the present value calculations of the bonds held in the fund. Since the ladder now has 1 bond for each month a "10-4" fund will hold ~70 bonds. So 70 bonds a month, 12 months a year, ~140 years = around 120,000 present value calculations for each simulation run.

Data Series

In addition to the problems mentioned above, having spent time with the available data, there are additional problems with trying to generate historical monthly returns like this:

- longinvest's simulator relies on the 12 month interest rate. But Shiller only reports that annually (not monthly). Linear interpolation is used to fill in the gaps.
- FRED has short-term rates available from 1919 onward but they are a mishmash data set of "3 to 6 month bond rates" in the NBER M1329AUSM193NNBR and M1329BUSM193NNBR datasets.
- FRED has 3-month Treasury rates (on the secondary market) available from 1934 onward.

This scarcity of data becomes more apparent when you chart the returns

Image

We can see that things look very different before and after 1950. (That's when FRED starts making available interest rates for a number of different terms, which lets us do a better job of figuring out the yield curve.)

It is also interesting to note that, just as looking at "calendar year" equity returns hides some of the volatility, the same is true for bonds. The red dot shows the calendar year return whereas the blue line is the rolling 12-month return. There are several times when things temporarily look quite bad for bonds (with up to a 10% annual loss) but they recover before the end-of-year so it disappears from the record when you only look at yearly results.

While certainly less volatile than stocks, there are quite a few (on the order of 8-10) times when bonds drop ~3% in single month.

Looking at monthly returns also provides a lens on the anxiety people would feel in real life. Few of us only look at account balances once a year. We check more frequently and we hear news reports constantly. There was a stretch around 1954-55 when a bond fund would have dropped in 12 out of 14 months. Shortly after, in 1956-57, the same thing happened in 13 out of 20 of months. In real life that would have meant over a year of CNBC talking about the death of bonds :D

And, actually, an exercise like this understates the problem. We're looking at total returns but in the real world people usually look at just the price return when freaking out. When we say "the simulated fund dropped -0.67% in October 1956" in the real world it probably would have been reported as an even bigger drop, since interest makes up such a large portion of the total return for a bond fund. (Shiller reports the interest rate back then in the 2.5%-3% range, so a single month's interest is something like 0.20%; which would make the "price drop" around .9% for a single month.)

The full spreadsheet of results is available at:

https://docs.google.com/spreadsheets/d/ ... sp=sharing

The jupyter book is available at:

https://github.com/hoostus/prime-harves ... ator.ipynb

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Sat Dec 24, 2016 4:02 pm

AlohaJoe wrote:A long time ago... I asked about monthly returns. After digesting longinvest's (annual) bond simulator I extended it to do monthly simulations.
AlohaJoe, this is terrific work. We're not leveraging monthly returns very much in our typical analysis, because the Simba spreadsheet keeps it simple, and solely provides annual returns, but I am keenly aware of the fact that this hides a lot of the reality that we live on a day to day basis, as well as distorting measures like standard deviation and correlation (say against stocks).

I think your work will find multiple useful applications. Did you give it a try with anything else than the 10-4 IT bond? There are many Barclays indices for specific durations that you can compare to.

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Mon Jan 02, 2017 4:16 pm

I have uploaded version 1.14 of the spreadsheet.

It is available online and to download from the links in the first post: viewtopic.php?f=10&t=179425#p2717165

Main changes:
  • Added NBER 20-year yield data (1942-1961).
  • Moved approximate yield calculations into a separate sheet.
As usual, comments are welcome.

Enjoy!
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Mon Feb 06, 2017 6:42 pm

I have uploaded version 1.15 of the spreadsheet.

It is available online and to download from the links in the first post.

Main changes:
  • Added FRED January 2017 data.
As usual, comments are welcome.

Enjoy!
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

danw70
Posts: 2
Joined: Sun Apr 27, 2014 9:55 pm

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by danw70 » Sun Jan 21, 2018 4:51 pm

What a great thread! You guys have done a great job estimating historical annual bond returns. I am trying to estimate the "daily" historical returns of an intermediate term treasury fund that is as similar as possible to the following fund:

iShares 7-10 Year Treasury Bond ETF
0.15% Expense Ratio
Avg Eff Duration 7.62
Avg Eff Maturity 8.36 Yrs

FRED has daily rates for multiple CMTs back to 1962 (1yr, 5yr, 10yr), which is the time frame I would like to estimate daily returns back to. I would appreciate any advice on the best approach to accomplishing this.

Thanks,
Dan

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Sun Jan 21, 2018 7:13 pm

Dan,
danw70 wrote:
Sun Jan 21, 2018 4:51 pm
I am trying to estimate the "daily" historical returns of an intermediate term treasury fund [...]

FRED has daily rates for multiple CMTs back to 1962 (1yr, 5yr, 10yr), which is the time frame I would like to estimate daily returns back to. I would appreciate any advice on the best approach to accomplishing this.
It's important to understand that the yields reported by FRED are approximations on (at least) two levels:
  1. There might not be a Treasury bond which matures in exactly 1 year, 5 years, or 10 years on each day. Yields are calculated by averaging yields on bonds maturing near a target maturity.
  2. Different Treasury bonds with different coupons maturing on the same day might have different yields. Yields are averaged.
We also know that pricing varies during the day, and that there's a bid-ask spread. None of these things show up in FRED's averaged numbers. There might be other issues (over the counter trading, etc.).

In another thread, I wrote a detailed post about the imprecision of the yield curve. I invite you to read it before undertaking such a project: What determines the shape of the yield curve?

I'm afraid that a calculation based on daily CMT yields would simply give us a false feeling of precision.
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Mon Feb 05, 2018 9:47 pm

I have uploaded version 1.16 of the spreadsheet.

It is available online and to download from the links in the first post.

Main changes:
  • Added FRED January 2018 data.
As usual, comments are welcome.

Enjoy!
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Thu Dec 27, 2018 9:16 pm

I have uploaded version 1.18 of the spreadsheet.

It is available online and to download from the links in the first post.

Main changes:
  • Added December 31st yields from FRED daily data, when available.
  • Added calculation to estimate zero-coupon yields.
  • Added an equal-par weighted zero-coupon bond fund (30 to 21-year).
As usual, comments are welcome.

Enjoy!
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

User avatar
Kevin M
Posts: 10387
Joined: Mon Jun 29, 2009 3:24 pm
Contact:

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by Kevin M » Thu Dec 27, 2018 10:22 pm

longinvest wrote:
Sun Jan 21, 2018 7:13 pm
It's important to understand that the yields reported by FRED are approximations on (at least) two levels:
  1. There might not be a Treasury bond which matures in exactly 1 year, 5 years, or 10 years on each day. Yields are calculated by averaging yields on bonds maturing near a target maturity.
  2. Different Treasury bonds with different coupons maturing on the same day might have different yields. Yields are averaged.
Treasury uses mainly on the run (most recently issued) yields, so it is an approximation of a par-yield curve, with coupon rates reasonably close to yields. So the yields will be closer to more recently issued Treasuries. There also are liquidity issues, since on the run Treasuries typically are more liquid, so higher demand -> higher price -> lower yield.
We also know that pricing varies during the day, and that there's a bid-ask spread. None of these things show up in FRED's averaged numbers. There might be other issues (over the counter trading, etc.).
Treasury uses closing bid yields as inputs to their algorithm. Since bid yields are higher than ask yields (what you pay when you buy), these yields will tend to slightly overstate the yield you'll get when you buy.

Another issue is that yields can vary quite a bit based on quantity. The CMT yields will be closer to large quantity quotes at Fidelity and Vanguard (Schwab generally offers large-quantity pricing on small quantities).
In another thread, I wrote a detailed post about the imprecision of the yield curve. I invite you to read it before undertaking such a project: What determines the shape of the yield curve?

I'm afraid that a calculation based on daily CMT yields would simply give us a false feeling of precision.
The CMT yields really aren't bad to get a decent sense of the yield curve, but your points are good ones. One can construct a yield curve of sorts based on actual ask yield quotes, and you'll see large dispersion of yields at a given maturity (or for maturities that are close). Some of this is due to duration (which depends on the coupon rate as well as the term to maturity), but even adjusting for duration, you see significant dispersion of yields for a given duration (or durations that are close). Probably some of the dispersion is related to liquidity.

Kevin
Wiki ||.......|| Suggested format for Asking Portfolio Questions (edit original post)

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Wed Jan 02, 2019 5:44 pm

I have uploaded version 1.19 of the spreadsheet.

It is available online and to download from the links in the first post.

Main changes:
  • Added December 31, 2018 yields from FRED daily data.
As usual, comments are welcome.

Enjoy!
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by siamond » Mon Jan 07, 2019 10:10 am

Here is another possible modeling idea. I am pondering to add a couple of 'price-only' data series (e.g. S&P 500 and Total-Bonds, or maybe IT Treasuries) to the Simba backtesting spreadsheet, directly comparable to corresponding 'total-return' data series. So that we can break down total return in two components: capital return vs. income return.

There are enough people pondering about a distribution/income-only strategy that I think it would be worth making such numbers available for analysis. Not to endorse such (dubious) strategy, of course, but in line with the general Simba spirit of putting facts on the table, so that people can make up their own judgment about endorsing or disproving a given approach.

With the S&P 500, I can go way back in time, thanks to Prof. Shiller's data. For bonds, with the Barclays Aggregate Bond index, I can get annual returns from 1976+ for both Price and Total-Return data series. Unsurprisingly, the corresponding annualized CAGR over time for such Price series is very low (0.3%), but it does go up and down (-3% in 2018!).

Would there be a way to simulate such price series with some tweaks to the fund model (e.g. 10-2 or 10-4), so that we can go back further in time? Or provide a capital/income/total return breakdown?

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns

Post by longinvest » Mon Jan 07, 2019 10:50 am

Good idea. I could effectively break the total annual return of simulated funds into its capital and income components. I'll do that in the next release.
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest » Tue Jan 08, 2019 10:25 pm

I have uploaded version 1.20 of the Bond Fund Simulator spreadsheet.

It is available online and to download from the links in the first post.

Main changes:
  • Added capital and income returns.
As usual, comments are welcome.

Enjoy!
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by siamond » Thu Jan 10, 2019 11:59 pm

I ran a few tests, comparing the total/capital returns of two Vanguard bond funds and their index against the corresponding numbers predicted by longinvest's latest bond fund model. I use VUSTX (LT treasuries) and VBMFX (total-bonds). I also wanted to try IT treasuries (VFITX) and ST treasuries (VFISX), but unfortunately I couldn't find the corresponding price series for their index of reference, plus they have less years of existence, so I didn't proceed.

About VBMFX, the index of reference is Bloomberg Barclays US Aggregate Bond. Here is a growth chart illustrating the match between total returns between the fund, its index and the bond fund 10-2 model. It's a pretty good match ("/X" is the index; "/M" is the model).

Image

Now let's do the same, but comparing price series. A bit of a hiccup in 2008, but pretty good overall.

Image

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by siamond » Fri Jan 11, 2019 12:17 am

About VUSTX, the index of reference is Bloomberg Barclays US Treasury Long. Here is a growth chart illustrating the match between total returns between the fund, its index and the bond fund 30-11 model ("/X" is the index; "/M" is the model).

Image

Now let's do the same, but comparing price series. We can notice something strange. The model matches the real-life fund (VUSTX) pretty well for total returns, but matched the index much better for prices. The same wasn't true for VBMFX.

Image

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest » Fri Jan 11, 2019 9:56 am

It's important to remember that VUSTX isn't an index fund. It would be interesting to see a comparison of the model and price index with Vanguard Long-Term Treasury Index Fund Admiral Shares (VLGSX) over its short history (inception 03/01/2010).

Note that the historical total-return performance of VUSTX is practically identical to the performance of VLGSX since inception of the later. But it would be interesting to see if there's a difference in price returns.
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

User avatar
siamond
Posts: 4406
Joined: Mon May 28, 2012 5:50 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by siamond » Fri Jan 11, 2019 5:36 pm

I ran a few more tests and swapped a couple of messages with longinvest.

The bottomline is that Vanguard treasury bonds which are NOT index funds (e.g. VUSTX, VFITX, VFISX) seem to be more active than I would have expected, and their price series have a life of their own... It seems that they end up with similar total returns using rather different means than indices (or corresponding index funds). Trouble is we have less than a decade of returns for true bond index funds (e.g. VLGSX, VSIGX, VSBSX), so it isn't terribly significant to check the simulator's numbers against such limited reality. And corresponding indices (for IT/ST treasuries) do not have a PR (price-only) data series available in Morningstar.

I don't know what else to do to run a solid sanity check about the new income/capital breakdown from the simulator. I'd be inclined to just trust longinvest and call it a day...

Topic Author
longinvest
Posts: 3193
Joined: Sat Aug 11, 2012 8:44 am

Re: Historical Bond Returns - Shiller: From Rates to Returns [Bond Fund Simulator]

Post by longinvest » Fri Jan 11, 2019 5:46 pm

The model still provides a realistic approximation of bond fund investing; it's just not weighted to market caps and uses a coarse yearly capital reinvestment granularity.

Anyway, you'll be able to forward-test it, like we already have started doing by applying the model on new yields, after the fact, every year.
Bogleheads investment philosophy | Lifelong Portfolio: 25% each of (domestic / international) stocks / (nominal / inflation-indexed) bonds | VCN/VXC/VLB/ZRR

Post Reply