CPI - u calculation changed?

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Post Reply
User avatar
Topic Author
LH
Posts: 5490
Joined: Wed Mar 14, 2007 2:54 am

CPI - u calculation changed?

Post by LH »

Has the cpi - u calculation changed from what it was in the 1980s, 1990s?

I intermittantly read, that if 1980s, 1990s cpi calc was used, inflation would be around 6 percent today.

I am assuming they are refering to the methodology of cpi-u being changed?

Is this the case or not? (note I am aware of the different cpi's, chained cpi etc, but specifically wanted to know if 1980,1990s cpi-u = 2013 cpi-u calculation)

Thanks for help,

LH
Cyclone
Posts: 288
Joined: Sun Jan 10, 2010 9:48 pm

CPI - u calculation changed?

Post by Cyclone »

At least according to this website, the calculation has changed:

http://www.shadowstats.com/alternate_da ... ion-charts
MindBogler
Posts: 1031
Joined: Wed Apr 17, 2013 12:05 pm

Re: CPI - u calculation changed?

Post by MindBogler »

I like this site, I've been following it since it started as The Billion Prices Project at MIT.

http://www.pricestats.com/us-series
User avatar
Topic Author
LH
Posts: 5490
Joined: Wed Mar 14, 2007 2:54 am

Re: CPI - u calculation changed?

Post by LH »

Cyclone wrote:At least according to this website, the calculation has changed:

http://www.shadowstats.com/alternate_da ... ion-charts


Interesting, so if true, and if TIPS existed back in 80s, one bought TIPS then, then one would expect the 80s cpi-u to be used. This would result in TIPS paying out near 6-9 percent inflation adjustment now.... of course, the new inflation cpi-u is 1-2 percent range. A whopping difference.....

that cpi-u has been adjusted twice now, (per report) in the past 20-30 years, lends one to think it stands a good chance of being adjusted again.

Again, if true.

Its significant if one expects TIPS to pay out say 6 percent, and they only pay out 0-1 after adjustment.

How much protection expectationally will TIPS provide?

Or do we just assume no more downward revisions in cpi-u estimation methodology or inputs?
User avatar
nisiprius
Advisory Board
Posts: 42212
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: CPI - u calculation changed?

Post by nisiprius »

The CPI-U methodology has been revised more or less continuously all along.

One important change was the transition from housing prices to owner equivalent rent in 1983.
Another set took place following the report of the Boskin Commission in 1996.

I don't know how to deal with this, but shadowstats is bogus. They don't calculate what say imply they're calculating, and at this point it's pretty obvious that they made a gross screw-up in the calculation they did make. John Willliams promised a few years ago to have "an academic" review his methodology, but never did. The proof that shadowstats is not doing what it claims is this: before 1983 CPI was based actual housing prices. In 1983, this was changed to use owner equivalent rent. If shadowstats were really calculating CPI the way it was calculated in the past, you would see a deep notch in the blue SGS "alternate inflation" curve circa 2006 reflecting the housing collapse, and the gap between the two curves would narrow. The fact that they don't shows that the blue curve is NOT, as claimed, "the CPI as if it were calculated using the methodologies in place in 1980."

Image

The second point is that Williams obviously screwed up the calculations that he did make. He somehow confused the CPI index itself, and the annual percentage of inflation (which is the range of growth in the CPI). He is claiming not merely that the official figure for annual inflation is an underestimate, but that the amount of the underestimate has been cumulatively increasing every year.

The third point is that inflation is not 10%. The Billion Prices Project show that it is not 10%, it is about 2% and reasonably close to the official numbers. And anyone who has actually lived through 10% annual inflation can tell you that it is not. 10% inflation feels completely different from what we are seeing now.

Image
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
User avatar
Topic Author
LH
Posts: 5490
Joined: Wed Mar 14, 2007 2:54 am

Re: CPI - u calculation changed?

Post by LH »

Thanks, the shadow stat thing, nisi rebuttal, is exactly what I m looking for. Thanks for links.

In terms of the price thing, I thought it was done by MIT, and they quit doing it? Maybe conflating. It's up now.

So basically CPI u calc is constantly changing, and when they promise to pay TIPS CPI u inflation, it is very nonspecific thing, they change it all the time.
User avatar
Epsilon Delta
Posts: 8090
Joined: Thu Apr 28, 2011 7:00 pm

Re: CPI - u calculation changed?

Post by Epsilon Delta »

LH wrote: So basically CPI u calc is constantly changing, and when they promise to pay TIPS CPI u inflation, it is very nonspecific thing, they change it all the time.
The CPI is published by the Bureau of Labor Statistics, part of the Department of Labor. The only link they have with the US Treasury is the President. Sure they can conspire, but it would be a very big conspiracy.

Also there vigorous, international academic debate over CPI and other parts of the national accounts. As far as I can tell all of the changes made to the CPI over the years are within the main stream of academic opinion. Indeed it's the failure to make changes that is outside the consensus. You could go back and read the conspiracy theories when they changed form GNP to GDP as the measure of an economy.
Valuethinker
Posts: 41439
Joined: Fri May 11, 2007 11:07 am

Re: CPI - u calculation changed?

Post by Valuethinker »

Epsilon Delta wrote:
LH wrote: So basically CPI u calc is constantly changing, and when they promise to pay TIPS CPI u inflation, it is very nonspecific thing, they change it all the time.
The CPI is published by the Bureau of Labor Statistics, part of the Department of Labor. The only link they have with the US Treasury is the President. Sure they can conspire, but it would be a very big conspiracy.

Also there vigorous, international academic debate over CPI and other parts of the national accounts. As far as I can tell all of the changes made to the CPI over the years are within the main stream of academic opinion. Indeed it's the failure to make changes that is outside the consensus. You could go back and read the conspiracy theories when they changed form GNP to GDP as the measure of an economy.
And if you've ever worked in large multi-partite bureaucracies the greatest improbability is that 2 departments would ever collaborate on such a thing. Or as a friend of mine who worked for the US Department of Defence put it: 'you have to understand that from the Pentagon point of view there are countries hostile to American interests, like Iran, and there is the real enemy: the State Department' ;-).

I think if you want to see a conspiracy you will see a conspiracy-- I am just working on another case of that right now in a physical sciences field.

If you want to see that CPI is 'unfair', 'unbalanced' and 'arbitrary' then you will see that.

The psychological studies show no amount of additional information will shift that view. In fact, it will tend to confirm one in one's view-- the brain practices 'motivated reasoning' screening out data to the contrary.

Also the mindset that sees conspiracy. See any of Richard Curtis' documentaries. If you think things are conspiracies, you see conspiracies. Billion price project won't do it-- that must be rigged too-- MIT gets lots of money from the government, right? Tower 7, etc. (Dad was a structural engineer, so I have a particular grievance with that one-- before he died he could show me on a *slide rule* with some squared paper (technically 'graph paper' is log scale, apparently) what happened).

A very good friend of my spouse, a brilliant woman with a Phd from a recognized university (top 100 in the world). Multi millionaire businesswoman and property investor-- entirely self made. Hangs around with 'alternative' people, and believes that our secret rulers have something to do with owls, and notes how many new buildings in London look like owls.

My conspiracy is your craziness ;-).
Spirit Rider
Posts: 13627
Joined: Fri Mar 02, 2007 2:39 pm

Re: CPI - u calculation changed?

Post by Spirit Rider »

The biggest changes they have made over time is the change in the "basket" (model) of goods and services. This is how it should be. Over time the percentages of your expenses for individual items changes. So how that particular item's price changes affect the overall inflation rate changes over time. One key thing is the replacement of items with other items.

For example, twenty years ago a small percentage of people had cell phones. Now the opposite is true. It is the rare person who does not have a cell phone. However, it is becoming more common that people do not have landlines. So the change in the cost of cell phone services has a bigger impact on the average person's inflation rate than the change in land line services.

There is nothing nefarious going on here. They are doing their best to reflect the inflation rate for the economy as a whole. This is not that same as your individual inflation rate, because you have a different "basket" of goods and services. Is it perfect, of course not, it is being done by human beings. However, I truly believe that any RMS error is due to statistical or model errors and not bias.
User avatar
nisiprius
Advisory Board
Posts: 42212
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: CPI - u calculation changed?

Post by nisiprius »

LH, see also Accuracy of the CPI.

One particularly germane datum, copied and pasted from that article, is that unlike shadowstats, which implies that it is calculating CPI according to the 1980 method but isn't, the Bureau of Labor Statistics actually does calculate "a series known as CPI-U-RS, which tracks the CPI from 1978 to the present when computed by current methods. CPI-U-RS is lower than CPI-U, but only slightly. Over the period 1978-1998, the annualized difference was 0.45%. The stated theory behind the changes is, of course, that the older method was overstating inflation and that the changes were a correction. If, however, you believe the older method was the correct one, then the current method is understating inflation--but only by about 0.45%."

This means that if you actually did what shadowstats claims to be doing, the blue curve--ANNUAL inflation--would be above the red curve about about half a percent, fluctuating, but the gap between them would not systematically increase. On the other hand, if you were to make similar charts, not for the annual inflation rate, but for the (cumulative) CPI-U itself, then they would in fact show a systematic, cumulative divergence--but not a huge one.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
ourbrooks
Posts: 1575
Joined: Fri Nov 13, 2009 4:56 pm

Re: CPI - u calculation changed?

Post by ourbrooks »

It is also the case that inflation impacts are very different, depending on who you are. Let's suppose that average college tuition dropped substantially, dragging down the CPI-whatever with it. How many retirees would believe that the rate of inflation had actually dropped?

Clearly, over the past five or so years, the cost of purchasing a house has dropped and rents have not climbed proportionately. Housing is a large component of the CPI-U so the effect has been to lower the rate of inflation. Homeowners who haven't bought new houses won't see this change and even those who have might not receive an overall benefit because the selling price of their previous house has also dropped. Seeing the increases in, say, grocery bills, it's pretty easy to conclude that somehow the CPI-U isn't correctly measuring inflation.

Somehow, people seem to think that there's some absolute measure of inflation that's "real" inflation but there's no such thing. All of the CPI measures are based on weighted average baskets of goods and services, and any individual will only experience "average" inflation in the unlikely circumstance that they purchase goods and services in the same proportion as the weighted average.
User avatar
Topic Author
LH
Posts: 5490
Joined: Wed Mar 14, 2007 2:54 am

Re: CPI - u calculation changed?

Post by LH »

nisiprius wrote:LH, see also Accuracy of the CPI.

One particularly germane datum, copied and pasted from that article, is that unlike shadowstats, which implies that it is calculating CPI according to the 1980 method but isn't, the Bureau of Labor Statistics actually does calculate "a series known as CPI-U-RS, which tracks the CPI from 1978 to the present when computed by current methods. CPI-U-RS is lower than CPI-U, but only slightly. Over the period 1978-1998, the annualized difference was 0.45%. The stated theory behind the changes is, of course, that the older method was overstating inflation and that the changes were a correction. If, however, you believe the older method was the correct one, then the current method is understating inflation--but only by about 0.45%."

This means that if you actually did what shadowstats claims to be doing, the blue curve--ANNUAL inflation--would be above the red curve about about half a percent, fluctuating, but the gap between them would not systematically increase. On the other hand, if you were to make similar charts, not for the annual inflation rate, but for the (cumulative) CPI-U itself, then they would in fact show a systematic, cumulative divergence--but not a huge one.
thanks for the link.
http://www.bogleheads.org/wiki/Accuracy_of_the_CPI
Several changes have been made in the definition of the CPI: the change in measuring homeownership costs from an asset-based approach to owner equivalent rent in 1983; the adoption of the use of the geometric mean for averaging prices within narrow categories in 1999; and the use of hedonic regression models, starting in 1988. Each of these changes had the effect of reducing the CPI at the time it was introduced.
This kind of thing is in par with history that I have read. Currency gets debased, and the tendency for debasers is to minimize/hide the amount of debasement. Simple human nature. History is replete.

In terms of "conspiracy" and such (replying in general to other posts), thats just a mental block I posit, and is off topic in this post. It detracts to the discussion, and really, cannot even be discussed even if one wanted to. This post is not about "conspiracies" or "perfection" or anything like that, its just simply about the cpi-u, and if/how has it changed.

So far per the given information:

1) The CPI-u has in fact changed several times

2) Each change that has occured so far has resulted in a downward adjustment of CPI. (which would make TIPS, if they existed at that time, pay less)

Appear to be facts so far.

3) There is controversy about how much the downward adjustments are/were, and how much the changes have resulted in a lower cpi now, than according to methodology of the past.

appears to be fair to say.

Its an interesting topic. I see Nisi has been since 2010 interested in this topic on the wiki. Your help is appreciated.



Thanks for everyones help and input,

LH
User avatar
momar
Posts: 1359
Joined: Sun Nov 13, 2011 12:51 pm

Re: CPI - u calculation changed?

Post by momar »

Shadowstats is a joke. The guy just adds a number onto the CPI and says that's true inflation. If we really had inflation that much higher than officially reported, you would know it because your standard of living compared to 20 years ago would be dramatically lower.
"Index funds have a place in your portfolio, but you'll never beat the index with them." - Words of wisdom from a Fidelity rep
Cyclone
Posts: 288
Joined: Sun Jan 10, 2010 9:48 pm

Re: CPI - u calculation changed?

Post by Cyclone »

But I still don't get the owl thing.
ourbrooks
Posts: 1575
Joined: Fri Nov 13, 2009 4:56 pm

Re: CPI - u calculation changed?

Post by ourbrooks »

I note that lowering the CPI value has the effect of reducing government spending. Had the major CPI changes not been made, Social Security and government pensions would be higher than they are now. One could argue that the hidden motivation behind the CPI changes comes from those who would like to see government expenditures reduced.
Alternately, one could believe that the changes in the CPI were actually best effort attempts to make the index more accurately reflect the financial experiences of the U.S. population.

In any case, as others have pointed out, the actual effects of the differences are so slight are more than likely to be completely overwhelmed by individual differences in consumption patterns. For nearly everyone, the differences due to local real estate conditions will completely overwhelm a possible difference in TIPS returns.
User avatar
nisiprius
Advisory Board
Posts: 42212
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: CPI - u calculation changed?

Post by nisiprius »

LH wrote:This kind of thing is in par with history that I have read. Currency gets debased, and the tendency for debasers is to minimize/hide the amount of debasement.
Except that the calculation of the CPI, and the changes in the calculation of the CPI, are not hidden. It's not as if a committee emerges from a secret meeting and simply announces a CPI. All the methodology is public, the changes in methodlogy are public, the data is public and available on their website.

They calculate a Research Series that shows what the CPI numbers for 1978-1998 would have been if calculated according to to current methods. They show exactly what the difference between the old and new methods is. That is not the action of an agency trying to "minimize/hide the amount of debasement."

On the contrary, it is shadowstats who, up until 2008 anyway, published numbers without any disclosure of how they were arrived at. In 2008 shadowstats promised
I shall make available to the public in the not-too-distant future the detailed series and calculations of same in collaboration with someone in academia (a search is in progress), who will have the opportunity to review and replicate or to challenge the alternate CPI data I have been publishing, and who will be free to publish those findings with peer review,
Does anyone know whether that study was ever conducted?
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
User avatar
tetractys
Posts: 4836
Joined: Sat Mar 17, 2007 3:30 pm
Location: Along the Salish Sea

Re: CPI - u calculation changed?

Post by tetractys »

Williams' shadowy thoughts are disgorged here: <http://www.shadowstats.com/article/no-4 ... easurement.>

Fun for the waders, Tet
User avatar
tetractys
Posts: 4836
Joined: Sat Mar 17, 2007 3:30 pm
Location: Along the Salish Sea

Re: CPI - u calculation changed?

Post by tetractys »

LH wrote:2) Each change that has occured so far has resulted in a downward adjustment of CPI. (which would make TIPS, if they existed at that time, pay less)
This is unestablished, if I understand what your saying. Sorry can't give a source right now, but I recall a few years ago seeing statements claiming the "changes" (Refinements in measurement might be a better way to put it.) showed an increased CPI growth rate. For one thing, the BLS changes in methodology have made it possible since 2007 to discern the CPI index to three places, up from one since 1913. -- Tet
User avatar
Topic Author
LH
Posts: 5490
Joined: Wed Mar 14, 2007 2:54 am

Re: CPI - u calculation changed?

Post by LH »

nisiprius wrote:
LH wrote:This kind of thing is in par with history that I have read. Currency gets debased, and the tendency for debasers is to minimize/hide the amount of debasement.
Except that the calculation of the CPI, and the changes in the calculation of the CPI, are not hidden. It's not as if a committee emerges from a secret meeting and simply announces a CPI. All the methodology is public, the changes in methodlogy are public, the data is public and available on their website.

They calculate a Research Series that shows what the CPI numbers for 1978-1998 would have been if calculated according to to current methods. They show exactly what the difference between the old and new methods is. That is not the action of an agency trying to "minimize/hide the amount of debasement."

On the contrary, it is shadowstats who, up until 2008 anyway, published numbers without any disclosure of how they were arrived at. In 2008 shadowstats promised
I shall make available to the public in the not-too-distant future the detailed series and calculations of same in collaboration with someone in academia (a search is in progress), who will have the opportunity to review and replicate or to challenge the alternate CPI data I have been publishing, and who will be free to publish those findings with peer review,
Does anyone know whether that study was ever conducted?
1) historically, currency gets debased, goes to zero or worth of the substance its made out of. Check out every currency that has existed.... and make a ratio, all currency that has ever existed/extant currency. you will find a very large number. Or cleaner, just pick the ones that became worthless, and compare to todays extant currencies...... historically, the issuers of currency, tend to minimize and/or hide the debasement.

2) I do not care per se about shadowstats, if its false, fine. If you want to state they do not minimize CPI changes, like the change from cpi to chained cpi, fine, its peripheral to the issue at hand.

3)The point is not what the current method applied backward is, the point is what the prior methods, applied forward would be, to see what the changes really amount to, an increase, no real change, or decrease in stated inflation.

What does an owner of TIPS care, if in say 2020 to 2040, he got paid by the methodology now in place in 2013, what the methodology is in 2060 applied backward in time? Nada. now if in 2030 there is a change in methodology, he WILL care about that 2030 methodology change from 2030 to 2040. He still will not give a darn about the 2060 method from 2030 to 2040 is.

The investor, and the cola'ed pensioner, care about the going forward in real time risk of diminishing CPI calculations....... This is the issue in reality.

So if the 2060 new CPI method results in whatever applied backwards, who really cares pedal to the medal back in 2040? There is no money from it, its meaningless.

What does have meaning, is the 1980s, 1990s methodology applied forward to now. Because the changes, which result in a downward change by the wiki you helped make, states all changes went downward, and therefor, result in less money paid out one supposes (I could see how this could not be the case, maybe it was down just initial time, then evened out to no change, etc, not fully specified).

So the question is, not what some theoretical backwards in time revision, but the forwards in time what where the actual effect that COLA people, and TIPS people, will experience in their pocketbooks.

I mean I cant fault you for it, its neat and such academically. Its a nice distraction. But whats really interesting and pertainant to know, is what happened in reality to people as a result of methodology changes.

I you could explain the FORWARD application of 1980s methodology, 1990s methodology, what would cpi have been under the various "old" methodologies, versus the new. Its not clear to me that backward applying it answers the question......

look if there have been say 4 cpi methodology changes since 1980
1
2
3
4

the real answer is to simply list out what each method inflation rate cpi would be started in the year it was created, to the current date 2103...... You could see what each cpi would have been, you could see, had the cpi calc not been changed, what people would have been paid in hard cold reality.

If its so open, so transparent, so easy, where is the info?

I mean it may be out there...... but where? Thats the formulation that makes the most sense. Its also the one that really shows what hits, -=if any=- cola'ed people took, and TIPS people would have taken.

So does that exist? If not why not? Because if its not there, AND its not hidden, well........... I do not know what to make of that at all. I do not know why this discussion is even necessary, its very curious is it not?

Doesnt seem to be a hard question for the BLS to simply spit that out. So nothing hidden, fine, whats the answer?

What kind of real financial hit, if any, did people take as a result of the methodology changes going forward?

1980 cpi annually 7,7, 7 8, 10, 10 7, 5, 5, 5, 5 ,5, 2,3,4,5,2,3,4,2,3,4,2,3,4,2,3,2,1,1,1,1,...............2103 number.
1987 new cpi annually 5, 5, 5, 5 ,5, 2,3,4,5,2,3,4,2,3,4,2,3,4,2,3,2,1,1,1,1,................2103 number.
1990 new cpi annually 5 ,5, 2,3,4,5,2,3,4,2,3,4,2,3,4,2,3,2,1,1,1,1,................2103 number.
1995 new cpi annually 5,2,3,4,2,3,4,2,3,4,2,3,2,1,1,1,1,................2103 number.

for an incomplete made up sample of what I am talking about.

with each cpi method starting in year it was put in place, continuing up to 2013.

Post that, that tells people what the actual financial meaning of the cpi changes are/were.

That can put some meaning on TIPS cpi calculation risk.

If its so open, transparent, I don't know why it hasn't, as far as I can tell, been done?

thanks for help, sorry for lack of brevity/editing down,

LH
User avatar
Topic Author
LH
Posts: 5490
Joined: Wed Mar 14, 2007 2:54 am

Re: CPI - u calculation changed?

Post by LH »

tetractys wrote:
LH wrote:2) Each change that has occured so far has resulted in a downward adjustment of CPI. (which would make TIPS, if they existed at that time, pay less)
This is unestablished, if I understand what your saying. Sorry can't give a source right now, but I recall a few years ago seeing statements claiming the "changes" (Refinements in measurement might be a better way to put it.) showed an increased CPI growth rate. For one thing, the BLS changes in methodology have made it possible since 2007 to discern the CPI index to three places, up from one since 1913. -- Tet
As the above lengthy post....

WHY is it unestablished?

It should be easy with an open, nonhidden published methodology to just list out the yearly calculations of each cpi method from creation time to current time.......

The juxtaposition of the lack of info, and the supposed completely open public methodology puzzles me.

Academics, government, wall street..... No one? Open nonhidden data and methods of various cpi-u? Went to the trouble to list out a backtest index applied in reverse time from its creation, but not to simply list out the actual previously extant cpi's going forward? So a synthetic backward meaningless future cpi list versus the cpi methods that would have paid out dollars in reality to actual people had they been not been canceled by the latest newer cpi

Really?

I find that highly curious if that is the case. And so far, that appears to be the case?


Thanks,

LH
Post Reply