Here is my attempt to answer your questions: Using this calculator: http://www.dinkytown.net/java/AnnualReturn.html punching in $35 for 1/1/1971 and $1655 for today, I come up with a 9.616% nominal annualized return from 1971 to today. Taking inflation into account, calculated so many different ways, I'm not sure what the real return is, but perhaps 4% annually?
In regard to the time prior to 1971, I don't know why people would argue we only can measure the real return of gold during the current period. I think we can measure with some qualifications, the real return of gold before that period. We would have to keep in mind that gold was $20.67 per ounce in 1802 and then went to $35 in 1935. Using the same calculator this is a nominal return of .385% annually through 1935. Was there a general deflationary tendency during that period, I believe so...nisiprius posted about this period...I really don't have the specifics, but let's say there was an average 2% deflation over that time...
A few observations.
1. Gold does have a long term positive real return.
2. Deflation did occur in the 1800s ... but probably only about -1 percent per year (estimate of CPI by Fed http://www.minneapolisfed.org/community_education/teacher/calc/hist1800.cfm?)
3. Gold's fluctuations are large.
Holding gold for diversification purposes makes sense given points 1 and 3, but only in moderation. I think this is Scott Burn's realization.