The issue about moving frequently is important. In the US as I understand it agents still charge 5-6%? In addition there is a 'bid offer spread' on houses that you don't see: you seldom get the 'market' price for a house, you get a slight discount representing whatever the buyer negotiates. And then when you buy, you have the same problem (ie you sell at the bid price, and buy at the offer/ask price, and there is probably typically a 3-4% gap between the two). Add it all up, you can lose 10% every time you move house. Huge transactions costs.
I don't have enough information to fully quantify the above. But my sense is we are losing c. 10% of our value every time we sell (and then buy) another house.
That's obvious if you sell and then buy stocks: it's on the screen. But NOT so obvious with houses, where the bid offer spread is not quoted, and there is a lot of noise in pricing. But my contention is the noise (where you sell for more than it's worth, buy for less than it's worth) will on average, average out, it's not a durable source of gain of wealth to you. Sometimes you win, sometimes you lose, but on average there must be a 3-5% bid offer spread reflecting liquidity and transactions costs.
It really underlines the dangers of moving frequently. I am on my second home, and hope that in my life there is not a third (there probably is). Agents fees are far, far lower here (sub 3%). But there is stamp duty (4%-- a huge cost) plus everything else.
It seems to me all indices of returns for retail housing are skewed because they cannot take these factors fully into account. We are not, generally, 'buy and hold' investors in our own homes-- life forces us to move. Your returns as a 'fund investor' in your house are nothing like the returns reported by the indices (which don't adjust for taxes, maintenance, stamp duty, agents costs etc.).
Houses in London have outperformed just about any financial asset long term-- +6% pa real increment 1994-2009 (plus rental return if you were a landlord). Nonetheless I still think housing is a difficult bet against stocks given the factors of transactions costs. Note that house prices (and rents) are still going up here (rest of the UK outside the SE commuter belt, fallen c. 10-12% real since 2008).
A number of US single family dwelling REITs are coming to market- -PE and hedge funds which gobbled up unsold Sun Belt dwellings during the housing bust. Historically REITs are apartments, so this is an entirely new asset class. It remains to be seen whether the relatively high administration and capex requirements of single family residential housing are properly valued at IPO. But the Prospectuses should have very interesting information on this new asset class, including estimates I hope of SFH returns, historically.