The problem is that even the experts whose job it is to predict the market's future earnings-per-share have a terrible track record of forecasting this metric just one year in advance, much less 10 years in the future. Though not totally discounting the rational calculations of future fundamentals, I'm sticking with overzealous optimism and pessimism as a significant driver of long-term multiple changes.
Future earnings estimates are basically a joke. They are made by analysts in the business of marketing equity and historically on a consistent basis overestimate earnings on average by 10%. That's good for their business models, investors who actually believe in analysts estimates are more likely to justify the purchase of richly valued stock based on presumed future growth in earnings. But in truth the problem is much worse than the average 10% error, because there is a very wide dispersion of errors, some wildly optimistic, some much less so, according to the prevailing emotional investment climate. In the the tech boom, long term earnings estimates were wildly overstated relative to what turned out to be reality. They and employed successfully to justify outrageous PEs as the bubble expanded. There seemed to be no end in sight for the massively growing profits of Lucent or Nokia or Cisco.
There are macroeconomic conditions for sure when next years earnings are expected to improve, no doubt about that. There are also times when they are expected to suffer. But in both conditions there is considerable emotional overlay on top of a rational analysis, partly from the conflict of interest agenda of buy side financial firms who generate profits from equity sales and partly from the natural emotional swings of herd human behavior.
Currently in the market there is considerable optimism on future earnings growth. The following is a quote from an article on earnings estimates for next year 2018 from Seeking Alpha:
Earnings are expected to rise an estimated 22 percent to $131.36 by the end of 2018, from the trialing-twelve-months estimates of $107.61. At 23.5 times 2018 earnings, the S&P 500 would be valued at 3087, or roughly 3,100.
22% earnings growth is quite robust and they're predicting generous increases in the S&P. Macroeconomic conditions are good now, no doubt, but achieving that level of earnings growth is quite unlikely IMO. It is however used to justify stock current and future price appreciation. The general optimism and complacency among investors now is reflected in generous but not bubble-like equity valuations and also in the Volatility Index VIX which is at or very near its lowest point in history. Essentially no one expects significant inflation or a recession and most expect the continuation of slow growth, low inflation, and increasing corporate profits (the Goldilocks scenario). The optimists may be right, it may continue unabated. The happy news may continue. The driving factor for our generous valuations and low VIX, however, is not just fundamental economic reality which, granted, looks okay, but a large measure of positive emotion/sentiment based on 8+ years of bull market. Earlier along in that 8+ year run, many investors were skeptical of it, loaded up on bonds yielding almost nothing, and the bull kept running in spite of a lot of money watching from the sidelines. A lot of that sideline money has finally given up on pessimism and bought into risk assets, driving up their prices. That's where we are now IMO. Historically, pervasive optimism and investor complacency tends to happen in the late stages of a bull market. That doesn't mean it will collapse soon, but rather that at some point in the future, air will start leaking out the positive emotional balloon and PE expansion will likely moderate, cease, or even reverse.