New Paper ". . . and the Cross-Section of Expected Returns"
New Paper ". . . and the Cross-Section of Expected Returns"
I don't have access to the full paper but others might and it sounds like it should interest those doubting any future factors or probably and recent ones or perhaps any factors at all...
http://www.nber.org/papers/w20592?utm_c ... source=ntw
Hundreds of papers and hundreds of factors attempt to explain the cross-section of expected returns. Given this extensive data mining, it does not make any economic or statistical sense to use the usual significance criteria for a newly discovered factor, e.g., a t-ratio greater than 2.0. However, what hurdle should be used for current research? Our paper introduces a multiple testing framework and provides a time series of historical significance cutoffs from the first empirical tests in 1967 to today. Our new method allows for correlation among the tests as well as missing data. We also project forward 20 years assuming the rate of factor production remains similar to the experience of the last few years. The estimation of our model suggests that a newly discovered factor needs to clear a much higher hurdle, with a t-ratio greater than 3.0. Echoing a recent disturbing conclusion in the medical literature, we argue that most claimed research findings in financial economics are likely false.
http://www.nber.org/papers/w20592?utm_c ... source=ntw
Hundreds of papers and hundreds of factors attempt to explain the cross-section of expected returns. Given this extensive data mining, it does not make any economic or statistical sense to use the usual significance criteria for a newly discovered factor, e.g., a t-ratio greater than 2.0. However, what hurdle should be used for current research? Our paper introduces a multiple testing framework and provides a time series of historical significance cutoffs from the first empirical tests in 1967 to today. Our new method allows for correlation among the tests as well as missing data. We also project forward 20 years assuming the rate of factor production remains similar to the experience of the last few years. The estimation of our model suggests that a newly discovered factor needs to clear a much higher hurdle, with a t-ratio greater than 3.0. Echoing a recent disturbing conclusion in the medical literature, we argue that most claimed research findings in financial economics are likely false.
A man is rich in proportion to the number of things he can afford to let alone.
-
- Posts: 16022
- Joined: Thu Feb 22, 2007 7:28 am
- Location: St Louis MO
Re: New Paper ". . . and the Cross-Section of Expected Retur
Yes with all the computing paper and the mischief of possibility of data mining (no theory just search for a correlation and then make one up after) that perhaps in terms of tstats 3 or even 4 is the new 2.
Having said that, IMO the same hurdles should apply --you want evidence to be persistent and pervasive, and better if also across asset classes as well as time, economic regimes, regions. And better if risk story vs. behavioral, so more likely to persist IMO, though human behavior doesn't appear to change and there are limits to arbitrage and costs/fears of margin which prevent market from "fixing" mispricing.
Larry
Having said that, IMO the same hurdles should apply --you want evidence to be persistent and pervasive, and better if also across asset classes as well as time, economic regimes, regions. And better if risk story vs. behavioral, so more likely to persist IMO, though human behavior doesn't appear to change and there are limits to arbitrage and costs/fears of margin which prevent market from "fixing" mispricing.
Larry
Re: New Paper ". . . and the Cross-Section of Expected Retur
Would it be better not to publish in journals papers with t-ratios less than 3? Even if you did, you could not stop people from posting papers on sites such as SSRN, and practitioners of quant finance do not wait until something has been published in a journal to do their own analyses. If the cost of implementing a factor tilt, including both transaction costs and taxes, is low, a belief that is 2/3 likely to be real may be enough to give it some weight.matjen wrote:I don't have access to the full paper but others might and it sounds like it should interest those doubting any future factors or probably and recent ones or perhaps any factors at all...
http://www.nber.org/papers/w20592?utm_c ... source=ntw
Hundreds of papers and hundreds of factors attempt to explain the cross-section of expected returns. Given this extensive data mining, it does not make any economic or statistical sense to use the usual significance criteria for a newly discovered factor, e.g., a t-ratio greater than 2.0. However, what hurdle should be used for current research? Our paper introduces a multiple testing framework and provides a time series of historical significance cutoffs from the first empirical tests in 1967 to today. Our new method allows for correlation among the tests as well as missing data. We also project forward 20 years assuming the rate of factor production remains similar to the experience of the last few years. The estimation of our model suggests that a newly discovered factor needs to clear a much higher hurdle, with a t-ratio greater than 3.0. Echoing a recent disturbing conclusion in the medical literature, we argue that most claimed research findings in financial economics are likely false.
Re: New Paper ". . . and the Cross-Section of Expected Retur
Ah, the beauty of owning the whole market. You get exposure to all the factors, whether real or not, whether discovered or not.
Paul
Paul
When times are good, investors tend to forget about risk and focus on opportunity. When times are bad, investors tend to forget about opportunity and focus on risk.
Re: New Paper ". . . and the Cross-Section of Expected Retur
The paper has been linked in this thread. Interesting that value and momentum far exceed their threshold, while market beta barely does.
- nisiprius
- Advisory Board
- Posts: 52105
- Joined: Thu Jul 26, 2007 9:33 am
- Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry
Re: New Paper ". . . and the Cross-Section of Expected Retur
Why "t-stats" at all?
"Student's" t-test was invented in 1908 for making statistical measurements of beer quality--that is to say industrial process control. It makes a huge boatload of strong assumptions which actually apply quite well in a wide range of real-world situations. For example, when you have good reason to believe that something you are measuring follows a normal distribution because it is influenced by many small things that have a nearly additive effect; and when you know enough about the situation to say that you are dealing, not only with measurements that follow a normal distribution, but measurements that are independent of each other. There are many situations in science and in industrial process control where this model is reasonable.
In financial data these assumptions don't pass the laugh test.
Meanwhile, in the century since 1908, statisticians have been developing a whole armamentarium of tests that are valid and can be used in situations where traditional parametric statistics are invalid. There are whole textbooks dealing non-parametric statistics. There is a whole field of "sequential tests," which apply to the situation where you are continuously acquiring data and what to know when you can stop--when you have enough. I don't personally know more than that they exist, but I do know that much--and they are not obscure.
If I look at, let's say, the University of Chicago, I find that it has an entire department of statistics, with 21 professors, and at least 18 courses, including one in "Nonparametric Inference," one in "Longitudinal Data Analysis" ("Longitudinal data consist of multiple measures over time on a sample of individuals. This type of data occurs extensively in both observational and experimental biomedical and public health studies, as well as in studies in sociology and applied economics,") one in "High-Dimensional Statistics" ("statistical problems where the number of variables is very large. Classical statistical methods and theory often fail in such settings...")
I'd have thought that economics and finance would be a hotbed of research on how to address statistical testing of financial data where few, if any, of the assumptions made in traditional statistic apply.
Of course, the tests that make fewer assumptions are less "powerful" than the traditional tests. That is to say, if you have good reason to believe a Student's "t" test is valid on your data, you can detect "significance" with that test when a Mann-Whitney test will not detect it. Conversely, if you have data that do not meet the requirements for a t-test, then the t-test will falsely report significant results when the data do not actually support it. It would seem that the right course of action is not to use an inappropriate test and arbitrarily increase the threshold, but to use an appropriate test.
There do seem to be e.g. courses in Statistical Techniques of Financial Data Analysis ("...Order statistics, Limit Theorems for extremes, Elementary stochastic processes such as Markov chains... Neyman-Pearson tests, likelihood ratio tests, and Wald tests... time series inference... non-parametric analyses... Testing hypotheses of independence, normality, homoscedascticity, and symmetry for returns, and the Bachelier and Mandelbrot models...")
This stuff seems to be out there. Why isn't it used?
"Student's" t-test was invented in 1908 for making statistical measurements of beer quality--that is to say industrial process control. It makes a huge boatload of strong assumptions which actually apply quite well in a wide range of real-world situations. For example, when you have good reason to believe that something you are measuring follows a normal distribution because it is influenced by many small things that have a nearly additive effect; and when you know enough about the situation to say that you are dealing, not only with measurements that follow a normal distribution, but measurements that are independent of each other. There are many situations in science and in industrial process control where this model is reasonable.
In financial data these assumptions don't pass the laugh test.
Meanwhile, in the century since 1908, statisticians have been developing a whole armamentarium of tests that are valid and can be used in situations where traditional parametric statistics are invalid. There are whole textbooks dealing non-parametric statistics. There is a whole field of "sequential tests," which apply to the situation where you are continuously acquiring data and what to know when you can stop--when you have enough. I don't personally know more than that they exist, but I do know that much--and they are not obscure.
If I look at, let's say, the University of Chicago, I find that it has an entire department of statistics, with 21 professors, and at least 18 courses, including one in "Nonparametric Inference," one in "Longitudinal Data Analysis" ("Longitudinal data consist of multiple measures over time on a sample of individuals. This type of data occurs extensively in both observational and experimental biomedical and public health studies, as well as in studies in sociology and applied economics,") one in "High-Dimensional Statistics" ("statistical problems where the number of variables is very large. Classical statistical methods and theory often fail in such settings...")
I'd have thought that economics and finance would be a hotbed of research on how to address statistical testing of financial data where few, if any, of the assumptions made in traditional statistic apply.
Of course, the tests that make fewer assumptions are less "powerful" than the traditional tests. That is to say, if you have good reason to believe a Student's "t" test is valid on your data, you can detect "significance" with that test when a Mann-Whitney test will not detect it. Conversely, if you have data that do not meet the requirements for a t-test, then the t-test will falsely report significant results when the data do not actually support it. It would seem that the right course of action is not to use an inappropriate test and arbitrarily increase the threshold, but to use an appropriate test.
There do seem to be e.g. courses in Statistical Techniques of Financial Data Analysis ("...Order statistics, Limit Theorems for extremes, Elementary stochastic processes such as Markov chains... Neyman-Pearson tests, likelihood ratio tests, and Wald tests... time series inference... non-parametric analyses... Testing hypotheses of independence, normality, homoscedascticity, and symmetry for returns, and the Bachelier and Mandelbrot models...")
This stuff seems to be out there. Why isn't it used?
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
Re: New Paper ". . . and the Cross-Section of Expected Retur
It isn't used because financial data are abysmal -- and yet we think that factor analyzing and multiple-regressing it is worth the time. Bill Sharpe is correct that you can't even trust that long term averages are meaningful: "Even if the Gods are kind and distributions never change - which is improbable - and even if you have lots of data, you can still be way off on the premium for the whole market. Anyone who thinks that looking at empirical data will resolve questions such as whether the premium for small growth stocks is different from large value stocks with any degree of precision is just kidding himself."
We don't know where we are, or where we're going -- but we're making good time.
Re: New Paper ". . . and the Cross-Section of Expected Retur
What data in the social sciences is of higher quality? Historical stock returns are measured precisely than historical GDP growth, for example. The science of psychology largely consists of extrapolating the results of experiments done on college students.Browser wrote:It isn't used because financial data are abysmal -- and yet we think that factor analyzing and multiple-regressing it is worth the time.
-
- Posts: 5
- Joined: Wed Oct 22, 2014 8:24 pm