I used a linear model the determine the best values for a and b, I don't know what the value(s) were exactly. I scaled pred_vol with values both above and below 1.Steve Reading wrote: ↑Mon Aug 31, 2020 4:54 pmOk but just to clarify, you only used b>1.0 above? And when you scaled pred_vol, was it by multiplying it by a number greater than 1.0?Uncorrelated wrote: ↑Mon Aug 31, 2020 3:15 pmExactly. I used a linear model (pred_vol = a + b * vix) to predict future volatility, and that results in worse out-of-sample utility. I also tried scaling the predicted volatility by various amounts, but a scaling factor of 1 gave the best results indicating that my linear model was already the best possible.Steve Reading wrote: ↑Mon Aug 31, 2020 9:53 amGotcha. So you tried multiplying the VIX by some constant factor greater than 1.0 and used that as the future volatility estimate, and the results were inferior as well?Uncorrelated wrote: ↑Mon Aug 31, 2020 1:52 am VIX is a measure of risk-neutral volatility (that means that it assumes options are priced by risk-neutral actors), but option traders are not risk neutral, this causes VIX to consistently under-estimate market volatility. Based on this explanation you might think that scaling VIX by some constant factor fixes this issue, but it turns out this doesn't work since the risk aversion of option traders changes quite substantially over time.

Linear model is a bit of a misnomer, the way I have implemented it has more similarities to quantile regression. My linear model significant outperforms various GARCH models terms of R^2 and out-of-sample performance (various time horizons. I'm not very knowledgeable about GARCH models but I tried multiple different models from a python package). I also tried using GARCH as an input to other models, but that didn't work either. I repeated the same process with various machine learning algorithms, none of them outperformed my simple linear model. I suppose the only thing I haven't tried yet are RNN's.Investing Lawyer wrote: ↑Mon Aug 31, 2020 10:30 pm If you want to predict volatility I would suggest using a GARCH model on sp500 returns. Personally, I've only seen GARCH modeling done on daily returns, but they've looked really good (haven't got around to it myself using longer dated sequences). Unfortunately with the typical GARCH(1,1) model you only get a prediction one period ahead. A linear model of VIX to predict the future volatility is not going to work. Least squares estimates are likely not consistent due to the absurd autocorrelation of the VIX and therefore, the residuals as well. If I wanted to model the VIX itself, I would consider only using an ARIMA model. I imagine if you run a least squares with the past value to predict the future and an ARIMA(1,0,1) you would get different estimates of the coefficients for the parameter for the lagged value. (Vix would likely need to be differenced before modeling so both models would be bad)

To summarize, I think I did my research pretty well