A mean variance framework for portfolio optimization

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Post Reply
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

A mean variance framework for portfolio optimization

Post by Uncorrelated »

Hello I'm introducing a mean-variance optimization framework for portfolio choice. This framework is intended to replace backtesting, and help you find the optimal portfolio choice under the following assumptions:
  • The expected return, volatility and correlations of various asset classes are known OR you believe that past returns are the best estimator of future returns.
  • Returns follow a normal or log-normal distribution.
  • You have a CRRA (constant relative risk aversion) utility function, or similar.
  • You are a mean-variance investor. That is, you only care about the probability distribution of outcomes.
This approach has several advantages over backtesting. It gives you the optimal allocation directly so you don't have to perform 50 different backtests to find the best one. Solves for all investor preferences simultaneously, instead of just one. Reduces the probability that results are misinterpreted (My dream is that I don't have to explain that small cap value has a higher realized return than total stock market ever again). Assumptions are made explicit. The assumptions can be tweaked which can help to reduce the impact of survivorship bias. The main disadvantage is that it makes a simplified distributional assumption, this is particularly troublesome when testing poorly diversified funds.

A quick primer on the CRRA utility function. An utility function specifies your personal trade-off between risk and reward. Suppose that you are presented with two choices:
A) you immediately receive a bonus of 10% of your current net worth.
B) you have X% chance of receiving a bonus of 20% of your net worth, otherwise nothing happens.
When X<50% it is obvious that A is the best option. When X = 100%, it is obvious that B is the best option. There is a specific value of X in which you think the two options are equivalent, that specific value of X completely defines your personal risk preferences. Basically, the utility function defines how much risk you are willing to take for a given reward.

Your coefficient or relative risk aversion is denoted by γ (gamma). Here are a few sample values of γ and the resulting (approximate) asset allocation:

γ=5 corresponds to an asset allocation of 40% stocks, 60% bonds
γ=4 corresponds to an asset allocation of 50% stocks, 50% bonds
γ=3 corresponds to an asset allocation of 65% stocks, 35% bonds
γ=2 corresponds to an asset allocation of 100% stocks
γ=1 corresponds to an asset allocation of 200% stocks
γ=0 corresponds to an asset allocation of infinitely many stocks.

Time horizon does not impact your risk aversion*. Age is probably unrelated to risk aversion. Your remaining human capital will have a big impact on your risk aversion. According to own calculations based on the theory discussed in the lifecycle investing book, your risk aversion becomes arbitrary low when your liquid net worth is negligible compared to your remaining human capital.

*according to some, some asset classes exhibit mean-reversing behavior which reduces the variance as the time horizon increases. This does impact your asset allocation, but is outside the scope of this topic.

Methodology:

I'll keep this short. If all assets are (log)normally distributed, then any linear combination of assets is also (log)normally distributed. If your asset allocation is normally distributed as X, the CRRA utility is approximately E[X] - .5 * γ * VAR[X]. To find the optimal portfolio for a given γ, we only have to figure out which asset allocation maximizes this equation. We repeat this for all γ to find the optimal portfolio for different investors.

It is possible to modify the optimizer to drop the assumption for (log)normally distributed returns, but this is somewhat computationally expensive and drastically limits the time period.

Trivial case:

For this example, we have estimated: (expected excess return / annualized standard deviation)
market returns at 6% / 15.5%, data from ken french 1963-2020
total fond market returns at 1.68% / 5%, data from simba's backtesting spreadsheet, 1934-2018
correlation between stocks and bonds = 0

Image

In the top pane, we see the results of the utility function with various γ. γ = 0 represents a risk-neutral investor, and can be interpreted as the traditional efficient frontier. The other values of γ represent the expected utility (technically, the certainty equivalent return) for investors with various coefficients of relative risk aversion, the colored x represents the location of the optimal portfolio.

In the bottom pane, you will find the composition of the optimal portfolio. To find your optimal portfolio, first look up the X coordinate of your "X" in the top pane, then look at the same X coordinate in the bottom pane. The bottom pane shows the composition of your optimal asset allocation.

The X axis is a bit wide to ensure we have sufficient axis space available when we investigate leveraged portfolio choice.

adding long term bonds:

For this example I have substituted total bond market for intermediate term bonds to ensure we only have exposure to the term premium (government bonds), not the credit premium (corporate bonds). We estimated:
market returns at 6% / 15.5%, data from ken french 1963-2020
ITT bond returns at 1.66% / 5.65%, data from simba's backtesting spreadsheet 1934-2018
LTT bond returns at 2.01% / 9.79%, data from simba's backtesting spreadsheet 1934-2018
correlation between stocks and bonds = 0
correlation between ITT and LTT = 0.8

Image

As you can see, the resulting asset allocation has changed slightly, but the top pane is nearly identical. Adding long term bonds looks like a micro-optimization. Just use total bond market.

leverage:

We substitute LTT for TMF (3x 20x treasury bonds) and add UPRO (3x leveraged S&P500).
TMF is simulated as having an expense ratio of 1.1% and an exposure of 3.0 on 20y treasury bonds.
UPRO is simulated with an expense ratio of 2% and an exposure of 3.0 on market and -.48 on smb.
The expected return, volatility and correlation of the size factor are taken from ken french data library based on FF5 model. The expected return of the factors was discounted by a factor of 2 to reduce the impact of survivorship bias inside the factor set.

Image

For investors with arbitrary low risk aversion, the optimal choice is 100% UPRO.
This specific scenario doesn't include LTT. But if we had included LTT, it's allocation would be 0%.
One particular thing to note is that the optimal ratio between TMF and UPRO varies according to your risk tolerance. There is no single ratio that is "optimal". The optimal ratio depends on your risk aversion. Strategies dat recommend a single ratio are suspect for this reason.
The asset allocation for γ = 2, which previously held a 100% stock allocation, now holds 62% total stock market, 16% UPRO and 22% TMF. This equates to a factor exposure of approximately 1.1 market and .6 term. This asset allocation has a certainty equivalent return of 4.4%, compared to 4.0% for the 100% stock allocation.
Options and futures are substantially cheaper than UPRO/TMF. You should probably use them instead of leveraged ETF's.
If we assume frequent rebalancing, the impact of volatility decay is probably negligible. In fact, if stock returns were independently distributed (not necessary normal), daily rebalancing would be optimal. If you actually want to use high leverage, you should probably optimize for the highest expected utility by using historical returns data. This way you can avoid making any distributional assumptions and simultaneously optimize for transaction costs and tax impact. Last time I checked I found that the choice between options/futures and leveraged ETF's depends on a lot of things.

Factor funds

We substitute leveraged funds for factor funds. We use Fama & French 5 factor model + momentum as our factor model. The correlation data was taken from A Five-Factor Asset Pricing Model. The expected return and volatility was calculated based on ken french 5-factor US 1963-2020 data. The expected return of FF5 factors was discounted by 2 and the expected return of momentum was discounted by a factor of 4 to reduce the impact of survivorship bias (and to be extra conservative, in general).

Value was simuated by IUSV (iShares Core S&P U.S. Value ETF). The expense ratio of this ETF is .04%. I estimated the factor exposure at .97 mkt, .03 smb, .3 hml, .15 rmw, .21 cma, -.02 mom.
Small cap Value was simuated by IJS ( iShares S&P Small-Cap 600 Value ETF). The expense ratio of this ETF is .25%. I estimated the factor exposure at 1.04 mkt, .86 smb, .21 hml, .19 rmw, .09 cma, -.05 mom.

Image

Even though we discounted the returns of factors by 2, factor funds still dominate ordinary index funds.
I use IJS and IUSV because they are relatively long running and simple. I don't necessarily recommend these funds.
With DFSVX thrown into the mix, the optimizer seems to have difficulty deciding whether IJS or DFSVX is better after expenses.
With these assumptions, everyone should factor tilt, not just those with low risk aversion.

This scenario suggests that a value tilt may be a better idea than a small cap value tilt for most individuals. I don't think that conclusion is statistically significant. I usually suggest 1/3 TSM 1/3 value 1/3 small cap value as a compromise, it's definitely possible to argue for more aggressive tilts.

Multifactor Factor funds

We add everyone's favorite multifactor fund, VFMF.

The factor exposure of VFMF was estimated as: .18 ter, .98 mkt, .47 smb, .36 hml, .16 rmw, .12 cma, .19 mom

Image

We see that VFMF mostly dominates small cap value. This makes sense, since VFMF is basically a slightly more efficient combination of value and small cap value. Surprisingly (to me), the optimizer still things that there is a value-add from adding a plain value fund.

factor funds + leverage

We use UPRO/TMF for leverage but add the possibility of factor funds.

Image

The earlier result in which we investigated UPRO/TMF is mostly unchanged, except that total stock market is replaced with factor funds.

Using historical factor returns

So far we have discounted all factor returns by a factor of 2 (momentum by 4). What happens if we don't?

Image

When assuming full factor returns, small cap value dominates TMF. We can infer that users that have high allocations to TMF have very optimistic views about future bond returns and/or very pessimistic views on future factor returns.


Comments:

All return figures above are excess returns vs T-bills.

The correct asset allocation depends on your personal risk preferences. One-size-fits-all Portfolio's (permanent portfolio, all-weather) can never work.

Unless you are willing to use leveraged bonds, don't try to cherry pick the bond market. Just use total bond market :wink: .

Even with very pessimistic forecasts (half the historical factor premia), factor funds dominate ordinary index funds. It would appear that in order to rationalize not investing in factor funds, you must be extremely skeptical of the Fama & French 3/5 factor model.

When you run a backtest, the ending balance (which is related to the CAGR) is calculated as:
ending balance = X1 * X2 * X3 * ... * Xn.
Or equivalently:
log(ending balance) = log(X1) + log(X2) + log(Xx) + ... + log(Xn)
Whereas the expected utility is calculated as:
expected utility = average(U(X1) + U(X2) + U(X3) + ... + U(Xn))
With U being the utility function. When your coefficient of relative risk aversion γ is 1, U(x) = log(x). Therefore, finding the max CAGR by repeatedly running backtests is equivalent to optimizing for max utility with γ = 1. That's why I said that this framework is intended to replace backtests: observing the CAGR value of a backtest is only meaningful if γ = 1, but it is very unlikely your personal coefficient of relative risk aversion is equal to 1!

With these assumptions it is optimal to rebalance continuously (it should be obvious why). The real question is which rebalance frequency or methodology is the best trade-off between the optimal asset allocation, tax impact, transaction costs and possible mean-reversing or momentum behavior on short time horizons.

Disadvantages:

I am aware that historical bond yields are far higher than current yields. I don't believe that necessarily implies that the term premium is a thing of the past. Anyway, the correct asset allocation for bonds is primarily determined by the difference between ITT and LTT, not their absolute return. If we assume E[ITT] = .66% and E[LTT] = 1.01% (down from 1.66% and 2.01%), then the allocation to TMF decreases (It's high expense ratio makes it very unattractive when expected bond returns are low), but the other scenario's are mostly unchanged.

Mean-variance optimization is optimal under the condition that you know the expected return, volatility and correlations of various asset classes or risk factors exactly. You can only estimate these characteristics with low accuracy. Attempting to perform mean-variance optimization with too many assets or too complicated models can (will) result in a massive amplification of measurement errors. You should check if the estimated variables are statistically significant before attempting mean-variance optimization. This is particularly important when it comes to alternative asset classes such as gold or REIT's. My philosophy is that you should select the asset classes or risk factors that you are most confident in (highest statistical confidence), and then run a mean-variance analysis or other expected utility analysis to select the correct asset allocation. If you skip the first step your results will be meaningless.

Our mean-variance optimizer assumes i.i.d. normally distributed returns. Real market returns are not independent and not normally distributed. aacalc.com has investigated the efficient frontier for small cap value without any distributional assumptions, and finds that without distributional assumptions, small cap value looks slightly better. Highly concentrated small cap value funds have extremely skewed return distributions and probably likely to perform much poorer than their standard deviation suggests. See How Diversification Impacts Investment Outcomes: A Case Study on Global Large Caps from DFA research for more background information. As mentioned earlier, it's possible to modify the optimizer to drop the normally distributed assumption.

We assume that FF5 fully explains return differences between diversified portfolio's by fitting a long-short factor model. But the funds we use are long-only. research shows that long-only factor funds are capable of capturing the factor premia, but the mismatch between a long-short factor model and long-only funds obviously introduces inaccuracies.

To minimize overfitting, I only used US data in this analysis. But you should really use international diversification. This type of analysis is not suited to determine which US/international split is optimal, because the difference between US and international is not statistically significant.


Source code:

Requires python 3 with packages numpy, scipy, matplotlib. Your assets can be defined around line 56 and 77. It should take around 10 seconds to run.

Code: Select all

# coding=utf-8
import numpy as np
import matplotlib.pyplot as plt
import scipy.optimize
import scipy.stats
from matplotlib.ticker import StrMethodFormatter


def asset_def(name, ter=0.0,
              mkt=0.0, smb=0.0, hml=0.0, rmw=0.0, cma=0.0, mom=0.0,
              itt=0.0, ltt=0.0):
    return name, {
        'TER': ter,
        'MKT': mkt,

        'SmB': smb,
        'HmL': hml,
        'RmW': rmw,
        'CmA': cma,

        'MOM': mom,

        'ITT': itt,
        'LTT': ltt,
    }


# return assumptions
# monthly arithmetic return, monthly std
factors = np.array([
    ['TER', -1 / 12.0, 0],
    ['MKT', .53, 4.44],

    # US factor data, 1963 - july 2020
    ['SmB', .21 / 2, 3.02],  # estimate only half the historical factor premia
    ['HmL', .25 / 2, 2.87],  # estimate only half the historical factor premia
    ['RmW', .26 / 2, 2.15],  # estimate only half the historical factor premia
    ['CmA', .26 / 2, 1.99],  # estimate only half the historical factor premia
    ['MOM',     .66 / 4, 4.7],  # estimate only a quarter the historical factor premia

    ['ITT', 1.66 / 12, 5.67 / 12**.5],
    ['LTT', 2.01 / 12, 9.79 / 12**.5],
])

factor_labels = list(factors[:, 0])
factor_mean = factors[:, 1].astype(np.float)
factor_std = factors[:, 2].astype(np.float)

factor_mean = np.array(factor_mean) / 100
factor_std = np.array(factor_std) / 100

print('factor mean & std (monthly)')
print(np.vstack((factor_labels, factor_mean*100, factor_std*100)).transpose())


cash_asset = asset_def('cash')
mkt_asset = asset_def('total stock market', mkt=1)
sp500_asset = asset_def('S&P 500', mkt=1, smb=-.16)

iusv_ff5_asset = asset_def('IUSV (value)', ter=0.04, mkt=.97,  smb=.03, hml=.30, rmw=.15, cma=.21, mom=-0.02)
ijs_ff5_asset = asset_def('IJS (scv)', ter=.25,   mkt=1.04, smb=.86, hml=.21, rmw=.19, cma=.09, mom=-0.05)
dfsvx_ff5_asset = asset_def('DFSVX (scv)', ter=.4,   mkt=1.03, smb=.87, hml=.40, rmw=.14, cma=.08, mom=-0.08)
vfmf_ff5_asset = asset_def('VFMF', ter=.18, mkt=.98, smb=.47, hml=.36, rmw=.16, cma=.12, mom=.19)

itt_asset = asset_def('ITT', ter=0, itt=1)
ltt_asset = asset_def('LTT', ter=0, ltt=1)
tmf_asset = asset_def('TMF', ter=1.1, ltt=3)
upro_asset = asset_def('UPRO', ter=2.0, mkt=3, smb=-.16 * 3)

BOUNDS = (0, 1)
CASH_BOUNDS = (0, 1)
SHORTING_COST = .5 / 100
X_AXIS = (0, .50)   # annualized standard deviation
Y_AXIS = (0, .12)   # annualized return

chosen_assets = [
    cash_asset,
    mkt_asset,
    itt_asset,
    iusv_ff5_asset,
    ijs_ff5_asset,
]

def asset_to_weight_array(asset):
    a = np.zeros(len(factor_labels))
    for k, v in asset[1].items():
        if k in factor_labels:
            a[list(factor_labels).index(k)] = v

    return a


asset_labels = [a[0] for a in chosen_assets]
assets = np.array([
    asset_to_weight_array(asset) for asset in chosen_assets
])


correlations = {
    # collected from FF, A Five-Factor Asset Pricing Model (2014)
    ('MKT', 'SmB'): .28,
    ('MKT', 'HmL'): -.30,
    ('MKT', 'RmW'): -.21,
    ('MKT', 'CmA'): -.39,

    ('SmB', 'HmL'): -.11,
    ('SmB', 'RmW'): -.36,
    ('SmB', 'CmA'): -.11,

    ('HmL', 'RmW'): .08,
    ('HmL', 'CmA'): .70,

    ('RmW', 'CmA'): -.11,

    ('ITT', 'LTT'): .8,
}

correlation_coef = np.identity(len(factor_labels))
for k, v in correlations.items():
    a, b = k
    try:
        a = factor_labels.index(a)
        b = factor_labels.index(b)
        correlation_coef[a, b] = v
        correlation_coef[b, a] = v
    except ValueError:
        pass


class Model:
    def __init__(self):
        self.init_weights = np.zeros(len(asset_labels), dtype=np.float)
        self.init_weights[0] = 1
        self.init_weights[:len(asset_labels)] = 1.0/len(asset_labels)

    def extract_weights(self, weights):
        return weights[:len(weights) / 2]

    def bounds(self):
        b = [BOUNDS for i in range(len(self.init_weights))]
        b[0] = CASH_BOUNDS
        return b

    def extract_weights(self, weights):
        return weights

    def get_mean(self, weights):
        factor_loadings = np.dot(weights, assets)
        mean = np.dot(factor_loadings, factor_mean)
        return mean

    def mean_and_std(self, weights):
        factor_loadings = np.dot(weights, assets)
        mean = np.dot(factor_loadings, factor_mean)
        std = np.array(factor_loadings * factor_std, ndmin=2)
        covariance_matrix = np.matmul(std.transpose(), std) * correlation_coef
        variance = np.sum(covariance_matrix)
        std = np.sqrt(variance)
        return mean, std

model = Model()
def find_weights(expected_std):

    w_bound = model.bounds()
    constraints = [
        ({'type': 'eq', 'fun': lambda w: sum(model.extract_weights(w)) - 1.}),      # weights must sum to 1
        ({'type': 'eq', 'fun': lambda w: model.mean_and_std(w)[1] - expected_std})  # std must be equal to expected_std
    ]

    w = scipy.optimize.minimize(
        lambda w: -model.get_mean(w) - np.sum(np.clip(w, -np.inf, 0)) * SHORTING_COST,
        model.init_weights,
        method='SLSQP',
        bounds=w_bound,
        constraints=constraints,
        tol=0.00000000005
    )

    return w.x

X = []
Y = []
asset_ratios = []

for expected_std in np.linspace(X_AXIS[0], X_AXIS[1], 200):
    vars = find_weights(expected_std / 12 **.5)
    weights = model.extract_weights(vars)

    if not (0.999 <= sum(weights) <= 1.001):
        print('fail')
        continue

    asset_ratios.append(weights)
    factor_loadings = np.matmul(weights, assets)
    mean, std = model.mean_and_std(vars)
    # print(expected_std, mean, std, expected_std - std, weights, factor_loadings)
    print("mean: {:.2%}, std: {:.2%}".format(mean * 12, std * 12 ** .5))
    Y.append(mean * 12)
    X.append(std * 12 ** .5)

asset_ratios = np.array(asset_ratios)


def isoeastic_utility(X, Y, gamma):
    return np.array(Y) - .5 * np.power(X, 2) * gamma

def plot_utility_function(X, Y, gamma, label, color):
    U = isoeastic_utility(X, Y, gamma)
    plt.plot(X, U, label=label, color=color)
    max_u = np.max(U)
    mask = max_u == U
    # draw a colored "X" at the maximum
    plt.scatter(np.array(X)[mask], U[mask], color=color, marker='x')



plt.subplots(2, 1, figsize=(8, 7))

plt.subplot('211')
plt.title('efficient frontier, for various coefficients of relative risk aversion')
plt.ylabel('expected utility')
plt.xlabel('risk (annualized stddev)')
plt.grid()
plt.xlim(X_AXIS)
plt.ylim(Y_AXIS)
plt.gca().yaxis.set_major_formatter(StrMethodFormatter('{x:,.0%}'))
plt.gca().xaxis.set_major_formatter(StrMethodFormatter('{x:,.0%}'))

plot_utility_function(X, Y, 0, u'γ = 0', 'C1')
plot_utility_function(X, Y, 0.5, u'γ = 0.5', 'C2')
plot_utility_function(X, Y, 1, u'γ = 1', 'C3')
plot_utility_function(X, Y, 2, u'γ = 2', 'C4')
plot_utility_function(X, Y, 3, u'γ = 3', 'C5')
plot_utility_function(X, Y, 5, u'γ = 5', 'C6')

plt.text(.30, .005, 'γ = 0 is the efficient frontier\nγ = 1 maximizes geometric growth,\nalso known as the kelly criterion',
    bbox=dict(boxstyle="square",
            ec=(.6, .6, .6),
            fc=(1, 1, 1),
            )
)
plt.legend(loc='upper left')


plt.subplot('212')
plt.title('asset weights')
plt.ylabel('asset weight')
plt.xlabel('risk (annualized stddev)')
plt.grid()
plt.xlim(X_AXIS)
plt.ylim((0, 1))
plt.gca().xaxis.set_major_formatter(StrMethodFormatter('{x:,.0%}'))

for i in range(len(assets)):
    plt.plot(X, asset_ratios[:, i], label=asset_labels[i])
plt.legend()

plt.subplots_adjust(hspace=.3)
plt.subplots_adjust(left=.1)
plt.subplots_adjust(right=.9)
plt.subplots_adjust(top=.95)
plt.subplots_adjust(bottom=.08)
plt.show()
chem
Posts: 63
Joined: Sun Apr 12, 2020 7:45 am

Re: A mean variance framework for portfolio optimization

Post by chem »

hooray for python code, nice post
toisvu
Posts: 6
Joined: Sun Mar 17, 2019 2:26 pm

Re: A mean variance framework for portfolio optimization

Post by toisvu »

Uncorrelated wrote: Sat Aug 08, 2020 3:06 pm Hello I'm introducing a mean-variance optimization framework for portfolio choice.
Thank you so so much! The code is easy to follow and it works like a charm (after installing a missing package: python3-tk). Cheers! :sharebeer
jarjarM
Posts: 303
Joined: Mon Jul 16, 2018 1:21 pm

Re: A mean variance framework for portfolio optimization

Post by jarjarM »

Code works great, thanks for sharing.
User avatar
cos
Posts: 234
Joined: Fri Aug 23, 2019 7:34 pm
Location: Boston
Contact:

Re: A mean variance framework for portfolio optimization

Post by cos »

Hooray! It's great to see this. Thanks for posting.

I see you got the correlation data from A Five-Factor Asset Pricing Model (Fama & French, 2014), but where did you get the returns and volatility?
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

cos wrote: Mon Aug 24, 2020 10:13 am Hooray! It's great to see this. Thanks for posting.

I see you got the correlation data from A Five-Factor Asset Pricing Model (Fama & French, 2014), but where did you get the returns and volatility?
I calculated the mean and stddev using Fama/French 5 Factors (2x3) from Ken French website, US version, monthly data.
hdas
Posts: 1342
Joined: Thu Jun 11, 2015 8:24 am

-----

Post by hdas »

-----
Last edited by hdas on Wed Sep 02, 2020 6:52 pm, edited 1 time in total.
....
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

hdas wrote: Mon Aug 24, 2020 1:34 pm
Uncorrelated wrote: Sat Aug 08, 2020 3:06 pm Hello I'm introducing a mean-variance optimization framework for portfolio choice.
Thank you for sharing. A quick question. You include leveraged individual components, but omitted leveraging the whole portfolio. So somebody can have a 50-50, leveraged 1.5x, 2x etc......Is that contemplated and I missed it?. H
That wouldn't make any sense. Although we are not sure why, there is considerable evidence that leveraging equities is more expensive than leveraging treasuries. If a 50/50 portfolio makes sense, then that ratio doesn't necessary make any sense if the entire portfolio was leveraged.

If you would like to perform mean variance optimization with unconstrained leverage, you can change the variables around line 78 to:
BOUNDS = (0, 3)
CASH_BOUNDS = (-2, 1)
SHORTING_COST = .5 / 100 / 12
Which would allow a max leverage of 3x with a presumed margin rate of 0.5% annually. (the shorting cost variable is applied monthly)

Another reason not to use unconstrained leverage is that factor funds are usually very expense or annoying to leverage.
petulant
Posts: 1901
Joined: Thu Sep 22, 2016 1:09 pm

Re: A mean variance framework for portfolio optimization

Post by petulant »

While this is very interesting, the actual graphs still suffer from issues related to backtesting in that they use forward-looking parameters for stocks based on historical performance. I take it that the same tool could be used with different parameters, though?
hdas
Posts: 1342
Joined: Thu Jun 11, 2015 8:24 am

-----

Post by hdas »

-----
Last edited by hdas on Wed Sep 02, 2020 6:52 pm, edited 1 time in total.
....
rkhusky
Posts: 10596
Joined: Thu Aug 18, 2011 8:09 pm

Re: A mean variance framework for portfolio optimization

Post by rkhusky »

And what are the results if small and value premiums are negative for the next 20 years? Or fluctuate but average to zero over the next 50 years?
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

petulant wrote: Mon Aug 24, 2020 3:50 pm While this is very interesting, the actual graphs still suffer from issues related to backtesting in that they use forward-looking parameters for stocks based on historical performance. I take it that the same tool could be used with different parameters, though?
The solutions from this method are only as good as the data you put into it. One advantage of this method is that you can change the assumptions to your liking. For example, if you're not sure about the value premium you can reduce the expected return of the value premium. If you're not sure about bond:stock correlations, you can set them to zero, etc. Ultimately the optimal portfolio depends on your return assumptions.
hdas wrote: Mon Aug 24, 2020 8:30 pm

So, it's possible that your optimizer spits something like 75% Stocks, 75% Bonds as an optimal allocation?. H
Yes that is possible if the assumptions are right. With the borrowing costs assumptions I use, the allocation would probably be 75% stocks (ETF), and 25% leveraged bonds (leveraged ETF or futures). If you want to know, you should run the code and fill in your return assumptions.
rkhusky wrote: Mon Aug 24, 2020 9:03 pm And what are the results if small and value premiums are negative for the next 20 years? Or fluctuate but average to zero over the next 50 years?
What do you mean? If you expect that the small and value premium are negative over the next 20 years, then obviously the correct allocation is in large cap growth.

If you expect that the small and value premium are positive but the realized premiums turn out to be negative, then choosing small and value was still the correct decision. The optimal allocation depends on the probability distribution of outcomes, and should be judged on that. You can't evaluate an investment decision based on it's outcome.
Seasonal
Posts: 2155
Joined: Sun May 21, 2017 1:49 pm

Re: A mean variance framework for portfolio optimization

Post by Seasonal »

Very nice, with the added bonus of Python code.

On mean variance generally, here's Bill Bernstein on the subject of basic mean-variance optimizers: http://www.efficientfrontier.com/ef/198/mvo198.htm and http://www.efficientfrontier.com/ef/497/mvo.htm which he found to be extremely sensitive to the data you put into them and end up picking whatever performed best among the inputs, as you cover in the immediately prior post.
rkhusky
Posts: 10596
Joined: Thu Aug 18, 2011 8:09 pm

Re: A mean variance framework for portfolio optimization

Post by rkhusky »

Uncorrelated wrote: Tue Aug 25, 2020 3:51 am
rkhusky wrote: Mon Aug 24, 2020 9:03 pm And what are the results if small and value premiums are negative for the next 20 years? Or fluctuate but average to zero over the next 50 years?
What do you mean? If you expect that the small and value premium are negative over the next 20 years, then obviously the correct allocation is in large cap growth.

If you expect that the small and value premium are positive but the realized premiums turn out to be negative, then choosing small and value was still the correct decision. The optimal allocation depends on the probability distribution of outcomes, and should be judged on that. You can't evaluate an investment decision based on it's outcome.
So, all the talk about correlations and factor analysis and mean-variance analysis are sort of beside the point. It all comes down to whether you believe that the future premiums will be positive or negative or zero.
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

rkhusky wrote: Tue Aug 25, 2020 6:44 am
Uncorrelated wrote: Tue Aug 25, 2020 3:51 am
rkhusky wrote: Mon Aug 24, 2020 9:03 pm And what are the results if small and value premiums are negative for the next 20 years? Or fluctuate but average to zero over the next 50 years?
What do you mean? If you expect that the small and value premium are negative over the next 20 years, then obviously the correct allocation is in large cap growth.

If you expect that the small and value premium are positive but the realized premiums turn out to be negative, then choosing small and value was still the correct decision. The optimal allocation depends on the probability distribution of outcomes, and should be judged on that. You can't evaluate an investment decision based on it's outcome.
So, all the talk about correlations and factor analysis and mean-variance analysis are sort of beside the point. It all comes down to whether you believe that the future premiums will be positive or negative or zero.
Exactly. The first step is to gather your assumptions, the second step is to determine the best asset allocation with those assumptions (for example with backtesting or mean-variance analysis). If you don't believe that the evidence for value is convincing, then it makes no sense to continue to do portfolio optimization with value. Neither mean-variance analysis or backtests are suitable methods to provide evidence for value, you'll need statistical tests and/or convincing risk-based arguments for that.

Similarly, I don't believe that US expected returns are different than international expected returns. Therefore, any backteset or MVO that attempts to compare US returns with international returns is meaningless (with my beliefs).
User avatar
patrick013
Posts: 3032
Joined: Mon Jul 13, 2015 7:49 pm

Re: A mean variance framework for portfolio optimization

Post by patrick013 »

Uncorrelated wrote: Sat Aug 08, 2020 3:06 pm
Hello I'm introducing a mean-variance optimization framework for portfolio choice.

...


It gives you the optimal allocation directly so you don't have to perform 50 different backtests to find the best one.

My answer to your answer.

Diversification: Is there any way to produce the same returns with less risk?
age in bonds, buy-and-hold, 10 year business cycle
chillpenguin
Posts: 32
Joined: Thu Jun 04, 2020 6:42 pm

Re: A mean variance framework for portfolio optimization

Post by chillpenguin »

This is really awesome, thanks for posting this.

One of the assumptions listed was "You have a CRRA (constant relative risk aversion) utility function, or similar". That's the part I'm stuck on. I currently am 100% stocks, and I've been considering adding in some leverage and/or factor tilts, so this post is right up my alley. But I'm not sure how to determine my CRRA.

How does one estimate their CRRA?

I'm young (late twenties) so I want to take a lot of risk while I have the luxury of time on my side. But I'm not sure how to figure out how much I should dial up the leverage/factors to increase the risks/rewards. I get that this framework here helps you decide on a portfolio once you know your CRRA, but I don't really know hot to determine my CRRA.

Any advice on this would be appreciated!

Thanks!
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

chillpenguin wrote: Tue Sep 01, 2020 1:44 am This is really awesome, thanks for posting this.

One of the assumptions listed was "You have a CRRA (constant relative risk aversion) utility function, or similar". That's the part I'm stuck on. I currently am 100% stocks, and I've been considering adding in some leverage and/or factor tilts, so this post is right up my alley. But I'm not sure how to determine my CRRA.

How does one estimate their CRRA?
This is a question that often turns up in Lifecycle Investing - Leveraging when young. The process is as follows:

1. Determine the asset allocation you plan to use when retired. For example, 60/40. Technically, the present value of social security should be included in the bond portion.

2. Use the following formula to determine your coefficient of relative risk aversion without correcting for human capital:
γ = r / (w * sigma^2)
w is the selected weight in stocks (for a 60/40 asset allocation, this is 0.6). r is the expected return on stocks (I use 0.06), sigma is the expected standard deviation of stocks (I use 0.16). For most users the γ is between 3 and 5.

3. Calculate the approximate asset allocation after correcting for human capital.
w2 = w * (1 + human_capital / liquid_net_worth)
w is the same value as in step 2. Human capital is the sum of all future retirement contributions. Liquid net worth is your current net worth. You can now calculate your corrected γ with the same formula as in step 2:
γ = r / (w2 * sigma^2)

This is the γ that should be used to select your current asset allocation. The theory behind this process is explained in the topic linked above. Basically, this asset allocation results in the highest expected utility when you reach retirement age. If you have other needs then unfortunately there is no straightforward way to determine your γ, so you just have to pick something that looks good.
Seasonal
Posts: 2155
Joined: Sun May 21, 2017 1:49 pm

Re: A mean variance framework for portfolio optimization

Post by Seasonal »

Am I reading this correctly - in order to determine the γ that should be used to select your current asset allocation you have to estimate your future asset allocation? If so, why would someone who is looking for a way to select their current asset allocation be any better at selecting their future asset allocation?
chillpenguin
Posts: 32
Joined: Thu Jun 04, 2020 6:42 pm

Re: A mean variance framework for portfolio optimization

Post by chillpenguin »

Thanks again Uncorrelated!

Seasonal sort of beat me to my next question though. How do I figure out step 1?
Uncorrelated wrote: Tue Sep 01, 2020 4:59 am 1. Determine the asset allocation you plan to use when retired. For example, 60/40. Technically, the present value of social security should be included in the bond portion.
I will check out that lifecycle investing link. Also, every step after step 1 makes a lot of sense, so I feel a lot closer now to being able to apply the mean variance framework.

Lets say I do have a cost of living number in mind for when I retire. Lets also say I have a target age I want to retire by, as well as an estimate for yearly contributions while I am still working.

Given those numbers, is there a straightforward way to determine what my asset allocation should be when I retire? Assuming the goal is to minimize the chances of running out of money while always being able to withdraw the cost of living amount.

Edit: I read the wiki page on Asset Allocation, and it does not answer the question. So far everything I have found on deciding on asset allocation is pretty vague and subjective. I see the whole "depends on ability, willingness, and need to take risk" thing a lot. But I'm looking for something more formulaic (which is what attracted me to this mean variance framework in the first place).
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

Seasonal wrote: Tue Sep 01, 2020 7:00 am Am I reading this correctly - in order to determine the γ that should be used to select your current asset allocation you have to estimate your future asset allocation? If so, why would someone who is looking for a way to select their current asset allocation be any better at selecting their future asset allocation?
Yes, that is correct.

Asking people for their asset allocation in retirement is a trick to make them evaluate their risk aversion with some consistent frame of reference. Even though you might not be very good at determining your risk aversion in retirement, this approach prevents investors from re-evaluating their risk aversion every time they look at their asset allocation.

This post from another topic does a much better job explaining the advantages of this process:
gordoni2 wrote: Determining the relative risk aversion coefficient to use to determine an asset allocation may at first appear to require circular reasoning, but doing so has real value.

First, it translates a decision about acceptable portfolio losses into something far more meaningful, uncertainty in retirement consumption.

Second, it makes the asset allocation decision self consistent over the lifecycle. Without the risk aversion framework you are left guessing what you should do, not once, but at every point in the lifecycle.

It is still a difficult decision to make. For the financial calculator I am currently working on I attempt to help the user decide by showing them the consequences of the RRA decision: for RRA=6 you can get a mean consumption of $50,000 with a standard deviation of $10,000, and this is what the distribution of outcomes looks like; or for RRA=1.5 you can get a mean consumption of $70,000 with a standard deviation of $30,000, and this is what the distribution of outcomes looks like.

Deriving an asset allocation by looking at the long term retirement consumption consequences seems far more meaningful than deriving an asset allocation from the short term psychological consequences of the commonly asked "how would you feel after a 50% drop in your investment portfolio value".


chillpenguin wrote: Tue Sep 01, 2020 1:50 pm Lets say I do have a cost of living number in mind for when I retire. Lets also say I have a target age I want to retire by, as well as an estimate for yearly contributions while I am still working.

Given those numbers, is there a straightforward way to determine what my asset allocation should be when I retire? Assuming the goal is to minimize the chances of running out of money while always being able to withdraw the cost of living amount.
I don't think that's possible, there is no way to avoid specifying your RRA. Investing is always uncertain, no investment plan can guarantee a certain retirement income.

Suppose that you target a retirement income of $20k/year. Your investments did poorly and you only obtained a retirement income of $18k/year. What is the difference in happiness between those different scenario's? Would you rather have a guaranteed 18.5k/year retirement income or 50% chance at either 18k and 20k? To answer these questions you need to know your personal RRA.
chillpenguin
Posts: 32
Joined: Thu Jun 04, 2020 6:42 pm

Re: A mean variance framework for portfolio optimization

Post by chillpenguin »

Uncorrelated wrote: Tue Sep 01, 2020 3:19 pm I don't think that's possible, there is no way to avoid specifying your RRA. Investing is always uncertain, no investment plan can guarantee a certain retirement income.

Suppose that you target a retirement income of $20k/year. Your investments did poorly and you only obtained a retirement income of $18k/year. What is the difference in happiness between those different scenario's? Would you rather have a guaranteed 18.5k/year retirement income or 50% chance at either 18k and 20k? To answer these questions you need to know your personal RRA.
Ok fair enough, I accept that there will be an aspect of subjective choice no matter what.

So to estimate ones CRRA, one must first decide upon their asset allocation they plan to use when retired.

To estimate the retirement asset allocation, one must first estimate their "personal RRA".

So how do I estimate my personal RRA?

Also,
Uncorrelated wrote: Tue Sep 01, 2020 3:19 pm Investing is always uncertain, no investment plan can guarantee a certain retirement income.
I understand nothing is guaranteed, but surely there is a way to minimize the chance of running out of money. Isn't there?

For example, lets say I retire with 500K, and I plan to make 20K withdrawals each year. Is there not a way to determine the AA that minimizes the chance of failure (running out of funds before death)? I would assume a given AA might predict a 50% chance of success, and another AA might predict an 60% chance of success. In which case I would prefer the latter.

Is this not a reasonable way to approach finding ones retirement AA?
User avatar
Steve Reading
Posts: 2518
Joined: Fri Nov 16, 2018 10:20 pm

Re: A mean variance framework for portfolio optimization

Post by Steve Reading »

chillpenguin wrote: Tue Sep 01, 2020 5:02 pm
Uncorrelated wrote: Tue Sep 01, 2020 3:19 pm I don't think that's possible, there is no way to avoid specifying your RRA. Investing is always uncertain, no investment plan can guarantee a certain retirement income.

Suppose that you target a retirement income of $20k/year. Your investments did poorly and you only obtained a retirement income of $18k/year. What is the difference in happiness between those different scenario's? Would you rather have a guaranteed 18.5k/year retirement income or 50% chance at either 18k and 20k? To answer these questions you need to know your personal RRA.
Ok fair enough, I accept that there will be an aspect of subjective choice no matter what.

So to estimate ones CRRA, one must first decide upon their asset allocation they plan to use when retired.

To estimate the retirement asset allocation, one must first estimate their "personal RRA".

So how do I estimate my personal RRA?
It's very simple. Imagine you are a retiree with a nest egg. I will now flip a coin. If it lands heads, you will double that nest egg. If it lands tails, you will lose X% of your nest egg. What is the largest X where you would still be willing to take this bet?

Once you have the number, plug it into the spreadsheet here:
http://www.lifecycleinvesting.net/Resou ... lator.xlsx

And out comes your coefficient of RRA. Voila!

(Note that the spreadsheet asks a similar question, but with a salary. It's ultimately the same question but aimed at those with little savings today, where their salary is basically their main asset. At the end of the day, the question is just asking how much you're willing to lose of all of your current assets, for an equal chance to double them.)
"... so high a present discounted value of wealth, it is only prudent for him to put more into common stocks compared to his present tangible wealth, borrowing if necessary" - Paul Samuelson
dharrythomas
Posts: 1080
Joined: Tue Jun 19, 2007 4:46 pm

Re: A mean variance framework for portfolio optimization

Post by dharrythomas »

I’m relatively simple. We’ve been discussing this for years. MVO will not work for you at the times you need it. As the author said, MVO works best when you know the returns, the variance, and the correlations.

It is an elegant model, full of figures and graphs and you can calculate precisely what the efficient frontier was in the past. You are guessing about future returns, volatility, and correlation. Except during a crisis when often all correlations converge on 1, where everything crashes together. Never forget “All models are wrong, some are useful!”

Don’t fool yourself with the precision of the outputs. They are only as valid as the accuracy of the estimates you plugged into the models. This is subjective judgment that is often mistaken for objective because the output is displayed with precise math.

The best way to approach this is to slightly over save, set a conservative AA, and withdrawal rate, and be flexible in adjusting as you go along.
User avatar
patrick013
Posts: 3032
Joined: Mon Jul 13, 2015 7:49 pm

Re: A mean variance framework for portfolio optimization

Post by patrick013 »

Which is exactly why single beta AA's can and
should prevail.

They make delightful efficient frontiers. :)
age in bonds, buy-and-hold, 10 year business cycle
User avatar
grabiner
Advisory Board
Posts: 28296
Joined: Tue Feb 20, 2007 11:58 pm
Location: Columbia, MD

Re: A mean variance framework for portfolio optimization

Post by grabiner »

Uncorrelated wrote: Tue Sep 01, 2020 4:59 am
chillpenguin wrote: Tue Sep 01, 2020 1:44 am This is really awesome, thanks for posting this.

One of the assumptions listed was "You have a CRRA (constant relative risk aversion) utility function, or similar". That's the part I'm stuck on. I currently am 100% stocks, and I've been considering adding in some leverage and/or factor tilts, so this post is right up my alley. But I'm not sure how to determine my CRRA.

How does one estimate their CRRA?
This is a question that often turns up in Lifecycle Investing - Leveraging when young. The process is as follows:

1. Determine the asset allocation you plan to use when retired. For example, 60/40. Technically, the present value of social security should be included in the bond portion.

2. Use the following formula to determine your coefficient of relative risk aversion without correcting for human capital:
γ = r / (w * sigma^2)
w is the selected weight in stocks (for a 60/40 asset allocation, this is 0.6). r is the expected return on stocks (I use 0.06), sigma is the expected standard deviation of stocks (I use 0.16). For most users the γ is between 3 and 5.

3. Calculate the approximate asset allocation after correcting for human capital.
w2 = w * (1 + human_capital / liquid_net_worth)
w is the same value as in step 2. Human capital is the sum of all future retirement contributions. Liquid net worth is your current net worth. You can now calculate your corrected γ with the same formula as in step 2:
γ = r / (w2 * sigma^2)

This is the γ that should be used to select your current asset allocation. The theory behind this process is explained in the topic linked above. Basically, this asset allocation results in the highest expected utility when you reach retirement age. If you have other needs then unfortunately there is no straightforward way to determine your γ, so you just have to pick something that looks good.
One problem with this model is that it assumes the human capital is constant. If your human capital is correlated with your investments (that is, you may lose your job or have your salary decline in a downturn), then you should be more conservative than this model indicates. And even if your human capital is not correlated with your investments, it has its own risks, such as disability and eldercare expenses.
Wiki David Grabiner
rkhusky
Posts: 10596
Joined: Thu Aug 18, 2011 8:09 pm

Re: A mean variance framework for portfolio optimization

Post by rkhusky »

chillpenguin wrote: Tue Sep 01, 2020 5:02 pm
Uncorrelated wrote: Tue Sep 01, 2020 3:19 pm Investing is always uncertain, no investment plan can guarantee a certain retirement income.
I understand nothing is guaranteed, but surely there is a way to minimize the chance of running out of money. Isn't there?

For example, lets say I retire with 500K, and I plan to make 20K withdrawals each year. Is there not a way to determine the AA that minimizes the chance of failure (running out of funds before death)? I would assume a given AA might predict a 50% chance of success, and another AA might predict an 60% chance of success. In which case I would prefer the latter.

Is this not a reasonable way to approach finding ones retirement AA?
One can see how certain AA's did in the past using a tool like FIRECalc, but no one knows what will happen in the future. FIRECalc provides numbers like "in 50% of past 30-year periods your portfolio would not be depleted using the given AA and withdrawal rate".

Most often, the best FIRECalc portfolios are 100% stock. Do you have the stomach for that? Are you willing to gamble your retirement on the future being sufficiently close to the past?

If all you need is $20K nominal, then your portfolio will last 25 years if you keep your portfolio in cash. If you invest in CD's and savings accounts you should get a little growth, so maybe you make it to 30 years if interest rates increase some. If you need $20K real, then you need an investment that keeps up with inflation. If you need more than CD's and savings accounts can provide, you probably need to go with some stocks.
User avatar
Steve Reading
Posts: 2518
Joined: Fri Nov 16, 2018 10:20 pm

Re: A mean variance framework for portfolio optimization

Post by Steve Reading »

grabiner wrote: Tue Sep 01, 2020 8:39 pm
Uncorrelated wrote: Tue Sep 01, 2020 4:59 am
chillpenguin wrote: Tue Sep 01, 2020 1:44 am This is really awesome, thanks for posting this.

One of the assumptions listed was "You have a CRRA (constant relative risk aversion) utility function, or similar". That's the part I'm stuck on. I currently am 100% stocks, and I've been considering adding in some leverage and/or factor tilts, so this post is right up my alley. But I'm not sure how to determine my CRRA.

How does one estimate their CRRA?
This is a question that often turns up in Lifecycle Investing - Leveraging when young. The process is as follows:

1. Determine the asset allocation you plan to use when retired. For example, 60/40. Technically, the present value of social security should be included in the bond portion.

2. Use the following formula to determine your coefficient of relative risk aversion without correcting for human capital:
γ = r / (w * sigma^2)
w is the selected weight in stocks (for a 60/40 asset allocation, this is 0.6). r is the expected return on stocks (I use 0.06), sigma is the expected standard deviation of stocks (I use 0.16). For most users the γ is between 3 and 5.

3. Calculate the approximate asset allocation after correcting for human capital.
w2 = w * (1 + human_capital / liquid_net_worth)
w is the same value as in step 2. Human capital is the sum of all future retirement contributions. Liquid net worth is your current net worth. You can now calculate your corrected γ with the same formula as in step 2:
γ = r / (w2 * sigma^2)

This is the γ that should be used to select your current asset allocation. The theory behind this process is explained in the topic linked above. Basically, this asset allocation results in the highest expected utility when you reach retirement age. If you have other needs then unfortunately there is no straightforward way to determine your γ, so you just have to pick something that looks good.
One problem with this model is that it assumes the human capital is constant. If your human capital is correlated with your investments (that is, you may lose your job or have your salary decline in a downturn), then you should be more conservative than this model indicates. And even if your human capital is not correlated with your investments, it has its own risks, such as disability and eldercare expenses.
Some studies show that people's income are, for the most part, basically uncorrelated with the market (except for some specific ones like bankruptcy lawyers or wallstreet asset managers).

Moreover, most of us need and will save. Whether I use my current job, or I switch jobs, or I get fired and find another, or I move, etc etc I will accumulate a nest egg one way or another. In other words, "human capital" isn't just your current salary at your current job. It is the future savings contributions that you will produce through your labor. To the extent you have decided you will produce those savings come hell or high water, then human capital is very much a "safe asset".

Of course, like you say, it makes sense to account for your career path's correlation to the market to some extent. My point is that it might be much more uncorrelated to stocks than many might initially think.
"... so high a present discounted value of wealth, it is only prudent for him to put more into common stocks compared to his present tangible wealth, borrowing if necessary" - Paul Samuelson
langlands
Posts: 611
Joined: Wed Apr 03, 2019 10:05 pm

Re: A mean variance framework for portfolio optimization

Post by langlands »

chillpenguin wrote: Tue Sep 01, 2020 5:02 pm
Uncorrelated wrote: Tue Sep 01, 2020 3:19 pm I don't think that's possible, there is no way to avoid specifying your RRA. Investing is always uncertain, no investment plan can guarantee a certain retirement income.

Suppose that you target a retirement income of $20k/year. Your investments did poorly and you only obtained a retirement income of $18k/year. What is the difference in happiness between those different scenario's? Would you rather have a guaranteed 18.5k/year retirement income or 50% chance at either 18k and 20k? To answer these questions you need to know your personal RRA.
Ok fair enough, I accept that there will be an aspect of subjective choice no matter what.

So to estimate ones CRRA, one must first decide upon their asset allocation they plan to use when retired.

To estimate the retirement asset allocation, one must first estimate their "personal RRA".

So how do I estimate my personal RRA?

Also,
Uncorrelated wrote: Tue Sep 01, 2020 3:19 pm Investing is always uncertain, no investment plan can guarantee a certain retirement income.
I understand nothing is guaranteed, but surely there is a way to minimize the chance of running out of money. Isn't there?

For example, lets say I retire with 500K, and I plan to make 20K withdrawals each year. Is there not a way to determine the AA that minimizes the chance of failure (running out of funds before death)? I would assume a given AA might predict a 50% chance of success, and another AA might predict an 60% chance of success. In which case I would prefer the latter.

Is this not a reasonable way to approach finding ones retirement AA?
Uncorrelated actually has a topic I think that specifically addresses such a utility function: viewtopic.php?f=10&t=293469.

I'm actually a little confused now on the relationship between this kind of "cutoff" utility function and the CRRA class of utility functions. I guess it has something to do with how ambitious your "cutoff" is. If you want to retire living the high life, you'll be forced to take on more risk and your gamma will be smaller. If you have less ostentatious material wants, you can afford more risk aversion (higher gamma). You'll need a pretty good estimate of market returns to get this mapping between the two types of utilities though.

On second thought, this kind of utility function has some rather undesirable properties though so you should think carefully about if it's what you want. If you've "won the game" as many people here like to say, then this utility function will tell you to sensibly reduce risk substantially since you have more than you need in retirement. But if you're behind and retirement is approaching, are you really going to risk it all to maximize the probability you can get your 20K per year? So you need to think carefully about how bad 19K is, how bad 18K is etc. Such is the basic plight of not being rich. There are no guarantees and you have to make do.
Last edited by langlands on Tue Sep 01, 2020 11:55 pm, edited 2 times in total.
User avatar
Gort
Posts: 641
Joined: Sat Mar 17, 2007 5:07 pm

Re: A mean variance framework for portfolio optimization

Post by Gort »

Deleted
Last edited by Gort on Wed Sep 02, 2020 6:14 am, edited 1 time in total.
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

chillpenguin wrote: Tue Sep 01, 2020 5:02 pm
Uncorrelated wrote: Tue Sep 01, 2020 3:19 pm Investing is always uncertain, no investment plan can guarantee a certain retirement income.
I understand nothing is guaranteed, but surely there is a way to minimize the chance of running out of money. Isn't there?

For example, lets say I retire with 500K, and I plan to make 20K withdrawals each year. Is there not a way to determine the AA that minimizes the chance of failure (running out of funds before death)? I would assume a given AA might predict a 50% chance of success, and another AA might predict an 60% chance of success. In which case I would prefer the latter.

Is this not a reasonable way to approach finding ones retirement AA?
That's possible, I calculated that specific asset allocation in this post (see the first image). However, the utility function "minimize the probability of running out of money" is very crude and unlikely to map to actual investor risk preferences. If it is your goal to retire with an annual spending of $20k/year and poor market returns make that goal unattainable by the time you retire, then you will probably reconsider that goal. It's not very useful to optimize for a goal that is likely to be changed, I didn't realize that until after I made the topic above.


The work of Gordon Irlam (https://www.aacalc.com/about) uses utility functions to select the right asset allocation in retirement. I highly recommend looking at some of the figures in Floor and Upside Investing in Retirement with Nominal SPIAs to get an idea how the asset allocation changes over time. Unfortunately his work only focuses on the asset allocation in retirement, he doesn't say which asset allocation should be used before you retire.
chillpenguin
Posts: 32
Joined: Thu Jun 04, 2020 6:42 pm

Re: A mean variance framework for portfolio optimization

Post by chillpenguin »

Thanks everybody this is all super helpful. I checked out those links posted and I have a much better idea how this all works now. It is quite fascinating how something can appear so objective on the one hand, but ultimately really does come down to subjective preferences, and those subjective preferences can be fairly complicated to figure out!

Overall I think I have a rough estimate of my RRA now. Thanks again everyone!
chillpenguin
Posts: 32
Joined: Thu Jun 04, 2020 6:42 pm

Re: A mean variance framework for portfolio optimization

Post by chillpenguin »

Hey Uncorrelated,

I defined 2 assets like so:

Code: Select all

ntsx_asset = asset_def('NTSX', ter=.2, mkt=0.9, ltt=0.3, itt=.3, smb=-.16 * 0.9)
mkt_asset = asset_def('total stock market', ter=0.03, mkt=1)
Here are the relevant return assumptions:

Code: Select all

    ['TER', -1 / 12.0, 0],
    ['MKT', .53, 4.44],
    ['SmB', .21 / 3, 3.02], 
    ['ITT', 1.66 / 12, 5.67 / 12**.5],
    ['LTT', 2.01 / 12, 9.79 / 12**.5],
When I graph it, I get an interesting result. NTSX is preferred right up until the riskiest preference, where all of a sudden TSM is preferred.

I can't figure out why that would be. The differences between the funds (as I have it here) are that NTSX has an extra 0.3 on both ltt and itt, and only 0.9 on mkt instead of 1.

The return assumption for mkt is .53 (6.36% per year), and the return for ITT and LTT is 1.66% and 2.01% per year, respectively.

My question is, why is it that TSM would be preferred in the riskiest scenarios, and NTSX preferred otherwise? I would think that the leverage component of NTSX would make it riskier. Wouldn't a crash in stocks and bonds cause a larger drop in NTSX due to the leverage aspect? Also wouldn't the additional 0.6 on the bond side increase returns in general despite the minus 0.1 on the mkt side?

I'm thinking maybe it has to do with the volatility of mkt compared to itt and ltt. 0.1 mkt can perhaps return more than 0.3 ltt + 0.3 itt with the given std values? Also, forgive my ignorance, but what is the math formula for calculating the distribution of possible returns given the monthly arithmetic return and the monthly std. If I knew how to do that I could verify my hypothesis myself without asking.

Let me know if I'm thinking this through correctly. I guess I might have answered my own question, but I want to double check that my reasoning is solid.

Thanks!

Note: I make no claims that the coefficients I used make up an accurate representation of the actual fund NTSX. I was just playing around.
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

chillpenguin wrote: Sat Sep 05, 2020 11:55 pm Hey Uncorrelated,

Sorry for the delay. Let's have a closer look at the results we get from your assumptions:

Image

We can observe from the bottom pane that 100% NTSX has an expected volatility around 14%, and 100% TSM has an expected volatility of around 16%.

In the top pane, we see that the expected return actually decreases if we target more than 14% volatility. For users with normal risk preferences (γ >= 0), it is better to have 100% NTSX than 100% TSM. Not shown in the image above, individuals with γ = -1 (risk-loving) would prefer 100% TSM to NTSX.
chillpenguin
Posts: 32
Joined: Thu Jun 04, 2020 6:42 pm

Re: A mean variance framework for portfolio optimization

Post by chillpenguin »

Hey thanks for getting back to me. I think I mostly figured it out, but you pointed out some things just now that I hadn't considered.

Good point about how the top pane shows us that expected returns decrease past a certain point, and how any investor with γ >= 0 would prefer 100% NTSX over 100% TSM. I also hadn't considered that γ could be negative. I was thinking it would asymptotically prefer more risk as it approaches 0.
Rad Dad
Posts: 30
Joined: Fri Aug 09, 2019 9:48 pm

Re: A mean variance framework for portfolio optimization

Post by Rad Dad »

chillpenguin wrote: Thu Sep 24, 2020 12:49 am Hey thanks for getting back to me. I think I mostly figured it out, but you pointed out some things just now that I hadn't considered.

Good point about how the top pane shows us that expected returns decrease past a certain point, and how any investor with γ >= 0 would prefer 100% NTSX over 100% TSM. I also hadn't considered that γ could be negative. I was thinking it would asymptotically prefer more risk as it approaches 0.
Hi Uncorrelated. I know that earlier in this thread you had the optimization framework for TSM, UPRO, and TMF. After seeing the TSM/NTSX comparison I was wondering if you could show the optimization framework for NTSX, UPRO, and TMF?
chillpenguin
Posts: 32
Joined: Thu Jun 04, 2020 6:42 pm

Re: A mean variance framework for portfolio optimization

Post by chillpenguin »

Rad Dad wrote: Sun Oct 04, 2020 5:16 pm
chillpenguin wrote: Thu Sep 24, 2020 12:49 am Hey thanks for getting back to me. I think I mostly figured it out, but you pointed out some things just now that I hadn't considered.

Good point about how the top pane shows us that expected returns decrease past a certain point, and how any investor with γ >= 0 would prefer 100% NTSX over 100% TSM. I also hadn't considered that γ could be negative. I was thinking it would asymptotically prefer more risk as it approaches 0.
Hi Uncorrelated. I know that earlier in this thread you had the optimization framework for TSM, UPRO, and TMF. After seeing the TSM/NTSX comparison I was wondering if you could show the optimization framework for NTSX, UPRO, and TMF?
I'm not Uncorrelated, but since I got this set up I don't mind running it for you. Here is what I got:

Image

Let me know if you want me to use different numbers. After all, we can enter in any assumptions we want.

Here are the numbers I used:

NTSX:
expense ratio: 0.2
market exposure (beta): 0.9
long term treasury exposure: 0.3
intermediate term treasury exposure: 0.3

UPRO:
expense ratio: 2.0
market exposure (beta): 3.0
size factor (SmB): -.16 * 3

Note: I used SmB premium of 1/3 of the historical premium to be conservative

TMF:
expense ratio: 1.1
long term treasury exposure: 3.0
Rad Dad
Posts: 30
Joined: Fri Aug 09, 2019 9:48 pm

Re: A mean variance framework for portfolio optimization

Post by Rad Dad »

chillpenguin wrote: Mon Oct 05, 2020 10:49 pm
Rad Dad wrote: Sun Oct 04, 2020 5:16 pm
chillpenguin wrote: Thu Sep 24, 2020 12:49 am Hey thanks for getting back to me. I think I mostly figured it out, but you pointed out some things just now that I hadn't considered.

Good point about how the top pane shows us that expected returns decrease past a certain point, and how any investor with γ >= 0 would prefer 100% NTSX over 100% TSM. I also hadn't considered that γ could be negative. I was thinking it would asymptotically prefer more risk as it approaches 0.
Hi Uncorrelated. I know that earlier in this thread you had the optimization framework for TSM, UPRO, and TMF. After seeing the TSM/NTSX comparison I was wondering if you could show the optimization framework for NTSX, UPRO, and TMF?
I'm not Uncorrelated, but since I got this set up I don't mind running it for you. Here is what I got:

Image

Let me know if you want me to use different numbers. After all, we can enter in any assumptions we want.

Here are the numbers I used:

NTSX:
expense ratio: 0.2
market exposure (beta): 0.9
long term treasury exposure: 0.3
intermediate term treasury exposure: 0.3

UPRO:
expense ratio: 2.0
market exposure (beta): 3.0
size factor (SmB): -.16 * 3

Note: I used SmB premium of 1/3 of the historical premium to be conservative

TMF:
expense ratio: 1.1
long term treasury exposure: 3.0
That’s great. Thank you very much!
User avatar
cos
Posts: 234
Joined: Fri Aug 23, 2019 7:34 pm
Location: Boston
Contact:

Re: A mean variance framework for portfolio optimization

Post by cos »

In your opinion, should factors be discounted? Why or why not? I'm not understanding the motivations there.

Regardless, if so, looking at the second-to-last graph, would the following be optimal for an investor with γ = 1 and desire for both international and factor exposure?
60% UPRO / 20% TMF / (10% AVDE / 5% AVEM) / 5% AVDV

Is that even sufficiently internationally diversified? Should I overweight emerging markets?

Are there better funds available than the ones I chose? If you were a γ = 1 investor, what would your portfolio look like?

Sorry for a second round of questions, by the way. I'm just dying to pick your brain. Again, thanks in advance for whatever advice you have!
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

cos wrote: Thu Oct 15, 2020 2:30 am In your opinion, should factors be discounted? Why or why not? I'm not understanding the motivations there.

Regardless, if so, looking at the second-to-last graph, would the following be optimal for an investor with γ = 1 and desire for both international and factor exposure?
60% UPRO / 20% TMF / (10% AVDE / 5% AVEM) / 5% AVDV

Is that even sufficiently internationally diversified? Should I overweight emerging markets?

Are there better funds available than the ones I chose? If you were a γ = 1 investor, what would your portfolio look like?

Sorry for a second round of questions, by the way. I'm just dying to pick your brain. Again, thanks in advance for whatever advice you have!
In my opinion you should discount factors. The motivation is that most factors realize lower premiums after publication, on average around 50% lower. But you can always argue for a lower discount if you believe value is sufficiently battle-tested. You can also argue for a higher discount if you think the underlying drivers of value are likely to be arbitraged away.

For international diversification, I would always start with market cap weight. There simply is no compelling reason to over-weight any geographic region. You should over-weight geographic regions if there is strong evidence they have higher expected returns, the available evidence simply doesn't hold up to academic standards.

If I was an investor with γ = 1, I would probably allocate my real money towards VFMF and an international factor fund, and then use options to leverage the rest of the market. The optimal leverage appears to be around 2x, so in practical terms that would be ~40% VFMF, ~40% some international factor fund, 120% VT (vanguard total world stock, with options). The numbers can be adjusted depending on how much cash collateral you are comfortable with.

Image
with 50% factor discount. Adding TMF, IJS or IUSV doesn't meaningfully alter the results in the γ = 1 region.

An alternative approach is to leverage the factor funds themselves, but that doesn't appear to be doable at low costs.

Just in case you're wondering about the tax implication: I don't know, my country has a wealth tax instead of dividend/capital gains tax. This greatly simplifies tax decisions.
User avatar
cos
Posts: 234
Joined: Fri Aug 23, 2019 7:34 pm
Location: Boston
Contact:

Re: A mean variance framework for portfolio optimization

Post by cos »

Under what license are you releasing that code in the OP? I'd like to make some modifications and share them on GitHub, but I'd rather not do so without your explicit permission.
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

cos wrote: Fri Oct 30, 2020 5:52 pm Under what license are you releasing that code in the OP? I'd like to make some modifications and share them on GitHub, but I'd rather not do so without your explicit permission.
I dedicate this code snippet to the public domain. Feel free to copy, modify and publish it in any way you see fit.
RovenSkyfall
Posts: 104
Joined: Wed Apr 01, 2020 11:40 am

Re: A mean variance framework for portfolio optimization

Post by RovenSkyfall »

When you calculate the factor premium is that just the calculated long SCV? Isnt the FF premia long Small cap short Large cap and long value and short growth? Is there data to suggest that just long SCV also has a premium? That is a question I have been wondering recently.
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

RovenSkyfall wrote: Tue Nov 17, 2020 4:10 pm When you calculate the factor premium is that just the calculated long SCV? Isnt the FF premia long Small cap short Large cap and long value and short growth? Is there data to suggest that just long SCV also has a premium? That is a question I have been wondering recently.
The factor regression is calculated on the long-short factors, that's the way everybody does it.

There are a few papers investigating long-only factors, e.g. When Equity Factors Drop Their Shorts. I also like the analysis provided in Robert Novy-Marx Quality Investing. These results seem to indicate long-only funds are capable of capturing a premium.
RovenSkyfall
Posts: 104
Joined: Wed Apr 01, 2020 11:40 am

Re: A mean variance framework for portfolio optimization

Post by RovenSkyfall »

Uncorrelated wrote: Tue Nov 17, 2020 5:19 pm
RovenSkyfall wrote: Tue Nov 17, 2020 4:10 pm When you calculate the factor premium is that just the calculated long SCV? Isnt the FF premia long Small cap short Large cap and long value and short growth? Is there data to suggest that just long SCV also has a premium? That is a question I have been wondering recently.
The factor regression is calculated on the long-short factors, that's the way everybody does it.

There are a few papers investigating long-only factors, e.g. When Equity Factors Drop Their Shorts. I also like the analysis provided in Robert Novy-Marx Quality Investing. These results seem to indicate long-only funds are capable of capturing a premium.
To clarify -- ETFs that most people hold like IJS are just long SCV, correct?
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

RovenSkyfall wrote: Tue Nov 17, 2020 9:59 pm
Uncorrelated wrote: Tue Nov 17, 2020 5:19 pm
RovenSkyfall wrote: Tue Nov 17, 2020 4:10 pm When you calculate the factor premium is that just the calculated long SCV? Isnt the FF premia long Small cap short Large cap and long value and short growth? Is there data to suggest that just long SCV also has a premium? That is a question I have been wondering recently.
The factor regression is calculated on the long-short factors, that's the way everybody does it.

There are a few papers investigating long-only factors, e.g. When Equity Factors Drop Their Shorts. I also like the analysis provided in Robert Novy-Marx Quality Investing. These results seem to indicate long-only funds are capable of capturing a premium.
To clarify -- ETFs that most people hold like IJS are just long SCV, correct?

Yes, most funds are long-only. There are a few long-short funds (sometimes called market neutral funds), but they tend to be expensive.
RovenSkyfall
Posts: 104
Joined: Wed Apr 01, 2020 11:40 am

Re: A mean variance framework for portfolio optimization

Post by RovenSkyfall »

Uncorrelated wrote: Wed Nov 18, 2020 2:30 pm
RovenSkyfall wrote: Tue Nov 17, 2020 9:59 pm
Uncorrelated wrote: Tue Nov 17, 2020 5:19 pm
RovenSkyfall wrote: Tue Nov 17, 2020 4:10 pm When you calculate the factor premium is that just the calculated long SCV? Isnt the FF premia long Small cap short Large cap and long value and short growth? Is there data to suggest that just long SCV also has a premium? That is a question I have been wondering recently.
The factor regression is calculated on the long-short factors, that's the way everybody does it.

There are a few papers investigating long-only factors, e.g. When Equity Factors Drop Their Shorts. I also like the analysis provided in Robert Novy-Marx Quality Investing. These results seem to indicate long-only funds are capable of capturing a premium.
To clarify -- ETFs that most people hold like IJS are just long SCV, correct?

Yes, most funds are long-only. There are a few long-short funds (sometimes called market neutral funds), but they tend to be expensive.
And this is why you discounted by a factor of 2? And hence why they are underwheighted when LETFs are available in your mean variance portfolios?

Additionally, can you explain why VFMF is a more efficient combination of value and small value?

Thank you greatly for all of your explanations.
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

RovenSkyfall wrote: Wed Nov 18, 2020 3:46 pm

And this is why you discounted by a factor of 2? And hence why they are underwheighted when LETFs are available in your mean variance portfolios?

Additionally, can you explain why VFMF is a more efficient combination of value and small value?

Thank you greatly for all of your explanations.
I discount by a factor of 2 purely to account for overfitting. I expect factor returns to be lower in the future.

VFMF isn't necessary more efficient. But I think you should invest in both large value and small value. VFMF is the only single ETF that fits that bill as it's 1/3 small, 1/3 mid and 1/3 large. If you want to invest in both large value and small cap value with other ETF's, you'll need two funds instead of one.
User avatar
cos
Posts: 234
Joined: Fri Aug 23, 2019 7:34 pm
Location: Boston
Contact:

Re: A mean variance framework for portfolio optimization

Post by cos »

Uncorrelated wrote: Thu Oct 15, 2020 3:57 am If I was an investor with γ = 1, I would probably allocate my real money towards VFMF and an international factor fund, and then use options to leverage the rest of the market. The optimal leverage appears to be around 2x, so in practical terms that would be ~40% VFMF, ~40% some international factor fund, 120% VT (vanguard total world stock, with options). The numbers can be adjusted depending on how much cash collateral you are comfortable with.
Alright, so when I run the simulation myself, I see an allocation of ~55% VFMF / ~45% UPRO in the γ = 1 region (this is also visible in the plot you shared, quoted below). Therefore, wouldn't the optimal allocation with options be more like ~30% VFMF, ~30% international equivalent, ~140% VT? Out of curiosity, if not from this simulation, how did you decide that a 3:2 market-to-multifactor ratio is optimal?

See:
Image
Uncorrelated wrote: Thu Oct 15, 2020 3:57 am Image
with 50% factor discount. Adding TMF, IJS or IUSV doesn't meaningfully alter the results in the γ = 1 region.
Again, after running the code myself, I'm seeing something different. I get a ~15% allocation to TMF in the γ = 1 region. Is that really insignificant? Are you discounting TMF for some other reason?

Also, interestingly, it seems the market-to-multifactor ratio drifts even further away from 3:2, approaching 4:1 with the addition of TMF. Why do you think this might be? Is it a result of factor correlations?

See:
Image
User avatar
Topic Author
Uncorrelated
Posts: 1147
Joined: Sun Oct 13, 2019 3:16 pm

Re: A mean variance framework for portfolio optimization

Post by Uncorrelated »

cos wrote: Wed Dec 02, 2020 6:28 pm Alright, so when I run the simulation myself, I see an allocation of ~55% VFMF / ~45% UPRO in the γ = 1 region (this is also visible in the plot you shared, quoted below). Therefore, wouldn't the optimal allocation with options be more like ~30% VFMF, ~30% international equivalent, ~140% VT? Out of curiosity, if not from this simulation, how did you decide that a 3:2 market-to-multifactor ratio is optimal?
Well, that's easy: I used 80% of my real money to purchase factor funds, and the remaining 20% to purchase some highly leveraged options. If I would have purchased more factor funds, I wouldn't have any money left to purchase options :happy

Of course you can use portfolio margin, but I'm not a big fan of portfolio margin.
Again, after running the code myself, I'm seeing something different. I get a ~15% allocation to TMF in the γ = 1 region. Is that really insignificant? Are you discounting TMF for some other reason?
With TMF, the optimal allocation for γ = 1 results in a CER of 7.86%. Without TMF, the CER is 7.78%. I don't think it's worth juggling one more fund for such a small increase in CER.

I also think there are some reasons to expect at slightly lower equity and term premium in the future. And TMF is a lot more sensitive to estimation errors than leveraged equity because the expense ratio is larger relative to the expected return. The majority of this argument is based on gut feeling instead of hard mathematics, so take this with a pinch of salt.


Also, interestingly, it seems the market-to-multifactor ratio drifts even further away from 3:2, approaching 4:1 with the addition of TMF. Why do you think this might be? Is it a result of factor correlations?
I'm not sure, but I suspect the marginal benefit of factors decrease as more factors are added. And TERM is quite similar in strength as SmB and HmL.
Post Reply