140% Stocks (or why the Kelly criterion is cool)

[TradingPlaces]

...It is well known that the off-diagonal entries in that matrix are weakly negative, i.e., that bond-stock correlation is negative, at least historically...

Is it really "well known?"

Admittedly my trusty Ibbotson 2010 SBBI Classic Yearbook gets less and less trusty every year and I'm too cheap to buy a new one, but according to tables 6-2 and 6-4, p. 79, for 1926-2009, the correlations are as follows. "Large Company Stocks" is the S&P 500 and (supposedly comparable) predecessors. I'm going to bold "long-term government bonds" because that's typically what people cite for "bonds."

Inflation-adjusted series (table 6-4)

Correlation of large company stocks with:
--Long-term corporate bonds, +0.23
--Long-term government bonds, +0.11
--Intermediate-term government bonds, +0.08

Basic series (i.e. nominal) (table 6-2)

Correlation of large company stocks with:--Long-term corporate bonds, +0.17
--Long-term government bonds, +0.03
--Intermediate-term government bonds, -0.01

I still have not figured out where the frequently claims of negative correlation actually come from. As market timer suggests, from recent short term data series that include 2008-2009, maybe.

You got me. Anyway, that's not the key point. I think a lot of us here are missing the forest for the trees.

Like I said, make HISTORICALLY forward looking lookups: that's your edge.

But go and construct the kelly criterion trading strategy, in a spreadsheet.

[TradingPlaces]

...It is well known that the off-diagonal entries in that matrix are weakly negative, i.e., that bond-stock correlation is negative, at least historically...

Is it really "well known?"

Admittedly my trusty Ibbotson 2010 SBBI Classic Yearbook gets less and less trusty every year and I'm too cheap to buy a new one, but according to tables 6-2 and 6-4, p. 79, for 1926-2009, the correlations are as follows. "Large Company Stocks" is the S&P 500 and (supposedly comparable) predecessors. I'm going to bold "long-term government bonds" because that's typically what people cite for "bonds."

Inflation-adjusted series (table 6-4)

Correlation of large company stocks with:
--Long-term corporate bonds, +0.23
--Long-term government bonds, +0.11
--Intermediate-term government bonds, +0.08

Basic series (i.e. nominal) (table 6-2)

Correlation of large company stocks with:--Long-term corporate bonds, +0.17
--Long-term government bonds, +0.03
--Intermediate-term government bonds, -0.01

I still have not figured out where the frequently claims of negative correlation actually come from. As market timer suggests, from recent short term data series that include 2008-2009, maybe.

How is that correlation constructed? Daily?

If bonds and stocks both tend to go up over time, than the longer the correlation estimate unit window (daily to weekly to monthly to annual), then the more the longer positive return rate of both assets affect the correlation.

I think the correct covariance matrix to enter is the one that corresponds to your expected returns source. But I don't know what is the best practice in this respect.

[555]

I think the OP's main point is that "Kelly criterion is cool" rather than how to apply it in the real world.

[lack_ey]

But the point that the wrong assumptions and applications can lead to entirely wrong intuitions is an important one to explore.

[TradingPlaces]

nisiprius: here is a question for you.

Let's say that the CORRECT, forward-looking correlation between the two risk assets: stock index and bond index, is +20%.

However, you take my word for it, and enter -20%.

How would the Kelly criterion solution change?

I think by solving these types of variations one can get better understanding of what the Kelly criterion does.

Here is my intuition.

Let's say we are fooled, and instead of having 2 risky assets, we have one, as follows. The supposedly 2 risky assets: S and B, are in fact identical. Thus, the aggregate Kelly bet on both of these should be the same as using one risky asset. E.g., if Kelly Says bet 140% on Stocks, then any combination of U_1, U_2 such that U_1 + U_2 adds to 140% should be a solution.

Now, let's imagine a completely out of this world scenario: S and B have identical variance, both have positive returns, but are 100% negatively correlated. I think at the limit, Kelly would make a very very large leveraged bet. In particular, if S and B had identical returns and identical variance, but -100% correlation, then Kelly should bet an infinite amount on S and an infinite amount on B, because, well, you can't lose by betting the exact same amount on both.

Thus, if you input a lower correlation than the true correlation SHOULD be, then you make LARGER bet than you should.

In particular, if you believe me and enter -20% for correlation between S and B, your bets will come out LARGER.
If you input +20% for correlation, then your bets will come out SMALLER, relative to the previous sentence.

[TradingPlaces]

I think the last paragraph of OPs second post (the one that the original post says, "read on to find out") was essentially suggesting that 100% stock investors are closer to the Kelly. That, I think is an attempt to a real application to the Kelly criterion.

[JoMoney]

I think the OP's main point is that "Kelly criterion is cool" rather than how to apply it in the real world.

That's my belief. Along with the whole Markowitz mean-variance/Modern Portfolio theory. The premises required for a lot of the math to work with the stock market are false, so while it's interesting, trying to implement it with any kind of precision is just silly. The useful takeaway though, is as Backpacker said - that the riskiest portfolio or "bet" isn't necessarily the way to maximize returns, and it may in fact be worse than less risky options.
I like the illustration from this article:

This particular scenario is contrived, but does demonstrate potentially real scenarios. The picture demonstrates the outcomes of several portfolios of different Cash/Stock positions. Under this simulation, the "optimized" portfolio of 18% Cash / 82% Stock has nearly the same return as the 100% stock position, whereas the levered -100 Cash /200% Stock position was an utter failure despite the overall positive results for stocks.

http://www.cfapubs.org/doi/pdf/10.2469/cp.v2004.n2.3379
...the leveraged portfolio performs dismally. Why? Although it does have the highest monthly expected return, it has such high variability that the likelihood of losing a substantial proportion of capital is so high that it likely will cause the investor to rebuild from a low base and never really recoup losses...

Taking on greater risk in the form of more volatile/variable outcomes does not necessarily lead to higher returns then a less variable portfolio even if the expected return is higher. The sequence of those returns can be very important.

[backpacker]

I think the last paragraph of OPs second post (the one that the original post says, "read on to find out") was essentially suggesting that 100% stock investors are closer to the Kelly. That, I think is an attempt to a real application to the Kelly criterion.

(1) This is the idea that I thought was cool: Increasing your investment risk (and your mathematical expected returns) does not always increase the expected growth rate. In fact, it can turn the sign of the expected growth rate negative. In graphical form:



The x axis is how much you wager on the favorable bet. 2(delta) is the optimum in the long-term. It's the "speed limit" as I've been calling it. Move to the right by taking on more risk and you have a lower expected growth rate. The y axis is what you expect to make on the bet.

(2) There's a separate question about which portfolio is optimal. All stocks? 140% stocks? 60% stocks and 100% bonds? That's a hard question once we have leverage, multiple assets, non-normal distributions, dependent covariance, mean reversion, and all the other factors that effect actual returns.

Taking on greater risk in the form of more volatile/variable outcomes does not necessarily lead to higher returns then a less variable portfolio even if the expected return is higher. The sequence of those returns can be very important.

Thanks for the great chart! Yes, that exactly. Maximizing expected returns leads to disaster if there's too much volatility. And this even if the underlying asset performs well. I hadn't really grasped that fact until recently.

[packer16]

I have used Kelly in a much simpler context, that is how much should I weight a particular stock holding that is a given % undervalued. I am not sure if Kelly was intended for asset allocation but the discussion is interesting. I still do not know what the conclusion is in that context.

Packer

[JoMoney]

I'm going to point out that the above levered portfolio example may not be typical of the way some might maintain a levered portfolio unless they're using something like one of those daily updated leveraged ETF's.
The way many people attempt leveraging the portfolio doesn't actually "rebalance" it to the same proportions. The way a lot of people do it involves letting the portfolio get to a higher and higher proportion if things move against them, or less and less levered if things grow positive. Effectively a "Martingale" style bet.
If someone owned 100 shares of stock and then bought 50 more on leverage their position would be 150% of their initial balance.
If stocks went down 50% they might try to hang on to the 150 shares, but the position would be more like 200% levered.
Provided they were allowed to keep the position open, they might continue along these lines getting into a riskier and riskier position, hoping that by doubling down it will eventually turn around. I'm not sure that they would keep rebalancing the position to be precisely 150% or whatever fixed percentage of un-levered balance.

[ncole1]

Suppose you have $1,000. I offer you a bet on a fair coin. Heads you double your money. Tails I keep whatever you bet. You can make one bet a day for as long as you want. How much should you bet each day?

You could bet everything every chance you get. That would maximize your expected returns. It would not maximize your likely long-term returns. Eventually the coin will land tails and you will have no money.

There is an answer to how much you should bet if you want to maximize your long-term winnings. That answer is given by the Kelly criterion. What you do to find the optimal strategy is (a) tell the Kelly criterion what the odds of different outcomes are and (b) tell the Kelly criterion how much you gain or lose in each outcome. The formula then tells you how much to bet to maximize your long-term returns. For the coin flipping game, the answer is 25%. You should bet 25% of your "portfolio" on the game at any given time.

The Kelly criterion for stocks is a simple formula, assuming that returns are normally distributed. That formula is:

Code: Select all

(optimal percentage in stocks)=(average excess return of stocks over bonds)/(standard deviation of excess returns)^2

The historic excess return of the S&P 500 is about 5.6%. The standard deviation about 20%. So the Kelly criterion says to put 140% of your portfolio in stocks to maximize your likely long-term returns.

If you're paying attention, you noticed that I just made some silly assumptions. I assumed that stock returns are normal. They're not. I assumed that historic S&P 500 returns represent the real odds of investing in stocks. They don't. So 140% isn't really optimal. What is? That's the million dollar question.

Why does this matter then? In debates about 100% stocks, someone always says: Why 100% stocks? Why not 105%? Or 110%? The assumption being that you can always trade more risk for higher likely returns in the long run. So there is no place to draw the line, to point at which an investor who wants higher returns in the long-run can draw the line. That's completely false. If we knew the real odds of investing in stocks (i.e. if we knew the real return distribution), we could use the Kelly criterion to find the portfolio with the highest likely long-term returns. That portfolio is the "speed limit". If you take on more or less risk than that portfolio, you will likely get lower returns in the long-run. Who knows what the speed limit is. It's probably lower than 140%. But I would bet it's not too far from 100% stocks.

(This post is largely a summary of what I learned from these great notes on investing and favorable bets.)

Also, what do we mean by 140% stocks? Do we mean that the account is periodically rebalanced by selling stocks when they have fallen in value and then buying them back after they have gone up in value? How often is the rebalancing done? This will have an impact on the optimal allocation.

Many leveraged investors don't rebalance at all, in which case the compounded return on the portfolio at a constant debt/equity ratio is irrelevant. This applies, for example, to those who are investing in the stock market instead of paying off debt. They may have 140% stock at a given time but will not stay that way. They will do much better with leverage, since they are not buying high and selling low.

If our investor has installment debt (student loan, mortgage, or auto loan) and holds 140% stock in virtue of that, two-way rebalancing is impossible because once the debt is paid down it cannot be borrowed back without requalifying for a loan.

With a margin account or leverage via ITM call options, it's different, but also more vulnerable to fluctuations in interest rates.

[Johno]

I was under the impression that the optimal Kelly leverage was only 117% stocks.

It depends on what numbers you use for the "historical" numbers. The paper you linked to uses a higher standard deviation than I used. Kelly hates risk, so will recommend less margin if you tell him that there is more risk.

The paper Rob referenced, if I read it correctly, said the S&P had 'quasi normal distribution' with mean .058 and std dev .2160, which would give 124% by the formula you quoted, but the paper says a numerical solution to the relevant (more complicated) equation it uses is what gives 117%. But it's not a huge difference and as you noted, sigma squared in denominator means a noticeable difference between 20 and 22% std dev even in the simple formula.

This is an interesting thread and topic. Seems to me it makes sense to focus first at least on simple CAPM concept of long stock and riskless>100% stock no riskless>100+% stock and short riskless, with 'riskless' being ~zero duration cash/margin loan/stock futures implied borrowing rather than digress into cases where you leverage both stocks and (duration risky) bonds. And some of the statements made about latter case also surprised me, like correlations are stationary? I believe correlations are unstable and unpredictable. For example in the 2008-09 situation 'safe' bonds (prices) and stocks were negatively correlated, and perhaps in the long run past and future they're closer to uncorrelated, but 'in the long run we're all dead' whereas a 'risk parity' position could blow up in a case where a bond market collapse spooks the stock market. Lots of matrix math optimizing multi-asset leveraged positions based on assuming stationary correlations seems of limited use.

Whereas again the basic stock/cash Kelly criteria concept is at least interesting and illustrative about risk, even if personally I've no intention to be 100% stock let alone more than 100%. So far I just toyed with simulating the math of coin flip case to reproduce the result in the paper that optimum % of capital ventured betting at even odds on a coin (you know) actually comes up heads 53% of the time is 6%, 53%-47%. The interesting thing though is that if three players play alongside one another venturing 5%, 6% and 7%, the highest average log return goes to the 6% player as predicted, but the other two have a better result in a 2000 flip game substantially more often (say comparing results of several 100 different 2000 flip games). The ratio of return to std dev is also better at lower %'s than at the optimum. It's good food for thought about what you really want to maximize.

As noted, to expand the analysis to leveraged stocks you have to have good data about the borrowing rate; taking an average over a long period might distort things substantially. There's more work there than in creating a toy to play with coin flip cases, which is all I bothered to do.

[robert88]

Also, one thing to understand: even if the key parameters in the Kelly criterion do not change (EXPECTED returns and the covariance matrix), you WILL HAVE TO TRADE. Why? Because your bet ratios need to be KEPT CONSTANT relative to your portfolio size. I.e., if stocks rally 50%, and bonds go down 50%, you will have to SELL STOCKS and BUY BONDS. You decide how frequently you want to trade, but Kelly says: EVERY DAY!

I don't think Kelly says you have to trade every day. As I see it, Kelly is the optimal strategy if at least conceptually you're making a series of atomic bets, and the length of those bets can be arbitrary, as long as you're consistent(if you rebalance once a year, use average annual returns and the variance of annual returns in computing your initial leverage) and you can't face a margin call in that interval. There's probably some i.i.d assumption buried in there, but if anything I would expect annual returns to be closer to i.i.d than daily ones.

[backpacker]

The interesting thing though is that if three players play alongside one another venturing 5%, 6% and 7%, the highest average log return goes to the 6% player as predicted, but the other two have a better result in a 2000 flip game substantially more often (say comparing results of several 100 different 2000 flip games).

Both the 6% and the 7% players have better results in shorter games? I don't even have a guess for why that would be. I need to spend more time working through uber simple cases to get a feel for how this works.

[555]

The interesting thing though is that if three players play alongside one another venturing 5%, 6% and 7%, the highest average log return goes to the 6% player as predicted, but the other two have a better result in a 2000 flip game substantially more often (say comparing results of several 100 different 2000 flip games).

Both the 6%[sic? 5%] and the 7% players have better results in shorter games? I don't even have a guess for why that would be. I need to spend more time working through uber simple cases to get a feel for how this works.

My guess is that what is being claimed by Johno is that most of the time in "short" games (e.g. 2000 flips),
either the "5% bettor" comes 1st, the "6% bettor" comes 2nd, the "7% bettor" comes 3rd,
or the "7% bettor" comes 1st, the "6% bettor" comes 2nd, the "5% bettor" comes 3rd.

But for sufficiently long games, the "6% bettor" almost always comes 1st.

Right, Johno?

You started the thread backpacker, you should understand this one. No number crunching required.

[Johno]

To clarify, I ran 250 games of 2000 flips each. Though 6% consistently has the best avg log return over 250 games, by a few %, that bettor comes in first the fewest times. 250 is arbitrary as the number of runs where log return and total % of heads converge to barely moving between runs of 250, though number of wins still wobbles around a bit. But a typical result is that out of 250 games, the 5% bettor has the best outcome 108 times, 6% bettor 48 times, 7% bettor 94 times. Also, the ratio of log rtn/std dev of log return decreases monotonically with % wagered.

I don't suppose this indicates anything shocking, 5 and 7% aren't much different from 6%. The basic answer is clear enough, ie if you bet much over 10% per flip, as some might assume by intuition to be optimum, you're likely to lose money in 2000 flips despite betting at even odds on a coin bent in your favor, and 2000 is 'few' compared to total convergence but std dev of heads % among 2000 flips is still only around 1%. But the wrinkles in the statistics near the optimum suggest the need to think about what you'd really want to optimize in the coin flip game, and what you really do want to optimize in real investing.

[market timer]

To clarify, I ran 250 games of 2000 flips each. Though 6% consistently has the best avg log return over 250 games, by a few %, that bettor comes in first the fewest times. 250 is arbitrary as the number of runs where log return and total % of heads converge to barely moving between runs of 250, though number of wins still wobbles around a bit. But a typical result is that out of 250 games, the 5% bettor has the best outcome 108 times, 6% bettor 48 times, 7% bettor 94 times. Also, the ratio of log rtn/std dev of log return decreases monotonically with % wagered.

One complication in your setup is that you include three strategies. The Kelly strategy is optimal in any long run head-to-head comparison, i.e., 6% vs. 5%, or 6% vs. 7%. The three-way comparison is less meaningful. Consider the limiting case as the 5% strategy converges to 6% and 7% strategy converges to 6%. In the limit, the Kelly strategy would never come out on top, always losing to 6% - epsilon or 6% + epsilon.

[just frank]

The original Kelly paper says "The reason has nothing to do with the value function which he attached to his money, but merely with the fact that it is the logarithm which is additive in repeated bets and to which the law of large numbers applies." However, other people--I don't have Poundstone's book at hand at the moment but I think Samuelson was one of them--insist that a logarithmic utility function is a smuggled-in assumption somewhere.

Fascinating.

So, if the original number is the result of gaming a Normal distribution with exponential returns, does it follow that if returns were log-normally distributed, that the Kelly Criterion would be unbounded?

If I had to guess, I would imagine that returns (on many investments, not just equities) are log-normally distributed.

Since the truly wealthy are often entrepreneurs, I can suppose that returns on small businesses are log-normal, and often debt financed (through fund raising). If the above statement were true, you would expect very deep fundraising (like VC), a fat-tail on the subsequent wealth distribution, e.g a Pareto distribution, a large fraction of folks who went bust, and the difficulty of predicting the unicorns up front.

[Johno]

To clarify, I ran 250 games of 2000 flips each. Though 6% consistently has the best avg log return over 250 games, by a few %, that bettor comes in first the fewest times. 250 is arbitrary as the number of runs where log return and total % of heads converge to barely moving between runs of 250, though number of wins still wobbles around a bit. But a typical result is that out of 250 games, the 5% bettor has the best outcome 108 times, 6% bettor 48 times, 7% bettor 94 times. Also, the ratio of log rtn/std dev of log return decreases monotonically with % wagered.

One complication in your setup is that you include three strategies. The Kelly strategy is optimal in any long run head-to-head comparison, i.e., 6% vs. 5%, or 6% vs. 7%.

I'd look at it the other way around. In the real world people follow all kinds of strategies not just three. The case I gave illustrates I think one of the things real investors would tend to care about: how one's strategy stacks up against everyone else's strategies in a particular finite number of outcomes. And it could be true that the best of multiple strategies in average return also has the most wins: it is true for my example if you just move the low and high % cases below around 4% and above around 8%, though the return/std dev of return ratio advantage for the lower % remains.

[tadamsmar]

The original Kelly paper says "The reason has nothing to do with the value function which he attached to his money, but merely with the fact that it is the logarithm which is additive in repeated bets and to which the law of large numbers applies." However, other people--I don't have Poundstone's book at hand at the moment but I think Samuelson was one of them--insist that a logarithmic utility function is a smuggled-in assumption somewhere.

Rabin and Thaler argue that Samuelson botched it:

http://faculty.chicagobooth.edu/richard ... PV15N1.pdf

because there exists no utility that would cause a rational agent to reject one favorable bet and take a series of Kelly bets. Utility theory simply fails to account for rational human behavior. Hence the person who accepts Samuelson's argument is not following a utility. They are falling prey to myopic risk aversion. They have risk aversion to single bet, and they make the mistake of treating each bet as an isolated decision rather than consolidating the decisions and making at single decision to take the series of bets or not,

I think that refutes Samuelson.

But it brings up the question of when it is prudent to consolidate the decisions in a real world application. Their are issues, but Samuelson's argument is not an issue because it has been refuted.

[snowball]

The Kelly criterion maximizes the expected value of the logarithm of your total capital. If you are leveraged into an investment then your total capital can go to zero or even negative, in which case the logarithm is undefined and so then is the expected value. So I don't know how one can say a leveraged investment (like 140% stocks) satisfies the Kelly criterion when the metric the criterion seeks to optimize is undefined.

To put it another way, the Kelly criterion is often introduced by deciding how much to bet on a coin toss. Maybe you want to bet 100%, but then the you risk ruin on your first bet and then you won't be able to bet anymore, hence the Kelly strategy. The key point of the Kelly strategy is to avoid the possibility of ruin on every bet. If you are leveraged then you have the possibility of ruin, so I think cannot be a Kelly strategy.

[tadamsmar]

The Kelly criterion maximizes the expected value of the logarithm of your total capital. If you are leveraged into an investment then your total capital can go to zero or even negative, in which case the logarithm is undefined and so then is the expected value. So I don't know how one can say a leveraged investment (like 140% stocks) satisfies the Kelly criterion when the metric the criterion seeks to optimize is undefined.

To put it another way, the Kelly criterion is often introduced by deciding how much to bet on a coin toss. Maybe you want to bet 100%, but then the you risk ruin on your first bet and then you won't be able to bet anymore, hence the Kelly strategy. The key point of the Kelly strategy is to avoid the possibility of ruin on every bet. If you are leveraged then you have the possibility of ruin, so I think cannot be a Kelly strategy.

That can be said for any real world application of Kelly. Any strategy that has you keeping a proportion of your capital out of each bet has the "zero risk of ruin" quality. This does not guarantee that your capital cannot shrink to less than a penny.

The leverage examples are just thought experiments, they ignore margin calls and all that. They assume that "ruin" means losing all the borrowed money.

[tadamsmar]

The way to evaluate a potential real world Kelly bet is to figure out how many iterations of the bet you can reasonably accomplish and evaluate the strategy based on the characteristics of that finite series of bets. The Kelly strategy often looks good for a finite series. But I am not sure it looks good for the finite series we mortals face in retirement planning.

The Samuelson analysis is not valid because it is known to reject highly favorable finite series that no sane man would reject.

By the way, yes Virginia there is such a thing as time diversification, the Kelly logic proves it. The likelihood of failed growth becomes vanishingly small after a long period. But does it matter to us mortals?

[nisiprius]

The Kelly criterion maximizes the expected value of the logarithm of your total capital. If you are leveraged into an investment then your total capital can go to zero or even negative, in which case the logarithm is undefined and so then is the expected value. So I don't know how one can say a leveraged investment (like 140% stocks) satisfies the Kelly criterion when the metric the criterion seeks to optimize is undefined....

That is an interesting observation.

[nisiprius]

The original Kelly paper says "The reason has nothing to do with the value function which he attached to his money, but merely with the fact that it is the logarithm which is additive in repeated bets and to which the law of large numbers applies." However, other people--I don't have Poundstone's book at hand at the moment but I think Samuelson was one of them--insist that a logarithmic utility function is a smuggled-in assumption somewhere.

Rabin and Thaler argue that Samuelson botched it:

http://faculty.chicagobooth.edu/richard ... PV15N1.pdf

because there exists no utility that would cause a rational agent to reject one favorable bet and take a series of Kelly bets. Utility theory simply fails to account for rational human behavior. Hence the person who accepts Samuelson's argument is not following a utility. They are falling prey to myopic risk aversion. They have risk aversion to single bet, and they make the mistake of treating each bet as an isolated decision rather than consolidating the decisions and making at single decision to take the series of bets or not,

I think that refutes Samuelson.

But it brings up the question of when it is prudent to consolidate the decisions in a real world application. Their are issues, but Samuelson's argument is not an issue because it has been refuted.

Since that article was published in 2001 and Samuelson lived until 2009, it's possible that Samuelson responded.

[tadamsmar]

Since that article was published in 2001 and Samuelson lived until 2009, it's possible that Samuelson responded.

That article is discussed in Kahneman's recent book Thinking Fast and Slow. No response from Samuelson is mentioned there.

[nisiprius]

Reading Poundstone's book and these threads, the thing that strikes me is how curiously difficult the issue is to resolve. It's sort of like the Monty Hall problem, except that the Monty Hall problem, although tricky, does have a coherent solution and as far as I know all competent mathematicians agree on what the correct answer is. This one is weird. If it's just math, it should have been resolved a long time ago. Yet there's this definitely sense of rival ideologies or schools of thought.

There's some element in here that isn't just math, but I'm not quite sure what it is.

Clearly the elephant in the room is that in the world of investing, the Kelly criterion is founded on unobtainium--unlike some gambling situations, there is no way in the world to know the true odds or your true edge, and saying something like "well, OK, divide by two to allow for that uncertainty, that should be pretty good" isn't very convincing. How much do you allow for overconfidence, including your overconfidence in judging the amount to allow for overconfidence? What's the rational basis for choosing a factor of 2 or any other factor? (It's reminiscent of the six sigma nonsense, where they allow 1.5 sigma for Kentucky windage because there's a consensus view that that's a reasonable amount...)

But it's disturbing that fairly clear-headed people can't come to any agreed-on understanding of what the assumptions are or what it is that is actually being optimized. I'm asking the meta-question--why is the question so slippery?

In principle, clear-headed people ought to be able to reach the point of intelligent disagreement: "We all agree that the Kelly criterion assumes X, Y, and Z, and that given those assumptions, A, B, and C must follow. The disagreement is that some of us don't think X or Y or Z is valid." Instead it is all "the Kelly criterion assumes a logarithmic utility function, does not, does too" and "the Kelly criterion optimizes the mean frammis of the distribution of log whatsis, no it optimizes the mean log frammis of the distribution of the exponentially compounded whatsis, no it optimizes the risk-adjusted reward of the mean log frammis of the exponentially compounded whatsis..."

[backpacker]

The original Kelly paper says "The reason has nothing to do with the value function which he attached to his money, but merely with the fact that it is the logarithm which is additive in repeated bets and to which the law of large numbers applies." However, other people--I don't have Poundstone's book at hand at the moment but I think Samuelson was one of them--insist that a logarithmic utility function is a smuggled-in assumption somewhere.

Rabin and Thaler argue that Samuelson botched it:

http://faculty.chicagobooth.edu/richard ... PV15N1.pdf

Thanks for posting this!

Here's the basic case: Say that a Kelly gambler has a bankroll of $200. We offer him 11:10 odds on a fair coin landing heads. He is only going to bet a small percentage of his bankroll because the odds are only slightly in his favor. Samuelson says that we can think of the gambler as an expected utility maximizer who thinks that money has diminishing marginal value. That's right. But Thaler and Samuelson point out that the gambler would then also turn down a single bet where he loses $100 on tails and wins $2.5 billion on heads. No good!

[tadamsmar]

The original Kelly paper says "The reason has nothing to do with the value function which he attached to his money, but merely with the fact that it is the logarithm which is additive in repeated bets and to which the law of large numbers applies." However, other people--I don't have Poundstone's book at hand at the moment but I think Samuelson was one of them--insist that a logarithmic utility function is a smuggled-in assumption somewhere.

Rabin and Thaler argue that Samuelson botched it:

http://faculty.chicagobooth.edu/richard ... PV15N1.pdf

Thanks for posting this!

Here's the basic case: Say that a Kelly gambler has a bankroll of $200. We offer him 11:10 odds on a fair coin landing heads. He is only going to bet a small percentage of his bankroll because the odds are only slightly in his favor. Samuelson says that we can think of the gambler as an expected utility maximizer who thinks that money has diminishing marginal value. That's right. But Thaler and Samuelson point out that the gambler would then also turn down a single bet where he loses $100 on tails and wins $2.5 billion on heads. No good!

Rabin and Thaler made a cool move. Samuelson proved that the Kelly strategy does not work for a finite series of bets. Rabin and Thaler pointed out that Samuelson proof amounts to a reductio ad absurbdum proof against least one of his premises since it's barking obvious that the Kelly strategy does work.

[tadamsmar]

Reading Poundstone's book and these threads, the thing that strikes me is how curiously difficult the issue is to resolve.

Poundstone was apparently unaware of the Rabin and Thaler column.

It's sort of like the Monty Hall problem, except that the Monty Hall problem, although tricky, does have a coherent solution and as far as I know all competent mathematicians agree on what the correct answer is. This one is weird. If it's just math, it should have been resolved a long time ago. Yet there's this definitely sense of rival ideologies or schools of thought.

There's some element in here that isn't just math, but I'm not quite sure what it is.

Part of it is the different schools of thought over Utility Theory.

Clearly the elephant in the room is that in the world of investing, the Kelly criterion is founded on unobtainium--unlike some gambling situations, there is no way in the world to know the true odds or your true edge, and saying something like "well, OK, divide by two to allow for that uncertainty, that should be pretty good" isn't very convincing. How much do you allow for overconfidence, including your overconfidence in judging the amount to allow for overconfidence? What's the rational basis for choosing a factor of 2 or any other factor?

There is no distinction between gambling and investing. In both gambling and investing, there is a distinction between thought experiments and real-world applications. There are no true odds and true edge in real-world gambling. Boglehead investing is roughly equivalent to fractional Kelly. I see no real distinction between the two. Many roads lead to Dublin.

But it's disturbing that fairly clear-headed people can't come to any agreed-on understanding of what the assumptions are or what it is that is actually being optimized. I'm asking the meta-question--why is the question so slippery?

In principle, clear-headed people ought to be able to reach the point of intelligent disagreement: "We all agree that the Kelly criterion assumes X, Y, and Z, and that given those assumptions, A, B, and C must follow. The disagreement is that some of us don't think X or Y or Z is valid." Instead it is all "the Kelly criterion assumes a logarithmic utility function, does not, does too" and "the Kelly criterion optimizes the mean frammis of the distribution of log whatsis, no it optimizes the mean log frammis of the distribution of the exponentially compounded whatsis, no it optimizes the risk-adjusted reward of the mean log frammis of the exponentially compounded whatsis..."

I think that things got off track with people looking for a general refutation or a general defense of Kelly. There is no general refutation, there is no general defense. It works form some edges, odds, and finite series of bets, and not for others. I think the problem is that the length of the finite series is a hidden parameter. Everyone has ignored it. They try to prove general claims regardless of the value of this hidden parameter.

[nisiprius]

...Boglehead investing is roughly equivalent to fractional Kelly. I see no real distinction between the two. Many roads lead to Dublin...

But how do you decide what fraction to use in "fractional Kelly?" And isn't this sort of undercutting the whole idea that Kelly tells you how much to bet? What does it buy you to say "Intuitively I feel I should bet 1/Nth Kelly" versus "Intuitively, I feel that my asset allocation should be 50% stocks?"

I think the problem is that the length of the finite series is a hidden parameter. Everyone has ignored it. They try to prove general claims regardless of the value of this hidden parameter.

I like that... I wonder if it has any bearing on the popular question of "stocks for the long run," and whether risks stay the same, decrease (Siegel), or increase (Pastor and Stambaugh) for long holding periods.

[robert88]

(This post is largely a summary of what I learned from these great notes on investing and favorable bets.)

Those notes are wrong. The expectation of log(1+pX*) doesn't exist if pX* can be less than or equal to negative one. The actual Kelly criterion is probably 100%.

[backpacker]

(This post is largely a summary of what I learned from these great notes on investing and favorable bets.)

Those notes are wrong. The expectation of log(1+pX*) doesn't exist if pX* can be less than or equal to negative one. The actual Kelly criterion is probably 100%.

Why would pX* be negative? Say my bankroll is $100. Assign $0 a utility of 1,000, -$40 a utility of 960, $200 a utility of 1,200, and so on. Utility is an interval scale, so you can always add a constant to the utilities without changing the results.

[tadamsmar]

...Boglehead investing is roughly equivalent to fractional Kelly. I see no real distinction between the two. Many roads lead to Dublin...

But how do you decide what fraction to use in "fractional Kelly?" And isn't this sort of undercutting the whole idea that Kelly tells you how much to bet? What does it buy you to say "Intuitively I feel I should bet 1/Nth Kelly" versus "Intuitively, I feel that my asset allocation should be 50% stocks?"

Fractional Kelly is used when you doubt your estimate of the edge and/or the odds, so you use conservative estimates and the definition of conservative is in the application domain, all the Kelly Formula can do is help you estimate what uncertainty does to the growth rate. But there are also issues related to the number of iterations of the bet, not sure how to handle those, but I think Rabin and Thaler seem to suggest that you could apply your utility or risk aversion to the single comprehensive decision to make the series of bets.

[robert88]

(This post is largely a summary of what I learned from these great notes on investing and favorable bets.)

Those notes are wrong. The expectation of log(1+pX*) doesn't exist if pX* can be less than or equal to negative one. The actual Kelly criterion is probably 100%.

Why would pX* be negative? Say my bankroll is $100. Assign $0 a utility of 1,000, -$40 a utility of 960, $200 a utility of 1,200, and so on. Utility is an interval scale, so you can always add a constant to the utilities without changing the results.

Why are you bringing up utility? The Kelly Criterion does not include utility in its calculations.

[Canogapark66]

Without margin is there even a Low cost way to be 140% stocks?

[nisiprius]

...Why are you bringing up utility? The Kelly Criterion does not include utility in its calculations...

The amazing thing is that there seems to be serious disagreement about whether or not it does.

I don't think there's really any disagreement (I expect to be proved wrong about that instantly!) that if your goal is to maximize the expected number of dollars, and ignore risk, the optimum strategy is the obvious naïve strategy--make every bet as large as you possibly can. This is true despite the near-certainty of eventual ruin. The distribution of outcomes is extremely unpleasant, but the expectation is nevertheless high.

If you say instead that your goal is to maximize the expectation of the logarithm of the number of dollars, then, instantly, the optimum changes to the Kelly bet.

Some people seem to think that maximizing the expectation of the logarithm of the number of dollars is objectively right, and is what all investors should seek to be doing.

Incidentally I'm not sure where or how risk, and risk-adjusted reward, make their appearance in the Kelly criterion framing. What exactly is the measure of risk in the Kelly world, and what is the counterpart to the Sharpe ratio?

Should we actually want to maximize the expectation of the log of the number of dollars? Shouldn't it be a risk-adjusted expectation? Somehow?

[Johno]

As a further casual experiment, I back tested return from leveraged position in the S&P. I used SPTR (S&P 500 total return) index data from June 1988 because it's immediately available on CBOE's site. Data on 3 mo LIBOR for same period was also easy to get. I assumed a leveraged S&P position financed at 3 mo L+.25% (a rough proxy for the futures implied financing rate), rebalanced quarterly, no taxes or transactions costs. The return maximizing allocation for 6/1/1988-12/1/2014 is 190%. But for June 2000-Dec 2014 it's 100%. IOW under those assumptions, any leverage reduced the return in that period, by digging deeper holes in 2002 and 2009, without the big profits of the long bull run of the 90's.

In neither period does leverage run you into the ground until 275%, assuming the investor sticks with it to zero. If the investor quits in disgust after losing half the initial capital though, that happens at only around 116% in the shorter 2000-2014 period.

The above is a digression from theoretical math discussion, less general, and past performance is no gtee of future results, but it's an obviously possible set of returns.

[lee1026]

Incidentally I'm not sure where or how risk, and risk-adjusted reward, make their appearance in the Kelly criterion framing. What exactly is the measure of risk in the Kelly world, and what is the counterpart to the Sharpe ratio?

Sharpe ratio is independent of leverage, so all of the bets that we are discussing have the same Sharpe ratio. Similarly, the concept of risk adjusted returns doesn't apply. Traditionally, the argument is that you can leverage up or down for your desired risk as long as the risk adjusted returns are good. But if we are talking about how much leverage to use, that become a moot point very quickly.

[robert88]

...Why are you bringing up utility? The Kelly Criterion does not include utility in its calculations...

The amazing thing is that there seems to be serious disagreement about whether or not it does.

I don't think there's really any disagreement (I expect to be proved wrong about that instantly!) that if your goal is to maximize the expected number of dollars, and ignore risk, the optimum strategy is the obvious naïve strategy--make every bet as large as you possibly can. This is true despite the near-certainty of eventual ruin. The distribution of outcomes is extremely unpleasant, but the expectation is nevertheless high.

If you say instead that your goal is to maximize the expectation of the logarithm of the number of dollars, then, instantly, the optimum changes to the Kelly bet.

Some people seem to think that maximizing the expectation of the logarithm of the number of dollars is objectively right, and is what all investors should seek to be doing.

Incidentally I'm not sure where or how risk, and risk-adjusted reward, make their appearance in the Kelly criterion framing. What exactly is the measure of risk in the Kelly world, and what is the counterpart to the Sharpe ratio?

Should we actually want to maximize the expectation of the log of the number of dollars? Shouldn't it be a risk-adjusted expectation? Somehow?

I was replying to the OP, who seemed to claim that the Kelly formula allows you to make up and plug in utility values into the formula, which it doesn't do and pX* as used in the cited notes is the actual return not utility of the return. As to your point, here's the Samuelson paper, http://finance.martinsewell.com/money-m ... on1971.pdf, which says that the Kelly Criterion is suboptimal for maximizing the expected value with arbitrary utility functions. Kelly, I think, claims that with a probability approaching 1, his strategy will outperform every other strategy that involves betting a fixed percentage of your bankroll in an infinite number of bets. The biggest problem with Kelly is that it requires a uniform strategy, betting a fixed percentage of your bankroll instead of a variable percentage. It doesn't allow you to ever quit playing after "you've won the game." From my read of Samuelson's paper, if your utility of wealth has finite upper and lower bounds, then the Kelly criteria is the best solution of all the bad choices(having to maintain a constant betting ratio forever), even if your utility of wealth is not logarithmic.

[robert88]

I was under the impression that the optimal Kelly leverage was only 117% stocks. Here's from a post from a prior conversation.
.

Thorpe assumes that it's impossible for the stock market to experience a three sigma or greater event. That seems like a flawed assumption, especially since finance is known to exhibit fat tails.

[madbrain]

In a later book, they suggest a strategy based on LEAPS, but not because of risk, but because--to them, regrettably--401(k) plans do not offer the ability to use margin. "In the not too distant future, courts may determine that failing to offer employees the option to diversify temporarally" (by not offering young 401(k) participants a means of using 200% leverage) will fail the legal test of fiduciary responsibility.

Actually, my 401k brokerage window at Fidelity doesn't let me invest in LEAPS, or any other type of options. And I don't think they are failing the legal test of fiduciary responsibility, or will be found to have done so in the future.

[Johno]

Incidentally I'm not sure where or how risk, and risk-adjusted reward, make their appearance in the Kelly criterion framing. What exactly is the measure of risk in the Kelly world, and what is the counterpart to the Sharpe ratio?

Sharpe ratio is independent of leverage, so all of the bets that we are discussing have the same Sharpe ratio. Similarly, the concept of risk adjusted returns doesn't apply. Traditionally, the argument is that you can leverage up or down for your desired risk as long as the risk adjusted returns are good. But if we are talking about how much leverage to use, that become a moot point very quickly.

Sharpe ratio is independent of leverage in the quasi-single period CAPM world. But in the CAPM world the return also increases toward infinity with more and more leverage.

Seems this conversation can be ambiguous as to whether we're talking about coin flip game for which Kelly math is well explained in original paper, or adapting it to stocks. Latter is the more practical discussion of course, but stuff like ratio of return/(std dev of return) is also different between the two. I have the same question as Nispirus as to the correct analog of Sharpe ratio in coin flip game, but taking a pseudo-Sharpe ratio of just the (log rtn)/(std dev of log rtn) among 2000 flip games betting heads at even odds on a coin which comes up heads 53% of the time, that ratio strictly declines the more you wager, though the max expected log return is at 6% of capital wagered.

Back testing for SPTR leveraged at 3 mo LIBOR+25 quaterly rebalance, see above, the Sharpe Ratio is also not a constant with leverage.
June 1 1988>Dec 1 2014
Quasi Riskless (ie 3mo LIBOR flat) return: 4.04%
SPTR at 100%: 10.42%, annualized quarterly std dev of return: 16.33%, Sharpe Ratio, as in (stock-'riskless')/(std dev of risky)= .390
SPTR at 117%: 11.18%, std dev 19.35%, SR .369
SPTR at 140%: 12.03%, std dev 23.68%, SR .337
SPTR at 190%: 13%, std 34.72%, SR .258
SPTR at 200%: 12.99%, std 37.39%, SR .239
SPTR at 275%: goes bust 12/2008

For 6/1/200>12/1/2014:
LIBOR rtn: 2.28%
SPTR at 100%: 5.37%, std dev 19.08%, SR .161
SPTR at 117%: 5.25%, std dev 22.77%, SR .130
SPTR at 140%: 4.80%, std 28.12%, SR .090
SPTR at 190%: 2.28%, std 42.14%, SR 0
SPTR at 275%: goes bust 12/2008

I know graphs are more entertaining, sorry. Anyway these results like any historical results don't 'prove' anything about the future, nor would collecting them further back, whereas theoretical results out to infinity are meaningless (the usual dilemma). However I'd say the *general indication* is that a stock allocation somewhere in the low to mid 100%'s may be reasonable for an investor highly focused on return and not particularly concerned with annualized std dev noise. The later shorter period isn't the worst imaginable for leverage but pretty bad and the moderately 100%+ investor doesn't come out *that* much worse than 100% investor, and enough better in the longer period to possibly be worthwhile, depending on preference.

[technovelist]

As an aside, should the word "temporarily" be "temporally"?

Yes. Thanks. Fixed.

What is the other book?

"Lifecycle Investing," by Ayres and Nalebuff.

On first glance, doesn't this type of "strategy" ignore the risk, however small or large, of going totally belly up?

RM

Ayres and Nalebuff say that in their backtesting it didn't seem to happen very often, and that when it does it's not a serious problem for a young investor because they have the time and wherewithal to pick themselves up, dust themselves off, and and start all over again. As "market timer" seems to have done.

Ayres and Nalebuff argue that the superior average outcomes in their simulations are well worth the small risk of financial ruin when young. You will have to ask "market timer" if he agrees.

IIRC, "market timer" moved to Thailand for the lower cost of living. Which of course doesn't depend on his having gone bust previously.

[lee1026]

I know graphs are more entertaining, sorry. Anyway these results like any historical results don't 'prove' anything about the future, nor would collecting them further back, whereas theoretical results out to infinity are meaningless (the usual dilemma). However I'd say the *general indication* is that a stock allocation somewhere in the low to mid 100%'s may be reasonable for an investor highly focused on return and not particularly concerned with annualized std dev noise. The later shorter period isn't the worst imaginable for leverage but pretty bad and the moderately 100%+ investor doesn't come out *that* much worse than 100% investor, and enough better in the longer period to possibly be worthwhile, depending on preference.

Sure, but that is with a single data point. Consider the following game:

You may bet any amount X.
A coin is flipped. Heads, you get X*10 back. Tails, you lose everything.

This game will run for 30 iterations. You are allowed to change your stake at any time.

The expected value is maximized if you bet your entire bankroll every single time. But you will also go bust nearly every time. However, the roughly once in a billion time that you don't go bust, you end up with a rather lot of money, so the expected value is still extremely high. But if we decide to monte carlo and only do say, a million iterations, you may miss on that fact and conclude that betting some smaller amount so that you are less likely to go bust have a higher expected value.

[Johno]

I know graphs are more entertaining, sorry. Anyway these results like any historical results don't 'prove' anything about the future, nor would collecting them further back, whereas theoretical results out to infinity are meaningless (the usual dilemma). However I'd say the *general indication* is that a stock allocation somewhere in the low to mid 100%'s may be reasonable for an investor highly focused on return and not particularly concerned with annualized std dev noise. The later shorter period isn't the worst imaginable for leverage but pretty bad and the moderately 100%+ investor doesn't come out *that* much worse than 100% investor, and enough better in the longer period to possibly be worthwhile, depending on preference.

Sure, but that is with a single data point. Consider the following game:

You may bet any amount X.
A coin is flipped. Heads, you get X*10 back. Tails, you lose everything.

This game will run for 30 iterations. You are allowed to change your stake at any time.

The expected value is maximized if you bet your entire bankroll every single time. But you will also go bust nearly every time. However, the roughly once in a billion time that you don't go bust, you end up with a rather lot of money, so the expected value is still extremely high. But if we decide to monte carlo and only do say, a million iterations, you may miss on that fact and conclude that betting some smaller amount so that you are less likely to go bust have a higher expected value.

I'm not sure how an even more skewed coin game relates to the back test of stocks in the real world mentioned in the part of my post you quoted. Maybe I'm just confused by which part of my post you quoted for brevity's sake. But why would anyone ever care practically about what happens one in a billion times? The coin game the paper refers to and version of which I referred to earlier in last and earlier posts at least has in common with the real world that expected value doesn't extremely heavily depend on extremely unlikely outcomes. That's what makes it a useful if limited vehicle for thought about the real world. A game totally dominated by extremely unlikely outcomes is further divorced from the real world.

In a word, I don't get your point as it relates to what I said.

[lee1026]

But why would anyone ever care practically about what happens one in a billion times?

I doubt anyone would care practically; but it doesn't change the mathematical reality of expected value. It also explains the difference between the theoretical (more leverage on a winning trade = more money) and the truth in practice (too much leverage on a winning trade = bad results).

I wrote that to try to explain that difference.

[tadamsmar]

Samuelson's first paper concerning Kelly did not mention Kelly, but it became a foundation of this later attacks on the Kelly Criteria.

One of Samuelson's colleagues, who came to be called SC in some behavioral economics papers, said he would not bet on a single 50-50 coin flip where he lost $100 but gained $200 but he would welcome a series of 100 such bets. Samuelson saw that there was a contradiction in this. After 99 bets, SC faces the single bet scenario that he said he would reject, so he must reject it. But at 98 bets, he would know that he must reject the last, so he is again facing the single bet scenario. And so on. Samuelson concluded that he must reject the the series if he rejects the one. The papers in the behavioral economics literature concluded that he must accept the one if he accepts the series. Everyone agreed that SC is inconsistent and that he must give one one or the other of his beliefs.

Here is the paper: https://www.casact.org/pubs/forum/94sforum/94sf049.pdf

[bgf]

I just finished Fortune's Formula, just read through this entire thread, and my brain is mush.

Add onto the already incredibly long list another instance of 1) my math skillz are so so insufficient, 2) there are some really intelligent, knowledgeable people on this board, and 3) optimization in the context of the stock market is really really hard (impossible?).

thanks for all the awesome replies.