AI/Machine Learning/Deep Learning in Investing

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
Post Reply
Topic Author
Hopes101
Posts: 6
Joined: Tue Mar 21, 2017 6:42 am

AI/Machine Learning/Deep Learning in Investing

Post by Hopes101 »

I heard the programs are being developed in both academic and private setting. Are there any programs available to personal investors?
User avatar
happenstance
Posts: 305
Joined: Sun Jul 26, 2015 11:24 am
Location: NYC

Re: AI/Machine Learning/Deep Learning in Investing

Post by happenstance »

ML trading is really just a branch of quantitative/algorithmic trading, but rather than having the trading algorithm designed by humans, the algorithm is derived by a machine learning system. A stock trading ML model is ultimately about trying to make predictions over some set of features (e.g. stock price) given some input data.

As a stupid example, let's say there's a trading effect where if it's raining on Thursday, companies with a P/E of 12 or higher will dip after 1pm and then rebound on the following Tuesday. In an algorithmic trading model, the human algorithm designer has identified this effect and manually applies conditions on when to trade based on these criteria. For ML trading, the designer would have historical data feeds for those features (day of week, weather, P/E, stock price) and would then run an optimization to find short-term gains over a e.g. seven-day window. The ML system would then (hopefully) infer this raining-on-Thursday effect based on its inputs. Then given some real-time stock data on a Thursday, it would produce predictions about whether or not a stock would dip and rebound on Tuesday. On a simplistic level, ML is just fitting linear regressions. The WSJ did a series of articles on algorithmic and quantitative trading that explains a lot of this in more detail, of particular relevance is "How a Trading Algorithm Actually Works". Planet Money also did an episode about making a trading bot that is also somewhat interesting.

A personal investor can definitely do algorithmic trading at home. There are several companies that provide individuals an API with which to execute stock trades (Quantopian, Quantiacs, and Kite are just a few). The investor writes a trading bot that monitors the live data feeds for the features/criteria it uses, and then places a trade using those APIs when its threshold/criteria are met.

If a personal investor wanted to use ML, the mechanics would be similar to a human-designed algorithmic trading bot. But rather than using the human-set criteria, the bot would send the live data feeds into the ML model to receive its predictions for a goal the model was optimized on. The investor would need a software package to help build and train a ML model (TensorFlow is popular), historical input data for the features the model is being trained on, and a problem/question the ML model is optimizing for.

So yes, the tools to do ML trading as a personal investor are available, but there is not an off-the-shelf ML trading program. ML is highly dependent on the data the models are trained on and the features selected for training. And one big disadvantage the personal investor has in this space is that quality data feeds (both historical and live) are expensive.
acanthurus
Posts: 418
Joined: Sun Aug 04, 2013 8:02 am

Re: AI/Machine Learning/Deep Learning in Investing

Post by acanthurus »

Removed
Last edited by acanthurus on Mon Jan 01, 2018 4:38 am, edited 1 time in total.
User avatar
whodidntante
Posts: 13090
Joined: Thu Jan 21, 2016 10:11 pm
Location: outside the echo chamber

Re: AI/Machine Learning/Deep Learning in Investing

Post by whodidntante »

Deep learning tools have improved dramatically. Some of these are free and are available to you. So you can begin working with those. However, models (at least the good, profitable or business critical ones) are kept as closely guarded secrets. I just attended a machine learning conference and no one was really spilling the beans about the details of their models/learning networks. They speak and write their papers at a higher level than that.

Machine learning is used for solved intuitive problems which have previously been best solved with human intelligence. E.g. does this drawing look like a cat? Did was it more likely that she said "far" or "bar?" Is this stock fairly priced? Models have to be designed and trained to fit a problem. It is not an attempt to design a general purpose intelligence.

It's not necessary that learning network output is used to actually execute trades. It may just provide information to another network or a human. E.g., it's not necessarily high frequency or high speed trading just because there is an AI in the loop.
random_walker_77
Posts: 2207
Joined: Tue May 21, 2013 8:49 pm

Re: AI/Machine Learning/Deep Learning in Investing

Post by random_walker_77 »

This is an area of active research and development. On one hand, a lot is being discovered and it's gold rush time to go figure things out before everyone else, but that implies the exact opposite of off-the-shelf tools. If you're versed in deep learning and highly technical, maybe you can figure out something valuable in this area and figure out a way to beat the market. But at the same time, you'll be competing with a lot of talented developers, PhDs, and armies of well-funded technical talent. The odds may not exactly be in your favor?

The other approach is to buy a representative slice of the entire market w/ low cost index funds...
Ron Scott
Posts: 1090
Joined: Tue Apr 05, 2016 5:38 am

Re: AI/Machine Learning/Deep Learning in Investing

Post by Ron Scott »

random_walker_77 wrote: Sun Nov 19, 2017 10:20 pm This is an area of active research and development. On one hand, a lot is being discovered and it's gold rush time to go figure things out before everyone else, but that implies the exact opposite of off-the-shelf tools. If you're versed in deep learning and highly technical, maybe you can figure out something valuable in this area and figure out a way to beat the market. But at the same time, you'll be competing with a lot of talented developers, PhDs, and armies of well-funded technical talent. The odds may not exactly be in your favor?

The other approach is to buy a representative slice of the entire market w/ low cost index funds...
It seems to me that if a solid algorithm can be developed that actually beats the market for some reasonable period of time the competition to beat it will be fierce. Those who own the market with an index fund lose nothing from it obviously, because they will always get the return the market produces. The smart algorithm will take its profits from other (dumber) investors who are "tilting" the wrong way...and the dumber ones will wake up.

Should be interesting to watch. In the early Benjamin Graham days, value investing seemed to work very well, and was able to take profits from those who held overpriced stocks. Then the larger market caught on and got efficient and value funds didn't beat the market anymore. But I do not discount the possibility of a smart, AI-based end-run that can survive for some period of time.
Retirement is a game best played by those prepared for more volatility in the future than has been seen in the past. The solution is not to predict investment losses but to prepare for them.
random_walker_77
Posts: 2207
Joined: Tue May 21, 2013 8:49 pm

Re: AI/Machine Learning/Deep Learning in Investing

Post by random_walker_77 »

Not AI, but to give you an idea of the amount of money at play for a tiny edge in trading, there was news several years back about new internet cables being built at exorbitant cost just to save a few thousandths of a second on transmission latency. For similar reasons, I don't think it's a stretch to assume that there's a lot of resources being directed at applying Deep Learning to the stock market, and that this has been going on for years (certainly well before Deep Learning became popular in the media).

$300M to save 3ms between London and NY
http://www.telegraph.co.uk/technology/n ... conds.html

$300M to save 3ms between chicago and NY
https://www.forbes.com/forbes/2010/0927 ... 9982e1741a

Profiling the rise of Quants:
http://www.telegraph.co.uk/finance/1018 ... treet.html

Example of AI for finance / IBM:
https://www.cnbc.com/2017/06/16/ai-assa ... iness.html

Some of the folks seriously applying GPU's towards investing back in 2012/2013 surely were paying attention to deep learning years ago...

https://insidehpc.com/2014/05/gpu-compu ... ance-gpus/
random_walker_77
Posts: 2207
Joined: Tue May 21, 2013 8:49 pm

Re: AI/Machine Learning/Deep Learning in Investing

Post by random_walker_77 »

Ron Scott wrote: Sun Nov 19, 2017 10:52 pm
random_walker_77 wrote: Sun Nov 19, 2017 10:20 pm This is an area of active research and development. On one hand, a lot is being discovered and it's gold rush time to go figure things out before everyone else, but that implies the exact opposite of off-the-shelf tools. If you're versed in deep learning and highly technical, maybe you can figure out something valuable in this area and figure out a way to beat the market. But at the same time, you'll be competing with a lot of talented developers, PhDs, and armies of well-funded technical talent. The odds may not exactly be in your favor?

The other approach is to buy a representative slice of the entire market w/ low cost index funds...
It seems to me that if a solid algorithm can be developed that actually beats the market for some reasonable period of time the competition to beat it will be fierce. Those who own the market with an index fund lose nothing from it obviously, because they will always get the return the market produces. The smart algorithm will take its profits from other (dumber) investors who are "tilting" the wrong way...and the dumber ones will wake up.

Should be interesting to watch. In the early Benjamin Graham days, value investing seemed to work very well, and was able to take profits from those who held overpriced stocks. Then the larger market caught on and got efficient and value funds didn't beat the market anymore. But I do not discount the possibility of a smart, AI-based end-run that can survive for some period of time.
Agreed. Those who find a win will have to continue innovating to maintain an edge. This is one of those games that favor the well-funded firms that have lots of resources to throw at the problem.

To me, this reinforces the biggest strength of the index fund investing approach: you don't have to beat the market. Because beating the market consistently (i.e. not through luck) is really hard.
Valuethinker
Posts: 48954
Joined: Fri May 11, 2007 11:07 am

Re: AI/Machine Learning/Deep Learning in Investing

Post by Valuethinker »

Hopes101 wrote: Sat Nov 18, 2017 7:52 am I heard the programs are being developed in both academic and private setting. Are there any programs available to personal investors?
Not AFAIK.

It's worth reading Andrew Lo's book on Adaptive Markets.
Post Reply