Can passive index investing beat active AI investing

[Gecko10x]

Saw this good article on bbc showing chatgpt4 is not good at reasoning:

when robots can't riddle

got some interesting logic problems (some you've probably seen before) and how poorly chatgpt did answering these questions.

It's my understanding that ChatGPT and similar AI function via context-aware predictive text. In other words, based on the previous few words, they guess at the next one. AFAIK, there's zero reasoning happening, which is why I trust it about as far as I can throw it.

[Prokofiev]

.
Once AI investing hits the mainstream, we will ALL be able to beat the SP500 averages!

[BuglheadLuvsLondon]

Passive investing is taking your share of money from the money tree.

Active investing is taking your share of money from the money tree and then playing poker with it with other active investors.

Even if the poker players are superintelligent AIs, they are playing a zero sum game. They CAN'T "beat" passive investors as a group because poker doesn't actually generate any net wealth.

Greetings,

It certainly seems as though this is the correct answer. "The aggregate returns of all investors in the market must by necessity equal the total return of the market."

For active investors, it is a zero sum game with winners and losers. For index investors, it is simply the market return.

Cheers.

[protagonist]

Passive investing is taking your share of money from the money tree.

Active investing is taking your share of money from the money tree and then playing poker with it with other active investors.

Even if the poker players are superintelligent AIs, they are playing a zero sum game. They CAN'T "beat" passive investors as a group because poker doesn't actually generate any net wealth.

Well said.

I believe that active AI investing is going to make a pot of money--for the companies that bring "active AI investing" vehicles to market, over the next few years. Up until a "ten-sigma" tail risk shows up--situations that the risk models say should show up once in sextillions years, but happen several times a decade--and they crash for reasons that "nobody could possibly have foreseen."

At the moment, the AI models that are getting all the attention clearly are not exhibiting "intelligence" similar to human-style intelligence, and cannot deal with novel situations they didn't encounter during their training.

There are a few we can look at already:

AI Powered Equity ETF (AIEQ)
VanEck Social Sentiment ETF (BUZZ)
WisdomTree International AI Enhanced Value Fund (AIVI)
Qraft AI-Enhanced U.S. Large Cap Momentum ETF (AMOM)

Source



What did we used to say in grade school? "Three cheers, two jeers and a moldy pretzel." Except it's four jeers. You'd have thought at least one of them might have beaten the market, just by chance.

Acc. to your graph they not only didn't beat the market...they all LOST money.
Acc. to Google AI (haha), $10K invested in the S+P 500 index on 4/1/21 would be worth $13,800 today.
I would not have expected AI to do THAT badly. It would be hard for a money manager to underperform by that wide a margin even if they were trying to lose.
How all of their AI models managed to achieve this remarkable feat is beyond me, and interesting in itself.

[nisiprius]

There are a few we can look at already:

AI Powered Equity ETF (AIEQ)
VanEck Social Sentiment ETF (BUZZ)
WisdomTree International AI Enhanced Value Fund (AIVI)
Qraft AI-Enhanced U.S. Large Cap Momentum ETF (AMOM)

Source



What did we used to say in grade school? "Three cheers, two jeers and a moldy pretzel." Except it's four jeers. You'd have thought at least one of them might have beaten the market, just by chance.

Update:



All four of the AI-powered ETFs are still behind Total Stock.

[Tyler Aspect]

Once AI investing hits the mainstream, we will ALL be able to beat the SP500 averages!

No, I think it is the opposite. There are a very limited volume of market inefficiencies that the velocity traders siphons off in terms of profits. These are propriety traders physically located near major exchanges, with fast computers and dedicated network links. In no way are they able to support an open ended fund structure. After they are done we are left with the good old efficient market Bogleheads love.

[Random Musings]

.
Once AI investing hits the mainstream, we will ALL be able to beat the SP500 averages!

I believe that is the case only if you live near a particular lake. It's good in the hood there.

RM

[Tycoon]

Is there evidence that AI investing beats any other type of investing?

[anoop]

Can you define active AI investing? AI is being made out to be some kind of magic which it's not and never will be.

AI learns patterns but there are no faithful patterns when it comes to stocks. Otherwise, there would be tons of successful funds that would exploit that.

[exodusNH]

Please explain what kind of human intelligence is manifested by your example. As far as I can tell, it is knowledge base search.

I am convinced that Artificial Stupidity by the human being using AI will mess things up when it matters. Hence, passive indexing will win.

KlangFool

Well, in my experience as a software developer who uses AI daily across multiple domains, all I can say is that these things clearly exhibit a nascent ability to reason. It goes way beyond a knowledge base search.

In my experience as a software developer, it is clear that current AI is basically a super-complicated MadLib.

It is a statistical model that predicts words based on context.

That's not to say it's not impressive or that it isn't useful. It is both! But it is not "thinking" in any meaningful way.

[nisiprius]

Well, here we go. A nice real-world test. From Matt Levine's "Money Stuff" newsletter, available at no cost from Bloomberg.com by email subscription.

An Israeli regulator has cleared an artificial intelligence startup to launch a chatbot that offers stock-picking advice in partnership with a large bank, even as other governments have raised alarms that AI might destabilize financial markets if used widely in investing.

Tel Aviv-based Bridgewise has been given the green light by the Israel Securities Authority (ISA) to release a chatbot called Bridget later this month that can offer recommendations for which stocks to buy and sell in response to user queries. The startup is working with one of the country’s largest banks, Israel Discount Bank, to roll out the product. …

The regulator laid down several other conditions for approval, including that the company offering such an AI tool must have an investment license, be compensated on a fixed rate rather than based on how well the investments do, and abide by basic conflict of interest rules.

Founded in 2019, Bridgewise uses AI trained on historical data combined with real-time news to provide investment analysis to brokerages, wealth advisors and exchanges, including the Nasdaq, the London Stock Exchange and the Tel Aviv Stock Exchange. The company said its new chatbot has taken years of work to develop and has gone through significant testing to ensure the advice is accurate.

Levine comments:

“To ensure the advice is accurate”? Like … to ensure that the stocks go up? Was the testing “ask it what stocks to buy, then see if they go up”? Or something else?

Even if it "works" statistically, my own prediction is that real-world customers will not be happy with beating the market by (say) 0.5% per year. Regardless of how many disclaimers they read, they will expect near perfection because it's "AI" and be upset by every bad call.

[ScubaHogg]

Sharpe’s Arithmetic remains undefeated

https://www.google.com/search?q=sharpes ... ent=safari

[Elric]

Please explain what kind of human intelligence is manifested by your example. As far as I can tell, it is knowledge base search.

I am convinced that Artificial Stupidity by the human being using AI will mess things up when it matters. Hence, passive indexing will win.

KlangFool

Well, in my experience as a software developer who uses AI daily across multiple domains, all I can say is that these things clearly exhibit a nascent ability to reason. It goes way beyond a knowledge base search.

In my experience as a software developer, it is clear that current AI is basically a super-complicated MadLib.

It is a statistical model that predicts words based on context.

That's not to say it's not impressive or that it isn't useful. It is both! But it is not "thinking" in any meaningful way.

While Large Language Models (which you are describing )are the huge thing right now, they are only a subset of the field of Artificial Intelligence (AI). One might even say a subset of neural networks, which themselves are a subset of AI.

[Dusn]

Saw this good article on bbc showing chatgpt4 is not good at reasoning:

when robots can't riddle

got some interesting logic problems (some you've probably seen before) and how poorly chatgpt did answering these questions.

It's my understanding that ChatGPT and similar AI function via context-aware predictive text. In other words, based on the previous few words, they guess at the next one. AFAIK, there's zero reasoning happening, which is why I trust it about as far as I can throw it.

My experience has been the opposite. I’ve caught it very confidently making stuff up in my field of expertise and it hallucinates in a very human, logical, and intuitive way. It would be extremely hard for a human to catch some of its hallucinations.

And it’s not getting the hallucinations off the internet as far as I can tell. If I Google search the same question, I’ll only find the correct answer on the few websites with this specialized knowledge.

[pchalt]

For what it's worth, Blackrock developed/implemented Aladdin way back in 1993.

https://archive.md/wbju2

https://corbettreport.com/how-blackrock ... ld-part-3/

[Mrbogleheads]

We all know passive investing beat active investing in long term. Can it also beat AI quantitive investing

I'll let you now in 10 years

[nisiprius]

...Well, in my experience as a software developer who uses AI daily across multiple domains, all I can say is that these things clearly exhibit a nascent ability to reason...

What is a "nascent" ability to reason, and what objective evidence do you have that they "clearly" exhibit it other than your personal intuition? Chatbots trace their lineage at least back to Joseph Weizenbaum's ELIZA circa 1965; was the ability to reason nascent in an embryonic form then?

And if it is "nascent" when will it emerge? AI has had three years since unveiling of DALL-E to look at images of human hands. How long will it take them to deduce that they normally have five fingers?

Show a 3x4 grid of exactly twelve hands, each with exactly five fingers, positioned and presented in such a way that there is no overlap and no fingers are hidden.

Here is the 3x4 grid of twelve hands, each with exactly five fingers, clearly positioned with no overlap or hidden fingers.

(I have displayed the image exactly as cropped and presented by ChatGPT. My favorite is the third row, fourth image.)

[donaldfair71]

AI investing has an edge if the investor can limit new investors, a la Jim Simons and Renaissance.

Capacity unlimited? I would have a lot of doubts, just like Jim Simons did.

[Gecko10x]

Saw this good article on bbc showing chatgpt4 is not good at reasoning:

when robots can't riddle

got some interesting logic problems (some you've probably seen before) and how poorly chatgpt did answering these questions.

It's my understanding that ChatGPT and similar AI function via context-aware predictive text. In other words, based on the previous few words, they guess at the next one. AFAIK, there's zero reasoning happening, which is why I trust it about as far as I can throw it.

My experience has been the opposite. I’ve caught it very confidently making stuff up in my field of expertise and it hallucinates in a very human, logical, and intuitive way. It would be extremely hard for a human to catch some of its hallucinations.

And it’s not getting the hallucinations off the internet as far as I can tell. If I Google search the same question, I’ll only find the correct answer on the few websites with this specialized knowledge.

I don't think this is opposite of what I'm saying at all. My point was that it is completely making things up based on predictive word context and it's doing zero reasoning. So I do not trust it with anything where there's an objectively correct answer.

[BrooklynInvest]

AI is having a very positive effect on my net worth.

I just rewrote a website for a client that thought they could use Chat GPT for the project. Everything Chat GPT did was almost right, but nothing was actually right.

The gig bought me half a car!

[MrBobcat]

Once AI investing hits the mainstream, we will ALL be able to beat the SP500 averages!

No, I think it is the opposite. There are a very limited volume of market inefficiencies that the velocity traders siphons off in terms of profits. These are propriety traders physically located near major exchanges, with fast computers and dedicated network links. In no way are they able to support an open ended fund structure. After they are done we are left with the good old efficient market Bogleheads love.

I think you missed the imbedded facetiousness.

[exodusNH]

Please explain what kind of human intelligence is manifested by your example. As far as I can tell, it is knowledge base search.

I am convinced that Artificial Stupidity by the human being using AI will mess things up when it matters. Hence, passive indexing will win.

KlangFool

Well, in my experience as a software developer who uses AI daily across multiple domains, all I can say is that these things clearly exhibit a nascent ability to reason. It goes way beyond a knowledge base search.

In my experience as a software developer, it is clear that current AI is basically a super-complicated MadLib.

It is a statistical model that predicts words based on context.

That's not to say it's not impressive or that it isn't useful. It is both! But it is not "thinking" in any meaningful way.

While Large Language Models (which you are describing )are the huge thing right now, they are only a subset of the field of Artificial Intelligence (AI). One might even say a subset of neural networks, which themselves are a subset of AI.

Of course, but this is the element of AI that everyone is thinking about.

Could there be some neural network that's able to predict stock market movements? Sure, but once one is created, others will follow, which will negate the advantage.

The first mover will get a huge advantage, but that will fade quickly. Passive investors will benefit just like we do now with the active managers doing the price discovery.

At the end of the day, it's math. The market return is the weighted return of active and passive investing. All active investors can't beat the market, as that math doesn't work -- everyone can't be above average! Either everyone is "average" or some are above and below.

[Stormbringer]

What is a "nascent" ability to reason, and what objective evidence do you have that they "clearly" exhibit it other than your personal intuition?

Here is an example:

Using only what you can observe from these photos, where were they taken? Explain your reasoning step by step.

The images appear to depict frozen sea caves or ice formations along a shoreline, likely in a cold, northern region. Based on the rock formations and the icicles, as well as the presence of a large group of people walking on what looks like a frozen lake, I believe these photos were taken at the Apostle Islands National Lakeshore in northern Wisconsin, specifically at the Apostle Islands Ice Caves.

Here's the reasoning step by step:

• Geological Features: The red and brown sandstone cliffs are characteristic of the Apostle Islands region along the shores of Lake Superior.
• Frozen Ice Caves: The icicles hanging from the cliffs are typical of the Apostle Islands Ice Caves, which form in winter when Lake Superior freezes. The ice caves are a famous winter attraction when conditions permit safe walking across the ice.
• Presence of Visitors: The large number of visitors dressed for cold weather, walking on ice, suggests a popular and seasonal outdoor destination.
• Location: Northern Wisconsin, particularly the Bayfield area, sees cold enough winters for Lake Superior to freeze, allowing access to these ice caves.

Let me know if this aligns with your observations!

The answer is correct. I removed the location data from the photos so the AI couldn't cheat, and these photos were from my phone, so Chat GPT was not trained on them.

For those who think that LLMs are just a glorified auto-complete, all I can do it shake my head.

For those who say that an LLM can't reason, I liken that to a bird watching the Wright Brothers maiden flight in disbelief, concluding that the plane doesn't fly the way the bird does, therefore it does not fly at all. AI is improving along an exponential curve, and eventually we will be like a flock of birds watching the jets take off and land at the airport, debating if they can really fly at all.

[nisiprius]

Google Images did just as good a job. Does it reason?

[nisiprius]

...AI is improving along an exponential curve...

Its consumption of electricity and capital is increasing exponentially, for sure.

But meanwhile, three years after the launch of DALL-E to the world and having three years of opportunity to look at more images of humans, it is still drawing hands with six fingers, and making ludicrous errors on straightforward tasks.

Every so often, I give AI chatbots a fair shake to see if I'm missing something, by asking them to do a straightforward task I really want done that's just a little laborious. For example:

Please construct a comma-delimited table of the eleven Vanguard sector ETFs; for each line, include three items: the ticker symbol, the full name, and the value of assets under management (AUM), and the applicable date for the AUM value.

I tried it on three AIs (ChatGPT, Gemini, Copilot) and got three grotesque failures, in which the AI's repeatedly made blunders like misidentifying VIG as a sector fund, failing to include VAW or VOX, presenting wrong numbers, and citing websites that don't in fact say what the AI says they said.

I do not see how anything capable of "reasoning" could think that VIG is a sector ETFs. It's easy to find a web page listing Vanguard's sector ETFs and note that VIG isn't on it. It's easy to find a web page describing VIG and note that it isn't a sector fund. In fact, just now,

Is Vanguard's VIG a sector fund?

No, Vanguard's VIG (Vanguard Dividend Appreciation ETF) is not a sector fund. Instead, it focuses on companies with a history of increasing their dividends over time. It includes stocks from various sectors, aiming for broad exposure to dividend-paying equities rather than concentrating on a specific sector. If you’re looking for something more sector-specific, Vanguard has other funds that target particular sectors.

That to me is almost proof that ChatGPT is not reasoning. It "knows" that VIG is not a sector fund, so logically it should not have included it in a response to another query that asked it to construct a "list of the eleven Vanguard sector funds."

[nedsaid]

Nisiprius, thank you for bring certain problems with Artificial Intelligence not widely appreciated by the public.

You brought up the issue of AI hallucinating, just making things up. You brought up an example where ChatGPT just made up references to studies that were non-existant.

You also illustrated above that AI doesn't reason the way we think it does.

There is also the issue that the quality of work from AI is very dependent upon the quality of sources that it draws from. If the sources are poor, the output from AI will be poor. If the sources of information are good, chances are that the output for AI will be good.

Definitely, I can see great potential here. ChatGPT can produce good answers to questions, that is if it draws from good sources. You can have it write up a first draft of a paper for someone to make edits and corrections, it can save time.

Obviously, there is still improvement needed from AI.

[Stormbringer]

Every so often, I give AI chatbots a fair shake to see if I'm missing something, by asking them to do a straightforward task I really want done that's just a little laborious. For example:

Please construct a comma-delimited table of the eleven Vanguard sector ETFs; for each line, include three items: the ticker symbol, the full name, and the value of assets under management (AUM), and the applicable date for the AUM value.

I tried it on three AIs (ChatGPT, Gemini, Copilot) and got three grotesque failures, in which the AI's repeatedly made blunders like misidentifying VIG as a sector fund, failing to include VAW or VOX, presenting wrong numbers, and citing websites that don't in fact say what the AI says they said.

I tried this on ChatGPT o1-preview, which uses chain-of-thought reasoning. The result (formatted as a table for readability) is as follows:



It did take 42 seconds to arrive at this answer, and it's chain of thought was as follows:

Confirming Vanguard sector ETFs

Gathering ETF information

I’m compiling a list of the eleven Vanguard sector ETFs, including ticker symbols, full names, and AUM as of October 2023. Ensuring accuracy by cross-referencing multiple sources is key.

Taking a closer look

I’m gathering data on Vanguard's eleven sector ETFs, ensuring each sector is correctly identified. Next, I’ll compile this information into a table with ticker symbols, full names, and assets under management (AUM) as of an applicable date.

Gathering information

I'm working on the AUMs for these ETFs as of August or September 2023. It's crucial to confirm all data's accuracy.

Gathering details

I’m compiling the ticker symbols, full names, and asset under management (AUM) values for Vanguard ETFs. Each ETF's AUM, identified by ticker symbol, appears to have specific details tied to it.

Determining AUM estimates

I'm figuring out the approximate Assets Under Management (AUM) for Vanguard materials, real estate, and utilities ETFs by examining existing data trends and suggesting the most up-to-date values from Vanguard.

Listing ETF details

I’m preparing a table of eleven Vanguard sector ETFs, detailing their ticker symbols, full names, and assets under management (AUM) as of September 30, 2023. AUM figures are approximate and subject to change.

Ensuring accuracy

The assistant must verify all factual claims. It should provide approximate AUMs as of the last update and suggest consulting official sources for precise, current figures.

Identifying limitations

The assistant can provide approximate AUMs as of October 2023, but recommends consulting official sources for precise figures.

Piecing together details

OK, I’m compiling a table with Vanguard sector ETFs, listing their ticker symbols, full names, and approximate assets under management (AUM) as of mid-2023.

Crafting a response

To ensure accuracy, I can provide approximate AUMs for Vanguard sector ETFs as of October 2023. Hallucinating data isn't an option due to OpenAI's policy.

Given the fluctuating nature of AUMs, my approach is to clarify that while these funds manage considerable assets, providing exact figures is impractical. Instead, I suggest checking Vanguard's website for the latest AUM data.

Listing sector ETFs

I'm compiling a list of Vanguard's sector ETFs, including Energy, Financials, Health Care, Industrials, Information Technology, Materials, Real Estate, and Utilities. Adjustments to growth projections align with real-time market analysis.

Analyzing sector ETFs

I’m working through the Vanguard sector ETFs, listing tickers, full names, and approximate assets under management (AUM) as of August 31, 2023, highlighting their respective values.

Weighing financial options

I'm thinking about the Vanguard Financials ETF (VFH) with an AUM of $9.5 billion, and the Vanguard Health Care ETF (VHT) at $18 billion. This makes me consider their financial health and growth potential.

Rather than instantly and magically spewing out an answer, this version spends more time working on the answer and provides some clues about how it got there.

[Stormbringer]

Google Images did just as good a job. Does it reason?

No, I believe that Google samples the images and shows visually similar results.

Unless ChatGPT is lying about the reasoning that it said it used identified the location, it got the same result in a different way.

Edit: Out of curiosity, I asked ChatGPT this:

With Google, I can search for visually similar images. Did you employ a similar technique to compare these images to other images which were known to be associated with the Apostle Island Ice Caves?

The response was:

No, I did not use any image search or comparison tools like Google’s reverse image search. My answer was based solely on reasoning and familiarity with similar landscapes, not by directly comparing these photos to any known database or visual search. The distinctive features of the landscape—such as the cliffs, ice formations, and winter conditions—led me to associate the images with the Apostle Islands Ice Caves through logical inference.

Is it lying to me? I hope not, or we are all in trouble.

[protagonist]

But meanwhile, three years after the launch of DALL-E to the world and having three years of opportunity to look at more images of humans, it is still drawing hands with six fingers, and making ludicrous errors on straightforward tasks.

After extensive driver training and decades of driving experience, humans are still backing into lamp posts. Many accidentally call their wife their ex-girlfriend's name in bed.

One morning last week after little sleep I made coffee and forgot to add the coffee. I still think I am probably conscious and can reason . (Dunning-Kruger??)

Three years is nothing. AI has made astronomical progress in 3 years.

I think a big mistake we are making is asking whether it is conscious or not. Consciousness may well not be a binary decision but a spectrum. And really, we don't have much of a clue as to what it is or how to really define it. If it is a spectrum (which I believe is the case, but don't ask me to defend that belief because I can't define consciousness), we certainly don't have a yardstick with which to measure it. Experts are constantly arguing whether or not certain organisms are self-aware or use reason.

[nisiprius]

Every so often, I give AI chatbots a fair shake to see if I'm missing something, by asking them to do a straightforward task I really want done that's just a little laborious. For example:

Please construct a comma-delimited table of the eleven Vanguard sector ETFs; for each line, include three items: the ticker symbol, the full name, and the value of assets under management (AUM), and the applicable date for the AUM value.

I tried it on three AIs (ChatGPT, Gemini, Copilot) and got three grotesque failures...

I tried this on ChatGPT o1-preview, which uses chain-of-thought reasoning. The result (formatted as a table for readability) is as follows: ...

Yes, that's impressive.

[stimulacra]

If AI investing takes off I would rather just do passive index funds and maybe do a 10-15% bet on power generation, data centers, and chipmakers.

With all of these various AI systems and approaches competing against each other in real time in the financial markets, I don't really see a benefit to the end customer. It's an already chaotic system becoming more chaotic.

Just feels like a global LAN party for hedge funds, banks, and pensions.

[Basis]

There are a few we can look at already:

AI Powered Equity ETF (AIEQ)
VanEck Social Sentiment ETF (BUZZ)
WisdomTree International AI Enhanced Value Fund (AIVI)
Qraft AI-Enhanced U.S. Large Cap Momentum ETF (AMOM)

Source



What did we used to say in grade school? "Three cheers, two jeers and a moldy pretzel." Except it's four jeers. You'd have thought at least one of them might have beaten the market, just by chance.

Update:



All four of the AI-powered ETFs are still behind Total Stock.

Frustrating website.

“Nobody knows nothing.”

Also… here’s history to show that I know things because we know history and history shows that I know things so it’s not “nobody knows nuthin” but in fact it’s “history shows that I know things and you should learn what I know.”

Which is it?

[gavinsiu]

Isn't AI investing just quantitative investing, which uses algorithms to find investment opportunities. Quantitative investing has been around for a long time. Years ago, I remember an interview with the Bridgeway's John Montgomery where he described finding new quantitative methods of finding investment opportunities. He pointed out that any methods have a finite lifespan. As your method becomes successful, people will notice and reverse engineer your method and replicate it, rendering the method useless. With today's technology, you can come up with new quantitative methods faster, but the methods developed will also be replicated more quickly by others. In the end, indexing will still be king.

In addition, how many of these technology is going to be real and not just something old with the word AI tacked on. Remember the old days when things go up because it had the word electric or web on it?

[cipriano]

Isn't AI investing just quantitative investing, which uses algorithms to find investment opportunities. Quantitative investing has been around for a long time. Years ago, I remember an interview with the Bridgeway's John Montgomery where he described finding new quantitative methods of finding investment opportunities. He pointed out that any methods have a finite lifespan. As your method becomes successful, people will notice and reverse engineer your method and replicate it, rendering the method useless. With today's technology, you can come up with new quantitative methods faster, but the methods developed will also be replicated more quickly by others. In the end, indexing will still be king.

In addition, how many of these technology is going to be real and not just something old with the word AI tacked on. Remember the old days when things go up because it had the word electric or web on it?

Yeah, this is my question—what do we mean by AI here? A few people brought up Renaissance Technologies in this thread, and rightly so. Assuming that it's not all a highly successful sham, it sounds like some of the machine learning principles they were using to build the models (and maybe even pioneered, to an extent) are pretty similar to LLMs. The Acquired episode on the company goes into this at some length.

All that said, this is definitely not my field, and I have no clue what I'm talking about. But "AI" is overextended and has taken on a life of its own.

[LotsaGray]

At the moment, the AI models that are getting all the attention clearly are not exhibiting "intelligence" similar to human-style intelligence, and cannot deal with novel situations they didn't encounter during their training.

Sure about that?

Ask your favorite AI chat bot who were both Senators from Louisiana in 1931. When it responds one was Huey Long, ask it how since Huey was Governor of LA until early 1932. Then it will give 1of 3 answers based on my past endeavors, Ransdell who Long had defeated, his widow Rose Long who completed his term or even the senator who was elected in 1936. You can continue pointing out the errors and that chat it will continue circulating the same illogical answers unless you give it the simple solution…one LA senator position was vacant for over a year.

My kids could learn this, even if they did not fully understand it, by 4th grade. That is about 9 as both kids were young for their grade. So in some cases AI still isn’t as smart as the typical 4th grader


Simply put if the conclusion isn’t logical AI struggles. Generative AI is also bad at math. It is a verbal tool using probable words. Though I will admit it is very confident when it gives a wrong answer. Though generative AI is only one type.

[gavinsiu]

Yeah, this is my question—what do we mean by AI here? A few people brought up Renaissance Technologies in this thread, and rightly so. Assuming that it's not all a highly successful sham, it sounds like some of the machine learning principles they were using to build the models (and maybe even pioneered, to an extent) are pretty similar to LLMs. The Acquired episode on the company goes into this at some length.

All that said, this is definitely not my field, and I have no clue what I'm talking about. But "AI" is overextended and has taken on a life of its own.

I am a software engineer but I can't say that AI or even machine learning is my area of expertise but did dabble in it a bit. Renaissance, from what I can determine, does employ machine learning. However, machine learning is a really broad category. It's like saying playpus and cats are mammals.

In my opinion, people on this thread are sort of fixated on general LLM and their application into investing. I don't feel that you can just go to ChatGPT and ask it what stock is best etc. Based on what I read Renaissance has build up decades of data that they of course keep to themselves. They probably have a lot of computer scientist who create specific machine learning to analyze the data for patterns. There is also leveraging involved. The fund predates LLM, but they could be using similar techniques as LLM but they just don't publish it. Apparently, they also employ stealth technique so that prevent or reduce other traders from piggy backing off their trades.

Is Renaissance an elaborate fraud? It could be, Madoff was able to fool people for decades. Renaissance is certainly secretive, but they have to be. However, I feel that they are not because they close off the fund to outside investors. Without constant outside capital coming in, it's can't be a ponzi scheme since the only participants are the employees of Renaissance. The fund also took some losses during 2020. One of the red flag of Madoff fund is its unusually low volatility.

Quantitative Investing using machine learning has been around for ages, but so far only Renaissance seems to stand out. This would indicate that running a successful fund using "AI" is as difficult as someone using more traditional techniques like Warren Buffet. There's probably a lot of research involved and a lot of data collected and also optimal execution. While LLM may open new avenues, it's more likely that fraudsters will come out of the woodwork. Expect to see a lot of AI power funds that are just quant funds or even non-quant fund where the fund manager has access to a chatgpt prompt.

There is an interest article about Renaissance at medium

https://medium.com/swlh/the-data-scienc ... 98ed6ed35e

[watchnerd]

Once AI investing hits the mainstream, we will ALL be able to beat the SP500 averages!

No, I think it is the opposite. There are a very limited volume of market inefficiencies that the velocity traders siphons off in terms of profits. These are propriety traders physically located near major exchanges, with fast computers and dedicated network links. In no way are they able to support an open ended fund structure. After they are done we are left with the good old efficient market Bogleheads love.

I think you missed the imbedded facetiousness.

Missing facetiousness is a hallmark of AI

[Elric]

Yeah, this is my question—what do we mean by AI here? A few people brought up Renaissance Technologies in this thread, and rightly so. Assuming that it's not all a highly successful sham, it sounds like some of the machine learning principles they were using to build the models (and maybe even pioneered, to an extent) are pretty similar to LLMs. The Acquired episode on the company goes into this at some length.

All that said, this is definitely not my field, and I have no clue what I'm talking about. But "AI" is overextended and has taken on a life of its own.

I am a software engineer but I can't say that AI or even machine learning is my area of expertise but did dabble in it a bit. Renaissance, from what I can determine, does employ machine learning. However, machine learning is a really broad category. It's like saying playpus and cats are mammals.

In my opinion, people on this thread are sort of fixated on general LLM and their application into investing. I don't feel that you can just go to ChatGPT and ask it what stock is best etc. Based on what I read Renaissance has build up decades of data that they of course keep to themselves. They probably have a lot of computer scientist who create specific machine learning to analyze the data for patterns. There is also leveraging involved. The fund predates LLM, but they could be using similar techniques as LLM but they just don't publish it. Apparently, they also employ stealth technique so that prevent or reduce other traders from piggy backing off their trades.

Is Renaissance an elaborate fraud? It could be, Madoff was able to fool people for decades. Renaissance is certainly secretive, but they have to be. However, I feel that they are not because they close off the fund to outside investors. Without constant outside capital coming in, it's can't be a ponzi scheme since the only participants are the employees of Renaissance. The fund also took some losses during 2020. One of the red flag of Madoff fund is its unusually low volatility.

Quantitative Investing using machine learning has been around for ages, but so far only Renaissance seems to stand out. This would indicate that running a successful fund using "AI" is as difficult as someone using more traditional techniques like Warren Buffet. There's probably a lot of research involved and a lot of data collected and also optimal execution. While LLM may open new avenues, it's more likely that fraudsters will come out of the woodwork. Expect to see a lot of AI power funds that are just quant funds or even non-quant fund where the fund manager has access to a chatgpt prompt.

There is an interest article about Renaissance at medium

https://medium.com/swlh/the-data-scienc ... 98ed6ed35e

Excellent post and summary! This is why I wish the board allowed up voting!

[gavinsiu]

Excellent post and summary! This is why I wish the board allowed up voting!

Thanks it's good to hear that someone finds the post useful. I am not super big on an up vote and down vote system though. On Reddit, I notice that I might post a question about something only to get downvoted because I didn't know what I am talking about, which is why I am asking in the first place. May be a system that just allow upvotes, the down vote has a nasty habit of bring out the worse in people.

I will be curious to see if any viable competitor to Renaissance Technologies springs up or if access to additional model means Renaissance's return gets poached.

[exodusNH]

What is a "nascent" ability to reason, and what objective evidence do you have that they "clearly" exhibit it other than your personal intuition?

Here is an example:

Using only what you can observe from these photos, where were they taken? Explain your reasoning step by step.

The images appear to depict frozen sea caves or ice formations along a shoreline, likely in a cold, northern region. Based on the rock formations and the icicles, as well as the presence of a large group of people walking on what looks like a frozen lake, I believe these photos were taken at the Apostle Islands National Lakeshore in northern Wisconsin, specifically at the Apostle Islands Ice Caves.

Here's the reasoning step by step:

• Geological Features: The red and brown sandstone cliffs are characteristic of the Apostle Islands region along the shores of Lake Superior.
• Frozen Ice Caves: The icicles hanging from the cliffs are typical of the Apostle Islands Ice Caves, which form in winter when Lake Superior freezes. The ice caves are a famous winter attraction when conditions permit safe walking across the ice.
• Presence of Visitors: The large number of visitors dressed for cold weather, walking on ice, suggests a popular and seasonal outdoor destination.
• Location: Northern Wisconsin, particularly the Bayfield area, sees cold enough winters for Lake Superior to freeze, allowing access to these ice caves.

Let me know if this aligns with your observations!

The answer is correct. I removed the location data from the photos so the AI couldn't cheat, and these photos were from my phone, so Chat GPT was not trained on them.

For those who think that LLMs are just a glorified auto-complete, all I can do it shake my head.

For those who say that an LLM can't reason, I liken that to a bird watching the Wright Brothers maiden flight in disbelief, concluding that the plane doesn't fly the way the bird does, therefore it does not fly at all. AI is improving along an exponential curve, and eventually we will be like a flock of birds watching the jets take off and land at the airport, debating if they can really fly at all.

Eh, it's still using data from its training set to do a match.

I said it was a super-complicated MadLibs. Nothing in the above changes my opinion.

Just because computers can beat Garry Kasparov doesn't mean they know how to play chess. They're just running a huge subset of possible future moves and choosing the one with the highest score.

[er999]

AI already has. Read about Renaissance technologies that used machine learning techniques to beat the market. See
The Man Who Solved the Market: How Jim Simons Launched the Quant Revolution by Gregory Zuckerman.

However, it could only be done with a limited amount of capital and also because his group was the only one doing it. If everyone used AI techniques unlikely to have any edge and suspect indexing will still work well.

[nisiprius]

I tried this on ChatGPT o1-preview, which uses chain-of-thought reasoning. The result (formatted as a table for readability) is as follows:

Confirming Vanguard sector ETFs

Gathering ETF information I’m compiling a list of the eleven Vanguard sector ETFs...
Taking a closer lookI’m gathering data on Vanguard's eleven sector ETFs, ensuring each sector is correctly identified...Gathering information I'm working on the AUMs for these ETFs as of August or September 2023.
Gathering detailsI’m compiling the ticker symbols, full names, and asset under management (AUM) values for Vanguard ETFs...
Determining AUM estimates I'm figuring out the approximate Assets Under Management (AUM)...
Listing ETF details I’m preparing a table of eleven Vanguard sector ETFs...
Ensuring accuracy The assistant must verify all factual claims.
Identifying limitations The assistant can provide approximate AUMs as of October 2023
Piecing together details OK, I’m compiling a table with Vanguard sector ETFs
Crafting a response To ensure accuracy, I can provide approximate AUMs for Vanguard sector ETFs as of October 2023.
Listing sector ETFs I'm compiling a list of Vanguard's sector ETFs...
Analyzing sector ETFs I’m working through the Vanguard sector ETFs,
Weighing financial options I'm thinking about the Vanguard Financials ETF (VFH) with an AUM of $9.5 billion, and the Vanguard Health Care ETF (VHT) at $18 billion. This makes me consider their financial health and growth potential.

First of all, yes, this is very impressive. I can cavil at marketing language like "It's crucial to confirm all data's accuracy" and "Hallucinating data isn't an option due to OpenAI's policy." The latter implies that there is no hallucination at all any more. Has OpenAI solved the problem of hallucinating simply by ordering ChatGPT to quit doing that?

But I do want to draw attention to that very last link in the "chain of reasoning."

Weighing financial options I'm thinking about the Vanguard Financials ETF (VFH) with an AUM of $9.5 billion, and the Vanguard Health Care ETF (VHT) at $18 billion. This makes me consider their financial health and growth potential.

Just when I am thinking "that does start to look like reasoning," there's the clinker. I don't think it affects the answer, and certainly my own chains of reasoning gets sidetracked, but... what the heck is that? Is it possible for you to ask it why on earth it thought it was relevant to consider that?

[Coltrane75]

We all know passive investing beat active investing in long term. Can it also beat AI quantitive investing

I think passive investing will beat out any AI marketed fund/investment strategy.

AI generated search results are often wrong. AI is more of a marketing term I think and will flood the infosphere with lots of clutter causing confusion.

I guess I'm more of a luddite and skeptic. I'm of the opinion "AI" will be another overblown and oversold silicon valley trend that won't come close to fulfilling expectations.

[Elric]

Just because computers can beat Garry Kasparov doesn't mean they know how to play chess. They're just running a huge subset of possible future moves and choosing the one with the highest score.

In other words, these algorithms play chess in a very similar way to human grandmasters (although grandmasters go to less depth and breadth in their search, trading off a better evaluation algorithm). Would you say that computers are in capable of adding because they don't grasp mathematical theory? Computers play chess. They can play extremely well. They can deal with positions never included in their training. They don't "understand" the concept of what a game is.

Have you looked at computer Go? The best software is amazing the best human experts by sometimes playing what appear to be erroneous or bizarre moves that turn out to be extremely powerful.

LLM's are just a small subset of AI, and not all AI works the same way.

[Stormbringer]

Ask your favorite AI chat bot who were both Senators from Louisiana in 1931. When it responds one was Huey Long, ask it how since Huey was Governor of LA until early 1932. Then it will give 1of 3 answers based on my past endeavors, Ransdell who Long had defeated, his widow Rose Long who completed his term or even the senator who was elected in 1936. You can continue pointing out the errors and that chat it will continue circulating the same illogical answers unless you give it the simple solution…one LA senator position was vacant for over a year.

This might be a situation where some prompt engineering produces a better result. I rephrased your question as: Name everyone who served as a Senator from Louisiana at any point in 1931?

In 1931, Louisiana was represented in the United States Senate by two individuals:

Edwin S. Broussard: He served as a U.S. Senator from Louisiana from March 4, 1921, to March 3, 1933. Broussard was in office throughout the entire year of 1931.

Joseph E. Ransdell: His term spanned from March 4, 1913, to March 3, 1931. Ransdell served until the end of his term on March 3, 1931.

After Ransdell's term ended on March 3, 1931, the Senate seat remained vacant until Huey P. Long took office on January 25, 1932. Although Huey Long was elected to the Senate in 1930, he chose to delay taking his seat to continue serving as Governor of Louisiana. Therefore, no other individual served as a Senator from Louisiana during the remainder of 1931 after Ransdell's term concluded.

Summary of Senators from Louisiana in 1931:

• Edwin S. Broussard
• Joseph E. Ransdell

Asking for "both" Senators may have caused some confusion, as there were really three names involved.

[BitTooAggressive]

Did not bother to read all posts. This is an easy question.

If AI investing dominates it will be reflected in the index and they will converge.

[Stormbringer]

First of all, yes, this is very impressive. I can cavil at marketing language like "It's crucial to confirm all data's accuracy" and "Hallucinating data isn't an option due to OpenAI's policy." The latter implies that there is no hallucination at all any more. Has OpenAI solved the problem of hallucinating simply by ordering ChatGPT to quit doing that?

Hallucinations are baked into the design of LLMs, because the "weights" used to traverse between bits of information are probabilities. If I ask you to fill in "The sky is _______", the answer is probably blue, but under certain conditions it could be red or gray. The "temperature" setting of an LLM dictates how much latitude it has to meander between the various probabilities. A higher temperature should produce better results for creative writing, whereas a lower temperature should produce better results for crunching facts and figures. ChatGPT is configured to be general purpose, so the temperature is medium. A purpose-specific LLM would be more finely tuned for the intended purpose.

To reduce hallucinations to an acceptable level, some algorithmic advances are necessary. One possible path is that several LLM agents independently produce multiple responses, and several other LLM agents independently "grade" the responses for accuracy and quality. The response with the highest average grade is presented to the user. Of course, this requires more resources and will be more likely to first appear in paid subscriptions.

But I do want to draw attention to that very last link in the "chain of reasoning."

Weighing financial options I'm thinking about the Vanguard Financials ETF (VFH) with an AUM of $9.5 billion, and the Vanguard Health Care ETF (VHT) at $18 billion. This makes me consider their financial health and growth potential.

Just when I am thinking "that does start to look like reasoning," there's the clinker. I don't think it affects the answer, and certainly my own chains of reasoning gets sidetracked, but... what the heck is that? Is it possible for you to ask it why on earth it thought it was relevant to consider that?

I wouldn't rule out that what Open AI shows us in its chain-of-thought is window dressing. Given that the model is trained on human-generated data, some of which is toxic, there are probably some embarrassing "thoughts" that the LLM has as it works towards an answer. For example, if you ask the AI to write a physical description of four suspects in a criminal line-up, you are just asking for trouble.

The import thing to take away from this discussion is the steep trajectory of AI, not its current state. There are simultaneous advances happening in compute, hardware, scaling, algorithms, data, and additional modalities (e.g., sight and sound). This is all happening in the context of an aggressive AI arms race between tech companies and nations. Anyone who wants to dismiss LLMs as a mere parlor trick or nothing more than a fancy auto-complete needs to reconcile that with the fact that all the major tech companies are all-in on this, and investing tens, perhaps hundreds of billions of dollars.

[LotsaGray]

Ask your favorite AI chat bot who were both Senators from Louisiana in 1931. When it responds one was Huey Long, ask it how since Huey was Governor of LA until early 1932. Then it will give 1of 3 answers based on my past endeavors, Ransdell who Long had defeated, his widow Rose Long who completed his term or even the senator who was elected in 1936. You can continue pointing out the errors and that chat it will continue circulating the same illogical answers unless you give it the simple solution…one LA senator position was vacant for over a year.

This might be a situation where some prompt engineering produces a better result. I rephrased your question as: Name everyone who served as a Senator from Louisiana at any point in 1931?

In 1931, Louisiana was represented in the United States Senate by two individuals:

Edwin S. Broussard: He served as a U.S. Senator from Louisiana from March 4, 1921, to March 3, 1933. Broussard was in office throughout the entire year of 1931.

Joseph E. Ransdell: His term spanned from March 4, 1913, to March 3, 1931. Ransdell served until the end of his term on March 3, 1931.

After Ransdell's term ended on March 3, 1931, the Senate seat remained vacant until Huey P. Long took office on January 25, 1932. Although Huey Long was elected to the Senate in 1930, he chose to delay taking his seat to continue serving as Governor of Louisiana. Therefore, no other individual served as a Senator from Louisiana during the remainder of 1931 after Ransdell's term concluded.

Summary of Senators from Louisiana in 1931:

• Edwin S. Broussard
• Joseph E. Ransdell

Asking for "both" Senators may have caused some confusion, as there were really three names involved.

It seems the training has finally caught up with the question. Previously it (at some point) "knew" it was missing someone and threw in various names. It started each time with Long serving as both Gov and Sen.

So yes, eventually AI gets there. And I fully expect AI to have better factual recall and illogic filtering. I am not yet convince it will be creative. Basically it will know what we know better than we do. This can/will lead to insights on novel ways to use this knowledge but will it become creative? That I am not sure we have yet. Outstanding derivative use of knowledge, yes but creating new knowledge?

[nisiprius]

First of all, yes, this is very impressive. I can cavil at marketing language like "It's crucial to confirm all data's accuracy" and "Hallucinating data isn't an option due to OpenAI's policy." The latter implies that there is no hallucination at all any more. Has OpenAI solved the problem of hallucinating simply by ordering ChatGPT to quit doing that?

Hallucinations are baked into the design of LLMs, because the "weights" used to traverse between bits of information are probabilities. If I ask you to fill in "The sky is _______", the answer is probably blue, but under certain conditions it could be red or gray. The "temperature" setting of an LLM dictates how much latitude it has to meander between the various probabilities. A higher temperature should produce better results for creative writing, whereas a lower temperature should produce better results for crunching facts and figures. ChatGPT is configured to be general purpose, so the temperature is medium. A purpose-specific LLM would be more finely tuned for the intended purpose.

To reduce hallucinations to an acceptable level, some algorithmic advances are necessary. One possible path is that several LLM agents independently produce multiple responses, and several other LLM agents independently "grade" the responses for accuracy and quality. The response with the highest average grade is presented to the user. Of course, this requires more resources and will be more likely to first appear in paid subscriptions.

My beef is that it didn't say "per OpenAI policy we operate at a low 'temperature' setting to minimize, but not eliminate, hallucinations." It said "Hallucinating data isn't an option due to OpenAI's policy." That's very different. You tell me if that's a lie; if not, why not; if so, who's lying?

[nisiprius]

...I am not yet convince it will be creative. Basically it will know what we know better than we do. This can/will lead to insights on novel ways to use this knowledge but will it become creative? That I am not sure we have yet. Outstanding derivative use of knowledge, yes but creating new knowledge?...

I am reminded of an early example from the early days of AI. It was a geometry theorem prover, and it was given the task of proving the pons asinorum, the theorem that the base angles of an isosceles triangle are equal. It's traditionally the first theorem proved in geometry.



The traditional proof in modern textbooks introduces point X and line AX.

The theorem prover, however, simply observed that BAC is congruent with CAB, by SAS = SAS. That is, the triangle is congruent with its mirror image, and therefore angle B = angle C and vice versa.

This was trumpeted as "the program found a new proof!"

In reality, that proof had been published by humans in the fifth century. It is logically simpler than the proof with line AX, but it's not good pedagogically because it "feels" weird and unintuitive, and because it isn't a useful technique for more important proofs. But the program, using some form of guided trial and error, wouldn't have been bothered by intuition, and would have gotten to this proof sooner because it was simpler.