New version of PortfolioVisualizer?

Discuss all general (i.e. non-personal) investing questions and issues, investing news, and theory.
User avatar
txaggie
Posts: 146
Joined: Sat Sep 09, 2017 11:35 pm

Re: New version of PortfolioVisualizer?

Post by txaggie »

beachtech wrote: Sun May 19, 2024 6:35 am I’ve been slowly working on https://totalrealreturns.com/ as a side project for many years. Basically stock/ETF/mutual fund charts with dividend reinvestment, inflation adjustment, log scale, %/year trendline, etc.

What are the minimal set of features you’d need to make this a decent replacement?
beachtech, your totalreturns site is great! I really like it and have been using it for a while.

The minimal feature that I would like to see is accurate data. When I compare the total return chart sites that I use (Morningstar charts, portfolio visualizer, totalreturns.com, and testfol.io) it seems like only Morningstar and portfolio visualizer have accurate data that I can trust.

Take a look at the 30-year total return of DFA US Small Cap Value (DFSVX) fund vs Vanguard Total Stock Market (VTSMX) fund. Morningstar correctly reports US SCV outperforming US TSM while totalreturns.com and testfol.io both get it wrong.

Morningstar
Image
Image

testfol.io
Image

https://totalrealreturns.com/n/DFSVX,VT ... 2024-05-23
Image
comeinvest
Posts: 2877
Joined: Mon Mar 12, 2012 6:57 pm

Re: New version of PortfolioVisualizer?

Post by comeinvest »

txaggie wrote: Sun May 26, 2024 3:32 pm
beachtech wrote: Sun May 19, 2024 6:35 am I’ve been slowly working on https://totalrealreturns.com/ as a side project for many years. Basically stock/ETF/mutual fund charts with dividend reinvestment, inflation adjustment, log scale, %/year trendline, etc.

What are the minimal set of features you’d need to make this a decent replacement?
beachtech, your totalreturns site is great! I really like it and have been using it for a while.

The minimal feature that I would like to see is accurate data. When I compare the total return chart sites that I use (Morningstar charts, portfolio visualizer, totalreturns.com, and testfol.io) it seems like only Morningstar and portfolio visualizer have accurate data that I can trust.

Take a look at the 30-year total return of DFA US Small Cap Value (DFSVX) fund vs Vanguard Total Stock Market (VTSMX) fund. Morningstar correctly reports US SCV outperforming US TSM while totalreturns.com and testfol.io both get it wrong.
Perhaps totalreturns and testfol copied from each other, or use the same faulty data source ;) None of them reveal either their identity or their data sources on the site. Now with the identical bugs on hand, we can perhaps reverse-engineer which data source they use, or from which public site they scrape the data off.
KyleAAA
Posts: 9554
Joined: Wed Jul 01, 2009 5:35 pm
Contact:

Re: New version of PortfolioVisualizer?

Post by KyleAAA »

jbowman wrote: Sat May 25, 2024 7:44 am
KyleAAA wrote: Fri May 24, 2024 6:06 pm
jbowman wrote: Fri May 24, 2024 2:27 pm Presumably this tool is only using daily closing prices, which seem to be widely available for free for end users without any need to scrape.
They are not widely available for free for commercial use.
jbowman wrote: Fri May 24, 2024 2:27 pm edit: So depending on how many requests you are receiving, it could conceivably average out to fractions of pennies per use. Back of napkin math, you'd need ~100k uses per month to drive the cost down to a penny even if it cost you $1000 a month for data access. No clue how many requests PV services monthly but that is hardly a crazy number and that $1000 is apparently on the high side.
Sure, it COULD be fractions of a penny per use if your usage pattern and requirements support it. But a fraction of a penny many times is still a lot of money. And costs will typically scale with usage because that's usually part of the contract regardless of what it costs the data provider i.e. you're getting more value if you show the data to twice as many people, so they'll charge you more and the extra is all profit for them. And that's just one of the costs. $1000 definitely is NOT on the high side for comprehensive financial data. I've priced and negotiated with a number of providers for commercial products. You could easily pay over $100k per month for some complex use cases with high traffic. But even if it was only $1000 per month, few are going to be willing bear that cost for a non-commercial project. The hosting costs themselves are likely to be low for something like PV, probably something in the $500-1000 per month range. Then you have the labor costs to build and maintain it. Even if you do it yourself, your time still has value. Then there are the SaaS and software tools you need to pay for to run a small business like this. If I had to guess, my best estimate is their operating costs are something like $20-30k per month assuming they're extremely lean and don't have more than 1 or 2 employees.
I'm certainly not going to dispute that proprietary data can be expensive. Especially if you need accurate, detailed, fine-grain data and ongoing access to updates.

However, I feel like you the glossed over the part where there appears to be such services actively advertised for commercial use (i.e. not internal-use only) for the prices I listed to bulk download historical end of day closing prices.

Assuming that is all they need, then their data costs specifically would not be significant. And that shouldn't be surprising. If it actually cost them $100k/month to license the data, it would never have been offered for free.

It would be hard to make any assessment of the rest of their costs without knowing how many requests they're getting on average, but again it can't be that exorbitant or it wouldn't have been offered all this time for free sans advertising.
Sorry, but I've actually priced this stuff out in extreme detail. I know what I'm talking about. Go read all the licensing agreements and think through how the restrictions may or may not impact your implementation and get back to me. If you don't believe me, I'm BEGGING you to go design this tool and come back with an exact cost estimate. I've done this exercise. You could also just ask why PV doesn't use those commercial data sources you think are sufficient if they are, in fact, sufficient and cheap. Do they just not like money?

The data backing Morningstar costs them considerably more than $100k/month and they display it for free. So of course somebody would do that. They have/are. I never once said nor implied PV specifically paid $100k/month. In fact I explicitly said the opposite.
Last edited by KyleAAA on Mon May 27, 2024 1:57 am, edited 2 times in total.
mbouck
Posts: 149
Joined: Wed Jan 24, 2024 2:08 pm

Re: New version of PortfolioVisualizer?

Post by mbouck »

txaggie wrote: Sun May 26, 2024 3:32 pm it seems like only Morningstar and portfolio visualizer have accurate data that I can trust.

Take a look at the 30-year total return of DFA US Small Cap Value (DFSVX) fund vs Vanguard Total Stock Market (VTSMX) fund. Morningstar correctly reports US SCV outperforming US TSM while totalreturns.com and testfol.io both get it wrong.
Interesting that those two sites also charge for their service. Some things are worth paying for...
comeinvest
Posts: 2877
Joined: Mon Mar 12, 2012 6:57 pm

Re: New version of PortfolioVisualizer?

Post by comeinvest »

KyleAAA wrote: Mon May 27, 2024 1:09 am
jbowman wrote: Sat May 25, 2024 7:44 am
KyleAAA wrote: Fri May 24, 2024 6:06 pm
jbowman wrote: Fri May 24, 2024 2:27 pm Presumably this tool is only using daily closing prices, which seem to be widely available for free for end users without any need to scrape.
They are not widely available for free for commercial use.
jbowman wrote: Fri May 24, 2024 2:27 pm edit: So depending on how many requests you are receiving, it could conceivably average out to fractions of pennies per use. Back of napkin math, you'd need ~100k uses per month to drive the cost down to a penny even if it cost you $1000 a month for data access. No clue how many requests PV services monthly but that is hardly a crazy number and that $1000 is apparently on the high side.
Sure, it COULD be fractions of a penny per use if your usage pattern and requirements support it. But a fraction of a penny many times is still a lot of money. And costs will typically scale with usage because that's usually part of the contract regardless of what it costs the data provider i.e. you're getting more value if you show the data to twice as many people, so they'll charge you more and the extra is all profit for them. And that's just one of the costs. $1000 definitely is NOT on the high side for comprehensive financial data. I've priced and negotiated with a number of providers for commercial products. You could easily pay over $100k per month for some complex use cases with high traffic. But even if it was only $1000 per month, few are going to be willing bear that cost for a non-commercial project. The hosting costs themselves are likely to be low for something like PV, probably something in the $500-1000 per month range. Then you have the labor costs to build and maintain it. Even if you do it yourself, your time still has value. Then there are the SaaS and software tools you need to pay for to run a small business like this. If I had to guess, my best estimate is their operating costs are something like $20-30k per month assuming they're extremely lean and don't have more than 1 or 2 employees.
I'm certainly not going to dispute that proprietary data can be expensive. Especially if you need accurate, detailed, fine-grain data and ongoing access to updates.

However, I feel like you the glossed over the part where there appears to be such services actively advertised for commercial use (i.e. not internal-use only) for the prices I listed to bulk download historical end of day closing prices.

Assuming that is all they need, then their data costs specifically would not be significant. And that shouldn't be surprising. If it actually cost them $100k/month to license the data, it would never have been offered for free.

It would be hard to make any assessment of the rest of their costs without knowing how many requests they're getting on average, but again it can't be that exorbitant or it wouldn't have been offered all this time for free sans advertising.
Sorry, but I've actually priced this stuff out in extreme detail. I know what I'm talking about. Go read all the licensing agreements and think through how the restrictions may or may not impact your implementation and get back to me. If you don't believe me, I'm BEGGING you to go design this tool and come back with an exact cost estimate. I've done this exercise

The data backing Morningstar costs them considerably more than $100k/month and they display it for free. So of course somebody would do that. They have/are. I never once said nor implied PV specifically paid $100k/month. In fact I explicitly said the opposite.
It seems like the overwhelming bulk of the cost of building a PV clone would be the cost data. The rest could be either part of a hobby or volunteer or relatively small one-time effort, except some limited ongoing maintenance, if the data itself is already clean i.e. high quality; or would be a fixed cost or scale disproportionally less when the traffic increases.
Perhaps there should be more competition among data providers, or among the actual source ownership of the data, if the exchanges are at the top of the food chain who own the data. Nothing in this day and age scales with usage per item; or else we would still pay $0.02 per minute for modem dialup time or per kilobyte downloaded.
Last edited by comeinvest on Mon May 27, 2024 1:56 am, edited 1 time in total.
KyleAAA
Posts: 9554
Joined: Wed Jul 01, 2009 5:35 pm
Contact:

Re: New version of PortfolioVisualizer?

Post by KyleAAA »

comeinvest wrote: Mon May 27, 2024 1:52 am [ Nothing in this day and age scales with usage per item; or else we would still pay $0.02 per minute for modem dialup.
Who do you think is saying anything here scales with usage per item?
comeinvest
Posts: 2877
Joined: Mon Mar 12, 2012 6:57 pm

Re: New version of PortfolioVisualizer?

Post by comeinvest »

KyleAAA wrote: Mon May 27, 2024 1:55 am
comeinvest wrote: Mon May 27, 2024 1:52 am [ Nothing in this day and age scales with usage per item; or else we would still pay $0.02 per minute for modem dialup.
Who do you think is saying anything here scales with usage per item?
I thought you or someone else said a few posts up that the cost of data would scale with usage. "costs will typically scale with usage because that's usually part of the contract". Perhaps not linear, but what do I know. I'm learning as I'm reading this thread. I think there would be at least a few folks in this thread capable and willing to program a PV clone, if data was available and affordable.
User avatar
nisiprius
Advisory Board
Posts: 53155
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

PortfolioVisualizer down?

Post by nisiprius »

Hmmmmm....

Image
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
dcabler
Posts: 5060
Joined: Wed Feb 19, 2014 10:30 am
Location: TX

Re: New version of PortfolioVisualizer?

Post by dcabler »

I might have missed it in the thread, but the legacy UI is still available (for now, at least): https://legacy.portfoliovisualizer.com/

But it still has the 10 year limitation for backtesting without an account.

Cheers
Call_Me_Op
Posts: 9984
Joined: Mon Sep 07, 2009 2:57 pm
Location: Milky Way

Re: New version of PortfolioVisualizer?

Post by Call_Me_Op »

anagram wrote: Sat May 18, 2024 5:21 pm
avalpert1 wrote: Sat May 18, 2024 5:11 pm Looks like they are trying to monetize the toolset, can't really blame them for that.
Yup. Fee for use next.
Indeed. It seems the modern business model is to offer something for free for long enough to get a lot of people hooked, and then suddenly block access and request payment for the service. Back to the old Simba spreadsheet.

Even if something is not monetized, it seems that "good enough" is never left alone. There seems to always be the need to change everything (change for the sake of change), even things that are working fine - often with the result that the thing becomes much harder to use.
Best regards, -Op | | "In the middle of difficulty lies opportunity." Einstein
Call_Me_Op
Posts: 9984
Joined: Mon Sep 07, 2009 2:57 pm
Location: Milky Way

Re: New version of PortfolioVisualizer?

Post by Call_Me_Op »

dcabler wrote: Mon May 27, 2024 6:38 am I might have missed it in the thread, but the legacy UI is still available (for now, at least): https://legacy.portfoliovisualizer.com/

But it still has the 10 year limitation for backtesting without an account.

Cheers
I tried it and it didn't work - application error.
Best regards, -Op | | "In the middle of difficulty lies opportunity." Einstein
User avatar
privatefarmer
Posts: 780
Joined: Mon Sep 08, 2014 2:45 pm

Re: New version of PortfolioVisualizer?

Post by privatefarmer »

anyone know why PV is down? I hope they didn't go under or close the site... I just re-upped my expensive subscription with them...
dcabler
Posts: 5060
Joined: Wed Feb 19, 2014 10:30 am
Location: TX

Re: New version of PortfolioVisualizer?

Post by dcabler »

Call_Me_Op wrote: Mon May 27, 2024 6:52 am
dcabler wrote: Mon May 27, 2024 6:38 am I might have missed it in the thread, but the legacy UI is still available (for now, at least): https://legacy.portfoliovisualizer.com/

But it still has the 10 year limitation for backtesting without an account.

Cheers
I tried it and it didn't work - application error.
Worked for me just before I made the post when I had it backtest a particular portfolio using ETFs.

Cheers.
gavinsiu
Posts: 5392
Joined: Sun Nov 14, 2021 11:42 am

Re: New version of PortfolioVisualizer?

Post by gavinsiu »

Call_Me_Op wrote: Mon May 27, 2024 6:52 am I tried it and it didn't work - application error.
I suspect PV is down.
jbowman
Posts: 240
Joined: Sat Jan 09, 2021 4:34 pm

Re: New version of PortfolioVisualizer?

Post by jbowman »

KyleAAA wrote: Mon May 27, 2024 1:09 am
jbowman wrote: Sat May 25, 2024 7:44 am
KyleAAA wrote: Fri May 24, 2024 6:06 pm
jbowman wrote: Fri May 24, 2024 2:27 pm Presumably this tool is only using daily closing prices, which seem to be widely available for free for end users without any need to scrape.
They are not widely available for free for commercial use.
jbowman wrote: Fri May 24, 2024 2:27 pm edit: So depending on how many requests you are receiving, it could conceivably average out to fractions of pennies per use. Back of napkin math, you'd need ~100k uses per month to drive the cost down to a penny even if it cost you $1000 a month for data access. No clue how many requests PV services monthly but that is hardly a crazy number and that $1000 is apparently on the high side.
Sure, it COULD be fractions of a penny per use if your usage pattern and requirements support it. But a fraction of a penny many times is still a lot of money. And costs will typically scale with usage because that's usually part of the contract regardless of what it costs the data provider i.e. you're getting more value if you show the data to twice as many people, so they'll charge you more and the extra is all profit for them. And that's just one of the costs. $1000 definitely is NOT on the high side for comprehensive financial data. I've priced and negotiated with a number of providers for commercial products. You could easily pay over $100k per month for some complex use cases with high traffic. But even if it was only $1000 per month, few are going to be willing bear that cost for a non-commercial project. The hosting costs themselves are likely to be low for something like PV, probably something in the $500-1000 per month range. Then you have the labor costs to build and maintain it. Even if you do it yourself, your time still has value. Then there are the SaaS and software tools you need to pay for to run a small business like this. If I had to guess, my best estimate is their operating costs are something like $20-30k per month assuming they're extremely lean and don't have more than 1 or 2 employees.
I'm certainly not going to dispute that proprietary data can be expensive. Especially if you need accurate, detailed, fine-grain data and ongoing access to updates.

However, I feel like you the glossed over the part where there appears to be such services actively advertised for commercial use (i.e. not internal-use only) for the prices I listed to bulk download historical end of day closing prices.

Assuming that is all they need, then their data costs specifically would not be significant. And that shouldn't be surprising. If it actually cost them $100k/month to license the data, it would never have been offered for free.

It would be hard to make any assessment of the rest of their costs without knowing how many requests they're getting on average, but again it can't be that exorbitant or it wouldn't have been offered all this time for free sans advertising.
Sorry, but I've actually priced this stuff out in extreme detail. I know what I'm talking about. Go read all the licensing agreements and think through how the restrictions may or may not impact your implementation and get back to me. If you don't believe me, I'm BEGGING you to go design this tool and come back with an exact cost estimate. I've done this exercise. You could also just ask why PV doesn't use those commercial data sources you think are sufficient if they are, in fact, sufficient and cheap. Do they just not like money?

The data backing Morningstar costs them considerably more than $100k/month and they display it for free. So of course somebody would do that. They have/are. I never once said nor implied PV specifically paid $100k/month. In fact I explicitly said the opposite.
Ok, so what it sounds like you're saying is those services I mentioned are not sufficient. That's fine. You claim to have a lot of expertise in this, while I've thought about it for all of 10 minutes and a couple Google searches. It would be asinine for me to argue the point further.

I would however flip your question back around and ask why you are convinced they aren't using a service like that? Presumably, as you say, they like money. Is the quality of the data bad or do they actually need more fine-grabbed data than simple closing prices? What is your read on why they were offering expensive data for free without an obvious monetization strategy all these years?
beachtech
Posts: 98
Joined: Wed Sep 14, 2016 11:23 am
Contact:

Re: New version of PortfolioVisualizer?

Post by beachtech »

comeinvest wrote: Sun May 26, 2024 7:01 pm Perhaps totalreturns and testfol copied from each other, or use the same faulty data source ;) None of them reveal either their identity or their data sources on the site. Now with the identical bugs on hand, we can perhaps reverse-engineer which data source they use, or from which public site they scrape the data off.
As I mentioned earlier, https://totalrealreturns.com/ uses data from https://eodhd.com/

I appreciate txaggie bringing up data quality. I haven’t had time to look into this symbol DSFVX closely yet. But I don’t yet know for certain which data source is actually correct: all we know is that the outputs of these tools disagree. (For all I know, it’s at least possible that Morningstar and PV are wrong.) To get to the ground truth, we’d need to obtain (perhaps from DFA themselves?) a “known true” spreadsheet of all closing prices and distributions going back 30 years.

When I look at only calendar year 2023 (which means the total return from the close on 2022-12-30 through 2023-12-29):

https://totalrealreturns.com/n/DFSVX?st ... 2023-12-29 shows a total return of +19.0%

testfol.io shows the same ($11902.42 ending balance, or +19.02%)

DFA’s website shows +18.91%

Yahoo Finance shows +18.91%.

Zacks shows +18.93%.

Was not able to get data from Portfolio Visualizer since it’s down at the moment.

For 2023, these are all pretty close. It’s definitely possible that any discrepancy is in older data only.

If the fund companies would publish an official historical data spreadsheet with daily closing prices and distributions, this would be easy to figure out. Of course, if they all did that, we wouldn’t need the intermediary data aggregators at all. One can dream…
Hydromod
Posts: 1077
Joined: Tue Mar 26, 2019 10:21 pm

Re: New version of PortfolioVisualizer?

Post by Hydromod »

beachtech wrote: Mon May 27, 2024 9:44 am
comeinvest wrote: Sun May 26, 2024 7:01 pm Perhaps totalreturns and testfol copied from each other, or use the same faulty data source ;) None of them reveal either their identity or their data sources on the site. Now with the identical bugs on hand, we can perhaps reverse-engineer which data source they use, or from which public site they scrape the data off.
As I mentioned earlier, https://totalrealreturns.com/ uses data from https://eodhd.com/

I appreciate txaggie bringing up data quality. I haven’t had time to look into this symbol DSFVX closely yet. But I don’t yet know for certain which data source is actually correct: all we know is that the outputs of these tools disagree. (For all I know, it’s at least possible that Morningstar and PV are wrong.) To get to the ground truth, we’d need to obtain (perhaps from DFA themselves?) a “known true” spreadsheet of all closing prices and distributions going back 30 years.
I asked the creator of testfol.io about this issue. testfol.io uses EODHD, like totalreturns.

The response was that EODHD missed recording some capital gains as dividends, most or all in the 1990s. PSLDX from EODHD matches morningstar though, and PSLDX produces a lot of capital gains.

Getting the data exactly right seems to be a fraught thing, especially with older data. Dividends, splits, and capital gains are harder to get right. I've had issues with yahoo finance, for example. And I've heard complaints about how timely the data is when folded into feeds at the intraday scale (which is neither here nor there for bogleheads, but still).
Hydromod
Posts: 1077
Joined: Tue Mar 26, 2019 10:21 pm

Re: New version of PortfolioVisualizer?

Post by Hydromod »

EODHD has added the missing capital gains distributions to DSFVX. On a holiday. Everything should be in sync now.
User avatar
txaggie
Posts: 146
Joined: Sat Sep 09, 2017 11:35 pm

Re: New version of PortfolioVisualizer?

Post by txaggie »

Hydromod wrote: Mon May 27, 2024 12:11 pm EODHD has added the missing capital gains distributions to DSFVX. On a holiday. Everything should be in sync now.
Nice! I am impressed that this was corrected during a holiday. The DFSVX total return is already updated and displaying correctly on testfol.io.
Hydromod
Posts: 1077
Joined: Tue Mar 26, 2019 10:21 pm

Re: New version of PortfolioVisualizer?

Post by Hydromod »

txaggie wrote: Mon May 27, 2024 2:31 pm
Hydromod wrote: Mon May 27, 2024 12:11 pm EODHD has added the missing capital gains distributions to DSFVX. On a holiday. Everything should be in sync now.
Nice! I am impressed that this was corrected during a holiday. The DFSVX total return is already updated and displaying correctly on testfol.io.
It turns out that EODHD is a French company, so not a holiday after all. But still, getting it fixed in just a couple of hours is impressive.
Nathan Drake
Posts: 6414
Joined: Mon Apr 11, 2011 12:28 am

Re: New version of PortfolioVisualizer?

Post by Nathan Drake »

Hydromod wrote: Mon May 27, 2024 3:16 pm
txaggie wrote: Mon May 27, 2024 2:31 pm
Hydromod wrote: Mon May 27, 2024 12:11 pm EODHD has added the missing capital gains distributions to DSFVX. On a holiday. Everything should be in sync now.
Nice! I am impressed that this was corrected during a holiday. The DFSVX total return is already updated and displaying correctly on testfol.io.
It turns out that EODHD is a French company, so not a holiday after all. But still, getting it fixed in just a couple of hours is impressive.
Someone requested it to be fixed?
20% VOO | 20% VXUS | 20% AVUV | 20% AVDV | 20% AVES
Hydromod
Posts: 1077
Joined: Tue Mar 26, 2019 10:21 pm

Re: New version of PortfolioVisualizer?

Post by Hydromod »

Nathan Drake wrote: Tue May 28, 2024 11:12 am Someone requested it to be fixed?
I brought up the issue reported by TxAggie to the testfol.io mastermind, who identified the discrepancy, passed on the issue to EODHD, and verified that the fix from EODHD was correct.
User avatar
nisiprius
Advisory Board
Posts: 53155
Joined: Thu Jul 26, 2007 9:33 am
Location: The terrestrial, globular, planetary hunk of matter, flattened at the poles, is my abode.--O. Henry

Re: New version of PortfolioVisualizer?

Post by nisiprius »

I haven't taken more than a casual look but testfol.io looks pretty interesting.
Annual income twenty pounds, annual expenditure nineteen nineteen and six, result happiness; Annual income twenty pounds, annual expenditure twenty pounds ought and six, result misery.
KyleAAA
Posts: 9554
Joined: Wed Jul 01, 2009 5:35 pm
Contact:

Re: New version of PortfolioVisualizer?

Post by KyleAAA »

comeinvest wrote: Mon May 27, 2024 1:57 am
KyleAAA wrote: Mon May 27, 2024 1:55 am
comeinvest wrote: Mon May 27, 2024 1:52 am [ Nothing in this day and age scales with usage per item; or else we would still pay $0.02 per minute for modem dialup.
Who do you think is saying anything here scales with usage per item?
I thought you or someone else said a few posts up that the cost of data would scale with usage. "costs will typically scale with usage because that's usually part of the contract". Perhaps not linear, but what do I know. I'm learning as I'm reading this thread. I think there would be at least a few folks in this thread capable and willing to program a PV clone, if data was available and affordable.
That's often the way the contracts are structured, but not always. It isn't because of a technology limitation, though. And the scale-up usually isn't linear. But many times you pay more if you show more even if it cost them nothing. The backtest functionality alone would not be too bad to build as a hobby, although if you're using cheap data sources you will have data problems and there will be plenty tickers that just won't work for whatever reason. But the backtest functionality is just a very small part of PV. But sure, you could do it. I can program it myself if I had the time and energy to do so but it's much too large a project for me to want to take on as a hobby, and it would require quite a lot of effort to maintain. If you have the free time, go for it. It will be quite a bit worse than PV but maybe that's sufficient for your needs.
User avatar
arcticpineapplecorp.
Posts: 15727
Joined: Tue Mar 06, 2012 8:22 pm

Re: New version of PortfolioVisualizer?

Post by arcticpineapplecorp. »

am i missing something here?

I went to check on bnd returns ytd and vanguard is showing -1.87% (through 5/30/24) :

Image

but testfol.io is showing -4.43% CAGR (even though the balance is down -1.83% from $10,000 to $9817):

Image

source: https://testfol.io/?d=eJxdT8FKBDEM%2FZe ... %2BgnkYF5F

then i looked at voo and vanguard shows 10.38% (see above) but testfol.io is showing a CAGR of 31.70% even though the ending balance on $10k invested is $11,180 which would be 11.18%, not 31.70%

Image

source: https://testfol.io/?d=eJxdT8FKBDEM%2FZe ... 6H9QroSV5l

is testfol.io not ready for primetime yet? Is it just the CAGR is not to be trusted or am i missing something with regards to the CAGR calculation?
It's hard to accept the truth when the lies were exactly what you wanted to hear. Investing is simple, but not easy. Buy, hold & rebalance low cost index funds & manage taxable events. Asking Portfolio Questions | Wiki
bongo
Posts: 165
Joined: Wed Oct 25, 2023 11:35 am

Re: New version of PortfolioVisualizer?

Post by bongo »

arcticpineapplecorp. wrote: Thu May 30, 2024 9:51 pm is testfol.io not ready for primetime yet? Is it just the CAGR is not to be trusted or am i missing something with regards to the CAGR calculation?
The A in CAGR is 'annual', right?

also start your YTD Results from 1/1/2024 (or maybe 12/29/2023) instead of 1/2/2024.
dbr
Posts: 46857
Joined: Sun Mar 04, 2007 8:50 am

Re: New version of PortfolioVisualizer?

Post by dbr »

bongo wrote: Thu May 30, 2024 10:31 pm
arcticpineapplecorp. wrote: Thu May 30, 2024 9:51 pm is testfol.io not ready for primetime yet? Is it just the CAGR is not to be trusted or am i missing something with regards to the CAGR calculation?
The A in CAGR is 'annual', right?

also start your YTD Results from 1/1/2024 (or maybe 12/29/2023) instead of 1/2/2024.
Yes, it is annual. It should be ACAGR = annualized compound average growth rate It is both a type of average and it is annualized
User avatar
arcticpineapplecorp.
Posts: 15727
Joined: Tue Mar 06, 2012 8:22 pm

Re: New version of PortfolioVisualizer?

Post by arcticpineapplecorp. »

bongo wrote: Thu May 30, 2024 10:31 pm
arcticpineapplecorp. wrote: Thu May 30, 2024 9:51 pm is testfol.io not ready for primetime yet? Is it just the CAGR is not to be trusted or am i missing something with regards to the CAGR calculation?
The A in CAGR is 'annual', right?

also start your YTD Results from 1/1/2024 (or maybe 12/29/2023) instead of 1/2/2024.
i did start 1/1/2024 but it changed it to 1/2/24 automatically.

is it annualizing, i.e., assuming the rate of growth over 5 months...IF ANNUALIZED?
It's hard to accept the truth when the lies were exactly what you wanted to hear. Investing is simple, but not easy. Buy, hold & rebalance low cost index funds & manage taxable events. Asking Portfolio Questions | Wiki
the_wiki
Posts: 3456
Joined: Thu Jul 28, 2022 11:14 am

Re: New version of PortfolioVisualizer?

Post by the_wiki »

arcticpineapplecorp. wrote: Thu May 30, 2024 9:51 pm am i missing something here?

I went to check on bnd returns ytd and vanguard is showing -1.87% (through 5/30/24) :



but testfol.io is showing -4.43% CAGR (even though the balance is down -1.83% from $10,000 to $9817):


is testfol.io not ready for primetime yet? Is it just the CAGR is not to be trusted or am i missing something with regards to the CAGR calculation?
Since CAGR is an "Annual Growth" stat, you can't exactly calculate it on 5 months of data. So they are just likely dividing by 5/12 to get the annual rate at current growth.

That could obviously be misleading. I'd probably just report CAGR as N/A if period is less than 1 year. Reporting YTD returns is also misleading as the return by Dec 31 will most likely be very different.
Hyperchicken
Posts: 2123
Joined: Mon Mar 02, 2020 4:33 pm

Re: New version of PortfolioVisualizer?

Post by Hyperchicken »

the_wiki wrote: Fri May 31, 2024 12:14 pm Since CAGR is an "Annual Growth" stat, you can't exactly calculate it on 5 months of data. So they are just likely dividing by 5/12 to get the annual rate at current growth.

That could obviously be misleading. I'd probably just report CAGR as N/A if period is less than 1 year. Reporting YTD returns is also misleading as the return by Dec 31 will most likely be very different.
Sure you can. You can calculate CAGR from 1 day of data if you want. That of course is an extreme - the longer the time period the more meaningful CAGR becomes, but there is nothing special about 1 year of data.

Nothing "misleading" about YTD either as it literally says "to date".
User avatar
arcticpineapplecorp.
Posts: 15727
Joined: Tue Mar 06, 2012 8:22 pm

Re: New version of PortfolioVisualizer?

Post by arcticpineapplecorp. »

thanks all.
It's hard to accept the truth when the lies were exactly what you wanted to hear. Investing is simple, but not easy. Buy, hold & rebalance low cost index funds & manage taxable events. Asking Portfolio Questions | Wiki
Hydromod
Posts: 1077
Joined: Tue Mar 26, 2019 10:21 pm

Re: New version of PortfolioVisualizer?

Post by Hydromod »

the_wiki wrote: Fri May 31, 2024 12:14 pm
arcticpineapplecorp. wrote: Thu May 30, 2024 9:51 pm am i missing something here?

I went to check on bnd returns ytd and vanguard is showing -1.87% (through 5/30/24) :

but testfol.io is showing -4.43% CAGR (even though the balance is down -1.83% from $10,000 to $9817):
Since CAGR is an "Annual Growth" stat, you can't exactly calculate it on 5 months of data. So they are just likely dividing by 5/12 to get the annual rate at current growth.
1 - (1 - 0.0187)^(12/5) = 0.0443 = 4.43%
Skyflyerman
Posts: 21
Joined: Sat Mar 22, 2014 10:30 pm

Re: New version of PortfolioVisualizer?

Post by Skyflyerman »

So glad I printed our custom portfolio last month. This new format is difficult to navigate and a real mess...very limited too!

We've created a virtual money machine in retirement, with the following portfolio(s)...admittedly, very unBoglehead-like, but highly successful!

67%--Dividend-producing main portfolio 100% reinvested dividends. Heavy in oil and shipping and REITS+SCHD.
18%--Roth Portfolio. Created for 20 year, high octane growth, with some protections. 12%, 27% max drawdown.
QQQ--53%
VGT----8%
GLD---8%
TLT--20%
BRK.B--11%

15%--We sell Puts and Calls to create monthly income along with Social Security. Ave=28% return, annualized.

With the above portfolio, created on Portfolio Visualizer, we have a money machine that allows us to live very well in our retirement, and keep reinvesting dividends back into our main portfolio (untouched til need it one day, or passed on to family). It is a virtual money machine, because we live off 15% of our investments plus SS, while snow-balling our IRA {created from 401(k) when I retired as an airline pilot}, and Roth IRA. When you live off of 15% of your portfolio and SS, you statistically guarantee your financial future is free of stress and drawdowns. The higher the drawdown in the market, the higher our income through selling Puts and Calls.

This sort of money machine is not for everyone. I spend 2-3 hours a day in the market, most days, always learning, so there is a certain time commitment. However, with my free time, I enjoy family, friends, golf and motorcycling. I also run a retirement fund for a friend, with similar results.
mbouck
Posts: 149
Joined: Wed Jan 24, 2024 2:08 pm

Re: New version of PortfolioVisualizer?

Post by mbouck »

Skyflyerman wrote: Wed Jun 05, 2024 1:55 pm So glad I printed our custom portfolio last month. This new format is difficult to navigate and a real mess...very limited too!

We've created a virtual money machine in retirement, with the following portfolio(s)...admittedly, very unBoglehead-like, but highly successful!

67%--Dividend-producing main portfolio 100% reinvested dividends. Heavy in oil and shipping and REITS+SCHD.
18%--Roth Portfolio. Created for 20 year, high octane growth, with some protections. 12%, 27% max drawdown.
QQQ--53%
VGT----8%
GLD---8%
TLT--20%
BRK.B--11%

15%--We sell Puts and Calls to create monthly income along with Social Security. Ave=28% return, annualized.

With the above portfolio, created on Portfolio Visualizer, we have a money machine that allows us to live very well in our retirement, and keep reinvesting dividends back into our main portfolio (untouched til need it one day, or passed on to family). It is a virtual money machine, because we live off 15% of our investments plus SS, while snow-balling our IRA {created from 401(k) when I retired as an airline pilot}, and Roth IRA. When you live off of 15% of your portfolio and SS, you statistically guarantee your financial future is free of stress and drawdowns. The higher the drawdown in the market, the higher our income through selling Puts and Calls.

This sort of money machine is not for everyone. I spend 2-3 hours a day in the market, most days, always learning, so there is a certain time commitment. However, with my free time, I enjoy family, friends, golf and motorcycling. I also run a retirement fund for a friend, with similar results.
Agree it isn't very Boglehead but I can't fault your strategy. I'm long energy/transport/BDCs/gold/long bonds/PFIX/alternatives but I won't be detailing that strategy here where I will get shredded for non-compliance. Also working quite well.
quattro73
Posts: 289
Joined: Sun Jun 20, 2021 2:07 pm

Re: New version of PortfolioVisualizer?

Post by quattro73 »

mbouck wrote: Wed Jun 05, 2024 3:58 pm
Skyflyerman wrote: Wed Jun 05, 2024 1:55 pm So glad I printed our custom portfolio last month. This new format is difficult to navigate and a real mess...very limited too!

We've created a virtual money machine in retirement, with the following portfolio(s)...admittedly, very unBoglehead-like, but highly successful!

67%--Dividend-producing main portfolio 100% reinvested dividends. Heavy in oil and shipping and REITS+SCHD.
18%--Roth Portfolio. Created for 20 year, high octane growth, with some protections. 12%, 27% max drawdown.
QQQ--53%
VGT----8%
GLD---8%
TLT--20%
BRK.B--11%

15%--We sell Puts and Calls to create monthly income along with Social Security. Ave=28% return, annualized.

With the above portfolio, created on Portfolio Visualizer, we have a money machine that allows us to live very well in our retirement, and keep reinvesting dividends back into our main portfolio (untouched til need it one day, or passed on to family). It is a virtual money machine, because we live off 15% of our investments plus SS, while snow-balling our IRA {created from 401(k) when I retired as an airline pilot}, and Roth IRA. When you live off of 15% of your portfolio and SS, you statistically guarantee your financial future is free of stress and drawdowns. The higher the drawdown in the market, the higher our income through selling Puts and Calls.

This sort of money machine is not for everyone. I spend 2-3 hours a day in the market, most days, always learning, so there is a certain time commitment. However, with my free time, I enjoy family, friends, golf and motorcycling. I also run a retirement fund for a friend, with similar results.
Agree it isn't very Boglehead but I can't fault your strategy. I'm long energy/transport/BDCs/gold/long bonds/PFIX/alternatives but I won't be detailing that strategy here where I will get shredded for non-compliance. Also working quite well.
I’d like to see it!
mark_in_denver
Posts: 527
Joined: Thu Feb 26, 2015 7:36 pm

Re: New version of PortfolioVisualizer?

Post by mark_in_denver »

Wow, I just went to PV and witnessed the total mess. I will miss it.

Is there any alternative that is agreed on that makes a good replacement?? I see testfol.io mentioned several times, is this the best there is?
comeinvest
Posts: 2877
Joined: Mon Mar 12, 2012 6:57 pm

Re: New version of PortfolioVisualizer?

Post by comeinvest »

mark_in_denver wrote: Tue Jun 11, 2024 9:54 pm Wow, I just went to PV and witnessed the total mess. I will miss it.

Is there any alternative that is agreed on that makes a good replacement?? I see testfol.io mentioned several times, is this the best there is?
It's relatively good in my opinion; but unfortunately its data are missing a lot of corporate actions of stocks and funds.
mark_in_denver
Posts: 527
Joined: Thu Feb 26, 2015 7:36 pm

Re: New version of PortfolioVisualizer?

Post by mark_in_denver »

Does anyone know how to see total dividends returned on testfol.io ? So if I have a dividend paying stock and I go back 15 years, and select not to reinvest dividends is there a way to see how much I would have collected in dividends over that time?
Hydromod
Posts: 1077
Joined: Tue Mar 26, 2019 10:21 pm

Re: New version of PortfolioVisualizer?

Post by Hydromod »

mark_in_denver wrote: Wed Jun 12, 2024 12:37 am Does anyone know how to see total dividends returned on testfol.io ? So if I have a dividend paying stock and I go back 15 years, and select not to reinvest dividends is there a way to see how much I would have collected in dividends over that time?
It doesn't look like that stat is tracked. You might compare returns with and without reinvest dividends checked to get an idea.
Blue456
Posts: 2193
Joined: Tue Jun 04, 2019 5:46 am

Re: New version of PortfolioVisualizer?

Post by Blue456 »

Hydromod wrote: Wed Jun 12, 2024 8:52 am
mark_in_denver wrote: Wed Jun 12, 2024 12:37 am Does anyone know how to see total dividends returned on testfol.io ? So if I have a dividend paying stock and I go back 15 years, and select not to reinvest dividends is there a way to see how much I would have collected in dividends over that time?
It doesn't look like that stat is tracked. You might compare returns with and without reinvest dividends checked to get an idea.
I was looking for the same. That’s a very good idea. Thank you.
mark_in_denver
Posts: 527
Joined: Thu Feb 26, 2015 7:36 pm

Re: New version of PortfolioVisualizer?

Post by mark_in_denver »

Hydromod wrote: Wed Jun 12, 2024 8:52 am
mark_in_denver wrote: Wed Jun 12, 2024 12:37 am Does anyone know how to see total dividends returned on testfol.io ? So if I have a dividend paying stock and I go back 15 years, and select not to reinvest dividends is there a way to see how much I would have collected in dividends over that time?
It doesn't look like that stat is tracked. You might compare returns with and without reinvest dividends checked to get an idea.
Thanks, I was thinking the same.

This is a real loss, I really liked PV.
hnd
Posts: 1134
Joined: Mon Jun 22, 2020 11:43 am

Re: New version of PortfolioVisualizer?

Post by hnd »

beachtech wrote: Mon May 27, 2024 9:44 am
comeinvest wrote: Sun May 26, 2024 7:01 pm Perhaps totalreturns and testfol copied from each other, or use the same faulty data source ;) None of them reveal either their identity or their data sources on the site. Now with the identical bugs on hand, we can perhaps reverse-engineer which data source they use, or from which public site they scrape the data off.
As I mentioned earlier, https://totalrealreturns.com/ uses data from https://eodhd.com/

I appreciate txaggie bringing up data quality. I haven’t had time to look into this symbol DSFVX closely yet. But I don’t yet know for certain which data source is actually correct: all we know is that the outputs of these tools disagree. (For all I know, it’s at least possible that Morningstar and PV are wrong.) To get to the ground truth, we’d need to obtain (perhaps from DFA themselves?) a “known true” spreadsheet of all closing prices and distributions going back 30 years.

When I look at only calendar year 2023 (which means the total return from the close on 2022-12-30 through 2023-12-29):

https://totalrealreturns.com/n/DFSVX?st ... 2023-12-29 shows a total return of +19.0%

testfol.io shows the same ($11902.42 ending balance, or +19.02%)

DFA’s website shows +18.91%

Yahoo Finance shows +18.91%.

Zacks shows +18.93%.

Was not able to get data from Portfolio Visualizer since it’s down at the moment.

For 2023, these are all pretty close. It’s definitely possible that any discrepancy is in older data only.

If the fund companies would publish an official historical data spreadsheet with daily closing prices and distributions, this would be easy to figure out. Of course, if they all did that, we wouldn’t need the intermediary data aggregators at all. One can dream…
i think its older data. I was doing some backtesting in testfolio and found anything 20+ years out something is wonky. 20+ years of data seems accurate.
Hydromod
Posts: 1077
Joined: Tue Mar 26, 2019 10:21 pm

Re: New version of PortfolioVisualizer?

Post by Hydromod »

hnd wrote: Wed Jun 12, 2024 10:24 am i think its older data. I was doing some backtesting in testfolio and found anything 20+ years out something is wonky. 20+ years of data seems accurate.
EODHD (the data source) seems to have occasional glitches with certain dividends and splits in older data (although they quickly correct it when it is brought to their attention). Maybe something to do with older data formats; that can be a problem if the data is scanned from paper copies.
hnd
Posts: 1134
Joined: Mon Jun 22, 2020 11:43 am

Re: New version of PortfolioVisualizer?

Post by hnd »

Hydromod wrote: Wed Jun 12, 2024 10:31 am
hnd wrote: Wed Jun 12, 2024 10:24 am i think its older data. I was doing some backtesting in testfolio and found anything 20+ years out something is wonky. today minus 20 years of data seems accurate.
EODHD (the data source) seems to have occasional glitches with certain dividends and splits in older data (although they quickly correct it when it is brought to their attention). Maybe something to do with older data formats; that can be a problem if the data is scanned from paper copies.
bongo
Posts: 165
Joined: Wed Oct 25, 2023 11:35 am

Re: New version of PortfolioVisualizer?

Post by bongo »

Blue456 wrote: Wed Jun 12, 2024 9:00 am
Hydromod wrote: Wed Jun 12, 2024 8:52 am
mark_in_denver wrote: Wed Jun 12, 2024 12:37 am Does anyone know how to see total dividends returned on testfol.io ? So if I have a dividend paying stock and I go back 15 years, and select not to reinvest dividends is there a way to see how much I would have collected in dividends over that time?
It doesn't look like that stat is tracked. You might compare returns with and without reinvest dividends checked to get an idea.
I was looking for the same. That’s a very good idea. Thank you.
That won't work, of course, because of the compounding effect.

PV can still do what you want on a single stock if you break it into two runs. Seems the timespan just can't span the 10 year limit point (for today, that's June 2014), but before and after is fine. So do one run on your stock from 15 yrs ago until June 2014, and then do another run from July 2014 until today (reset the starting value of the second run to be the ending value of the first run). Then add up the Income column from the Annual Returns table from both runs.
comeinvest
Posts: 2877
Joined: Mon Mar 12, 2012 6:57 pm

Re: New version of PortfolioVisualizer?

Post by comeinvest »

testfol.io doesn't seem to get the NAV tickers of CEFs right; possibly the distributions are wrong. Examples: XNXCX and XNCAX, results differ from www.portfoliovisualizer.com for the period January 2022 to now.
Post Reply