Anything that is termed a tIRA is included in the pool of money that is considered for conversion to a Roth, no matter where it came from, what it's invested in, or where it's housed (Vanguard, Fidelity, etc.)
I'm very, very much a noob on this, but my take-away on this is that it's to your advantage if your portion of money in tIRA's isn't that big. If so, go ahead and convert your tIRA (or some portion thereof) before moving 401k's over to tIRA's. That way you won't have to move all your tax-advantaged accounts. If there are more funds (but not all) that you want to move to Roths, move only that amount to 401k's, and then switch.
Someone please tell me that I've read this right...
eta: I'm concerned about what you wrote here: "But then I realized 401K is not considered a TIRA because it is funded with pre-tax dollars." They're both funded with pre-tax dollars, and you'll pay income tax on both when you withdraw. The difference is that a 401k (or 403b, or TSP, etc.) is an employer-sponsored plan, where they may or may not match part of your contributions, and they control the selection of available investment options, vs a tIRA which is controlled by you (and by whichever IRA provider you choose, of course.) You can fund a 401k etc up to $17,500 ($23,000 if you're over 50), but an IRA is limited to $5,500 ($6,500 if you're over 60.)
So basically, they're both pre-tax-funded retirement plans, but 401k's and friends are employer-sponsored with much higher limits (but a limited selection of investment options), and IRA's are controlled by you but with much lower limits (but generally a lot more options.)
It's Roths, of whatever variety, that are funded by post-tax funds. So you will pay tax on whatever you put into a Roth, but when the time comes that you pull money back out, you don't pay tax on your original investment, and you don't pay tax on whatever growth has occurred on top of your original investment.