It's not the US govt that they are worried about. Institutions like Vanguard are worried that they may become liable to the regulations of a country like Japan if it appears that they are soliciting business in the foreign jurisdiction even if they don't have a business address there themselves.
That may be, but it isn't what they say. One broker, in particular, when summarily closing my account because of my foreign address attributed their policy change it to provisions of the PATRIOT Act, but as with the OP, they were not able to point to any provision in the Act that was forcing their decision. You'd think if their concerns were regulations in the foreign country they would just say so and be done with it.
As for the US mailing address, if you want to keep things above board, the important thing is to check the contract terms to see if they require US residency. Vanguard, for example, explicitly told me they require customers to be US
residents, therefore merely having a US mailing address would not be sufficient to open an account. Of course... people can still do that, us a mailing address, but it's a form of deception. Vanguard may even suspect that it's the case (wink wink, nod nod), but in the end, if there is any liability, they will claim to have been deceived, and burden will fall on the customer.
Personally I have no trouble with Vanguard's policy, though.They don't want my business, Schwab and Fidelity do. Fine. But I share the OP's frustration when a company says they can't do something for compliance reasons but then can't/won't explain just what it is they are trying to comply with.