With turning 50 this year, my wife is 52 and had a breast cancer scare as well (very early stages thankfully), we are seeing that life can be shorter than we think. I'm starting to feel that my super saver tendencies may lead to later disappointment. The neighborhood is continuing to deteriorate and honestly we feel like we deserve a bit more out of life in terms of our surroundings. We earn a very good income and likely could afford a home in the $900K range. I feel it's time to move to a nicer neighborhood and purchase a home in the $400-$450K range. A few things to note, we have no kids to worry about, we plan to spend every red cent before we pass

We have approximately $220K in taxable, and $670K in retirement accounts. Currently we save about $80K per year so we feel we are on track to reach our savings goals of $2M with our asset allocation as follows:
Total US Bond Market Index 22%
Hi Yield Corporate Fund 8%
Short Term Inflation Protected 8%
Total US Stock Market Index 37%
Total International Index 19%
REIT Index 6%
Prior to the new home purchase idea, we thought of earlier retirement but have both decided we'd be willing to work to age 62 to enjoy a better home. I feel regardless of this purchase we will still be in very good shape at age 62 even if we were to save $40-50K per year instead of the $80K. That $40-50K is just a lowball estimate, I assume we will save more.
Our plan is to buy the new home without selling the other first. If we sell our existing home ( we may rent it) my mortgage guy said that we could put as low as 10% down and avoid PMI with a 1.4% adder to the loan. We obviously could put down enough to avoid even the adder. If we do sell the home, we do have the option to use the proceeds of the sale towards the mortgage balance, then get a new monthly payment amount without having to refinance. We will likely take out a 30 year term at 3.50% or so and pay it as if it were a 12-15 mortgage. We like the idea of having the flexibility with a lower payment.
So my question is this. Based on our ages, what should we do as far as the down payment and or with the proceeds of the sale of our home? If we were younger, I think its a no brainer to go with a low down payment and invest the rest. Being that we are older, I'm a bit on the fence. I'd like some input from the above 45 crowd please

Thanks