The quote below neatly sums up the issue. You do NOT have to pay taxes on gifts larger than $15k per year per recipient; the amount over $15k merely counts against your $11.4 million (as of 2019; the amount is adjusted for inflation) lifetime exemption. And your spouse has their own $15k annual gift tax exclusion.
https://www.thebalance.com/gift-tax-exc ... on-3505656If you gift $120,000 to your daughter in 2019, $105,000 of the gift is taxable because it exceeds the $15,000 annual exclusion by that amount. You can either pay the gift tax in that year, or you can charge it to your lifetime exemption. If you do the latter, your $105,000 taxable amount reduces your 2019 lifetime exemption from $11.4 million to $11,295,000.
Gifts greater than $15k to a single donor must be reported using IRS Form 709 so they can track how much of your lifetime exemption you've used up.
So unless you're planning on giving and/or leaving behind an estate with a combined total exceeding $11.4 million, and double that if you're married, you don't have to pay the gift tax. You merely report gifts over $15k (or $30k if you're married).
I hope that this will clarify the issue for someone.