Should I Pay Off My Mortgage?

Having a paid off house has always been a goal. Living mortgage-free sounds awesome, but it can also put the brakes on financial growth. Any money put toward home equity is robbing investment accounts of hard working capital. After years of investing, we finally took some of our investment gains and paid off our house. This decision was a struggle for me, and I still think investing is better than paying off a mortgage in many cases.

Paying down debt isn’t just about the numbers

Compounding interest is an incredible force. Increasing returns by a little over 2% means having twice as much money after 30 years. Since I’d rather be a beach bum and let my dollars do all the hard work, I try to find the optimal way to deploy them.

In discussing whether I’d pay off my student loan debt (I’m not) I characterized my financial decisions as a struggle between two economic models. I call them Homo Sharkinus and Homo Chickenus. Both of these characters want to get the most out of my dollars, but handle risk differently. Sharkinus is always looking to maximize returns and doesn’t flinch when it comes to taking on risk. Chickenus on the other hand, can’t stomach much risk, and prefers the security of guaranteed returns.

We all need to find our own balance of risk

Each of us has our own mix of these characters, and when making financial decisions we need to find the right compromise to make them both happy. My wife doesn’t stress much about money and has a little more Sharkinus to her, meanwhile as someone who is a bit more OCD and prone to stress, I’m more Chickenus.

Even though these characters have opposing views on risk, there are some financial decisions where they can both come to an agreement.

When you shouldn’t pay off your mortgage

While working, our marginal tax rate was around 30%. That meant for every $10 that I could put in my pre-tax retirement account, I could only put $7 after taxes toward the mortgage. That’s 43% more that I could put in my 401k.

Of course, I’ll have to pay taxes on that 401k money eventually, but I plan on being in a much lower tax bracket when I withdraw it. In fact there is a good chance that I won’t have to pay any taxes on my 401k withdrawals. Not only is it important to take advantage of this tax arbitrage opportunity, but there are also ways to withdraw money from your 401k before retirement age. So even if you plan on retiring early, you can never have too much money in pre-tax accounts.

Since I stood to benefit by as much as 43%, both my Chickenus and Sharkinus agreed to first max out pre-tax contributions. And there’s one more low risk decision they agreed on.

Can you refinance?

Before even thinking about whether to pay down debt or invest, it’s worth considering a refinance. If you can reduce your mortgage interest rate by 1%, you should probably refinance. But depending on the size of the mortgage, even reducing the rate by as little as .5% could be worthwhile.

The key is to determine how much you will be saving in interest payments, then compare that with the refinancing fees. Last time I refinanced, I went from 5% to 4% rate on my mortgage and paid around $3,600 in fees. That 1% reduction on a $200,000 mortgage meant saving around $2,000 in interest each year. Well worth the fee I paid, so long as I held my mortgage for more than a couple of years.

Of course, if you refinance into another 30 year mortgage, you will have reduced payments, but the clock starts over again for another 30 years. This isn’t a problem since you’re saving on interest, and could just pay some additional principal each month to pay it off sooner.

But should you pay of your mortgage sooner, or invest?

Eventually I had extra cash to deploy even after maxing out my 401k and refinancing the mortgage. Now the decision to invest or pay down debt was getting harder.

My Chickenus loathed seeing how little of my mortgage payment was going toward the principal, and how much I was paying in interest. “Just a couple hundred extra dollars a month could set us forward years in paying off the mortgage, and those extra payments stored up as equity will keep saving us 4% a year guaranteed!”

My Sharkinus on the other hand was indifferent to the interest payments. “Who cares about all that interest, we can make up for it by investing in the market.” Over the stock markets history, the S&P500 has returned nearly 10% on average.

Generally Sharkinus wants to invest in the market before paying off any debt under 6%, and even wants to take out more debt to keep investing. The thing is, the market can be volatile, and there is no guarantee future returns will mirror what we’ve seen in the past.

While I did go back and forth, sometimes making additional principal payments, I mostly followed my Sharkinus and stuck with investing. Even though my Chickenus loathed the additional risk, I wanted to speed up my journey to financial independence as much as possible… Besides, I was just starting my career and would have plenty of time to recover if the investments didn’t work out.

The benefit of investing vs. paying down mortgage

Over a decade down the road, I took a look at how the Sharkinus route of investing vs. the Chickenus route of paying off the mortgage played out. Using historical data, I examined the effects of investing $1k a month into an S&P500 index fund vs. putting that money toward a $200k mortgage with 4% interest.

Had I followed my Chickenus tendencies and made additional mortgage payments of $1k a month, I would have just paid off the entire $200k mortgage and saved $28k in interest.

Sharkinus on the other hand, would have directed that additional $1k a month into an S&P500 index fund. This kind of investment over the last 12 years would have yielded just over $300k today with approximately $157k in gains. Not only does that beat the mortgage interest savings of $28k by a long shot, but I’d be able to pay off the house with the investment gains alone, which is kinda what we just did.

First, I started selling bonds…

As a lazy investor, all of my money has been going into an efficient three fund portfolioΒ – a portion of which is allocated to bonds. These bonds weren’t making more money than my mortgage was costing, but I figured they also served as an emergency fund. The bond interest was being taxed, but my mortgage interest was tax deductible, so in a way they cancelled each other out.

Last year however, there were some pretty big changes in the tax code. With the standard deduction increasing dramatically – now $24k for married filing jointly – our mortgage interest is no longer worth keeping as a deduction. That makes the difference between our bonds (which are taxed as income) and the mortgage interest we are paying, much larger.

At the beginning of the year, I started selling bonds from our brokerage accounts. The proceeds went towards our mortgage, and we and managed to pay off a bit more than half of our six figure balance.

Then I sold some stock…

With our money working hard for us, we got to a point where we could start taking it easy. Two years ago, after reaching financial independence, I quit my job. Two years before that, my wife quit her corporate job to teach at our local community college. Our income is now about 1/4 of what it had been, putting us in the first two tax brackets.

Being in the first two tax brackets has the benefit of 0% capital gains tax. To take advantage of this, we’ve been harvesting capital gains each year by selling stock and re-investing. But, as I’ve written earlier this year, I do believe the market has been getting high. Whether it ends up crashing, couch-locked, or getting even higher, many economists forecast that it will slightly under perform in the future.

The market might be having a bit too much fun

In addition, we already left our corporate jobs and are more in the preservation phase of our financial journey. With my Chickenus screaming for stability, we went against the policy we’d followed for over a decade. After selling some stocks, we paid off the last chunk of our mortgage.

But was it a good idea to pay off the mortgage?

While I’m stoked that we don’t have any more mortgage payments, under different circumstances, I probably wouldn’t have paid it off…

For example, if we were in a higher tax bracket, I’d still be maxing out any pre-tax contributions (HSA, 401k, 403b, 457, etc.) before anything else. I also wouldn’t be selling any stock investments if I was going to pay a 15% tax on the capital gains. However, I probably still would have sold the bonds that were in a brokerage account where it’s taxed as income, and put that toward the mortgage with a higher interest rate.

While there may be a market crash looming, it still pays to invest for the long term. Twelve years ago, we were heading into the great recession when I started investing instead of paying down the mortgage, and I’m glad I did.

Even though it might not be the financially optimal route that my Sharkinus wanted, paying off the mortgage was the right balance for us at this point in our lives. It’s one less thing my Chickenus has to worry about, and it feels pretty awesome to have a paid off house.

23 thoughts on “Should I Pay Off My Mortgage?

  1. We’re going through the same type of thing. Although it’s not a mortgage, it’s student loans and a car loan.

    But yeah we’re so close I think we make the choice to just kill it, then juice everything from there. But well use extra cash, and not liquidate anything.

  2. The struggle is real between paying off the house and investing. We’ve gone through the same strategies over the past few years and did a blend between a few of your ideas. We paid a good chunk to get the PMI off, refinanced to 2.875%, and after making sure to max out our tax-advantaged accounts, over the course of a few years when we had extra money past investing, we threw it at the house. Now we’re facing the same situation! With enough money to pay off the house in a handful of months, do we pay it off or put it to work? Not complaining though. Good problems to have. πŸ™‚

    I think after the big runup in the market in the past years, it’s been feeling a bit high. Of course we won’t pull any out, but instead of putting the EXTRA money in, we might just pay the house off to lower our expenses and give ourselves more cushion. Everyone has different risk tolerance, but that’s what would make us sleep easier at night. Thanks for the thoughts! Sounds similar to us.

    • Nice, that’s a really good rate, and good problems to have!

      I think your idea of allocating new cash toward equity is good option. Having the house paid off does help me sleep better at night πŸ™‚

  3. I’ll pay mine off this month. I feel doing so is more justified than ever before because of the higher standard deduction and how expensive the stock market has gotten.

    The question for most is: Will you have the discipline to save/invest the money you are saving from not having to make a mortgage payment?

    • Yes, you’re going to need that discipline, especially if you decide to invest instead of paying down debt. Don’t do it if you can’t muster up the discipline to stick with your investments when the market takes a tumble πŸ™‚

  4. Although you seem a little bit torn, from one mortgage free family to another just wanted to say congrats! I definitely agree that it’s a personal choice based on a variety of factors, both financial and psychological.

    Next month will be two years since we paid ours off, and it’s one of the best decisions we made for us. As a family, it has sent our savings rate into overdrive, has made us way more aggressive as investors, put less stress on our household, and with the personal pan pizza (aka little guy) about to make his arrival the last thing we wanted is another bill to pay πŸ˜‰

    • Thanks, Danny, and congrats to you too! A solid move with the personal pan pizza on his way πŸ˜‰

      You brought up a good point about having more confidence to invest aggressively. With the extra cash flow we can build our investments back up as well πŸ™‚

  5. Well congrats on your paid off house! There are many people that struggle with the decision between putting money in the house vs investing. Seems like you did a well balanced decision. Cash-flow wise it is also a nice thing to have. Plus, you sold some stock at the right time too!

  6. I have an easy decision to make whether to pay off my mortgage or not back then. It based on three words “Peace Of Mind”. I want to pay off my mortgage first before doing any investment. Best feeling ever when I wrote my last check to the bank πŸ™‚

  7. I paid off our mortgage last year. I thought of it more as diversification strategy. Putting money into stocks/bonds vs real estate. At these high stock price levels I thought it more prudent to put money into paying off the mortgage (investing in real estate) and then using the savings to max out the 401k and Roth’s thereby dollar cost averaging stock market investment. With the mortgage paid off I consider us to be 50/50 in stocks and real estate allowing me to invest all of our retirement funds into stocks while maintaining higher safety. It also means I don’t need to think about putting any money into bonds in this low interest rate environment.

    • Great strategy. I see it this way too. It allows additional investment in stocks and even if the economy gets REALLY ugly, my largest monthly expense it gone. It would only take $1-2k/mo to squeak by.

  8. Love the addition of color in your cartoons. Chickenus surfing Sharkinus is my favorite. After liquidating my mortgage free home, I invested a substantial portion in bonds and the dividends cover my rent. But after finding out about the taxes on bond dividends and tax free stock dividends in the in the lower tax brackets, I decided to rebalance my portfolio which is really easy in Vanguard. Thanks for the heads up!

  9. Great job. I think either option is great. I’m pretty sure nobody ever regrets paying off their mortgage. We’re keeping our mortgage for now because my wife still works. Once she stops working, then we’ll evaluate our option again. The tax deduction came in very handy in the past. It isn’t going to be as good this year, but we’ll get something. I’ll have to run the numbers first. Our state tax + property tax is more than $10,000.

  10. Mr CK,

    Thanks for your recent post. I’m trying to balance the desire to pay-down my $300,000 mortgage while investing. I follow a modified version of your “Three Fund” portfolio and overall my portfolio is up, which I am thankful for. Last year, I used some of the gains for our 20% mortgage down-payment.

    However, due to my occupation, I am not eligible for a 401K. I have a mandated pension program that takes 11% from my gross pay every month.

    Any suggestions about maxing out pre-tax income in my case (besides the traditional IRA)?

    By the way, congrats on paying off your mortgage. While the stock market may be unpredictable, being mortgage free is permanent and should give your family peace of mind.

  11. First off, Congrats. Ideally we should’ve taken a 15 year mortgage instead of a 30 year one. But we wanted to make sure one income covered all our expenses. So now we’ve taken the mid-way approach. Of the amount we could pay extra towards our mortgage, we use half of that towards extra payments and the other half towards investments.

    But there’s still years to go πŸ™‚

  12. what is your view on a person in their 60’s carrying a mortgage? 3 years ago we refinanced to 4% and only one of us is working

  13. Good post. I know refinancing to get rid of a PMI, for the least fortunate, is a great move too. Nothing like paying for the sins of other’s past!

    Thanks boomers! Kidding…

    I too have weighed paying off the mortgage faster, but my shark mind says invest! I need to focus on that, since the returns have historically been there.

    I just hate paying for borrowed money! Thanks for sharing your thoughts on this. It helps me with my decision made to invest in the market

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.