In 2012, I decided to quit my six-figure job in investment banking and retire at 34. I had amassed a net worth of about $3 million, generating roughly $80,000 in investment income per year.
For seven years, I lived a charmed life in early retirement with my wife, who also retired from her finance job three years after I did. Together, we earned roughly $250,000 in passive income streams per year, mostly from dividend-paying stocks, interest from savings, municipal bonds, and rental income.
We traveled to more than 20 countries. I spent time doing things I enjoyed, like coaching a high school tennis team and writing on Financial Samurai, the personal finance website I started in 2009.
Fast forward to the end of 2019, and my wife and I are no longer comfortable living the early retirement lifestyle, especially in a big, expensive city like San Francisco.
According to a report from the California Association of Realtors, you now need a minimum household income of $309,400 to afford a median-priced home in San Francisco, which is around $1.6 million. So even with our annual passive income of $250,000, we’re still short nearly $60,000 yearly.
To own our existing home without a mortgage, we could try to sell our other investments. But that would create a much higher net worth allocation toward real estate than desired. (Our goal is to keep real estate expenses to less than 35% of our income.) Furthermore, selling our investments would trigger a tax liability on capital gains.
Retiring from early retirement
When I left my job (and even got an excellent severance package that paid out all my deferred stock and cash compensation), I thought I could retire and be set for life. I was wrong, and I’m not afraid to admit it. If my wife and I don’t take action soon, we’ll no longer be able to live comfortably off of our passive income streams in San Francisco. Here are the main reasons I now need to retire from early retirement:
1. We had a child.
My wife and I never planned on becoming parents, but that changed in 2017 when we were blessed with a baby boy. Retiring without children is like a walk in the park compared with retiring with children. In addition to costs for diapers, clothes, toys, baby food, and occasional babysitting, our biggest concern is paying for our son’s education, especially now that he’s in preschool.
In San Francisco, the public school system starting in kindergarten is based on a lottery system, so even if you pay property taxes, your child isn’t guaranteed a spot in your neighborhood schools. Most parents are forced to pay big bucks to send their kids to private schools, but paying $30,000 or more doesn’t guarantee admission to top-rated private schools.
Don’t get me wrong: Fatherhood has been the most rewarding experience of my life, and we’re going to do whatever it takes to support our son’s needs and give him access to as many opportunities as possible.
2. I underestimated how low interest rates would go.
I believe in “low interest rates for life,” but I didn’t think the 10-year bond yield would ever drop below 1.5% in 2019. I thought we’d stay around 2.5%.
Now, instead of only needing $2 million in additional capital to generate $50,000 at 2.5%, I need $2.5 million to generate the same $50,000 in passive income at 2%. At 1.5%, the required capital to generate $50,000 in passive income is over $3.3 million. Seeing such a significant shift in the goalposts when you don’t want to take more investment risk is disconcerting.
The only way I benefited from the recent interest rate decline was by refinancing my mortgage to 2.6%. My cash flow is now about $13,000 greater per year. While the increase helps, it still doesn’t offset the expected investment income decline.
3. Rising health insurance premiums
In 2019, the monthly premium on our health insurance was $1,820. But last month, I received notice that our premium would rise to $1,940 in 2020. That’s $23,280 annually in health insurance premiums, excluding co-pays and co-insurance. This will be an even bigger problem if we decide to have more children later because the 2020 amount would go up by another $440 (to $2,560) per month.
I’ve finally reached a point where paying so much in health insurance premiums feels like highway robbery, especially since we’re a healthy family with no pre-existing conditions. While I understand that we have to help subsidize others who are less healthy, the cost has become untenable.
4. The bliss of early retirement didn’t last as long as I thought it would.
Early retirement made my life better in ways I couldn’t have imagined. I was able to fully commit to being a father, my relationship with my parents got better, I aged slower, and I even became more independent.After a few years, however, the bliss of taking a permanent vacation disappeared. I started getting the itch to do something more productive than play tennis and sleep.
Also, when you start paying $2,000 per month for preschool, not using your six to eight hours of free time to find ways to cover that cost doesn’t feel right.
How do I plan to retire again by 2022?
To get to $309,400 a year — and retire again by 2022 —I’ve decided to spend some time focusing on entrepreneurship. For too long, I’ve treated Financial Samurai like a hobby.
The website has grown to amass more than one million readers per month. My immediate goal over the next six months is to increase traffic and boost advertisement revenue by partnering with relevant financial institutions and creating more products and services (like my book on severance negotiations).
I also plan to apply for jobs at various tech, media, and financial firms. With the income from a full-time job, subsidized healthcare, and other benefits, I can accelerate my goal of accumulating extra capital to generate enough investment income.
Lastly, I will keep buying real estate in the middle of the United States, where prices are lower, cap rates are higher, and job growth is stronger.If all else fails, we will relocate to Honolulu in 2022, when our son is eligible for kindergarten. The median price for a single-family home in Honolulu is $835,000 — or 47% lower than in San Francisco. Meanwhile, the cost of private school education (around $25,000) is also 30% to 45% less.
Always plan for an unknown future.
My advice is to always plan several years ahead for those who want to retire early. Set a date for your exit plan and do whatever it takes to meet your goal. My biggest mistake was failing to consider how my lifestyle and needs would change — and what to do when those changes took place. Downturns do happen.
You can’t plan for every variable, but the more prepared you are, the greater your chances of achieving financial freedom. If you’ll excuse me, it’s time to turn in my early retirement membership card and return to the salt mines!
Sam Dogen worked in investment banking for 13 years before starting Financial Samurai, a personal finance website. He received a B.A. in Economics from the College of William & Mary and his MBA from the University of California, Berkeley. Sam has been featured in Forbes, The Wall Street Journal, The Chicago Tribune, and The Los Angeles Times.