Gil Baumgarten - Segment Wealth Management

Gil Baumgarten is the Founder and President of Segment Wealth Management, a top-10 firm in Houston. He is a nine-time recipient of the Top 1,200 Financial Advisors distinction and has been ranked among the 35 best advisors in Texas. Gil is the author of FOOLI$H, a book looking into the convoluted nature of the brokerage ecosystem and investor behavior.

Here's the link to Gil's personal website where you can see some of his creative work and especially the precision woodworking we spoke about in the podcast.

Interview transcript continues below

Portfolio tracker Sharesight tracks your trades, shows your true performance, and saves you time and money at tax time. Get 4 months free at this link 

Disclosure: The links provided are affiliate links. I will be paid a commission if you use this link to make a purchase. You will also usually receive a discount by using these links/coupon codes. I only recommend products and services that I use and trust myself.

INTERVIEW TRANSCRIPT

Chloe (2s):

Stocks for Beginners.

Gil Baumgarten (6s):

Daniel Kahneman proved that people experience a loss three times more acutely than they experience a similar dollar gain. So a profit of $10,000 feels like a three on a scale of one to 10 and a loss of $10,000 feels like a nine. And so when we're looking at a gain or loss of $10,000, we don't properly assess our own risk appetite because we're trying to avoid the pain of a loss more than we are trying to make a gain.

Phil Muscatello (36s):

Hi, and welcome back to Stocks for Beginners. I'm Phil Muscatello. Today I'm pleased to welcome Gil Baumgarten, one of the nation's top financial advisors. He's the founder and president of Segment Wealth Management. Since its inception in 2010, Segment Wealth Management has grown to a top 10 firm in Houston with over a billion dollars in client assets under advisement. Gil you're also an avid outdoorsman, award-winning woodworker and a family man, friend and colleague. Hi Gil.

Gil Baumgarten (1m 3s):

Thank you. Hello.

Phil Muscatello (1m 5s):

Thanks very much for joining me. Now, I know that many people come to this podcast because they've been attracted to so-called meme stocks like GameStop. And they kind of think that this is what the stock market is, and they're being attracted like moths to a flame. And I'm hoping that today we can show the brutal realities of good investing. So let's start off with your book Gil. It's called "Foolish". Why the name?

Gil Baumgarten (1m 29s):

Well because we find people do foolish things and I would consider buying meme stocks to be a foolish thing. People tend to do what fish do just before they get caught: they are chasing flashy things. And investors tend to chase flashy things and they get caught up in the same problems. So traditional long-term investing has much better compounding and better tax features. And we don't tend to chase trendy things.

Phil Muscatello (1m 55s):

So do you find that though? That people are coming to you because they've seen these kinds of gains that are being made in the stock market and that they think, well, why can't I do that as well?

Gil Baumgarten (2m 5s):

I hear people talk about it, but people don't ask me to do it for them. Our clients are already wealthy and they're more interested in staying wealthy than they are in getting wealthier. What it is that's so appealing about things like Bitcoin and meme stocks, you know, AMC and GameStop and whatever is these spectacular rates of return that can be had simply from the naivete of other investors driving the price up. There's only so many shares that are for sale every day and when there's more people trying to buy than there are people trying to sell, the price is going to go up. And those spectacular gains tend to be what entices the small investor.

Gil Baumgarten (2m 48s):

It doesn't usually entice the large investor. They've been stung too many times already.

Phil Muscatello (2m 53s):

You worked on wall street for a number of years. What period were you working in? What particular crisis did you live through?

Gil Baumgarten (3m 1s):

I started at EF Hutton in 1984. EF Hutton ultimately became what is today Morgan Stanley. So I stayed there from '84, until 2000. Moved to Paine Webber and Paine Webber became UBS. I stayed there until 2010 when I started my own financial advisory firm. So, you know, I saw the '87 market crash, the '99, 2000, the 2008 market crash, the flash crash in May of 2010. So I've seen all those and seen the ultimate recovery from it. So we tend to buy the dips.

Phil Muscatello (3m 39s):

Yeah. Is that what you learned from it? Buying the dips? Is that the main thing that you learnt or was there a psychology as well?

Gil Baumgarten (3m 45s):

Yeah, pretty much. You know, I've learned a lot of things. There are just some universal truths that tend to play out over and over again. One of those things is that market weakness tends to beget market strength. When most people believe that weakness begets more weakness. That's always a very temporary phenomenon. If you look back over history at market movements in the US Market, 81% of the time, the stock market goes up. That doesn't mean on a daily basis, but pick any 12 month time period, the returns are positive in 81% of all market environments. If you go back to periods where the market has just dropped 10% and calculate that as the start of your 12 month time period, it's 91%.

Gil Baumgarten (4m 32s):

So the moral of the story is buy the weakness.

Phil Muscatello (4m 35s):

Yeah. Even though that sometimes these are going to be very difficult times to buy because everyone's running for the doors and running for cover. So it's really a matter of will and holding your strength in yourself and your convictions as well isn't it?

Gil Baumgarten (4m 48s):

Well, a lot of it is what is it that you own and why is it that you own it? A lot of people only want to buy stocks so that they can sell them to somebody else more expensively later. And they lose sight of the fact that these are real companies with real products and real earnings and real employees that work for them. If you look at it as a business and you own a piece of a business, that's how Warren Buffet has used his money to make billions of dollars for himself and for others. Warren doesn't care what the value of it is today or tomorrow he's looking 10 or 20 years out. That's the way we approach it too. And that's the reason why Warren gets rich and most everybody else who attempts stock market speculation does not.

Phil Muscatello (5m 32s):

So let's cover the good, the bad and the ugly. What's the good of Wall Street? What's been the good aspect for you and your experience there?

Gil Baumgarten (5m 38s):

It is a fantastic place to compound wealth if you do it correctly. And in the US you know, we don't pay tax on unrealized gain. So if it increases the market value, it doesn't matter if you've held it for 10 years or 20 years. And in the US if you die with an appreciated asset, all of that appreciation is tax free. And so we manage our client money with a mind towards unrealized gain: the portion that is untaxed. And so that's what we're after.

Phil Muscatello (6m 9s):

And how about the bad and the ugly? What's the bad and the ugly of Wall Street?

Gil Baumgarten (6m 14s):

I think hidden costs and hidden motivations is probably the bad or the ugly, however you want to look at it. Wall Street likes to make money in multiple layers. They will have their salespeople talk to a client about a mutual fund for which they'll get a commission, a finder's fee, if you will. There'll be a 12b-1 fee that is an inflation of the underlying expense rate that is hidden from the client but gets kicked back to the brokerage firm. The brokerage firm will require that the mutual fund run trades across their trading desk, they'll elevate the trading commission costs, they'll do investment banking deals, they'll offer IPOs.

Gil Baumgarten (6m 55s):

They do all kinds of side deals with that industry in order to elevate what clients pay and monetize all the layers of that client relationship. So I think that part of the business is a little unseemly and unrecognizable by most investors who aren't paying attention.

Phil Muscatello (7m 13s):

So a new investor who is receiving that glossy brochure or the marketing for one of these products, how can they guard against these hidden costs? Are there any red flags or signals?

Gil Baumgarten (7m 25s):

Well, the first thing they should consider is whether active management or passive management is a better strategy for them. Active management makes the assumption that somebody is smart enough to parlay my money as an investor smarter than the market would. There's more mutual funds that exist or professionally managed pools than there are stocks for them to choose from. If you look at the fact that the typical mutual fund holds 90 stocks, don't they in fact all own the same thing? I mean there's not that many listed securities. So that being the case, it becomes a bit of a siphoning operation where everybody's trying to convince you that they are smarter than everybody else.

Gil Baumgarten (8m 7s):

I'm here to tell you that the market as a whole is going to provide a better rate of return than all the people who are trying to make decisions for a fee.

Phil Muscatello (8m 15s):

So would you suggest that first time investors look at index funds to start their investing?

Gil Baumgarten (8m 21s):

I do think index funds are a great place to be. I like ETFs as well. Mostly the indexed version of ETFs. They come in active management as well but ETFs have different tax treatment than open-ended mutual funds and index ETFs carry even a more powerful compounding methodology simply because they don't rearrange their holdings. And exchange traded funds don't offer co-mingled accounting with other shareholders. So repositioning the funds assets at the end of every day doesn't affect an ETF shareholder like an open-ended shareholder does. And it maximizes that unrealized gain that we are looking for, which increases the likelihood that gains are tax-free.

Phil Muscatello (9m 7s):

And it's also to do with the costs as well, because costs end up compounding as well.

Gil Baumgarten (9m 11s):

Absolutely, but that is not as big of a distinction. And when you're looking at open-ended fund structure versus ETF fund structure, the costs are essentially the same. It's the active management versus passive management that is the large driver of cost differential. Which is normally about 10 to 20 times higher in an active fund versus a passive fund.

Phil Muscatello (9m 38s):

So let's talk about the realities of investor behavior. What are some of the investor behaviors that you've seen and people should guard against?

Gil Baumgarten (9m 46s):

Well, we are hardwired to make bad decisions and we are hard wired because we have a fight or flight type of mentality. Which means that we only respond to negative stimulus and that negative stimulus could be the cash that we are holding while we're watching the market go higher. We have a negative stimulus that is the greed of the opportunity that is passing us by. And on the downside, we have the stimulus of having already lost money, which when we started this conversation, it was my methodology that you should buy the dips, not sell the dips. And most people are wired to sell the dips and buy the advances and it should be done the other way around.

Gil Baumgarten (10m 30s):

And so we're just not wired to be good investors. We have to learn that behavior.

Phil Muscatello (10m 35s):

What are some techniques for learning this behavior?

Gil Baumgarten (10m 37s):

Mostly the school of hard knocks. Otherwise you could also use an advisor like me or like anybody else who has a bias that has been established with, in my case, 37 years of doing some things right on something's wrong. And learning from that pattern of behavior and, you know, try to help clients make better choices.

Phil Muscatello (10m 57s):

So do you find that's part of your role as holding their hands when the markets are going through one of these periodic ructions?

Gil Baumgarten (11m 3s):

Well, John Bogle, the famous investor who headed up Vanguard and pretty much invented the index fund said that an advisor's most valuable role is keeping an investor in their seat. My most valuable role is not to outperform the stock market, but it is easy to outperform other people because other people do such a horrific job of managing their fear and greed mentalities. It becomes very exploitable for somebody like me. Who's more of a neutral observer.

Phil Muscatello (11m 36s):

Have you got any examples of clients that you've really had to work on to get them to stay in the seat?

Gil Baumgarten (11m 41s):

Well in the book, "Foolish", we talk about two clients who actually knew each other 60 years before I had done business with both of them. I knew they were both from Ohio, I just didn't know that they knew each other. And as it turns out, one of them had far better investment performance than the other we'll call him Alvin. And the other one was Barry, who had very poor investment performance. And they had just very different perspectives on how to manage their retirements. And Alvin ran circles around Barry because he was patient and optimistic. And Barry always thought that the sky was falling. Always thought that a transaction either to buy or to sell was going to solve this problem where sitting still really would have been the best thing for him to have done in most cases.

Gil Baumgarten (12m 29s):

He just wouldn't do it. So we get into that story in the book, which I think would be very illustrative of your question.

Phil Muscatello (12m 37s):

Fantastic. So in the notes that I was looking at for this interview prior, you were talking about designations and why investors may be confused by them. Now I've never heard the term designations. What does it mean? And what's confusing about them?

Gil Baumgarten (12m 52s):

Well, you know, there are financial planners and there are IMCA designations and there's certifications for financial planning, there's brokers who are calling themselves financial advisors, you have advisors who are calling themselves fiduciary, you have some advisors calling themselves fee only when they also earn commissions. It's extremely confusing. I would think for the typical investor to really understand the ecosystem of who is it, that's advising me and whose side are they on? And the advisory business does a very good job of obfuscating that by inflating their own expertise to garner the attention of new investors, when in fact that confidence may have been misplaced.

Phil Muscatello (13m 39s):

So what are some of the designations and what sort of questions should an investor ask to find out if the advisors interests are aligned with themselves?

Gil Baumgarten (13m 49s):

I'll answer that in two parts. First, there is a place for commissionable and conflicted advice. I don't believe that it is a good idea to have one of those people advise you on your life savings because that person has incentives that are built into their system. That probably are not in the client's best interest. I'm of the opinion, and it ultimately drove me out of the brokerage business, I'm of the opinion that the best way to serve the needs of a advisory relationship with clients is to go ahead and become a fiduciary. That required me to give up my licenses, that required me to lose the ability to charge a commission or a spread.

Gil Baumgarten (14m 32s):

I still earn fees but they have to be on a client statement. So if a client investor were to agree with that philosophy, the first thing where they would want to know is if the advisor is series seven license. If they have a series seven, then they work for a brokerage firm. Another that is a little less in your face, it's a lot more subtle, is to ask them if they earn 12b-1 fees. A 12b-1 fee is a kickback, plain and simple. And it is designated as a commission. Anybody who is a fiduciary advisor would not be allowed to earn a 12b-1. And if a investor asks that question, whoever it is, is going to be required to disclose whether they earn 12b-1, and those are kickbacks from mutual fund sales.

Gil Baumgarten (15m 22s):

So that separates the wheat from the chaff real quickly, right there.

Phil Muscatello (15m 26s):

Many people don't go to financial advisors because they see them as being expensive. However, then they go and make very expensive mistakes themselves. It can be viewed as an investment as well as a cost.

Gil Baumgarten (15m 38s):

Yes, I have a peer in my business who we compare notes about his business and my business. And one of the things that I remember him saying from 20 years ago is, you know, Gill, the S&P 500 is the fantastic return that nobody ever achieves. The reality is that you could own the S&P 500, if you would buy an index fund. But most people don't sit tight, they don't understand taxes, they want to buy it back lower. And another thing that is very prevalent is that because people experience the pain of a loss more painfully than they experience the pleasure of a gain, they don't calculate the odds correctly.

Gil Baumgarten (16m 20s):

Daniel Kahneman proved in years past that people experience a loss three times more acutely than they experience a similar dollar gain. So a profit of $10,000 feels like a 3 on a scale of 1 to 10, and a loss of $10,000 feels like a 9. And so when we're looking at a gain or loss of, you know, $10,000, we don't properly assess our own risk appetite because we're trying to avoid the pain of a loss more than we were trying to make a gain. So a financial advisor can be very powerful as a tool to call an investor out when they're exhibiting risk-avoiding behavior that is probably not in their best interest.

Gil Baumgarten (17m 6s):

That type of advisor relationship can be profoundly valuable far in excess of the fee. And there's also many studies that show that even though less optimal solutions are provided on brokerage platforms, the performance still runs circles around investors who go it alone and are therefore avoiding a fee, but are making much poorer investment choices and don't know the difference.

Phil Muscatello (17m 32s):

So you're kind of a therapist as well.

Gil Baumgarten (17m 33s):

90% of what I do is financial and psychological counseling. The money that we manage behind the scenes is almost a commodity. We take it for granted because we understand the ecosystem and we make choices on a very defined fashion. And people perceive that as the value add, that's not the value add. The value add is understanding what makes somebody tick and making them do the hard things. That's where the value is.

Phil Muscatello (18m 1s):

Let's talk about two portfolio theory. What is that?

Gil Baumgarten (18m 5s):

Well, we find that when clients mix risk assets and non-risk assets, they can't quite understand what it is. That's making them feel good or bad about their portfolio. They look at their statements and in an up market it looks like I'm trailing behind, in a downmarket maybe I should be losing less than I perceive. And when you mix that all up into a big pot, people lose the ability to attribute what it is that is causing them angst. We tend to separate that into multiple account numbers and you build a low risk portfolio and a higher risk portfolio so that you can discern, what is it in the good market that's doing really well?

Gil Baumgarten (18m 49s):

What is it in a bad market that's doing better? It allows people to discern. So we have people build two separate portfolios and we don't really like them to mix them into a common pot. Because as in the book we talk about there's something sour in my spaghetti and I don't know whether it's noodle or whether it's sauce. And so we're trying to help people distinguish that. And the way we do that is to split their portfolio into two components. And we weight that based on what the client is telling us. But we also weight it based on the personality characteristics of having gotten to know the client and what people say and what people do are two different things. And it's our job to understand how to apply that to a client and make them stick with it.

Phil Muscatello (19m 33s):

Are these portfolios weighted differently because of risk appetite or something like that?

Gil Baumgarten (19m 38s):

Yeah risk appetite, or income characteristics, or cashflow metrics, or there's also another dichotomy that exists that people want a hundred percent of the upside and they want somebody else to own the downside. That's just not practical because you can't lift your legs in the right rhythm to be in sync with whatever the opportunity set is going forward. So we tend to build static portfolios and then we rebalance them over time when one does far better. But we may meet a client who we don't think they have the risk appetite that they say that they have. And we might talk the client into a 50:50 stock and bond blend. And then we find that the first sign of trouble, March of 2020 in the market decides it wants to drop 25%, all of a sudden they start to try to back up and de-risk their portfolio.

Gil Baumgarten (20m 29s):

We're at that point trying to take money from the no risk pot and put it into the risk pot and they're trying to backpedal. So that's really, when all this theory comes to apply to someone, that's where the really good and bad decisions are typically made.

Phil Muscatello (20m 44s):

Gil this is a question without notice. You're obviously with a lot of high net worth individuals, but what would you say to someone who's really starting from a standing position? They don't have any money, they don't have any assets, they're young and they just getting started in investing. They want to get ahead wealth-wise. What sort of advice would you give them?

Gil Baumgarten (21m 3s):

They need to distinguish the difference between a speculation and an investment. If you go the investment route, as opposed to speculation route, you're much more likely to experience positive attributes. If you go the speculation route, which you're trying to swing for the home run on every pitch, you're just not going to have good results. Every once in a while you may nick the ball, but you're going to end up with a lot of foul balls and problems from that. So I would think that somebody should start from a position of suppressing their expenses. If they have no assets, all they have is some cash flow from their work, they need to cut back on their expenses to at least get the snowball rolling.

Gil Baumgarten (21m 45s):

Save a hundred dollars a month then ratchet it up to $500 a month. Try to get into business for yourself where you can take both the company's profit plus the profit you would've made as an employee. Ultimately, my income got kicked into high gear when I made the choice to leave the brokerage industry and build my own firm. I would encourage entrepreneurial-ism. Not everybody is wired that way you can still do just fine if you suppress your costs and have some residual. Ultimately wealth is the sum total of all the money you've never consumed. And if you're only making a couple of thousand dollars a month, if you can figure out a way to pair that back and save a hundred bucks and put it into something that's an investment or an index fund or something that can provide some stock market returns without being speculative, that's the best way to parlay that to get the ball rolling.

Gil Baumgarten (22m 37s):

So that's what I would suggest.

Phil Muscatello (22m 39s):

So you have a methodology for people to donate part of the stocks that have appreciated to charities. How does that work?

Gil Baumgarten (22m 46s):

People who are generous tend to be really good risk takers and people get rich from taking good risks and then they turn around and become generous again. So it's sort of like this self-perpetuating cycle. And so the IRS allows people, it's a little less deductible, but you can donate appreciated securities that contain a gain and get the same type of tax treatment as though you had donated cash. The difference is that cash is deductible up to 50% of your income and stock gifts are deductible up to 30% of your income. So most people are not donating 30% of their income every year.

Gil Baumgarten (23m 28s):

So there would be no difference. But if you're trapping a gain that would otherwise be taxable to you if you were to sell the position on the future, gifting those positions to your favorite charity is a lot more creative way to reduce your future tax bill than just simply giving cash. Some people might say, well I don't want to give up that security, give that security away and take the cash you were going to give the charity and buy the security back. What you're doing is you're incrementally stepping up the costs to trap the smallest amount of gain and use those highly appreciated securities as currency for charitable giving. That's the primary reason why we use individual securities as opposed to ETFs or mutual funds, because in our dividend strategy, which has had fantastic performance, it has not had the thousand percent gain that our Home Depot shares have had over the past 12 or 13 years.

Gil Baumgarten (24m 24s):

So if you can reach into your account and pull out your Home Depot shares, gift that to charity, turn around and put the cash back in the account and buy the Home Depot shares back. That is a way to rinse capital gain activity out of your account and it's fully sanctioned by the IRS within those limits and can be a very powerful tool. So there you have it.

Phil Muscatello (24m 44s):

So you're a man of many interests, but the one that I'm particularly interested in is precision woodworking. What is precision woodworking?

Gil Baumgarten (24m 51s):

Well, my firm Segment Wealth Management is named after a style of artwork that I do called segmentation. And it is the cutting of various colored species of wood, and then assembling them into a pattern. And you can see many examples of this on my website, my personal website and not my company website, which would be gilbaumgarten.com. That's one L in Gil. I have my artwork, my painting, I do oil, I do acrylics, I do sculpture. But precision woodwork would be, there's a bowl, an Ebony and maple bowl, and I like dark and light woods. So you'd have Ebony being almost black and maple being almost white.

Gil Baumgarten (25m 32s):

And I think there's 315 pieces of wood in that that are all assembled in a precise pattern. Yeah, that's what I like to do is I like to see the contrast and the ability to have all the points aligned just right so that all the pieces were assembled just perfectly.

Phil Muscatello (25m 50s):

That obviously gives you a great deal of satisfaction.

Gil Baumgarten (25m 52s):

Yeah, I do. I like it.

Phil Muscatello (25m 54s):

Yeah. So Gil, tell us about the book and where people can buy it and websites and how they can get in contact with you and find more about your world.

Gil Baumgarten (26m 1s):

Yeah. So "Foolish" is the name of the book. The subtitle is "How investors get worked up and worked over by the system". It's available on Amazon. It's also available on the Target website. And my firm's website is segmentwm.com. That's Segment Wealth Management, segmentwm.com. We also write a blog segmentwm.com/Blog. You can sign up there for free distribution of our periodic writings. We call them musings, just about taxes and portfolio strategy and interest rates. Just whatever pops into my head. I'll write out a, you know, four or 500 word dissertation on it.

Gil Baumgarten (26m 42s):

It goes out to everybody who's on the list. And the history of the ones that we've written, we cut it off at about five years. So the last five years of our writings are available online If anybody wants to look at it and see how accurate we were at prognosticating what was coming next?

Phil Muscatello (26m 57s):

Oh well look, we'll put all of those links in the notes and in the blog post. But Gil Baumgarten, thank you very much for joining us today.

Gil Baumgarten (27m 4s):

Thanks for having me.

Stocks for Beginners is for information and educational purposes only. It isn’t financial advice, and you shouldn’t buy or sell any investments based on what you’ve heard here. Any opinion or commentary is the view of the speaker only not Stocks for Beginners. This podcast doesn’t replace professional advice regarding your personal financial needs, circumstances or current situation.