Every industry is trying to figure out just how AI or Large Language Models can be used to do business. But Bridgewater Associates, the world's largest hedge fund, has already been at it for a long time. For years, it has explored AI and adjacent technologies in order to analyze data, test theories, develop novel investment strategies and help its employees make better decisions. But how does it actually use the tech in practice? And what's next going forward? On this episode, we speak with co-CIO Greg Jensen about both the possibilities and limitations of these advances. We also discuss markets and macro, and why he believes that investors are still too optimistic about the Federal Reserve's ability to get inflation back to target. This transcript has been lightly edited for clarity.
Key insights from the pod:
Bridgewater and Jensen’s relationship with AI — 3:18
Poker, Go and AI — 8:35
How Bridgewater is investing in AI? — 17:14
Differences in GPT iterations — 19:13
How to use AI for investing — 23:33
What AI means for human employees — 26:16
AI and reflexivity in markets — 29:28
What data is Bridgewater using? — 37:24
What Bridgewater got wrong about markets and the economy — 43:25
Are markets still too optimistic? — 51:32
–––
Tracy Alloway: (00:10)
Hello, and welcome to another episode of the Odd Lots podcast. I'm Tracy Alloway.
Joe Weisenthal: (00:14)
And I'm Joe Weisenthal.
Tracy: (00:16)
Joe, I think it's fair to say there is a lot of excitement about investing in AI. There is also a lot of excitement about using AI to invest.
Joe: (00:28)
Yes. I mean, I think there's like a new like Chat ETF I saw an ad for. I think I saw another project. It was like, “We're going to have ChatGPT pick the stocks for us.” And you know, I get it. It's kind of exciting and maybe there's some new way of like these super advanced digital brains that can beat the market, etc. But I don't totally get it.
Tracy: (00:51)
Well, I also feel like there's a tendency nowadays for people to talk about artificial intelligence in a sort of abstract manner. You hear people bring up AI almost as a synonym for just software at this point. I think you pointed out recently that the Kroger CEO mentioned “AI” like eight times on the earnings call. So a supermarket chain, right?
Joe: (01:16)
Yeah. And you know, it's like machine learning, tech, algebra, algorithms, it's all existed for a long time. Quantitative investing. But it feels like because of the excitement around a few specific consumer facing products that have been unveiled over the last six months and the way they've captured people's attention, you know, suddenly there's a lot of interest in how are companies using this tech to do something?
Tracy: (01:41)
Yeah. Well, I'm glad you mentioned that because today we really do have the perfect guest. This is someone we've actually spoken to about AI before. Last year, in fact. Someone who is at a firm that has a lot of experience using machine learning and AI of different types. And we're going to get into the differences between all those technologies. I'm very pleased to say we are going to be speaking once again with Greg Jensen, the co-Chief Investment Officer at Bridgewater Associates. So Greg, thank you so much for coming back on Odd Lots.
Greg Jensen: (02:16)
Yeah, it's great to be here. Exciting topic.
Tracy: (02:19)
Yeah, so I actually revisited our conversation from last year. I think it was in May of 2022. And you said two things that stuck out in retrospect. So number one, you said that markets had further to fall, which turned out to be correct. And two, you brought up artificial intelligence as a major point of interest for Bridgewater. And this was all before ChatGPT really became a thing and everyone started talking about AI at every single conference and earnings call and so on.
So I guess just to begin with, maybe you could lay the scene, going back to Joe's point in the intro, we are used to hearing these terms. So Bridgewater does machine learning and systematic strategies and quantitative trading strategies and AI and things like that. What's the difference between all of these things and how do they relate to each other at a firm like Bridgewater?
Greg: (03:18)
Yeah, great question. So I think to answer that, let me take a step back for a second and give you a little bit of my background because it all kind of comes together in a way to connect these different pieces.
So, you know, even as a kid or whatever, I was certainly interested in kind of translating and predicting things using some mix of my thinking and technology. So I can think back to in the late eighties using Strat-O-Matic baseball cards. I don't know if you know what they are, but programming them into computers to try to calculate the way to create the best baseball lineup and use that in fantasy baseball type situations. And similar things with poker and whatever, and trying to learn how to kind of use technology to combine with human intuition, to get at what was, different ways to create edges.
And then in college, when I heard about Bridgewater, Bridgewater was a tiny place at the time. But the basic idea was that there was a place where we were trying to understand the world, trying to predict what was next, but doing that by taking human intuition and translating that into algorithms to predict what was next. It kind of mixed two things that I loved. I loved to try to understand the world, and I love the idea of having to discipline, to write down what you believed and stress test what you believed and utilize that, right?
So if you go back, and this is now in the nineties, kind of where artificial intelligence was at the time, most of the focus was still on expert systems, was still on the notion that you could take human intuition, you could translate that into algorithms. And if you did enough of that, if you kept kind of representing things in symbolic algorithms, that you could build enough human knowledge to get kind of a superpowered human.
And Bridgewater was a rare example of where that worked, where given the focus of trying to predict what was next in markets, given the incredible investment that we made, creating the technology to take human intuition and translate that into algorithms and stress test. It's an incredibly successful expert system, essentially, that was built over the years. I'd say probably the most profitable expert system out there.
And that's really what Bridgewater has been about, which is building this great technology to help us take human intuition out of the brain, get it into technology where it's both then readable by, let's say investment experts, but also runs on a technology basis. And that's kind of where algorithms, let's say, the mix of algorithms and human intuition. And it was really important, you know. If you go through the history of our competitors, they're littered by people that tried to do something more statistical. Meaning that they would take the data, run regressions, and then after regressions, let's say basic machine learning techniques, to predict the future.
And the problem that always had is that there wasn't enough data. The truth is that market data isn't like the data in the physical world in the sense that you only have one run through human history. You don't have very many cycles. Debt cycles could take 70 years to play out. Economic cycles tend to play out around for seven years. There's just not enough data to represent the world.
And secondly, the game changes as participants learn. So the existence of algorithms, as an example, changed the nature of markets such that the history that preceded it was less and less relevant to the world you're living in. So those are big problems with, let's say, a more pure statistical technique to markets.
So you had to get to a world where statistical techniques or machine learning could substitute for human intuition. And that's really where kind of the exciting leaps are now. That you're getting closer. It's not totally there, but you're much closer than you've ever been, where large language models actually allow a path to something that at least mimics human intuition, if not is human intuition.
And that you can then combine that with other techniques and suddenly you have a much more powerful set of tools that can deal – or at least take a big leap forward – on dealing with the problem of very small data sets and the fact that the world changes as people learn in a way that up until the big breakthroughs in large language models, I think were much further away.
So that's a huge change in the limits of ways that statistical machine learning could affect something with small amounts of data. Something where the future varies from the past. All of those problems, we’re closer to having at least ways to take on more and more of what humans have done at Bridgewater, what humans generally do in investment management firms. And that's a huge leap forward that's going on now.
Joe: (08:09)
I have one very short, quick question. I realized just now that not long after we talked to you last year, last spring, like a month later, you won your first World Series of Poker bracelet. So congratulations on that. I only say that because you mentioned poker. Did you play the World Series this year?
Greg: (08:27)
I'm heading out actually after this…
Joe: (08:30)
Okay. Congrats.
Tracy: (08:33)
Good luck!
Joe: (08:34)
Yeah, good luck.
Greg: (08:35)
Yeah. And it kind of connects to this because I never get to, I don't get to play very much poker, but I really studied what machines were learning about poker. So much has been learned in the last five years, 10 years. And, you know, basically trying to translate that into intuitions that I could use. You know that basically can't actually replicate poker in a very complex way, but you can pull the concepts out, right?
And this actually mirrors part of what we're doing at Bridgewater, which is that as you get to computer generated theories, that if you can pull the concepts out of these complex algorithms, you can make more of an assessment, human assessment, of whether they make sense and what the problems might be. And that's really a big deal.
So there's actually a link between what I'm doing in poker – imperfectly for sure – and many of the concepts that we're trying to apply at Bridgewater. And like you said, we had talked kind of before the LLMs had really hit the public scene.
I mean, just to give you a little bit of background for me. You know, if you go back to 2012, first off, we brought Dave Ferrucci, who had run the Watson project at IBM that had beat Jeopardy, into Bridgewater. And that was a time when I was trying to experiment with, okay, what can we do with more machine learning techniques? And Dave was trying to take what he had done to win at Jeopardy, but actually put in more of a reasoning engine. Because while what happened on Jeopardy was impressive, it was pure data. It had no idea why it was doing what it was doing. And therefore really a lot of the path with Watson or whatever was going to be very hard to move forward with because at its end, it was just statistical and it didn't really have any reasoning capability.
So Dave came to Bridgewater and later partnered with Bridgewater to roll out a company, Elemental Cognition, that focused on using large language models, etc., but overlaying a reasoning engine that essentially helps with things like hallucinations that large language models have and focus on how do, what does human reasoning and how does it work and how does it limit views that are unlikely to be true? So that's one thing.
And then in 2016 or ‘17, I was introduced to OpenAI and actually as they transitioned from a charity to a company, I was in that first round and I met a lot of the people and looked hard at their vision using scale, technical scale, to build general intelligence and build reasoning.
So I both was working with Dave Ferrucci and sort of understood many of the people at OpenAI at the time and moving forward with those things. And then I was literally the first check for Anthropic, another large language model, kind of [made by] people that had been at OpenAI.
And so I've been passionate about this, trying to take different paths to how will we build a reasoning engine to overlay on statistical things and couple different approaches that were being applied at the time. And obviously they've panned out to a different degree, but many things are coming together now to say, okay, you can actually – in a way, at a pace and a speed humans could never do – you can replicate human reasoning. And that's a huge deal. And if you could really break through that, you could start to apply it in so many ways in our industry, I believe, and obviously way beyond our industry.
Joe: (12:23)
You talked about earlier generations trying to embed human knowledge. And I'm wondering, you know, if an analogy is like, I remember when Deep Blue came out. They had all the grandmasters sort of work with IBM to come up with this great computer program that was basically as good or eventually better than Gary Kasparov. But then the next generation of chess computers didn't even have the grandmasters playing. It just learned the game from ground-up and crushed those previous generations. Is that sort of what we're talking about here with the transition from earlier engines to the new sort of LLM-focus? Which is, the reasoning comes out of the computer rather than having to be taught directly by the experts.
Greg: (13:08)
Yeah. I think something like that is happening, right? You got that in chess because once you had the ability, you had enough data and enough compute, you were able to do enough sampling that you got to the point where the pure data process with, you know, good human intuition on how to build that data process. But a data process was able to beat those rules-based things.
Now chess, unlike markets is, you know, a little bit more static in the sense that while there are adversaries – and the adversaries they'll try to learn your weaknesses – it's more static in the rules of the game are steady and those types of things so that sampling could work.
Although what's interesting, I love Go because it is an analogy to some of the problems that pop up and will pop up. If you take AlphaGo, right. On the Go game, Go – after chess obviously – but Google was able to create this game that was beating the pros and radically beating the pros, killing everybody and getting better and better and better.
Although, you know, – I don't know how up to date you are – but then there was this loophole in it where another person who is a mediocre Go player but a computer scientist who thought there might be a hole in this super-AI, used a little program to find the hole. And what it illustrated was the AI had no idea how to play the game. Because the mistake the AI was prone to was a mistake a six-year-old playing Go would never make, wherein if you made a large enough encircling – I don’t know if you know how Go works – but if you encircle the other guys' pieces, you eliminate them all.
And something that would never work in a human game is you make a really big circle. And because it never came up in human games and because when they perturbed human games and started playing computer against computer, they basically started with a seed of human games. They never perturbed it enough to try this out to try a massive circle. And a human would never let the massive circle happen.
It's so easy to defend against, but actually the best Go algorithm in the world allowed it to happen, right? And now a mediocre Go player with a little bit of AI found a way to beat this incredible Go game. Again, because the Go algorithm at that time had this tremendous amount of data, but the things that weren't in his data, it wasn't aware of, and it wasn't in any deep sense understanding the principles of the game.
So that's the type of, you know, data problem you can have even with a massive amount of data play, you know, millions and millions of games. But to play every possible Go board, you'd have to, there are more possible Go boards than there are atoms in the universe. So it was never going to calculate every possibility. And it never got to reasoning, right?
And therefore, that was a weakness. And on the other hand, had you mixed that, blended that, even with a basic reasoning that a language model could come up with – understanding the rules of Go and being able to talk about it – there's an element of knowing those things that humans already know that's possible with a blend of, let's say a statistical technique like AlphaGo was using and a reasoner to prevent these types of mistakes.
Tracy: (16:22)
I like that story because it makes me think I have a chance against the super smart supercomputer. Okay, that's kind of comforting. But I definitely want to ask you more about weaknesses in AI and large language models. But maybe before we do, you know, just sort of setting the groundwork once again, but when we see headlines like “Bridgewater Restructures, Will Put More Focus on AI,” what does that mean exactly?
What does it mean for a firm, an investment firm like Bridgewater, to build up resources in AI? And then secondly, could you walk us through a concrete example of how AI would be deployed in a particular trading strategy? I feel like the more concrete we can get with this, the more helpful it'll be.
Greg: (17:14)
Yeah. Great. So I think that as we restructured one of the things that, as we've made the transition at Bridgewater, you know, from Ray having the key ownership to ownership at a board level and that transition, we have done something we hadn't done in the past, which is essentially retain earnings in a very significant way, which allows us to invest in things that, you know, aren't going to be profitable right away, but are the big long-term bets that we're making. And certainly recognizing that there's a way to reinvent a lot of what we do using AI machine learning techniques to improve what we're doing to understand the world accelerate that.
So, and specifically what we've done on the AI ML side is we've set up this venture. Essentially there's 17 of us with me leading it. You know, I'm still very much involved in the core of Bridgewater, but the 16 others are a hundred percent dedicated to kind of reinventing Bridgewater in a way with machine learning.
We're going to have a fund specifically run by machine learning techniques, which will take me into, Tracy, what kind of strategies you could do. You know, that's what we're working on right now in that lab and pressing the edges of what AI's capable of, machine learning is capable of.
Now, right now there are big problems, right? You take large language models and they have two types of problems. One thing is the basic problem is they are more trained on structure of language. So they usually return something that looks like good structural language. They don't always return accurate answers. So that's a problem. It hallucinates, it makes things up because it's more focused on the structure of what word or what concept would come next than whether it's accurate in what concept comes next.
Tracy: (19:00)
Can I just say, when I hear “AI hallucinations” it becomes so science fiction for me. It's very like robots “dream of electric sheep” kind of. It's just so surreal.
Greg: (19:13)
Yeah, well, I mean in this case you can imagine what's happening, right? Because it's just what it's trained on, right? So if you're just, basically the basic concept is; give it any stream of words and it'll predict based on having read everything that has ever been read what comes next, right? And that if it's a little bit wrong in what comes next, it can misfire and give you something that sounds like something that could come next, but actually is wrong.
You know, it is just what it's trained on, right? It's trained to predict the next word, slight errors that create those types of issues. Now the algorithm is pretty remarkable particularly, as I said, I've been tracking OpenAI as an investor for a long time and looking at their technology for a long time.
And you know, up until, there's GPT-1, GPT-2, GPT-3 and many versions in-between. And then at GPT-3, it started to have some use. GPT-1 and GPT-2 were, you know, barely coherent. GPT-3 was, you know, somewhat usable for certain tasks.
GPT-3.5, which is what ChatGPT is, you know, got to a certain level. Like on Bridgewater's internal tests, you suddenly got to the point where it was able to answer our investment associate tests at the level of first year IA, right around with ChatGPT-3.5 and Anthropic’s most recent Claude. And then GPT-4 was able to do significantly better. And these are, you know, at least what we thought were conceptual tests significantly better than our average, you know, first year investment associate that went through training. And similarly it's able to take the LSAT and do well, etc.
So it can be basically pretty smart. It is pretty smart on a wide variety of things with errors, but pretty smart on a wide variety of, whether it's the MCAT or the LSAT or Bridgewater's internal tests or whatever. A whole wide variety of things. This is a big deal that it can achieve all of those kind of academic things. And yet it's still 80th percentile kind of thing on a lot of those things, which is remarkable to be 80th percentile on many, many different things.
But at the same time, it's 80th percentile for a reason. There are flaws, meaning it's not one hundredth percentile. And so that leads to, you need to find a way to work through those flaws, right? And that's really where you know, so if somebody's going to use large language models to pick stocks, I think that's hopeless. That is just a hopeless path. But if you use large language models to create some theories – it can theorize about things – and you use other techniques to judge those theories and you iterate between them to create a sort of an artificial reasoner where language models are good at certainly generating theories, any theories that already exist in human knowledge, and putting those things connected together. They're bad at determining whether they're true.
But there are other ways to pair it with statistical models and other types of AI to combine those together. And that's really what we're focused on, which is combining large language models that are bad at precision with statistical models that are good at being precise about the past, but terrible about the future.
And combining those together, you start to build an ecosystem that can achieve, I believe can achieve the types of things that Bridgewater analysts, combined with our stress-testing process and compounding understanding process at Bridgewater, can do, but it can do it at so much more scale.
Because all of a sudden if you have an 80th percentile investment associate, technologically, you have, you know, millions of them at once. And if you have the ability to control their hallucinations and their errors by having a rigorous statistical backdrop, you could do a tremendous amount at a rapid rate. And that's really what we're doing in our lab and proving out that process can work.
Tracy: (23:13)
Oh, I see. So is the idea that AI could possibly generate theses or ideas that can then be rigorously, you know, statistically fact-checked by either the humans or, you know, existing algorithms and data sets? Is that the idea?
Greg: (23:33)
Yeah. And then the idea goes further, but yes, that's the start. Language models can do that. Statistical AI can then take theories and generate whether those have at least been true in the past and what the flaws with them are and refine them, offer suggestions on how to do them differently, which then you could dialogue with.
So then the other strength the language model has that humans are weaker at is, now take a complex statistical model and talk about what it's doing. And there's ways to train language models to do that then allow sort of a judgment to say, okay, now let's think about what's happening here and reason over what's happening.
So you use, the way we've modeled this kind of out is that language models can come up with potential theories. Now there's a limit to that. It's not the most creative thing in the world, although it's theory at scale, for sure. And again, that’s language models with good, you know, you have to tune your language models in a certain way so it's not straight out of the box. But then you can use statistical things to control that.
Then you can use language models again to take what's coming out of that statistical engine and talk about it with a human or other machine learning agents and kind of report back on what you're finding and what that is and the types of theories that are out there that might run contrary to what you believe, which can lead to more tests and other things.
So that's the loop that I'm very excited about and as I said, up until, statistical AI was limited because it was focused on the data of markets. Where language models, the good thing is it has a much better sense of something that a statistical model wouldn't really have.
A statistical model of markets doesn't get the concept of greed. Language models pretty much understand the concept of greed. They've read everything that's ever been written about greed and fear and whatever. So now they can start to think about statistical results in the context of the human condition that generates those results. Big deal. And really a radical difference.
Joe: (25:39)
Let me ask you one very simple question, and it might be one that speaks to an anxiety of listeners. If already GPT can perform at maybe the type of level that a high-quality first-year or second-year associate or analyst at Bridgewater can do, does it mean fewer hires in the future – humans being hired at Bridgewater? Or does it mean the same number or more humans doing even more? Is it a replacement? What does it mean for the type of person that would've been, 10 years ago, a first-year employee at Bridgewater?
Greg: (26:16)
What I think people should expect at Bridgewater and just generally, is things are changing quick. That really requires people to be capable of playing whatever role is necessary in order to do that, right? Like if you go back, the clock at Bridgewater, when I started, or just before that, right? We were, you know, we were using egg timers, like we had rules on how to trade, but we were using egg timers and humans to do these things. And over time, computers could do more and more.
We kind of got to this point where it was, I'd say, kind of humans settled into the role of intuition and idea-generation. And we used computers for memory and for constantly running those rules accurately, etc. That was a transition that half, it got to 50/50, technology and people. And now this is another leap, right?
And it's definitely true that it's going to change the roles that investment associates play. Now exactly how, and you still need– for as far foreseeable future – you're going to want people around that, out that working on those things, there's edges that these techniques I'm describing certainly won't do well for an extended period of time. And there's how to build the ecosystem of these machine learning agents, etc.
And so what I've found, and certainly the people in the lab, you want people who are curious about these new technologies. You want to utilize them. And that's going to be really part of the future of work. I think it's going to be very hard in any knowledge industry to not utilize these.
And we're seeing this huge breakthrough in coding, right? That that is so democratizing in a sense that you really need to know what you want to code more than you need to know coding, you know, and that's a big breakthrough. So a bunch of people that weren't as well trained or as capable in C++ or in Python or whatever can suddenly get what they want so much faster.
So all of a sudden, the skill sets are changing and they're changing in ways that I think are a surprise to many because it's actually a lot of the knowledge work, a lot of the things where, you know, content creating and whatever that I think people thought would be later in computer replacement that are happening faster.
So the main thing is, I'd say right now there's so much in flux, that having flexible, the more you need flexible generalists who can have an eye towards this, an eye towards the goal, and be able to utilize whatever tools are necessary to get there. That's really where I think, you know, you're seeing a fair amount of change quickly.
Tracy: (28:56)
So you mentioned earlier that just the existence of machine learning can impact both the current environment and the future. So I think you said that the future data points aren't going to look like the past data points simply because machine learning exists. Does that sort of reflexivity between machine learning/AI and markets become more of an issue as AI and machine learning becomes more and more popular and more in entrenched?
Greg: (29:28)
Yeah, I think it's a big deal, right? And I think it's both something that's going to cause accidents and something I'm super excited about. Obviously, I'm excited about the power of this. I think there's ways to utilize it really well. And also, there will be a lot of mistakes.
Like you're saying, there will be funds that will, you know, use GPT to pick stocks and not really deeply understanding what's happening and what the weaknesses of that might be. There are already plenty of times where statistical, pure statistical, because there's not enough data, you're not building with those issue fundamental issues in mind…
You know, not that it was directly in markets, but in the housing market, what Zillow did is a great example, right? Zillow goes out and uses an AI technique – that wasn't fit for purpose, for what it's worth – but they use an AI technique to predict housing prices and then go into the market to start buying houses that they think are undervalued, right?
And they have a couple problems. One is while they had a ton of housing data, it was over a relatively short period of time. So even though they had what looked like tons of data points, because they have the price of every house and everywhere and whatever, there's still a macro cycle that affects everything that was underestimated in what they did.
And secondly, they underestimated what it would be like in theory versus in practice when it's actually an adversarial market. Every time they won an auction, there was something about that particular lot that the other people bidding on that lot knew, that they didn't.
And so it ended up obviously being a huge problem for Zillow and they kind of had a big impact on the real estate market and then a big failure. And that's the kind of thing you're going to see over and over again if, because the basic problem that the data that you're looking at isn't necessarily the data you'll face in real world, you're not facing the adversarial problem when you're looking at that data the way they were.
A statistical technique that's very good at seasonality and trend following might not be very good at understanding macro cycles and so on. So that was another case where, you know, Zillow is a case and I think we'll see it over and over again, where the recognition that it's not as simple as taking machine learning out of the pack and applying it to this problem. Even when there's a ton of data, right?
Some of the places where there is a lot more machine learning going on, very short term trading, arguably is better for machine learning because there's a lot of data and you can learn faster over that data. And there's some merit to that. And in terms of tangible places – this is now years ago – but where we started applying some of these techniques were in things like monitoring our transaction costs and looking for patterns in shorter term data. Because there's a lot more data.
But on the other hand, the data often, it's like having the data of your heart rate for your whole life. You can feel like, wow, this is, yeah, I've got every heartbeat for, you know, 49 years. That seems like a lot of data, but it’s totally irrelevant when you've heart attack. So that even when there's lots of data, it could be misleading and those are the types of issues that will lead to these techniques having huge problems, which means it's not out-of-the-box, AI is going to solve all these problems.
You really, and this comes back to you have to understand the tools, what they're good at, what they're bad at, and put them together in a way that uses what they're good at and protects them from what they're bad at.
Now, nothing, no process we're coming up with will do that perfectly, but the more and more you could do that, I think the more and more you could become, let's say, better than humans at that. Because humans have many of those fallibilities or versions of those fallibilities that these processes will have. And that'll be the question of how far we can take that and how much human judgment is better than those things, which is stuff, you know, we'll be experimenting with as we go along.
Joe: (33:32)
So you know, one thing that your founder Ray Dalio years ago, like sort of, he wrote down a set of rules. You've talked about this before. He wrote down a set of rules about how he understood the sort of the machine of the markets to work.
And one of the issues with AI – and I think you're sort of been getting at this – is that AI legibility and the understanding of like, okay, you put in, you pose a query to a large language model, it creates some output. You don't really know like what it did to get there. And so that's, you know, that's sort of different than dealing with a human analyst.
You could say, well, did you think about that? Did you think about that? Can you talk a little bit more about the sort of, I don't know if that's a weakness or how do you sort of get around the fact that it's still difficult to query an AI model and say “How did you arrive at X or Y conclusion?”
Greg: (34:25)
Yeah, and I think that's really important. But also something that's more and more breakable. Cause even with humans, one of the things, one of the places where, I think there are a lot of areas where Bridgewater has a strength, right? Bridgewater has a strength and we never went from a statistical model, so we built data based on what we needed for reasoning, and as a result we have a better, longer cleaner database than I think anybody has.
We've been thinking through this problem that you're referring to, which is how do you actually get out what somebody means? You'd be surprised how hard it is to truly get from a human. Humans don't actually know why their synapses do what they do. They actually, when you ask somebody to describe something, you get some partial version of what they're thinking.
If you took like an intuitive trader and you start peeling back all the reasons, that's very hard. We've been doing that for a long time and have an expertise in doing it. And I would say that humans don't even know what they're doing often. But you can, there are ways to, you know, like you're saying query and force questions: “What about this? What about that?” That will help pull out human intuition.
And what you find with machine learning algorithms, if you get good at this is, and this is going back to 2016-2017, has been critical to my work, is there's a way that you can query machine learning algorithms like you query [humans] Like it's different, but the concept is the same, as how you query humans to get at why they really believe what they believe.
And as I was saying, I think there's actually elements of large language models interpreting what statistical AI is doing that allows that process to accelerate. And I think it's very critical. You really want to know because that's the way you find the flaws.
If you go back to my Go example and you can think about, if you can query a model and think about what it has done and what it hasn't done, then you can figure out what data's missing, right? And you need to set up adversarial techniques in order to keep querying an algorithm for what it's doing.
And again, I think that's still an area of research, but a process that's moving along quickly to basically get to the point where the standard is, even though a machine learning technique might be doing something very different than a human is, that it can still explain itself. And it might not perfectly explain itself just like humans don't perfectly explain themselves, but to a very high degree of confidence across a wide range of outcomes so that you have a sense of what's going on is possible. And that's part of the design of what we're putting in, which is, well, how do you query it? How do you give it more information, remove information, etc., see how it changes its mind to determine roughly what's going on.
Tracy: (37:00)
You know, you mentioned the data sets there and I guess it's a cliche nowadays to say, “well, a model is only as good as the data that it's trained on,” but it's a cliche because it's true. Do you use your own internal data for the large language models or where are you actually pulling in data from? And then secondly, like what type of data have you found so far is most useful for these types of projects?
Greg: (37:24)
Well, I think the things that are most interesting to us, A)e're trying to learn things that we don't already know. So we're being careful about what kind of Bridgewater knowledge we put in here because it's not that helpful if we reinvent Bridgewater. Somewhat helpful, but it's not as helpful as, let's say, reinventing everything that we don't know about. That other people have thought about, etc.
And so in the lab right now, at least we're focused on not making this too Bridgewater-centric on purpose because it's in that way we'll learn things that we don't already know. And if you just fed a Bridgewater information, which we may well do – that could be a productivity enhancing thing – but you'll quickly, you know, produce something very similar to Bridgewater.
Whereas what's been amazing so far is we're producing good results by Bridgewater standards, but different, very, very different conclusions and different thoughts than what we have internally. So I think that's point 1 choice.
Now on raw data and cleaning data and how you put together data. Now we are benefiting from Bridgewater scale on that. That's a big deal that over the years, again, precisely because we took human intuition and said, “What data do we need to replicate that intuition?” We have a unique database where if everybody else is pulling from Datastream, Bloomberg, etc. We put together the data we needed to feed our intuitions. Oftentimes that data didn't exist. We had to figure out the way to create it.
And also we’re big believers that you need to stress test across a very long period of time. So we have much longer data histories. Now those things are certainly valuable. In a context of small data, any quantity of data, understanding the data and being able to therefore, for a given theory, find appropriate unoptimized data. Those are big deals and that we are using. And you know, that does allow us to move forward more.
And on the large language models, there's still a lot of work to be done, but you certainly can train through reinforcement learning to, you know, to make sure that they're not making mistakes that you know about. And so there's ways to do that. Now we've been trying to avoid that for the reasons I was describing before. I avoid doing too much of that, injecting our own knowledge, and use external sources to do that. But that's still part of you know, part of the tool set that will be available that, yes, you can train it more directly on things you already believe to be true if you want to do that. And that certainly will lead to answers that replicate your thinking more quickly.
Tracy: (39:59)
So just on this point one thing I wanted to get your opinion on is; how good is AI at predicting big turning points or structural breaks in market regimes? Because I don't know about you, Joe, but one of the first things I did with ChatGPT was I asked it to write, you know, a financial news article about inflation just to see whether our jobs were in danger. And you could tell that it was trained on not quite current data. It was talking about how inflation has been stubbornly low for many years and the Fed is trying to get it to the 2% target. But how good is AI at predicting those regime changes? Because if you're running, you know, a macro fund, I imagine that one of the important things that you need to do is try to figure out when something is fundamentally changing in the market.
Greg: (40:51)
Yeah. And I'd say terrible if you use it in the sense that you're using it right? That it's a little bit like saying, well how good are people at that? Well, people are pretty darn bad at that, right? That doesn't mean that there isn't a way or some people who could do such a thing, right? So AI, it's hard to just think about AI as a thing or think of, okay, well if I'm just going to use ChatGPT for that, you're exactly right. ChatGPT as it comes out-of-the-box is only trained over to a certain history and it doesn't care. Like unless you know how to make it care, it doesn't care that it's just answering a question about inflation based on everything it's ever read about inflation. Time isn't even that important unless you make time be very important to it and predicting.
And so, you have to know how to use the tools to generate the type of outcome that you're describing. So do I think like AI out-of-the-box will do that? No, absolutely not. It'll be awful at that. Are there ways to take what's in embedded in AI to come up with a way to do that? Embedded in language models and if you combine that with statistical tools? Yeah, there's a path there, but it's not going to be as simple as open up ChatGPT and ask it that question. There's more involved. But it is helpful to have an analyst that's read everything that was ever produced, even if they stopped reading in 2022 or in 2021, I should say. There's a way to use that, but you have to use it correctly and not misuse it in order to try to generate that answer.
Joe: (42:22)
All right. So I can't just ask a large language model “when will inflation get back to the Fed's target?” But I'm not speaking to a large language model. I'm speaking to a CIO at Bridgewater. And I am curious, you know, we do want to talk a little macro.
Before we sort of, I'm not going to directly ask you when inflation will be back the Fed's target, but what strikes me about the last year and since the last time we talked that's really blowing my mind, is that rate hikes have been a lot faster than people expected. Inflation is hotter than people expected. The unemployment rate is lower than people expected.
What is it that people misunderstood a year ago about the economic machine such that the Fed has hiked rates much faster than people expected, and yet it's been surprisingly ineffective at cooling things down. And to this day, there seems to be a surprising amount of economic momentum with Fed funds at like five and a half percent?
Greg: (43:25)
Yeah, it's a great question. I have a bunch of thoughts on it. You know, certainly I can't speak for all people, but I can speak for myself, I've been wrong about a bunch of those things. So just to talk about what I certainly, and let's say we at Bridgewater, didn't nail.
Like you're saying, I thought the degree of, and certainly we had understood in our statistical models or whatever, we knew that we could easily be wrong, but that the degree of tightening was fast and high relative to history and that any tightening like this in the past had led to significant downturns, although the lead lag is somewhat variable and still possible. That's right. But I think a lot of things happened different than I expected.
Usually when, let's say, as they were last year, stocks were falling and short rates were rising, that formula in history always led to the personal savings rate rising, people seeing a higher interest rates available to them, asset prices falling, housing slowing down, etc. Usually people save more money, which meant there was less revenue for companies, which meant there were layoffs, which meant savings rates rose more when the employment market weakened. And you know, a recession was caused through that mechanism.
And what's happened in this period is that I think, and now I could be wrong, that normal impact of the higher interest rate and wealth effect impact was offset by the fact that wealth had been changed so radically in the 2020-21 period by fiscal policy. And that we had fiscal policy as extreme as the war, and the ripple, the length to which that disrupted, let's say, those other relationships was interesting. The degree of it was interesting.
I think there's ways we should have, you know looking back now, I think there are reasons that we should have, I should have known that. And some people were pointing to that. But that created much less of a reaction in household balance in household savings rates as you normally did. You came out of the recession with better balance sheets than ever. People were willing to dissave. So even as rates climbed – and actually debt growth collapsed as it normally would – but what simultaneously collapsed outside of debt is, or let's say increased, was the willingness to spend down the cash that that households had built up. And that cash doesn't just disappear when one person spends it. It goes on to others' balance sheets, whether it's corporate balance sheets, other household balance sheets.
And so that what's been happening, it appears, is that money has been spinning around in a way that made the rate hike have much less impact than I believe it would have had pre-Covid if you had anything like that rate hike. On top of that, within the US economy in particular, corporates had extended their duration so the impact is taking longer. The effect on corporates, although I think it's happening, but it is taking longer. And so, there are a few other things.
And then obviously the benefit of when nominal, what did happen is rate rise has created a decline in nominal demand, but that has mostly shown up in inflation. So nominal demand has fallen pretty much as much as I've expected. It has been more inflation falling than real growth falling. Which again, I think there are reasons that's the case. But before there was this massive demand shock from the what the Fed, what the central banks and the Treasury had done to get everybody's balance sheets up, and supply was struggling to keep up with this massive demand shock. And now demand's falling, but supply's still catching up to that old level. So on net, real growth has come out stronger.
Now I could see all that in the rear-view mirror. I didn't by any means predict that that would be the way it would play out. But I think that's why you've had this stubborn strength in the economy and that's created a certain amount of stability.
Now equities have rallied significantly since then. Some of the negative wealth effects have eased. At the same time though, a lot of that excess cash that was on balance sheets has been distributed. So there's a mix of pressures here that looking forward, you know, we do think inflation's still coming down a bit.
Although on net, we've entered what we think is a more inflationary environment such that 2% inflation probably more likely to be more of a bottom than a cap. And we do think fiscal policy as the way to deal with the recessions is probably politically the more likely outcome than, let's say, moving back in the next recession to more QE. Fiscal policy is a lot more inflationary and effective in a sense of stimulating growth quickly, as we've seen.
So I think you're going to see, you know, a world where we are still adjusting to a higher inflation, a world that is de-globalizing. Although everything we're talking about on the productivity front, maybe machine learning, changes that we'll see. But largely ex a major productivity miracle, I think de-globalization, the move towards fiscal policy has changed the inflation path in a way that markets haven't fully adjusted to. Because markets right now believe the Fed is totally credible, that inflation is going to return to target basically with very little problems.
When we measure the pressures, we don't think so. We think it's going to be much more challenging to get inflation where markets expect it. The impact on earnings is going to be a lot more negative than the markets are currently expecting and it's going to take longer and be harder. So big differences between what we're seeing and expecting, and what the markets are currently pricing.
Tracy: (49:00)
So I think last year you were talking about the possibility of a recession in 2023. Is that off the table now? So you're still positioned, it sounds like, for a level of higher inflation, but it sounds like maybe you're a bit more optimistic on the growth front?
Greg: (49:19)
Yeah, we've been wrong on growth. So I'd say, look, we think it's going to be a struggle. We're in a state of disequilibrium in the sense that relative to a given level of growth, we think the level of inflation to the Fed target, that they're going to have difficulty achieving growth and inflation at the level they want and are going to have to give on something.
In the short run, I think that's leading to, you know, higher rates. The expectation that the massive easing is coming is unlikely. The Fed is going to continue to have to be tighter longer than the markets expect. And so that's bad for, you know, let's say bonds and long-dated short rates. It's also probably bad for equities.
And at the same time we think growth will be struggling. It's nominal growth slowing. I think nominal growth is going to continue to slow and as nominal growth slows while you're more in stickier inflation, things like wage growth and some of the service areas, more sticky inflation, you get more of a challenge as nominal growth falls for it to just flow through to inflation.
So my view is you end up with growth disappointing a bit and inflation disappointing on the high side a bit. Ending up, you know, probably bad for bonds and probably, you know, a little bit bad for equities. And generally weak growth. And if that weak growth starts to translate into rising savings rate, you could easily end up into a recession and one that's going to be difficult to deal with, you know.
But yeah, I'd say we've tamed – I've tamed and we've tamed at Bridgewater – to some degree our view on growth. While still negative, not as extreme as it appeared. And it's a more gradual process that's unfolding.
And then on the inflation front, while we expected a quick decline inflation as nominal GDP fell, we do think we're in the range where you're in the much more stubborn part of inflation. It’s going to be harder to continue to get those inflation falls going forward.
Joe: (51:11)
So just to be clear though, you do think there is a gap between either what the market sees in terms of how much more work the Fed is going to have to do, or what the Fed thinks how much more work the Fed is going to have to do, and what basically you think the Fed is going to have to do if it actually is serious about getting inflation back to something resembling its target?
Greg: (51:32)
Yeah, I think so. I mean, I'd say the Fed seems a little bit more realistic than the markets on what it's going to take. But right. We think that's right. That when you look at what the markets are saying that it's super optimistic. It could come true. You do need, essentially, to get an equity rally from here, you have to have lower rates fairly quickly, into a world where earnings are pretty good. That's kind of the discounted line. To get above that you need even more than that.
And I think that line’s super optimistic relative to what we're, you know, what we measure. And again, I'm using the words, but I'm describing a process that's based on studying, you know, hundreds of years of economic history and how these linkages work and building all of that into a systematic process. But just spitting out kind of the output of that, is that it doesn't appear that the Fed will be able to achieve that and that we're in this disequilibrium where you still have more inflation relative to growth and you don't have an easy way to close that gap.
So we'll see. We've been wrong about that in terms of at least what the market outcomes have been for the last six months or so, after having been incredibly right for an extended period of time. And that's part of it. We get a lot of things wrong and that's normal. But I think when you break down why we got it wrong and the ways in which that, you know, we've learned from that and the ways in which our processes have taken in new information, still leads to this view that that the markets are overly optimistic about how easy that's going to be.
Tracy: (53:05)
All right. Well, Greg, we appreciate you coming on and outlining your thought process, both around the markets and AI and how you're actually deploying this new technology. So really appreciate it. Thank thanks for coming back on the show!
Greg: (53:17)
My pleasure. Good to talk to you.
Joe: (53:18)
Good luck in Vegas. Bring home another bracelet.
Greg: (53:23)
I'll try.
Joe: (53:25)
Thanks Greg. That was great.
Tracy: (53:39)
So Joe, I feel like I have a slightly better conception of exactly how this kind of technology can be used for investing. So the idea of maybe you have the AI models come up with theses or ideas that could then be rigorously fact-checked because all the AIs are hallucinating and things like that. That makes some sense.
Joe: (54:05)
Yes, absolutely. And I think, you know, you asked the question that's like, “Can AI do our jobs?” and I don't think the answer is yes. And I think it's like, can the AI replace the stock picker? It doesn't sound like the [answer] is yes. But can the AI…
Tracy: (54:20)
Augment…
Joe: (54:21)
… Augment the way someone is thinking, come up with theories that then can be rapidly tested, have that sort of go back and forth, and sort of do some of the work that currently sort of like junior analysts do in terms of testing ideas and stuff like that. You could see how it could be a force multiplier at a large fund.
Tracy: (54:42)
Yeah. But I mean, to that sort of turning point question, that also seems to be the big weakness here, is that, if you have an algorithm or a model that's been trained on years and years of prior data, so rates going lower and lower, and inflation staying below 2%. It seems very difficult to project what might change.
Joe: (55:06)
Which to Greg's point, humans aren't very good at that either, but you would hope, like that's what we want! To just be able to ask ChatGPT or whatever, you know. I'm using that as like a stand-in for this technology.
Tracy: (55:18)
Yeah. Or maybe you ask AI “what would you need to see in order to start taking the prospect of regime change seriously?” I don’t know.
Joe: (55:27)
Yeah. I mean he talked about this idea of the sort of adversarial way of thinking about it, which I think is like really important. And he pointed out the sort of like disaster of the home iBuyers…
Tracy: (55:39)
Yes, the Zillow analogy!
Joe: (55:39)
And then they got adversely selected because it's like, well, if Zillow is in the market, we know they're going to overpay. And so everyone suddenly dumps all the homes on Zillow. And it was not anticipating its own role in the market, in response to your question, which I think is like a really interesting dimension to all of this.
Tracy: (55:58)
Yeah. That sort of reflexivity between the models and the markets, I think we're probably going to be hearing a lot more about in the future. On that note, shall we leave it there?
Joe: (56:07)
Let's leave it there.