We're in the midst of an AI mania of sorts. In public markets, investors are placing bets on the companies perceived as being the winners of this new wave of computing. Companies that aren't even in "tech" are touting their AI bonafides. And of course, in private markets, every venture capitalist suddenly seems to be pivoting to AI in some way or another. But who will actually win? Will it be the big incumbents? Can those incumbents be disrupted? Will it be the companies who have access to unique datasets? Or will it be whoever has the most computing power? On this episode, we speak with Josh Wolfe, co-founder of Lux Capital, who has been investing in the space for several years, long before it was trendy. He talks about where he's placing his bets and how he's thinking about identifying winners. This transcript has been lightly-edited for clarity.
Key insights from the pod:
Why are people excited about AI now? — 4:08
Software vs. hardware — 8:41
Tie-ups with AI startups vs. acquisitions — 13:55
The role of data in AI — 18:14
Regulatory scrutiny and AI — 22:31
Due diligence in AI and competition — 28:35
The impact of AI on coding — 38:14
Lessons for VCs after the crash — 40:51
Why invest in AI now? — 46:04
---
Joe Weisenthal (00:10):
Hello and welcome to another episode of the Odd Lots podcast. I'm Joe Weisenthal.
Tracy Alloway (00:15):
And I'm Tracy Alloway.
Joe (00:17):
Tracy, I mean, I think it goes without saying that the appetite and interest in anything related to AI and making money from AI continues unabated.
Tracy (00:27):
Yes, I think that's accurate. Well, it's interesting because you kind of see it from two different sides at the moment. So there are a lot of companies that are talking about investing internally in AI technology and then there are a lot of investors talking about investing in AI in one way or another.
Joe (00:46):
Yes. I mean, needless to say, you like throw a dart at Nasdaq stocks and they're talking about the way AI incorporated. It's also funny because you read the earnings transcript of like a grocery store chain.
Tracy (00:59):
Yes!
Joe (00:59):
And the CTO will be talking about how AI is going to help them.
Tracy (01:04):
Wasn't this Kroger?
Joe (01:06):
I think so, and to be fair, I think they actually kind of legitimately have been investing, so it's not totally… but they mentioned AI, one company mentioned it like a dozen times.
Tracy (01:17):
I'm pretty sure it was Kroger.
Joe (01:18):
Yeah.
Tracy (01:19):
But I think I've said this on the podcast before, it does feel like there are some people out there who at this point are basically using AI as a synonym for any type of software. “We use software, we're using AI.” But it begs the question of how to smartly invest in a technology that clearly a lot of people are enthused about, but it's also kind of hard to disaggregate a lot of the marketing from the reality.
Joe (01:46):
The thing that gets me and that I'm still trying to wrap my head around too is, especially for the big tech incumbents and I'm thinking of like Alphabet or Google, they have an amazing business model right now, right? Like people search for something and then you're telling the machine exactly what you are looking for and then the machine knows, okay, well here's some ads that we can put.
And I know obviously Google is very front of the curve in terms of AI tech and they have their own large language models and all that stuff, but does anyone know that this is going to turn into a money making thing for them? Will it be anywhere close to this amazing money printing machine that they built when they built the Google search box?
Tracy (02:29):
Well, I think that's a really good question and also so far we have seen the incumbents come out as the big winners of a lot of this new technology. And I think A) that's been unexpected. If you'd asked someone five years ago who was going to be the big winner in AI, I don't think anyone would've said Microsoft.
Joe (02:49):
Right.
Tracy (02:50):
So that also raises the question of, okay, how do the giants monetize this? To your point.
Joe (02:55):
Yes.
Tracy (02:55):
And then secondly, are there going to be new players who somehow come in and find a better way to do it?
Joe (03:01):
Right, like how disruptive will it be?
Tracy (03:04):
Yep.
Joe (03:04):
Well, I am excited because I believe we do have the perfect guest. We are going to be speaking with Josh Wolfe, founding partner and managing director at Lux Capital, who has been investing in AI since long before it was cool, long before everyone started asking ChatGPT to, you know, write a song about the Fed and the style of Johnny Cash or whatever.
Tracy (03:29):
Hey, I feel seen.
Joe (03:30):
No, I'm describing myself. I'm describing both of us. Long before we were all doing that and so he is going to talk to us about how he thinks about making money in AI and where the value is going to accrue in identifying investments. So Josh, thank you so much for coming on Odd Lots
Josh Wolfe (03:47):
Joe, Tracy, great to be with you.
Joe (03:49):
Can I ask you a question for real? We obviously have this boom etc., but the tech's been around, did people basically just get excited because like someone finally put a good UX on top of this technology for a while and suddenly they're like “Oh my God.” Was that sort of what catalyzed this current stage of enthusiasm?
Josh (04:08):
I think the lay answer is what catalyzed this was a sense of conjuring magic. People felt like they were effectively casting spells, like you said, whether it was a Johnny Cash song conjured to talk about the Fed, but it's a feeling that somebody had a superpower.
So I think that's what catalyzed it, where it was a feeling of positive surprise where people were like “Oh my God, I just created magic” and that classic cliché that any sufficiently advanced tech is indistinguishable from magic, this felt like magic.
Now the roots of it go back over a decade and that's what the public doesn't see and part of that is the bones and part of it is the brains, the tech infrastructure. So you start with the GPUs. Now we've known computing for decades and Intel was the dominant force. Intel made CPU -- central processing units -- and there was this thing off on the side that was just doing graphic processing and it was for video games.
I've got this mental model, this framework where a lot of people say that the most dangerous words in investing are "this time is different." I actually think that there's a secret that people can follow, which is that the most valuable words are whenever you hear a parent say "it will rot your brain," that basically presages, it predicts the next $10 billion industry.
So think about this, I mean literally 1960s, those hip-shaking Johnny Cash, Elvis, rock and roll, it'll rot your brain. Boom. $10 billion industry, '70s, personal computers, chat rooms, '90s, the internet, these online chat rooms and then gaming. “My God, these kids are turning into couch potatoes, get them off the video games.” The video game players of yesterday are today's robotic surgeons and drone pilots.
But what's really important is the tech that was underlying that, these massively multiplayer games and people demanding ever higher video resolution and PlayStation competing with Xbox, competing with Nintendo. It created these chips that took Nvidia from a $15 billion market cap at the time when Intel was $150 billion a few years ago and today it's a trillion-dollar business.
We had invested in this company going back about eight years, it was four people off the Stanford campus, your classic garage, and they were literally in a garage at the SLAC, the Stanford linear accelerator, sort of secret group. And they were trying to develop self-driving cars and we had put about 25 million into this small team called Zoox, which was a zoo for robotics, it was a silly name. And we go in there and I see all these people playing video games and I start to get a little bit upset. I say, we just put a lot of money into this company, what are all these engineers doing? And the founder turned to me and said, no you don't understand the cars that you see outside on these tracks that are running around, they're ingesting information at the rate of one second per second, what we call reality. And they're taking lidar and radar and visual cues and thermal sensors and vibrational sensors and all of that and they're processing it.
But inside these rooms, air conditioned, these people are not playing video games. This is not Grand Theft or this is not Call of Duty. The machines are actually running simulations and they are training the cars, thousands of simulations a second and the machines don't know the difference between reality and simulation. And I was like “Oh my god, okay, this is pretty amazing.” What are these things running on? And they said, “Those two guys over there are from this company Nvidia, and we have chips that aren't on the market yet and they are able to do processing like never before.”
And that sent us on this path as investors in AI and from there we found this amazing team that was developing like the NextGen GPUs and, it was a guy, Naveen Rao, he had a company called Nervana Systems. Intel buys him within a year of us investing for about $350 million, becomes the core kernel of Intel's AI system, we go back to that guy again and just last week Databricks bought his new company called Mosaic for $1.3 billion.
Joe (08:08):
Congratulations.
Josh (08:09):
Thank you, wild ride but you just try to find these people that have this irreverent view and they sort of see the future and they invent it and you get behind them.
Tracy (08:18):
So maybe if I could just step back for a second, and this will maybe tell us a little bit more about what you're doing in the space, but how do you evaluate AI opportunities between the hardware -- so the chips where there so far seems to have been a lot of excitement and activity -- versus some of the software and the sort of underlying models?
Josh (08:41):
Today Nvidia really has a lead, it's very hard for people to compete. Obviously, there's all kinds of considerations of geopolitical dependency, and TSMC and ASML and who's helping to make these chips. But there's an entire very sophisticated stack of semi-cap equipment manufacturing, IP design so that people can make these chips, get in the chips themselves. And these things are very expensive, I mean, these H100 chips from Nvidia, $100,000, they are in scarce supply.
One of the other really interesting things right now in this chip domain that people should watch for, and then I'll tell you where I think Nvidia is actually quite vulnerable and they're not just pure monopoly here. Anytime that there's hype in a sector, just like you were talking about, you know, Kroger's adding AI to their name. You saw this in the dot coms, you saw it in the internet, you saw it in mobile.
Tracy (09:30):
Blockchain, Long Island Iced Tea .
Josh (09:32):
Exactly. And, you know, look, they lower the cost of capital, they take advantage of people's irrationality, they capitalize and what happens in every field, the hype gets high, the cost of capital gets low, hundreds if not thousands of new companies get funded, 99% of them fail.
And from that detritus, it becomes the combinatorial fodder for the next wave. So interestingly with crypto, crypto people were clamoring for these GPUs. They couldn't get enough of them so that they could do Bitcoin mining.
As that market went hyperbolic and then crashed, now you have all these excess GPUs and you're starting to read headlines about the Bitcoin miners that are selling their GPUs now to the AI researchers and they're able to do it now at cents on the dollar of what they paid before.
So on the hardware side, that's very interesting but our bet is that there's something different that's happening than betting on the next chip. Okay, there's Moore's Law, which everybody knows about. There's Rock's Law, which basically is that the cost of these semiconductor fabs increases exponentially even as our chips get cheaper and faster.
Joe (10:33):
Wait, sorry, which law?
Josh (10:36):
Rock's, named after Arthur Rock, who was one of the first VCs, one of the first funders of Intel. An East Coast, OG VC. So they called it Rock's Law because basically the cost to build these fabs to make the chips keeps getting more expensive every year and a half or two years. It used to be $100 million, then it's $1 billion, then it's $10 billion and so on.
Okay, why is that relevant? To make these foundation models that we're all using behind the scenes – GPT-3 and GPT-4 and what comes next? It used to be a few million dollars, maybe $10 million to train GPT-3. GPT-4 is estimated in the low hundreds of millions of dollars and whatever comes next, people believe, is going to cost about a billion dollars. Why? Because they have to buy all these Nvidia chips. So Nvidia is telling everybody, you have to get these A100 chips we make, these H100 chips we make.
The reality is actually that there's this interesting vulnerability. This is where we make our speculative bets. Now we might be wrong, but this is what we're betting, we're betting that that's not going to be the case. That it's not going to be just the domain of OpenAI, that it's not going to be Anthropic, that it's not going to be just the big giants. And we'll talk about how those guys are all intertwined, like you said, with the Microsofts and the Googles and the Metas, etc., because that's an interesting dynamic.
Nvidia has a language, a computer language that people program on and it's called Cuda, C-U-D-A and this has been the dominant form, but it's really vulnerable. And it's vulnerable, interestingly, because of Facebook, Meta, they came up with this language called PyTorch and a lot of the developers are moving to PyTorch, it's open source, it allows people to do a lot of the AI processing, but hardware agnostic, meaning you don't have to use an Nvidia chip. If you use an Nvidia chip, you have to program on Cuda.
These guys are saying we can use an AMD chip, we can use an ASIC, an application specific integrated circuit. They're saying we're not going to be beholden to this. So there's two competing software languages that are emerging sort of quietly. One is called PyTorch and one is called Triton, and Triton is from OpenAI. People probably trust that one a little bit less and PyTorch is totally open source but originated from Meta, which is really interesting.
Joe (13:04):
Man, I'm already learning a lot because I was not familiar with PyTorch. So obviously as you mentioned, and we talk about GPT-3, GPT-4, ChatGPT, this magical search box that got everyone's attention, what came out of OpenAI, but there are others that are building chatbots.
How do you think about winners either at the foundational model level, like if we started Odd Lots GPT, is that a worthwhile area? Or are you thinking like some of these problems are kind of solved and it makes more sense to focus on building something on top of one of these core winners, like GPT-4, and build some sort of app specific application for an industry that uses a foundational model that already exists?
Josh (13:55):
You're thinking exactly right and you should be a VC, because we're betting on the ladder that you're going to start to get these generalized models, which are wowing everybody. Although, you start to look at some of the usage pattern, classic thing, right? People get really excited about the thing, then it starts to die off. Maybe it's the summer, but maybe it's reached a little bit of a plateau of incremental interest.
Okay, so let's break this down in terms of the models. You've got OpenAI, which will continue to invest a huge amount of money, continue to develop models, continue to wow people, their next thing will be multimodal. Instead of just text or voice transcription, you'll start to have all kinds of interesting things where, like you said, make me the Johnny Cash song, give me a full music video, print out all kinds of crazy images.
You know, it'll do four different things and that'll be really limited by people's creativity, but it's going to be general. Now, one of the problems with the general stuff, GPT-4 was trained on the public internet and part of the problem with the public internet is that you got a lot of information, but you also have a lot of misinformation. It was trained on Reddit and Twitter and all kinds of repositories of public info and so it's going to hallucinate, it's going to give you BS answers.
So you have people saying, okay, that's a problem, there's whitespace, let me solve it, and it's probably going to be financial, and healthcare is our guess, where you get very specialized models that need to have high accuracy and they're going to be smaller models. So instead of these giant models, they're going to be smaller, more bespoke, more industry verticalized.
Even Bloomberg, Bloomberg GPT people are really fascinated by what that's going to portend because you have a proprietary data set, you've got a locked in user base and sort of a social network and you have reliable high-quality data. So I think that's going to be the next wave. It's going to happen in financial data. My bet is on Bloomberg, it's going to be in healthcare with some of the major healthcare systems. And I think that's sort of the next wave.
Now when you look at the big players today with the big foundation models, OpenAI, if you're really honest about it, they're captive to Microsoft. Microsoft did an incredibly clever deal. They knew that there's no way that the DOJ or the FTC would allow them to actually acquire OpenAI. So they structured a deal in a way that they effectively control it.
But without doing an acquisition, you look at Google, they're closely tied up with Anthropic and a lot of these deals are interesting because what happens is the company gets a giant equity investment. In this case, I think Anthropic got about $300 million from Google, but that money sort of round trips. Google gets equity, Anthropic gets cash, that cash then goes back to Google and is spent on compute. So they get to book it as revenue in Google Cloud.
Now Meta is really interesting because just like you said before, Tracy, nobody would've thought Microsoft was the leader or would be a leader in AI. Meta has been under Congressional scrutiny and has been the sort of evil villain of consumer and social media and disrupting and destroying our democracy and all this stuff. And then they made this bet on the Metaverse, which nobody cares about.
But this idea of this Fediverse that they're starting to talk about with Threads, it's really interesting because they are embracing this idea of open source. Now they're not doing it benevolently because they think it's a good thing, it's in their self-interest. They want to be the sort of network that is connected to everything else. They don't want to be siloed and they get to use it as a little bit of a sheen to stave off the regulatory scrutiny and the public criticism.
But I think you have to watch Meta really closely in the coming weeks, new releases of open source models that are going to really compete with OpenAI, lots of partnerships with interesting companies knowing that they themselves couldn't possibly do an acquisition.
Tracy (17:42):
You mentioned data just then, and this is something I've been thinking about, but when it comes to AI technology, what's the most important factor? Is it access to reliable data, as you mentioned, maybe reliable and exclusive sources of big data or is it the underlying modeling technology. And I guess another way of framing it is, are the big winners going to just be companies like banks, insurers that have huge data sets that they can do things with?
Josh (18:14):
I think so. I think that today it has been a little bit of ignorance arbitrage meaning the people that really were in the know were the model makers, the people that could design the algorithms to do the predictive analysis and make the models. All those models are either held proprietary in the case of like OpenAI or in the case of one of our companies, which has one of the most powerful repositories and one of the most ridiculous names, Hugging Face.
One of my partners, amazing guy, Brandon Reeves, he says, there's these French PhD computer scientists and mathematicians, they're hanging out in Brooklyn and they've got this company called Hugging Face. And here's the irony, they started out as a chatbot, you know almost like Joaquin Phoenix in Her, and then they became this open source repository for all the models and now hundreds of thousands of models including corporate models that are hosted there and constantly improving and it's all open source.
And the irony is that OpenAI started as this open model company has become the world's greatest chatbot. So it's sort of an inverse. So Hugging Face is making these models and they were the beneficiary very early on of everybody trying to deploy the models. They could run them on Hugging Face, they could use the cloud compute that they provide.
And now, Tracy, to your point, you're starting to see people saying, okay, the thing that we want to do is build on top of all of these models. What was expensive and scarce and rare before was the compute and the algorithms. And those are becoming increasingly abundant. So what is scarce today? Reliable data and proprietary data. And the data sets, like you said, could be big banks, could be consumer data, could be Amazon retail, spending information could be Spotify with user's behavior. It could be healthcare systems appropriately anonymized and protected and compliant with HIPAA.
But being able to collect all this information and have it do high quality inference and training. So you have to train the models on the data and then you have to be able to do the predictability, which is the inference from somebody putting in a prompt. I will say the area that we're probably the most excited about, which is not something that the everyday layperson is going to spend time doing, they're not going to be making those Johnny Cash songs or conjuring images on Midjourney and Dall-E, is biology.
And the key breakthrough here is there's something in all these models called the context window. And all it means is basically how much information you can put in. And if you ever tried to take like a long transcript, let's say of Odd Lots and throw it into a context window, it might say, “Oh, it's too long,” right? The context window for OpenAI has been about 8,000 tokens. Anthropic now has 100,000 and that's growing.
What that means is the amount of data that you can put into a single prompt is growing exponentially. If you think about the human genome, if you think about genetic data where you have millions of tokens that you need to be able to put into this effectively, that is the next domain where you're able to put in huge amounts of information and do all kinds of predictive things from designing new proteins to discovering drugs.
And that's an area where not only are the markets enormous, the information and the expertise is very narrow and specialized and I think it's going to completely upturn pharma and biotech in a giant way.
Joe (21:32):
Wow, that's really interesting. You mentioned, you're talking about some of these investments that the hyperscalers, the tech giants have made, and I hadn't really appreciated that dynamic before. It's sort of easier to link, it's easier for Microsoft to link up with an OpenAI than to make a big acquisition that's going to get on the headlines for regulators. It's easier for Alphabet and Anthropic, etc., and also the point about how a lot of that cash just comes back in terms of all these companies compute bill is extremely interesting.
As a VC can you talk a little bit about this dynamic? There seems to be a lot because of this, of corporate VC in AI specifically and how that sort of changes the game as a non-corporate VC or as an independent VC firm when you're thinking about evaluating companies, the presence of these big, the VC arms of the large corporations and how that sort of changes the game?
Josh (22:31):
Great question and I'll give you sort of three quick angles here. The first is how we think about ultimately making money and exiting as a VC and how these people play in the ecosystem. The second is a related question about how do we ultimately exit our companies? Meaning how do we sell them to a large incumbent when you have all this congressional scrutiny and it's very improbable today that Microsoft or Meta could do a big acquisition, it's just regulatorily improbable.
And then the third is a geopolitical angle here that I think is actually going to change that. So on the first one, I always say that it sounds a little bit cheesy, but we do a lot of hard science investing, a lot of deep tech investing and I like to say like the first law of thermodynamics -- energy is not created or destroyed -- risk and value are not created or destroyed, they just change form.
And every risk that I can identify in an early stage company, AI or Biotech or Aerospace, whatever it is, if I can kill that risk, if I can actually say, “Okay, there's financing risk or tech risk or management risk or product risk or customer risk,” whatever it is, kill that risk, a later investor coming after us should pay a higher price and demand a lower quantum of return because they're taking less risk and I should get rewarded for taking the early risk.
So I sort of think about it as destroying risk to create value. Why do I say that? Because if we take an early stage risk in a company to prove that the tech works, I want those corporate VCs coming in, I want them coming in, I want them paying a higher price than we did, providing a lower cost of equity than we did and helping to both validate and create some competition.
So, I'll give you an example, Runwayml, bunch of interesting scientists, one of them was an intern back in the day at Hugging Face, became a co-founder of this company, Runway. Runway basically said we can take the cutting edge models that we're developing, they actually were the developer of stable diffusion, and we're going to make videos. We're going to start with two second videos, you talk to the CEO there, Chris, he will say within the next two years you will have a full feature movie that is entirely generated by people sitting at a computer and just prompting angles, lighting, actors, expressions, it's a little bit hard to fathom. It's like looking at YouTube when it was 240 pixels versus like 8K today. But it's going to happen.
Joe (24:55):
A totally full feature Hollywood film, everything perfect except the hands.
Josh (25:02):
Exactly. Although I think they'll get the hands right and there'll even be some unique special effects, but the sound, the lighting, the angles, everything, I think we're two years from something that actually you'll be like “Oh my God, that was made by AI?” And it'll probably be a shorter film, but it's coming.
Okay, why do I say that? You just had $140 million financing announced a week or two ago; Google, Nvidia and Salesforce and those are three great companies. One is on the data side, one is on the sort of strategic side, one is on the hardware side that wants them to use their compute. All of those guys are now linked with this company. And so Google's competitors are looking at this and Salesforce doesn't want to be left behind and Nvidia is looking and AMD is looking.
And so, the more corporate strategic folks that you get in, the more competitive juices start to flow, and it increases the chance for the founders and for us, that not only do you get good strategic partners, but you set up competitive dynamic for future exits.
And so that's typically, you know, great companies get bought and they get bought because there's competitive fervor from a corpdev person at one of the big companies that says “We can't let our competitors get this.” Okay, so that goes to the second thing, which is against this regulatory backdrop, who's going to allow these big companies to actually buy these small companies? And I'm hopeful, okay, this is wishful thinking, this is more prescriptive than observant, I think that the regime today is very focused post-2016 and the election and the chaos and social media and all the abuses, particularly that you saw at Facebook with users and fake information and misinformation. I think that we are turning our targets of attention from Congress on the wrong targets.
I think that focusing on the domestic industry and trying to slow it down and prevent acquisition, prevent failures and prevent these companies from buying and competing is exactly what some of our peer adversaries overseas would love. China and particularly the CCP would love nothing more than for AI in the US to slow down and for all of these iterations and experiments to have problems and for there to be a disincentive for VCs to want to fund these things because they'll never get out.
And I actually think that you'll see some sea change coming in the next few quarters, year or two where people say, okay, wait a second, it isn't that we've met the enemy and he is us, we actually have to have domestic competitiveness and one of the great assets that the country has is competitive, great technology companies, and we need to let them thrive so that we can compete particularly with China's CCP.
Tracy (27:34):
Just going back to what you said about fully AI-generated movie. When I hear something like that, it sounds incredibly exciting, it also sounds very sci-fi and difficult to wrap my head around in various ways, but it kind of leads into a very basic question, which is what is it like to invest in AI right now? So how are you actually doing your due diligence?
If someone comes to you with an opportunity for investing in a new technology, is it like all of us sat here in the office playing around with Chat GPT, is that basically the thrust of due diligence on this technology or something else? And then secondly, how competitive is it right now, from a venture capital perspective to get in on some of these investments? Because I imagine given the level of excitement, there is a lot of money crowding into this space.
Josh (28:35):
So the latter I'll answer first, which is it's very competitive. I mean, anybody that can write a check is a competitor. Now, if you are a founder, just like if you are a star high school athlete or a star high school scholar, you want to go to the places that reflect the quality of your craft. And so you might want to go to Yale or Princeton or Stanford or you might want to go to Vanderbilt, Duke or Michigan and play ball.
And so I think it's the same thing where great founders want to work with great firms and Lux, Sequoia, Andreesen and a handful of others have brands that confer to a founder that we are highly selective, that we have a great network that we can be value add, but anybody can fund any of these companies.
There's always somebody that's got a roommate who's got a mother or father that gave them some money and they became early investors in this company, and they made a ton and so our view is that we are competing on one hand with everybody.
Now, the second thing is that I always say there's this five-year psychological bias, which is that you want to be invested today, where we were five years ago and so I'm trying to figure out what's the next thing three, four, five years ahead that people don't yet appreciate. So I mentioned biology, now all the listeners can go out and try to find the next models in biology, but it's a harder, more complex thing. And I'm confident that there's a fewer number of investors that actually understand or have the networks of the connection.
So we have some slight competitive advantage there when we're evaluating these things. You're looking at the credibility of the founders. Many of them happen to be in academic published papers. So you can see for example, the people behind Runway who published the papers that led to stable diffusion, or the team that came out of Google that published the paper on transformers, not the robots of course, like Optimus Prime, but the underlying algorithms that led to Chat GPT. Every one of the people on those papers have basically gone on to start companies and raise money.
People that were at OpenAI have the pedigree, they learned what works, they went and started Anthropic. And so there’s this sort of, just like if you go back in finance to the Drexel days where they spawned Apollo, Carlisle and Jefferies and all these, it's the same sort of thing. There's a diaspora that's coming out of a small group of people and you can reference the credibility and then yes, you sit with them and you look at what their demos are and we like to say that we believe before others understand.
And so when we had that Runway team in, or we had the Hugging Face team in, it was very raw and very crude and you have to sort of squint and see the future that they're seeing and then we don't fully fund companies. You give a little bit of money and you say how much money will accomplish what, in what period of time and who will care? Are we going to get paid for the risk that we're taking in funding you?
But I would say right now, if you're funding anything that is application-focused, anything that is to your earlier point in the wrappers around the user interface, most of those things are just features, they're not companies. Most of those are going to be competed away by a hundred other examples. A great example of that is, and I can't even remember the name of it now, but there was something that went out and it was an app and you could pay $20 and it would give you a hundred versions of yourself, you know, as a comic book hero and a cowboy and a black and white.
Joe (31:55):
I think Tracy and I paid for that. I think we did. I think we signed up for the one month version of that. I don't know, I probably forgot to cancel.
Josh (32:01):
And it spiked and then it's done. And you're going to have tons of those things where it spikes and it's done and it ends up being a feature integrated into -- going back to the earliest point you made, many of the big tech companies, the Adobes and the Microsoft, it's going to be in all of their suites.
I have one contrarian take here, which is a bit odd as an investor, as part of a partnership who's funding the deep tech roots of the semiconductors, the infrastructure, the networks, the models, the algorithms and despite doing all of that, I actually have a view that what we are doing right now, humans talking, even though it's through digital communications in an analog way, that is actually going to become the scarce thing. You are going to be flooded, I mean utterly inundated by emails, texts, tweets that are not written by humans.
And that's not like two years away, that's like two weeks. The increasing percentage of the communications that you receive, even by the way, from people that you know and love and trust are not going to be written by them. They're not going to be spoken by them. You know, voice is going to be the next domain where it's going to be very hard. You're going to be getting a voicemail and the voicemail was not actually spoken by your spouse, your cousin or kid.
I've already trained a model on my child and I was able to trick my wife on it, it was funny and scary. The point of this is if that happens, and as that happens, you will start to grow increasingly distrustful of many of the communications you get. It'll be this form of chat phishing is what I call it. And you'll start to just pine for in-person communications and there will be private clubs that form where people come and no devices and you just know that you're talking to a human because increasingly that will be scarce. So the great irony here is the flood of money, talent and productivity in AI and deep technology is probably going to bring us closer to our innate humanity.
Joe (33:58):
It sounds like the answer is Tracy and I need to do a lot more Odd Lots pub quizzes and other live events. That sounds good to me. Can I ask a question though? If all you have to do is sort of be a graduate from one of the right universities and have your name on some paper that's on like the archive website, what does that mean for recruitment from the companies that you are investing in and when they want to go out and hire someone, how is the challenge? Like no, that person who they probably want to hire can also raise $100 million. And how difficult is it to recruit if there's just so much money going to a potentially smart founder?
Josh (34:41):
You know, this has been the plague if you're an investor for arguably the past decade in tech broadly, and it's a weird phenomenon. And what I mean by that plague is everybody thought that they could raise money and they could, they all had a friend, a colleague, a former associate or roommate who raised money and they literally had this reaction: “He or she just raised that money at that valuation? What an idiot, I can go and do that.”
And so that set a comparable for them to say, I'm going to go do it. And you get this sort of collective craze and so talent gets diffuse and disperse, it's increasing the cost of hiring employees, valuations are going up, money is being misallocated, like the whole thing. All of that crashed about a year and a half ago once rates started rising and the SPAC boom ended and all that, except for this one domain of AI.
And so, if you actually look at the hiring data and you have to parse this to see whether it's signals that they're posting jobs or they're actually hiring, but all of the layoffs that you saw, the tens of thousands at Meta and Google and Microsoft, etc., you're starting to see this spike back up in some of the data and what are they doing? They're hiring, whether it's low level jobs for data entry and processing and cleaning or whether it's cutting edge algorithmic design for AI. There was an existential panic at Google and it's been reported, when OpenAI came out, the all hands deck meetings that people had of “My God, we have to throw a ton of talent and money at this so that we don't get left behind.” So right now it's a bit of an utter craze.
You have to be really careful. Most of the incumbents are the winners and there's going to be some small companies that end up with these really interesting novel approaches that are not raising a ton of money almost out of necessity. They're doing these cheap, very focused models and arguably, and here's another interesting thing, training on distributed computing. Instead of having these big centralized clusters of compute, we for example, backed a company called Together Compute. And his basic hypothesis was, can I train very sophisticated models on all of the excess compute from the crypto craze or from idle computers and can I do it for a fraction of the cost of what OpenAI did? And the answer was yes.
And so you're going to see lots of these small companies that come and I always say whenever the DOJ comes in and starts looking at monopoly concerns from the big companies, it's never the DOJ that disrupts the big company that people are concerned with. It's always some small competitor. It happened with Microsoft in the late nineties, Google came along, it wasn't the DOJ, it was Google. It happened with Facebook and Google, it's happened with OpenAI and now Facebook and it'll happen again and it'll be four guys or girls in a room in Croatia, Singapore, Mexico City or Silicon Valley that come up with a crazy new thing that disrupts the big incumbent.
Tracy (37:49):
What impact do you see of AI on how the tech industry/VC is organized or operating at the moment? Like do you see people start to respond to the idea that maybe a lot of coding is going to be done by AI in the future? Are people sort of like reorganizing themselves or reorienting themselves ahead of some of this technology?
Josh (38:14):
Definitely. When you look at the Copilot, which is both from people like GitHub and OpenAI and others, it's basically how can we either help you code or how can we completely, if you look at Code Interpreter, completely write the code for you?
You just described in general lay language what you want to do and it will in Python create the code. So pretty much every company will have a form of computer programmers in the software that they use, whether it's open source or proprietary, that is constantly developing and iterating their own applications. And it'll touch everything from customer service to radiology and x-ray image analysis to Bloomberg queries and people will be able to basically have superpowers.
What it means is you'll have the elite coders that are sort of always on the edge trying to figure out the next thing and then you'll have the average coders that are basically really leveled up and almost indistinguishable from the prior elite coders.
So I do think that what once was really scarce and really valuable, which was top notch, what people call 10X coders is now going to become increasingly commodity and people will start looking for example, now the real value is not can you code for the current moment? It's like, are you an amazing prompt engineer?
Like I can't draw, I can't code, I can't write 12 lines stanzas, but I can prompt pretty well. And so it shifts the capability to the creativity of somebody that really wants to describe and control the machine. Again, almost more like casting a spell than the person that actually has the discreet technical capability.
Joe (39:57):
Can I ask, you mentioned that roughly a year and a half ago, or maybe two years ago, the model that had been working, like this incredible trade, this incredible line go up decade or whatever for tech and VC, did sort of crumble to some extent and we saw the big plunge of the Nasdaq.
And I'm sure there were tons of funds raised in 2021 and 2020 that are like deep underwater and all that stuff and everyone knows that. When you're at the table now looking at companies, is there still pain and paranoia and fear from that? Has the pain of that been internalized in the way or is it, you know what? We're just back in it, the game's back on, FOMO motorcycle back on let's go. Like how much are there scars from the crash of 2021?
Josh (40:51):
I think people that put a lot of money to work in 2020, 2021, feel a lot of pain. They invested at record high multiples, they made the presumption, which was a fair presumption to make, that if you fund something, there's going to be later stage capital or a robust public market to follow you on. All of that is gone, so I think we went from what everybody called FOMO, fear of missing out, to what I call SOBS, which was the shame of being suckered.
Joe (41:20):
Can I just say there's no shame and, you know, people buy it, it's hard to know what the top is, but I think about the person who paid half a million dollars or millions of dollars, for the Bored Ape NFTs, that's the ultimate sob. You can never live that down. Okay, keep going. Sorry, I just thought about that recently and that was in my head.
Josh (41:39):
But, you know, I got a friend Zac Bissonette who wrote the book back in the day…
Tracy (41:44):
Yeah, we’ve had him on the show. Zac's amazing.
Joe (41:45):
Yeah, I like Zac.
Josh (41:46):
So, this is one of the things that I love to say, which is not some crazy personal insight. It's just an observable truth, which is that technologies change, businesses change, rules change and policies change. Human nature is a constant, greed and fear is a constant. That is what made Buffett and Munger brilliant. You know, it's what Howard Mark's chronicles all the time. It's what you guys cover, the excesses of human emotion.
And so a lot of this is actually capturing, where are people not paying attention? Where is attention scarce? Because where attention is scarce, valuations are going to be low. And we always say, they're contrarian investors, we want people to agree with us, just later. And that's the key.
Going back to, to your question, you know, we went from FOMO, fear of missing out, to SOBS, the shame of being suckered, and there was an important reason for that, which was the disappearance of two major players at least symbolically, and that was SoftBank and Tiger.
And why was that important? Because you had a venture firm maybe a decade ago that said the price you pay for a company doesn't really matter because there's only 10 companies that matter amongst all the ones that are funded. And if you would've funded LinkedIn or Facebook at $5 billion, $10 billion or $20 billion, it wouldn't have mattered, right?
And so that set a precedent that I think was a bit insidious and dangerous to say the price doesn't matter, you just have to be in the right companies. Of course, that's only obvious in hindsight. So you had lots of people that lost valuation discipline, it skewed control and leverage to the founders over from the investors, and you saw weak governance and you saw fraud and excess and all that kind of stuff. And it's starting to wash out, not fully, but the disappearance of those two players symbolically, they were the top ticking marginal price setting investors.
SoftBank was paying insane prices. And of course they were all kind of shenanigans of them marking up their own book and pricing up again and all that kind of stuff. And Tiger sort of took a passive indexation approach, which was something that was widespread in the public markets, but they did it in the private markets and they said, we're just going to be in all the companies and the winners will make up for the losers and it'll work.
When those kinds of players disappear, now all of a sudden you have a more rational scrutinizing market of people who are afraid of paying excess prices, feel like they need to get a better deal. You're seeing down rounds in companies, you have a morale spin and decline where employees now have underwater stock and need to be refreshed. And here's where things get really interesting.
We went from this domain where I called it the megas and the minnows. The megas were the giant funds that were $10 billion+ and they were writing these giant checks, and the minnows were the thousands of small sub-hundred million dollar funds that were just doing all the seed investing. Both of those guys have been squeezed out.
And so now you have a smaller base of capital, you can see it in the data. LPs have pulled back, the champagne has stopped flowing down the pyramid of glasses, GPs are struggling to raise capital. We closed a $1.2 billion fund in 10 weeks, which for us was amazing and is signal of great support of our LPs and great founders.
A lot of funds out there right now are downsizing, it's taking them a lot longer to raise and all of that is a rational reaction to a retraction. So I don't think you see the same FOMO, I think you see a lot more fear. People don't want to pay higher prices. The only area where there's a an exception is inside of AI.
Tracy (45:18):
I just have one more question, and you sort of touched on this earlier where you were talking about parallels between now and the sort of dot com era and the idea that maybe eventually, some big winners will emerge from this new technology, whether it's AI or the internet as it was in the late 1990s, early 2000s. What's the case for investing in AI right now, rather than waiting a little bit to see where the dust settles, maybe wait to see who those big winners are, or maybe at the very least get a little bit more clarity on how this whole thing is going to be structured or organized?
Josh (46:04):
The argument for waiting is by the time you know, it's already fully factored into a price. The contrary to that is you pay a high price for cheery consensus, as Buffet historically said. And so if everybody agrees that Nvidia is the winner, you know, that to me gives me pause for concern.
Jensen is running high, he's got the iconic leather black jacket. He's becoming the next prophet of tech. Those are all signals that are like the classic Sports Illustrated curse, simple reversion to the mean, like what happens, where's the vulnerability there to me is the question.
And I gave you guys and listeners a clue, which is that Cuda, their language system is vulnerable to these other ones of PyTorch from open-source originated from Meta and Triton from OpenAI, and that means that AMD could actually come from behind and start to take share.
It's something that people are skeptical about. So I would say that if you're thinking about investing now, it's too late. It really is. Again, five-year psychological bias, you want to be invested, five years ago where everybody wants to be today and vice versa.
So I'd be thinking about what are the improbable things that are likely to happen in the next wave? I'll give you one company that I think is interesting, Lux is not invested in, it's a public company and we do private, but Cloudflare. If you go back to the internet, early days, one of the winners in the infrastructure was Akamai, the people that were sort of caching and they were helping to shape the structure of the internet. Cloudflare is very interesting because they have a lot of compute infrastructure at the edge of the network, and you hear about this in sort of a hype way.
Sometimes the edge, edge inference, edge compute, it's a real thing. Very simply, you're talking on a mobile device or you're on your computer, right now you have to go up to the cloud in the cloud, which is basically a bunch of servers somewhere with high bandwidth interconnectivity processes. Then you have another domain which is on device, so you do something, the models get smaller, the chips get better on your Apple device or your Android or your iPad.
You're able to run the AI model there. Cloudflare is cashing a lot of these models and hosting them very close to the users, and they're doing it in thousands or tens of thousands of places all over the world. So I think that they, probably a $20 billion market cap company, $1 billion revenue, 50-60%, growth. I think that they might be poised and aren't one of the names that are on the tips of people's tongues that are benefiting, but we see them in all the infrastructure behind a lot of our companies.
Joe (48:37):
Interesting. A little investment tip for people listening.
Josh (48:42):
Yeah, it's just do your work, investigate it, but it's something that is just not on the front page. And I think that they're poised in the same way that if I go back 10 years when I'm in that room in our startup, and I got the benefit of this legal inside information of seeing these guys from Nvidia making these chips that were the soul of the new machine in the proverbial Tracy Kidder sense, I just see that this infrastructure from folks like Cloudflare is probably going to win.
Joe (49:06):
So I just have one more question as well, and it actually also is sort of on the public market side, but you know, going back to a company like Alphabet and I sort of talked about this in my introduction and obviously they've made a lot of AI investments and they've been doing research for a long time.
Nonetheless, though, like the core business for now and for probably at least the medium term is going to be what we call Google.com or something like that and enter a search and get served a really compelling ad because it's very good at that.
In your view, how confident should people be that some of these big companies can actually produce revenue and income? I mean inference is a lot costlier, I presume, than a typical search query. We don't know what the advertising is going to look around it, etc., we don't really know, do you think it's obvious that these big companies are going to find ways to actually sell something profitably from this tech?
Josh (50:08):
I do. If there's good, strong leadership, and that sounds like a weasel answer, but historically if you look at Satya and Microsoft and you look at Google, I just feel like Google was run by the inmates for a very long time, and this competitive, near existential threat from OpenAI has given a sense of urgency for them to refocus and say, “Okay, we have to stop with all of the social stuff that is happening internally and we have to really focus on what our roots were.”
Google hasn't really had a killer product, I mean, a true new product in over 10 years. But what's interesting is YouTube is a big winning, I mean, that was a great acquisition. It's a thriving product, it's generating a lot of money. Hopefully they don't go crazy and spend like everybody else in the streaming wars.
But just like Facebook, right? Facebook.com is dead, right? What makes money for Facebook is everything else; the Instagram and WhatsApp, and if Threads takes off, you know, who knows? And they're able to capture some modicum of the enterprise value that has been destroyed by Elon with Twitter. So it's all of these ancillary product categories inside of the mothership that I think that people are cranking and figuring out how do we make this work?
Google's prominence and search, I think is going to persist. I think it'll extend into other domains. I think it's less likely to be threatened by a lot of the AI stuff, they'll integrate it, Bard when it first launched sucked. Now it's not bad, incremental search results are pretty good. But the corpus of information that they have from my photos to my emails, to my calendar, I'm pretty locked in.
And I'm relatively trusting of Google. I'm also relatively, if not high trusting, of Apple. And I've historically been very low trusting of Meta. I always say that whenever Meta launches a product, the one feature that it lacks is trust. And I think they're realizing even if it's a little bit of a showcase facade for both the regulators and the critics, that they really have to double down on trust. And one of the ways to do that is a lot of open source stuff. So really watch for Meta to embrace open source in a giant way.
Joe (52:13):
Josh Wolfe, Lux Capital, that was a great conversation, great overview of the market right now. Thank you so much for coming on Odd Lots. Got to have you back again.
Josh (52:23):
Joe, Tracy, great to be with you.
Joe (52:37):
Tracy, can I just say, I don't know, listeners might know I'm an amateur songwriter and my only goal is to get something published before the computers are just like so good at it. Maybe I have a window of like a year or two. I just want to have like one public, one something, someone singing one of my songs, and then the computers can do their thing.
Tracy (53:02):
I mean, I do think this is kind of the most disturbing aspect of this whole AI discussion, which is that so far it seems to mostly apply to the fun stuff; songwriting, poetry, making movies, and we're still sort of doing all the dredge work ourselves.
But that was a really interesting conversation. So, I don’t know, I take Josh's point about getting in early on some of this, but I'm looking at a chart of Google since the IPO and if you got in, in 2007-2008, I think you'd still be okay. You would've missed maybe the life-changing money but you'd still be up significantly on your investment. So I do wonder, obviously there's a lot of excitement around the prospects of AI and what it means for various companies, but I also feel like if you waited for the dust to settle a little bit, you wouldn't necessarily be automatically losing out.
Joe (54:08):
I like how your question and your point here is basically questioning the entire premise of venture capital.
Tracy (54:16):
Of venture capital?
Joe (54:18):
I like how that is actually the entire subtext of the question.
Tracy (54:21):
Hey, after 2022 I think that's a valid question.
Joe (54:25):
Why not just wait for it all to go public?
Tracy (54:27):
Yeah, it's fine. It’s fine.
Joe (54:28):
The other thing which I hadn't appreciated, which I thought was really interesting, obviously I know that Microsoft OpenAI, Google or Alphabet Anthropic, but the way in which some of this may be a function of the regulatory environment. I had not really like appreciated that and why like, okay, it’s going to be hard to make big acquisitions. So you just invest in companies who spend most of their money with you.
Tracy (54:55):
Yeah.
New Speaker (54:56):
I hadn't appreciated that element.
Tracy (54:58):
I think that was a really interesting angle and actually explains a lot of the choices and decisions that are being made at the moment. Because sometimes you look at them and you're like this is interesting, but I'm not sure I completely see what's happening here. But if you look at it from a regulatory/reputational angle, it makes a lot of sense.
Joe (55:18):
You know what I'm really excited about?
Tracy (55:20):
What?
Joe (55:20):
Bloomberg GPT, Josh said it's going to be one of the winners.
Tracy (55:25):
I feel like we should be doing a disclaimer here.
Joe (55:29):
We work for Bloomberg.
Tracy (55:29):
It's fairly obvious. We work for Bloomberg.
Joe (55:30):
The disclaimer is we work for Bloomberg, but we did not tell Josh to say that Bloomberg GPT. But this point about who has actually high quality data is interesting.
Tracy (55:39):
Yeah and I would say so far a lot of the excitement is around the chip makers. Some of the incumbents, like Microsoft, I haven't seen people get really excited about like insurance companies as an AI play yet but I think there's something there.
The other thing I wanted to say, and you asked this question about the user interface and I actually think it's really important in the story here, and this is where I would draw a parallel with blockchain and crypto, which is the interesting thing about crypto was that you could participate in this as a sort of normal person. You know, you could open a wallet of some sort and buy whatever your preferred cryptocurrency is so you could participate in it. And I think having something like OpenAI and various other models that you can play around with, clearly has drawn in that additional interest.
Joe (56:35):
Oh yeah.
Tracy (56:35):
That is a big part of it.
Joe (56:36):
Absolutely. I do think we've all had that jaw dropping moment, which like, you didn't really get with crypto. It's like, yeah, you could do it. But then it's like, okay, now I have this coin in my wallet.
Tracy (56:46):
Right. Well, that's true.
New Speaker (56:48):
But then literally it takes you 10 seconds to like be blown away. It is just so powerful.
Tracy (56:54):
Yeah, shall we leave it there?
Joe (56:55):
Let's leave it there.
You can follow Josh Wolfe on Twitter at
@wolfejosh
.