00:36:48 (someone): So pharmaceutical companies have patents, which means they have a monopoly on certain drugs over a period of time and they can make a lot of money. But it's often a challenge to identify people who need their drugs, particularly for those that are relatively rare. There's a push in that industry toward Blockbuster that, hey, if you hit 65, you're going to need this drug pretty much no matter what. We don't have to worry about the price. It's not quite world based, but really mass market. That's where a lot of the opportunity is. Now, imagine we have an AI that does better diagnosis. And in particular, it diagnoses a disease that it's not that rare, but it's hard to identify. And so you have a prediction machine that identifies it. You have a drug that helps cure it, or at least manage that disease in the presence of a good prediction machine. The pharma company can now sell a lot more, increasing their profit, and also help a lot more consumers. This isn't a pharma company exploitation. This is, hey, you know, because we have a better prediction technology, we can diagnose people at scale more efficiently for this previously seemingly rare disease, any production capacity on that kind of drug becomes much, much more profitable. There's a lot of this kind of opportunity, which is once you have better prediction, you can do things that you couldn't do before. And if you have the assets that are a compliment to prediction that could take advantage of it, there's huge money making opportunity.

00:14:33 (someone): So I think people are pretty comfortable with AIs recommending stuff to them. The next phase is what we call sort of AI enabled, which is completely changing a workflow in an industry to the point where the AI is essentially doing it for you with some supervision. So an example of this is the car insurance example I brought up before where the AI is actually making the repair replace decision. Now there is a human in the loop just making sure the decisions are right and sort of correcting one in every ten, one in every hundred, one in every thousand as the AI gets better. But it's making a decision that has real impact on the business and if it gets it wrong the whole system is sort of worthless. The fourth phase, which is really exciting and where I spend most of my time, is in this whole area where AI is able to do things that we just can't do as humans, make decisions we can't make. And so these are particularly decisions in complex systems like energy grids or healthcare systems or biological systems. And at this point in time, The AI technology is not necessarily good enough to rely on to run a power grid. And we don't really trust it for that reason. And so the adoption of those technologies is still probably a few years off. And so in a sense, we're spending a little bit less time there than the third phase. But in a sense, we're starting to spend more time there because we're looking three, five years out.

00:15:56 (someone): Maybe we could try to generically draw one of these companies, kind of what these companies tend to look like.

00:11:19 (someone): Now, of course, not all peer-reviewed or whatever else, but anyway, the industry is super competitive. I think we can talk about differentiation in the industry, perhaps, as another topic. But to answer the second part of your question, what would you look for as an LP, I mean, I think there are two things to consider. One is the obvious, which is, are they good money managers? And I say money managers rather than good pickers or whatever else, because I think that's a clear differentiator in the market today. There are a lot of people starting funds that haven't managed money before. And that is a set of experiences that you need. to make sure you don't lose money. It's somewhat obvious to someone coming from outside the VC industry, but it's not all particularly obvious to some that are in the industry today. And I think the second thing is, from an LP perspective, and I'm not an LP, so it's not necessarily my place to say this, but the world is becoming a lot more dynamic, and having a flexible model where you can co-invest, you can jump in and out of certain situations, special situations, you can do later stage rounds for funds that are early stage, I think given that fees aren't going anywhere, they're not really budging, I think that's the way around the fee drag in the industry. If you assume returns are going to be compressed, then you've got to get around it another way, which is fees.

00:12:29 (someone): Let's talk through the stages that you've talked about that AI will sort of be deployed in a general sense, starting with consumer.

00:02:11 Patrick O'Shaughnessy: My guest today is Alexander Wang, the CEO and founder of Scale AI. Alexander founded Scale in 2016, having been inspired to accelerate the development of AI through his work at Quora and his studies at MIT. Specifically, Alexander realized there was a lack of infrastructure solutions for producing high quality data, the lifeblood for AI models. Today, Scale provides data solutions to leading AI teams at Meta, Microsoft, OpenAI, Flexport, the US Air Force, and many others. This time last year, the business was valued at over $7 billion. Our conversation is a primer on AI. We discuss the building blocks beneath successful artificial intelligence, AI's role in both the public and private sector, and why data is the new code. We also cover the similarities and differences between AI and software from an investing perspective, and what inspiration scale takes from AWS. Please enjoy my great discussion with Alexander Wang. So Alex, we're going to talk about every dimension, I think, of AI, artificial intelligence, machine learning, all the things that it's going to impact. I'd like to structure our conversation from sort of the widest angle down to the most specific, which will probably be around your specific business and product. But given what's going on in the world today, I think it's probably an appropriate place to start with the role that AI, data, machine learning, et cetera, play in geopolitics and foreign policy.

00:09:18 (someone): Fundamentally, AI technology, modern deep learning, was initially developed primarily in the private sector through a lot of incredible research that was done at Google and DeepMind and OpenAI and Facebook and these incredible firms, as well as a lot of contemporary research done in China, the clear other major location where a lot of great artificial intelligence research has been done. And a lot of that research, there's been this very fast integration of that into very particular use cases in China. Facial recognition technology, I think, is one of the primary examples where there's companies like SenseTime, Face++, et cetera, that are primarily contractors of the Chinese government, have built world-class computer vision algorithms for facial recognition for use by the Chinese government in furthering a bunch of national objectives that they have. We may have questions from a humanitarian perspective, like the work being done with the Uyghurs. That all has happened in China. Then at the same time, I think this is starting to change now, but over the past, call it five years in the United States, there's been this very unclear relationship between the private sector and the public sector or the private sector and the government around the use of AI for applications like defense and intelligence. The Google Cloud Project Maven conflict was probably the most visible, most clear example of this. But as a general rule, I think that there's not the same level of partnership between the best in class technologists in the United States as well as the government. That is a significant cultural problem that needs to be solved in the United States for us to really be well positioned in the future. with respect to artificial intelligence. I think, broadly speaking, we're in the very early days with all of this technology. The speed of innovation in artificial intelligence is incredibly, incredibly fast.

00:00:00 Patrick O'Shaughnessy: This episode of Invest Like the Best is sponsored by Catalyst. Catalyst is the leading destination for public company data and analysis. Founded by a former buy-side analyst who encountered friction sourcing, building, and updating models, Catalyst is now used by over 400 institutions, including the largest money managers globally, and by a number of guests on the show. With detailed company-specific models and data on virtually every public company, Catalyst clients are able to ramp up faster, update models instantly, and incorporate the highest quality fundamental data into any workflow. If you're a professional equity investor and haven't talked to Catalyst recently, you should give them a shout. Learn more and try Catalyst for yourself at catalyst.com slash Patrick. That's C-A-N-A-L-Y-S-T.com slash Patrick. If you're curious to hear more about Catalyst, stay tuned at the end of the episode where I talk to Catalyst customer Brandon Weir from BWCP to discuss Catalyst in more detail. This episode is brought to you by Lemon.io. The team at Lemon.io has built a network of Eastern European developers ready to pair with fast-growing startups. We have faced challenges hiring engineering talent for various projects, and Lemon.io offered developers for one-off projects, developers for full start-to-finish product development, or developers that could be add-ons to an existing team.

00:44:23

Patrick O'Shaughnessy: As an investor, I'm really curious how you are, whether it's different or the same as you've always viewed this as an investor. But the first time we talked to you, I had some really interesting thoughts on just the things you look for in young companies. Even since Intercom started, the friction and expectation for starting companies has changed. There's so many of them. YC batches are so much bigger. If there's a whiff of an opportunity all of a sudden, You get 20 people leaving their jobs to start a company. It's amazing. It's great. The world benefits from this for sure. But I think it makes investing harder, especially given sometimes the prices are quite high. How have you evolved as an investor? What are the things you're looking for? Any philosophical changes over your investing career? And then I'll obviously talk specifically about the change that LLMs have on that too.

00:45:10

(someone): Early days, I used to get fooled a lot by just a great looking product. I mean, I feel this probably the wrong word because some of them actually worked out pretty well. But just I thought if you could build software, that that was enough almost.

00:01:29 (someone): Zeta invests in companies which build software that use artificial intelligence methods like machine learning to predict and prescribe outcomes. Ash's combined experience as a founder, entrepreneur, and investor give him the perfect background to discuss with us one of the hottest topics in business and investing. This conversation is useful for anyone trying to evolve their own way of dealing with data. Of particular interest are the ways in which Ash and his team evaluate data sets and how they think about competitive advantage in this new world, where Ash advocates a new term to replace the concept of moat, something he calls loops. If we can use data to do things better than humans, or if we can supercharge our intuition with predictive models, we can harness the power of this new technology. What Ash has taught me is that data itself is dumb, but great data sets can represent the fuel for incredible companies. Let's dive into how that may be. Please enjoy this conversation on how AI is changing business and how we might profit from that change. So Ash, you run one of the more focused and specific strategies of anybody that I've come across. A good place to start would be a quick description of the strategy itself, the types of businesses that you're looking for, and then we will dive into the very interesting topic that is AI.

00:02:41 (someone): Yeah, so essentially we're investing in what we think is a fundamental shift in computing, and therefore a shift in the technology industry, and therefore a shift in how you invest in the technology industry. And that is investing in things that don't just give you calculations quickly or put things in and out of a database quickly, but investing in things that give you predictions. And those predictions are super relevant to your business or create like real value for your business. So we invest in those sorts of companies.

00:48:43 (someone): What does that ultimate prediction look like? What are we trying to predict? And then you go back and think through what data do we have already that's going to help that and what data do we need? Now in second stage, what data do we need? Well, now how do you go about collecting data? Strategy number one is you can buy it. Sometimes it might be out there. But more commonly, you can create it. And you can often create it by launching products into the market that don't represent the system level change, but that represent either a point solution or an application solution on the way. In the auto industry, we think a lot about autonomous vehicles. The company that launches the first autonomous vehicle, and they can start collecting data at scale from that, is going to have this beneficial feedback loop. And they're going to collect more and more and more data. It's going to be hard for anybody to compete. But in order to actually launch that autonomous vehicle, you need to have enough data in the first place that you overcome the regulator and you're creating a safe and not dangerous car. And so far, that's been a meaningful barrier, hasn't it? So what's the strategy? The strategy is you embed sensors into all the cars you have on the road, even if you're not, as an automotive company, an autonomous vehicle. You don't have autonomous driving yet. And we've seen Tesla do this. They have cars with all sorts of sensors that are driven by humans.

00:00:54 Patrick O'Shaughnessy: The team at Lemon.io has built a network of Eastern European developers ready to pair with fast-growing startups. We have faced challenges hiring engineering talent for various projects, and Lemon.io offered developers for one-off projects, developers for full start-to-finish product development, or developers that could be add-ons to an existing team. Check out Lemon.io to learn more. Hello and welcome everyone. I'm Patrick O'Shaughnessy, and this is Invest Like The Best. This show is an open-ended exploration of markets, ideas, stories, and strategies that will help you better invest both your time and your money. Invest Like The Best is part of the Colossus family of podcasts, and you can access all our podcasts, including edited transcripts, show notes, and other resources to keep learning at joincolossus.com.

00:01:46 (someone): Patrick O'Shaughnessy is the CEO of O'Shaughnessy Asset Management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of O'Shaughnessy Asset Management. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of O'Shaughnessy Asset Management may maintain positions in the securities discussed in this podcast.

00:02:11 Patrick O'Shaughnessy: My guest today is Alexander Wang, the CEO and founder of Scale AI. Alexander founded Scale in 2016, having been inspired to accelerate the development of AI through his work at Quora and his studies at MIT.

00:00:00 (someone): This podcast is sponsored by CFA Institute, the global association of investment professionals whose mission is to lead the investment profession by promoting the highest standards of ethics, education, and professional excellence for the ultimate benefit of society. CFA Institute serves a global community of investment professionals working to build an investment industry where investors' interests come first, financial markets function at their best, and economies grow. The Chartered Financial Analyst credential is the most respected and recognized investment management designation in the world. The views expressed in this podcast do not necessarily represent the views of CFA Institute. Hello and welcome everyone. I'm Patrick O'Shaughnessy and this is Invest Like The Best. This show is an open-ended exploration of markets, ideas, methods, stories, and of strategies that will help you better invest both your time and your money. You can learn more and stay up to date at InvestorFieldGuide.com.

00:00:59 (someone): Patrick O'Shaughnessy is the CEO of O'Shaughnessy Asset Management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of O'Shaughnessy Asset Management. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of O'Shaughnessy Asset Management may maintain positions in the securities discussed in this podcast.

00:01:24 (someone): My guest this week is Ash Fontana, a managing partner at venture capital firm Zeta. Zeta invests in companies which build software that use artificial intelligence methods like machine learning to predict and prescribe outcomes. Ash's combined experience as a founder, entrepreneur, and investor give him the perfect background to discuss with us one of the hottest topics in business and investing.

00:36:10

(someone): Now i sit down in front of twelve suggested articles that are fully spec'd out. Now i'm starting to think this could be replacing a workflow. Everything else is just a little bit of nicety along the way i think.

00:36:21

Patrick O'Shaughnessy: As an investor how much have you seen that is exciting in this spirit.

00:36:26

(someone): I would have to say I see much more of the, what I would call like the easiest API call type AI. That is honestly, right now, the majority of what I see is not amazing. It's very much just, we also know how to summarize a piece of text or change the tone of a piece of text or whatever. And that could be like, we're a sales tool, but we can guess the opening introduction in your paragraph or whatever. And I think those things are basically dead in the water. I think stuff like say what Rewind AI are doing, I think is far more powerful. Drinking in a massive amount of data and then giving me a natural query engine on top of it. I think that's like really exciting. I think some of the visual stuff I've seen, be it like mid-journey or be it like, there's one I've invested in called Kettle, where like, again, you just describe the visual of what you want and they produce a vector. And what's really cool about a vector is you can actually tweak it yourself then if it's not exactly what you want it. I've seen really good examples of that type of thing.

00:38:57 (someone): And quality means AI is fundamentally used to make predictions, and this is the variance from zero error. So going from 96% accurate to 98% accurate is a doubling of quality. That creates all these powerful feedback loops for large tech companies, where if data quantity drives AI quality, and you have a dominant market share position, you have this feedback loop where you have the most users, they're generating the most data, that's training your algorithms, and so your algorithms are improving at a faster rate than anyone else's can improve. So I think when you put those things together, it's why you've seen less mean reversion. So if you had an algorithmic breakthrough that changed that, whether it's in GANs or CNNs or whatever it is, generative adversarial neural networks and convolutional neural networks, a lot of research is going into solving this problem. That would probably introduce more mean reversion within technology. And then the second thing is, why did value work in addition to those fundamental principles? There was this human element where you were perceived as taking career risk if you owned a really unpopular stock. And algorithms don't feel career-esque. And it's pretty easy to put into an algorithm, hey, this is the bottom decile. Within Quant specifically, I think a lot of AI has been applied to it with great success, but by probably a few players. Renaissance, I would bet what is at the core of their success is proprietary data. They went around the world and bought up proprietary data sets with 50 to 100 year licenses. And they intersected those with the stock market. And they're running all sorts of incredible pattern recognition.

00:05:47

(someone): Then we went to general availability, I think in late May or early June. And that's been a wild ride. Just seeing how the bot almost always is outperforming what our expectations of it are in terms of like the complexity, the nuance. Customers will send us seven questions, not knowing they're talking to a bot. Seven questions would nested. If this, then how do I, that. Fin just blitzes through it. Genuine conversations that might've taken a support rep an hour to aggregate all the information for it. Fin is just blitzing it. And there are some shocking stats. We've seen customers see 50% of their support volume drop. We've seen on average, most customers who turn Fin on would know how to work, literally clicking an on button. are getting 15, 20, 25 percent of their support volume just going away straight away. And it's just been crazy to see a product where we knew we had done all the right stuff on our side, but you're still crossing your fingers going, I hope it works well in the aviation industry. Sure enough, it does. So it's been wild to see it up front. And we've continued and obviously we can talk more about it. But yeah, we're now asking ourselves, what's the next tier of this? How do we make it more powerful? How do we make it interoperate with humans better, et cetera?

00:00:54 Patrick O'Shaughnessy: The team at Lemon.io has built a network of Eastern European developers ready to pair with fast-growing startups. We have faced challenges hiring engineering talent for various projects, and Lemon.io offered developers for one-off projects, developers for full start-to-finish product development, or developers that could be add-ons to an existing team. Check out Lemon.io to learn more. Hello and welcome everyone. I'm Patrick O'Shaughnessy, and this is Invest Like The Best. This show is an open-ended exploration of markets, ideas, stories, and strategies that will help you better invest both your time and your money. Invest Like The Best is part of the Colossus family of podcasts, and you can access all our podcasts, including edited transcripts, show notes, and other resources to keep learning at joincolossus.com.

00:01:46 (someone): Patrick O'Shaughnessy is the CEO of O'Shaughnessy Asset Management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of O'Shaughnessy Asset Management. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of O'Shaughnessy Asset Management may maintain positions in the securities discussed in this podcast.

00:02:11 Patrick O'Shaughnessy: My guest today is Alexander Wang, the CEO and founder of Scale AI. Alexander founded Scale in 2016, having been inspired to accelerate the development of AI through his work at Quora and his studies at MIT.

00:25:10 (someone): Your code is what will differentiate your product versus your competitor's product, or your processes versus your competitor's processes, et cetera. But then as more and more of the software in the world is written, infused with AI, using AI, or over time the interfaces shift to AI interfaces, an Alexa-like interface, for example, as that shift happens, as you go from 99.99 to 90.10 or 80.20 or even 50.50 over time, the vector of differentiation totally shifts to data and the datasets that you have access to. And so what that means is that your strategic differentiator, to your point, as a firm, is going to be primarily based off of, what are my existing data assets? And then what is the engine by which I'm constantly producing new, insightful differential data to power these core algorithms that are effectively powering my business? And these algorithms at the core that will power the future of business, I think are relatively core. I think there's definitely algorithms around automating business processes that are going to result in significantly more profitable firms over time. There's going to be algorithms that are based around customer recommendations and customer lifecycle, which is a lot of the algorithms that we've seen to date. Imagine TikTok recommendation algorithm, but for every economic interaction or every economic transaction in your life, that is constantly identifying the perfect next thing that you may want to transact with. And that is going to exist across every firm or every industry is basically going to be have to build their version of that. And that's going to result in significantly more efficient trade.

00:00:00 (someone): This podcast is sponsored by CFA Institute, the global association of investment professionals whose mission is to lead the investment profession by promoting the highest standards of ethics, education, and professional excellence for the ultimate benefit of society. CFA Institute serves a global community of investment professionals working to build an investment industry where investors' interests come first, financial markets function at their best, and economies grow. The Chartered Financial Analyst credential is the most respected and recognized investment management designation in the world. The views expressed in this podcast do not necessarily represent the views of CFA Institute. Hello and welcome everyone. I'm Patrick O'Shaughnessy and this is Invest Like The Best. This show is an open-ended exploration of markets, ideas, methods, stories, and of strategies that will help you better invest both your time and your money. You can learn more and stay up to date at InvestorFieldGuide.com.

00:00:59 (someone): Patrick O'Shaughnessy is a principal and portfolio manager at O'Shaughnessy Asset Management. All opinions expressed by Patrick and podcast guests are solely their own opinions and do not reflect the opinion of O'Shaughnessy Asset Management. This podcast is for informational purposes only and should not be relied upon as a basis for investment decisions. Clients of O'Shaughnessy Asset Management may maintain positions in the securities discussed in this podcast.

00:01:21 (someone): My guest this week is Jeremiah Loewen. Jeremiah is a childhood friend of mine who has been a sounding board for me throughout my career. We have conversations like the one you're about to hear about once a month, and in all those conversations, just like this one, you can hear me just trying to keep up.

00:35:46 Patrick O'Shaughnessy: Is it going to be the incumbents? Is it going to be the disruptors? Is it going to be some combination? Where are all the great businesses? Where are all the bad businesses? Some of these businesses that have taken off probably are not going to be good in the long run because they're so easy to copy, et cetera. I just love these dynamics of power and disruption.

00:36:02 (someone): In any gold rush, for lack of a better metaphor, people benefit by providing the equipment. There's going to be huge opportunities for companies that are providing the compute power underlying AI. Right now that looks like it's the traditional big tech companies and they're providing the underlying compute power and as the demands for compute power grow, there's going to be opportunities.

00:36:25 Patrick O'Shaughnessy: Basically everyone's buying cloud that's all running NVIDIA GPUs.

00:36:29 (someone): Exactly. If you make chips or if you provide cloud services, good for you. There's going to be a handful of opportunities there for sure. Second point is maybe more interestingly, there's a whole bunch of business opportunities that are constrained by bad bridge. My favorite example is a thing about pharmaceuticals. So pharmaceutical companies have patents, which means they have a monopoly on certain drugs over a period of time and they can make a lot of money. But it's often a challenge to identify people who need their drugs, particularly for those that are relatively rare.

00:51:45 (someone): Yeah, it's in the realm of, I guess I call societal systems, and it's in the realm of these AI-enabled systems that they're making decisions that, again, we just can't make because it's just so complex. And so I work with this company called Invenia, and they are optimizing the five biggest power grids in the US. And so what they're doing is They're improving our 24-hour-ahead, our day-ahead prediction of demand and supply. And if you think about it, if we get that wrong on the downside, if we don't supply enough power a day ahead, there are blackouts. There are blackouts everywhere. But if we supply too much, then that power has to go somewhere. The electrons are being created, they've got to move somewhere, and we sink it into the ocean. And so it's amazing how much impact they're having by building these incredibly complex ensemble of machine learning systems to predict the weather and how power physically moves on this part of the grid versus that part of the grid. And if you add a battery on this part of the grid, can you store it for a bit longer? How does that affect the delay and provision of power? So they build all these systems and make all these predictions, and they save so much CO2. So I'm really excited about systems like that in energy, in health care, and in smart cities, routing people and traffic and emergency vehicles and just resources and supplies better. throughout society.

00:53:05 (someone): You've obviously mentioned the key variables, several interesting lists of key variables for evaluating things.

00:00:00 Patrick O'Shaughnessy: This episode of Invest Like the Best is sponsored by Catalyst. Catalyst is the leading destination for public company data and analysis. Founded by a former buy-side analyst who encountered friction sourcing, building, and updating models, Catalyst is now used by over 400 institutions, including the largest money managers globally, and by a number of guests on the show. With detailed company-specific models and data on virtually every public company, Catalyst clients are able to ramp up faster, update models instantly, and incorporate the highest quality fundamental data into any workflow. If you're a professional equity investor and haven't talked to Catalyst recently, you should give them a shout. Learn more and try Catalyst for yourself at catalyst.com slash Patrick. That's C-A-N-A-L-Y-S-T.com slash Patrick. If you're curious to hear more about Catalyst, stay tuned at the end of the episode where I talk to Catalyst customer Brandon Weir from BWCP to discuss Catalyst in more detail. This episode is brought to you by Lemon.io. The team at Lemon.io has built a network of Eastern European developers ready to pair with fast-growing startups. We have faced challenges hiring engineering talent for various projects, and Lemon.io offered developers for one-off projects, developers for full start-to-finish product development, or developers that could be add-ons to an existing team.

00:58:52 (someone): And so if all we're delivering them is companies with this sort of technology, our base rate of success is just going to be higher. There's a secular trend there. So there's that. A really interesting conversation I had with someone yesterday was around, why do it? Why translate? Because as in, why let later stage investors come in and invest in your companies? And why put all this effort into helping them understand these companies so that they can invest some money into it? Because what you're really doing there is you're solving for a market inefficiency, which is that the market doesn't understand why this company is so valuable. But you're letting someone else profit from resolving that inefficiency. It's sort of silly. The reason we do it is because, one, we don't necessarily have funds that are big enough to fund a company ad infinitum. And we also would like some other investors to get involved in the company because they bring to bear some experience sometimes. And, you know, it's a good test for the entrepreneur to, like, see if they can present, articulate their company's vision and strategy and whatever else to the market and get feedback on that. So there are reasons why you pitch other investors for follow-on rounds, but it's sort of a funny thing. I don't think someone coming from another industry would, like, intuitively think that, oh, every one of our companies should be funded by a later stage investor. They would probably think, I have to keep funding that myself. And in a sense, that's what a lot of really successful investors have done.

00:33:28 (someone): And that is a really interesting area of research and a really important area of research, not just for regulatory reasons, but also for business decisioning, which is making models decomposable. So it gives you this prediction, but what were the steps to get to that prediction? And in a really deep neural net, 800,000 layers, whatever, that's pretty hard to decompose all those steps. But in simpler methods, more regression-based methods and basic optimization methods, you can actually break it down into the steps and go, oh, OK, there was a bias at this step in the credit decision or whatever else because it was fed this bit of data at this point. So regulation will probably follow technology. Step one will probably be just understanding it. And then really after that, the third part of this answer is, It really depends on what we want as a society. It becomes like this, I don't mean to introduce such a nebulous question, but it really depends on whether we want to understand complex systems or not. Do we really want to understand how the energy system works and weather and biological systems and Humanity and ecological systems, do we want to understand that? Because if we do, we're going to have to some degree, not a complete degree, let AIs run a bit loose in those domains. And that's going to be risky. But the payoff is huge, because what we do know is that we absolutely don't understand those systems today. We've got no hope. As human beings, we cannot throw that many variables into our head and compute them.