We could be looking at the beginning of the end of the internet as we know it, where AI systems, its software is the primary users, consumers, producers on the internet. >> I call it an economy of agents. In an economy of agents, decisions are being made primarily by AI. >> What happens to a market [music] economy when there are no longer people as the primary operators? Do we get to that perfectly competitive market or are we looking at something that gets kind of chaotic? They're going to do things that we never imagined they were going to do. >> If the agent makes a mistake or colludes, who becomes liable in that world? We really don't have a regulatory [music] infrastructure in place for all these AI agents to go out there and now start participating in the economy. The timeline Is it possible we could have an economy of AI agents 2026, 2027? That's actually my version of existential risk. If we release that really quickly without any infrastructure in place, >> [music] >> we just crash the economy. What is an economy operated by billions of AI agents going to become? And how should we possibly think about governing and regulating that world? Professor Gillian Hadfield is at the forefront of this research. She holds a JD and PhD in economics from Stanford University. She's a distinguished professor of AI alignment and governance at John Hopkins. She's a professor of law at the University of Toronto and currently serves as an advisor to leading organizations such as Google. I'm Shanna Bovell and this is [music] I've got questions. I want to start by grounding us in just how different the world we're walking into is going to look. So, right now when we think about our digital lives or our financial lives, I'm the one shopping for a pair of jeans online or I'm the one applying for a credit card and I compare prices, I compare offers. If I'm on eBay, maybe I go go and forth and try to negotiate with the person doing the sale, and even if I'm using an AI chatbot, which I use a lot now to discover new products or places, I'm the shopper. I'm the negotiator and the transactor at the end of the day. Your work argues that all of that is about to change, and we're heading into what you call an economy of AI agents. Can you paint a picture of what that means and what that would look like? But before I start, let me just make clear that this is where we could be headed. And I think there's two things that affect whether or not we actually get there, and I just want everybody to have that in mind. One is, do we decide we want to go there? Maybe we can't choose a different path, but I think there are still decisions to be made. And the second is, I think we're still wondering whether or not some of the capabilities that are necessary to get there are, you know, on the near horizon. But let's think about the world that the developers are building, the investors believe they're putting billions of dollars into. What would that look like? That agentic economy. So, if you have an agent, you basically tell this piece of software in your that's in your computer or on the web that you're connected to, "I'm looking for this kind of shoe. Here's a picture. Go get it for me." And then in the background, you're not paying any attention anymore. In the background, the agent software is out there searching, looking for what's available, is comparing that to maybe uh information that the agent has about what you like, what colors you like, what styles you don't like, what brands you like, and then is actually can be making that purchase directly. And if it has to, you know, if it if it's a context where you negotiate a price, it's negotiating that price. It's maybe arranging delivery. Maybe it's even signing agreements. I mean, you wouldn't do this with a pair of shoes, but if you're buying a different kind of a piece of equipment, for example, where you had to sign an agreement about, you know, your legal rights with respect to how that product worked, it's doing all of that. So, with an agent, you're sort of saying, "Here's what I'm looking for. Go do it for me." It's like hiring a personal assistant and giving them the information about what you're looking for, and they go out, find it, research it, buy it for you, and bring it bring it back. So, technically, what you're describing, if >> all goes well in the vision of what investors and AI companies are trying to build, we could be looking at the beginning of the end of the internet as we know it. So, if we think about our internet as infrastructure built for humans by humans, we could be moving towards a world where AI systems, it's software is the primary users, consumers, producers on the internet. That's the vision. I call it, you know, an economy of of agents, and it's the idea that the things that we associate with what we do as humans, right? We own firms, and we make decisions about what to produce and how to price it, and how to ship it out, and we make decisions as consumers or firms buying supplies, buying inputs. Uh we're we're looking for a pair of shoes. We figure out where it is we go. We may be doing that online, a lot of that now, but nonetheless, we're the ones who are making all those choices. And in an economy of agents, if that's where we end up, those decisions are being made primarily by AI, by software. So, making decisions about what to produce, making decisions about how to price, making decisions about how to ship it out, making decisions about what websites, if they're still websites to visit, what products to buy. Now, obviously, that doesn't mean that humans couldn't be coming in and saying, "Well, wait a second, I want a review before you purchase something, or I want a review before you decide to produce something, but the more we move towards this vision of uh AI running firms, for example, which is part of the vision in OpenAI's stages of AI, the idea that you could have AI that's running an entire firm, or the idea that you've got an AI that's managing all the purchasing for a for a company, we go further and further from the point at which a human is said, "Here's what I want you to do." So, yeah, that's the economy of agents. And I want to come back to the future of the firm, because that gets really interesting in your research. But when you think about the whole foundation of a market economy, right? It's supply, it's demand, and that's based on millions of individuals making human choices, and then prices emerge dynamically based on our self-interest, or that's at least Adam Smith's vision of the invisible hand. But if you think about an agentic economy, the entity engaging with price and product, it's not human anymore. So, what happens to a market economy when there are no longer people as the primary operators in it? How do we even think about something like price when it's dynamic and there's AI systems making all of those decisions? Yeah, well, remember price is just a mechanism for coordinating all those decisions that consumers and firms are making, right? So, it it's signal, it's information to consumers and firms. So, the idea that it might not be price per se, but rather another mechanism that AI is using to coordinate those decisions. It would be valuation of some kind, but it might not be price in the way we think of it, right? Like price as in something that's going to be stable for some period of time. We're already seeing that with algorithmic pricing, or if people have had the experience of you know, two people who call a an Uber or Lyft and get different prices, or we all suspect that the price of the airline ticket is responding to our searches and you know what we've previously bought that somehow we may be getting very individualized pricing which starts to not feel like a price. So So it's it it's a world where those fundamental So it's great to think about like those fundamentals of Adam Smith or our original theorems about the way competitive markets work. And those real first principles that what the market is doing through prices and through the individual decisions of all the consumers and all the firms profit maximizing firms that that is leading to a set of choices about what to produce, how much to produce, and then how to allocate it across all possible consumers. That's what our prices are doing. So when I think about the future of the economy, I'm thinking about Okay, now if those decisions are being made those out production and allocation decisions are being made by AI by AI systems that are quite a distance from being instructed by a human what does allocation and distribution look like? We have to start thinking about how do those AI systems actually make those decisions? Will they be making those decisions in the same way that that that humans human consumers human firms are? And it really calls into question a division like marketing or advertising. That has to fundamentally change because you're no longer necessarily marketing to a human that's going to see your products. You're marketing to AI agents that would be the the first in line to interact with that. So your products may not even get in front of the consumers that you're targeting. You're now indirectly targeting that human through their team of AI systems which changes how we think about price. And even those psychological tricks of $9.99 versus $10 and we're more likely to purchase the thing that says $9.99. None of that works for agents, right? They don't fall for those types of those types of scams and psychological hooks. So, if I was to put this into an example for someone that's kind of thinking, how does this actually work? If I wanted to buy something in skin care, let's say, my AI agent would never sleep and it can consistently go and try to find the best deal. And if I'm just looking for a particular ingredient skin care, I may think that I'm going to find that at the local Walmart, but it could keep bargaining and find it somewhere else in the world cheaper, better. So, does this become like this race to the bottom in some way in a world where we have teams of AI agents that are never sleeping and always trying to find us the best deal and another and another potential product, a better product? Well, let's break that down. So, I think it's really important to sort of like like that that image of I'm looking for this thing. So, you as the human have an intent. And, you know, the fundamental thing that companies are trying to build with AI, and you can think about this like with Google Search. This is how Google would describe what it's always been trying to do with search is to get closer to what are you really looking for? You know, what given what you typed into the search bar, what are you really looking for? What's your intent? And like I say, pricing, prices, the way we experience them now, websites, marketing, prices, in some ways those are all shaped by the transaction costs of how we get that information, how the seller tries to connect with your intent. And of course, what's happening with an AI agent is the AI agent has other ways of connecting with your intent cuz has lots more information about you and has different ways of connecting with the seller to say, well, this is what I'm really looking for and you can really get that that kind of transaction we just don't see today. Now, you asked if it would be a race to the bottom. I guess I'm wondering what you're thinking. What's the race to the bottom? An AI agent can consistently or continuously try to find better prices somewhere else or a better version of that product. Um and it could negotiate with AI agents and say the person that I'm representing, these are their preferences. This is what they want. I'm going to go somewhere else or I the agent can go somewhere else that has an alternative version, a substitute, but it's the next best thing. So, could we see agents consistently, constantly negotiating and bargaining and the market really becomes this really strange emergent new system. Interesting. So, I I think often when we talk about a race to the bottom, we're talking about something bad. But if what we were talking about there is consumers that are able to get prices that actually um number one, they they can find products that really satisfy what it is they're truly looking for. And you can get prices that are getting down to marginal cost and don't have, you know, just pure profit in them. Um you know, that's actually our theory of how competitive markets work and how they work well. To make sure that we're building the stuff that everybody wants and it's getting into the hands of the highest valued user. I mean, that's the theory of the perfectly competitive market. So, I actually don't know if these results, this is what I'm sort of encouraging my economist colleagues that were who are still doing theory to to think about. What happens in that economy when it is AI systems that are making all those decisions? But I think it's exactly the question you're you're framing, which is is it getting us to the, you know, something close to the perfectly competitive market because we've taken out all the imperfections that we get from transaction costs and, you know, you sort of referred to like the the scams and so on with the exploitation of, you know, psychological weaknesses for humans or just our transaction costs, our search costs. Do we get to that perfectly competitive market, or are we looking at something that gets kind of chaotic because it's actually now, well, there's now this gap between what I said I wanted and what the AI is now out there searching for, negotiating for without checking back with me and for me to evaluate. Like, I don't know if you've ever had the experience you go out you're quite sure what it is you want to buy and after a little while you go, you know, that's really not what I need. I really need something different than I thought I did when I started. So, I think there's there's a lot of questions to ask about how well would this work, but it's an absolutely critical question that I don't think enough economists are asking. How does the economy work when you start handing over large shares of economic decision-making, core economic decision-making to AI agents? Right, there's that a potential world was beneficial to us where all of that greenwashing and all of those we promise these products are clean or we promise these products are sustainable. In a world where you have an agent that can take on all that complexity, it could go through all of those ingredients and say, well, actually this product doesn't actually deliver on the values that the company claims. So, there's a world in which these systems actually get us closer to what it is that we're after, but then there's also a world in which there could be that wedge between what you intend to to purchase or what you think you want and what the agent actually delivers. And I don't just have to have one agent. I could have a thousand agents that are working for me, buying for me, negotiating for me. How do we think about ownership and liability and mistakes? So, if I have a team of agents that's shopping for me, how do can people or companies verify that that agent belongs to me? If the agent makes a mistake or colludes or cheats in some way as it's buying something for me, who becomes liable in that world? Oh, this is a big open question and it's one of the things that worries me a lot about how fast we're moving on the agentic side of things because we really don't have the what I call legal or regulatory infrastructure in place for all these AI agents to go out there and now start participating in the economy and I think that's the way we have to think about it. So, your first question is getting our framing right to say, you know, we're used to thinking of AI as a technology, a tool that humans use to accomplish their objectives. But, if we start thinking about that true agentic world where it's a long time, there's a lot of autonomous behavior by the AI, that's a new actor in the economy. And we currently don't have any systems in place for attaching identity or registration legal structure to those AI agents. So, I always like to remind people that you know, we now live in an environment where anybody who's participating in the economy has an identity and is registered in one way or another. So, we have identity for humans and if you you know, checked into a hotel recently or you know, you've had to show your ID when you when you check in. Right? We have addresses, we have ID. If we want to get a job, we have to fill out the forms that say we're authorized to work in the jurisdiction. Uh for corporations, we have a whole legal structure that makes sure that they have a unique name and they're registered with the state in a public document so that if I buy something from Acme Corporation in California, I can go look up with the Secretary of State, who is Acme Corporation, and importantly, where do I file the law papers that start an action against them? If I think they haven't kept up with their end of the bargain, or they've sold me a dangerous product, or they've taken my IP or something. And we just don't have that in place for AI systems right now, for agents. We don't have any identification of agents. So, when you describe that thousands of agents out doing your bidding, right? If they're actually participating in purchasing, so in transactions, there are there are people and companies on the other side of those transactions who don't really have a way of saying, "Well, wait a second, who did I just sell that to? And are they in a state I'm not allowed to sell that product in?" Or, you know, have they legally bound themselves to pay me later if they if they you know, if it was a delayed payment structure of some kind. Um so, we don't have any of that legal infrastructure in place. People have ideas about how that's going to work. It's the invisible part of doing law. Right? The invisible part of doing law is we have identity systems, registration schemes. We have all this structure, infrastructure, that our accountability systems then hook into. And it's funny cuz it was through reading your research that I recognized how much of the human identity and corporate identity we just invented all of that, right? The idea of needing a last name and your social security number. So, all of these different ways and methods be formalized and institutionalized the idea of the human identity. And it was for market purposes, right? We think, "Oh, this is this is my family origin, and this is our tree, and this is our last name." But, it actually served a very specific purpose to know who you are in reference to the state, in case you need to get recruited to an army, or in case you trying to buy and sell land. So, now when you think about this world, we have 8 billion agents such as humans, right? We have 8 billion humans, and for the most part, we know or can get to the bottom of most people. In a world with AI agents, we don't have those systems, but it seems like you're saying maybe that's how we have to think about agents. They should have identities and licenses and registration the way you register your corporation. Now, anyone can start a company tomorrow, that's not a problem. You fill out the forms, and your company then can go and exist. But, how do you think about that with agents? And so, is that the logic that you're starting to see in a world with AI agents, or that's what you think we should be aspiring towards? It's a really important point, too, that you're you're bringing out that we invented these systems of identity for humans. We invented systems, governments invented systems of last names, and then we we It's a It's a fundamental function of government to oversee the identity process, you know, to make sure we've got the birth certificates, and you've got the passports, and the driver's licenses, and then the registration schemes for for corporations. And it's that it's that artificial quality, the fact that this isn't natural identity. I mean, we take it so for granted, that's how successful these systems are. Um I often find when I'm talking about this, with people have to really remind them that, you know, corporations are not natural objects. And we obviously feel very much that who we are is a natural object, but our names and our our ability to say, "This is who I am, and I'm not that other person," right, in interactions, that that's something that we've created through law. So, I think for AI agents, the thing here is not to start by saying, "Well, where are the boundaries technologically between these agents?" So, you just said, like, maybe I said have a thousand agents out there. But what you might have is government might create that you Sinead register, like you get a you get a number for your agents. Right? You you get an identification and you've registered and and that's an official thing. And then you basically you have a a thousand instances of agents that are identified with that number. And the key thing about that number is that if I enter into a transaction with you, I can go to a public database that says, "Oh, when I entered into this transaction, the AI on the other side of the transaction proved its identity with this number." And then I was able to go and check a public document that was overseen by government. Right? A reliable entity. You could have some of this arise privately, but ultimately I think it's a government um function. And I can go confirm, "Oh, that interaction I just had with a piece of software was connected to Sinead. So that if if we've said that you Sinead are responsible for everything that your agents do, we've got a way of structuring that. Right? We've got we've created the system for that. And it's not a natural thing that's going to come out of the boundaries of the technology, but rather we're going to impose it on top. We're going to design it on top. And that's what we've done with human identity and corporate identity. And I want to come back to the idea of can your agent continue to exist and and go on without you the way a company can? But you you wrote about this example and I think Mustafa Suleyman had also referenced a world in which theoretically I could build a team of agents or buy an agent and tell it, "Okay, I want to make a million dollars in the market uh in 30 days. I have 100k. Please go make that happen. Do the best that you can. Don't break the law. I would I would appreciate if you didn't do that. And call me again in 30 days." And then you could send off your agent to go do that. What does that example force us to confront? Because I could also do that with a human. I could hire a really great strategist and say, "Make me a million bucks. I have 100k." There's nothing wrong with that. Theoretically, I could also do that with AI agent. So, how does that complicate the picture? The AI go make me rich? And if the AI actually does something that it does cheat, some of it could have been a bug or a flaw in how the AI was trained. But in a world where these AI systems would maybe be licensed to you or have a way to to track that agent back to me, how do we think about liability and accountability and what agents should and shouldn't be able to do? Yes. This So, this is Mustafa Suleyman's modern Turing test. Right? Is an agent capable of taking that general instruction from you and coming back with the the 10x return on on the investment? And what I like to emphasize is that if that really is where we're headed, and this is why I point to that example as this is where all those billions of dollars are getting spent is to try and build that. That's what we we This is what the industry believes it is building. And I think it's important to take that seriously and say, "Well, let's let's suppose we did build that. What what does the world look like?" And so, I always emphasize just what would that mean in terms of what that agent is doing? Well, it's entering into contracts. It's It's doing consumer product research. It's It might be negotiating prices, setting prices. It might have to do compliance work and file documents with the state and all of these things that the human agent would have to do if you decided to hand over 100,000 and just say, "See you later. Uh bring me back bring me back a million." And I think the question of where is the liability? And and we don't need to talk about liability for for things like running over small children in the in the intersection only. Markets are built on legal rules, contracts, IP, property, right? There's just There's no such thing as a stable market without all of that, without some expectation that, "Oh, I've actually done something when I reached this agreement with the supplier that I have recourse if uh uh if the product isn't good or if it causes harm or if it isn't get delivered in time." And they have recourse if if if I don't pay up um or I, you know, say things about the private information they shared with me during the transaction and so on. That's All of that is built on the idea that there's legal accountability, liability in some sense for for those behaviors. And we just really have not sorted that out. I mean, when I when I have conversations with people who are thinking about agents people are imagining that, "Oh, well, it's all 100% going to come back to whoever it was who sent that agent out there." But number one, we actually don't have the law that makes that really clear. There's background law we could appeal to. But it's going to be very uncertain to figure out is going is going to be lots of gaps. And because our AI agents are not human agents they're alien entities they don't actually think like we do. And so they're going to do things that we never imagined they were going to do. And I think we're going to see a world where actually we have some real trouble holding the human behind the agent responsible for what the agent did because we have a hard time holding humans liable for things they couldn't anticipate, couldn't foresee, couldn't do anything different other than not send the agent out. I mean, we could just say, "Look, unless you're willing to take every single risk." But I think our systems are not likely to um not not likely to do that effectively. But nonetheless, there I did law and economics for a very long time. Um and I think we just haven't really thought through what the shape of that liability should be. And you can see those types of scenarios that we're heading straight for them. I mean, in a world where AI agents can operate somewhat reliably, you can imagine most people saying, "Figure out a way to make me money and come back when it's done and just try not to break the law." And in a world where you're paying 20 bucks a month to rent that agent or whatever the pricing scheme is, that is probably what's headed in some ways for markets and financial institutions and I don't really hear anybody taking that seriously. It sounds like a fantastical example, but that's actually likely where we're headed. And the second point that you make in your work, the corporate identity can continue after the person who founded it has passed on or maybe they sell their shares. So, if I want to start a company, I don't have to be liable for what the company if it's a corporation. I can change liability. So, if I'm not doing anything wrong, you can actually sue the corporation, not always just the people behind it. And if I leave the company, the corporation has the right to continue to exist without me um until the corporation maybe stops making profit. Are we going to have to create some type of personhood cuz I think people forget that the corporation actually is a type of personhood for these AI systems, so they can actually engage in in moving capital and they could theoretically exist past the lifespan of the person who created them, the same way companies can. I think these are these are all the big open questions that I I I sort of hope we can sort of rally many, many more economists to start thinking about right now because it's happening so fast and we need to be thinking is true. And that's why people have to take seriously this thing that feels sci-fi, although it feels less sci-fi every month, right? That we're headed to that world of an economy of of AI agents. And so this is this is the question. So I I and and I I often say I I don't actually know if this is the right thing to do to create legal personhood for AI agents, although I have a prediction that that's where we will end up. And the reason is because we created, as you pointed out, for for corporations. Right? The idea that I don't have to go find the owners, the managers of the corporation if they've breached the contract or taken my IP or put a dangerous product out. The more we are seeing AI agents that are operating autonomously, sort of a long distance from the original instruction from a human, and the more complex those systems. I mean, those systems are quickly going to be We imagine it right now it's all us, you know, individuals with an agent or a team of agents. But it's going to be aggregations. It's going to be collections. It's going to be firms. You know, even a couple of years ago, I think I read that that, you know, Sam Altman had a bet with other CEOs about how long it would be before a founder could create a billion-dollar valuation company without a single employee. Right? Because the founder could just say, "Here's my vision. Here's my idea. I can I can instruct all these agents to do this." And it's so these are going to be complex entities, I think. And the idea right now that what you're going to have to do is trace all the behaviors back to uh the original human or the original firm. Um I think there's going to be a lot of pressure to say, "Well, I No, I signed the contract with agent 472B96X." Right? And I And And that's that's all they need to know in order to file my lawsuit. But the thing is that to file the lawsuit means there has to be what we call legal personhood. I mean, and it's really important cuz the word personhood gets us thinking about consciousness and sentience and, you know, like real what it meaningful intelligence and No, corporations are legal persons and all that means is that they can be sued and they can file lawsuits. That's what we mean by legal personhood. And if we don't want to give them other rights, don't give them free speech rights, don't give them voting rights. That's we can absolutely do that and we we know countries go in different directions on how many rights they give corporations. Like do they give free speech rights to corporations? But narrowly speaking, the idea of legal personhood is just can file the lawsuit and can be sued in court. And so we can connect legal responsibility, accountability directly to the entity, the actor that has entered into the transaction. And so the reason I predict we end up there is is kind of a transaction cost story, which I think that's why we did it with corporations. You know, and uh my guess is that's going to that's the There's going to be a lot of pressure to say, "I don't want to have to uh find the people behind it." Like you pointed out, if it's more complex, they've disappeared, they're no longer around. Um I can't prove that they knew everything about what the agent was going to do. The agent did weird things. We're probably going to want that lawsuit directly with the agent, which means that agent also has to have assets. And that's a whole other thing, right? The agent actually has capital that it owns. Um and it could get very complex. Agents may spin up other agents to help them complete certain certain projects or or or tasks. You said you think that's where we're headed, but you don't think it's where we should go. What did you mean by that? I mean, I don't have an opinion yet on whether that's the optimal way to do it. Because I think we just don't know enough. So, one of the things that we haven't mentioned yet about this challenge of the economy of AI agents is just the fundamental alignment problem. Which is that we we currently still don't have a reliable way of making sure that if I have constructed an agent or instructed an agent, "Here's what I want you to do." And I've expressed that in some way, and that I can have confidence that the agent is going to go do that and not do weird stuff. Not do dangerous stuff. Not do stuff that I would never want it to do. And this is the fundamental alignment problem. It was the first thing I ever wrote in the AI area, sort of drawing on background in economics and law, which is look, we know this when we hire humans. We can never write down the complete contract that says, "Here's what I want you to do in every every situation, every possible circumstance." But we know how to manage that with humans cuz there's a lot of norms and legal rules that kind of fill in the gaps. And uh in this early paper, we um and this is with Dylan Hadfield-Menell, we um we analogize the alignment problem to that incomplete contracting problem that said, well, the alignment problem is the same sort of thing. We don't know how to tell the the robot, here's what I want you to do exactly. It's always going to be um incomplete, and so how are we going to have that confidence that the that the agent, and I think this is this is relevant to your questions about how the economy will work and how liability will work. So, the reason I don't have an answer to the question right now, what's the optimal thing, should we create legal personhood or not for AI agents, is I don't think we yet know what the shape is of that that alignment problem. How bad is it going to be? What are we going to be able to do? Um and then and you know, it's possible we shouldn't be sending those things out there at all because they're going to do things that we can't control and predict. And the alignment problem, and for anyone who's listening that hasn't heard of it, uh this is a problem that a lot of computer scientists are trying to focus on right now. Uh how do you ensure that when you give an AI instructions, it's going to complete that goal but in a way that's aligned with how a human would have. And uh the example that I like to use is you may ask your AI agent to do something really simple, book you a reservation at a restaurant. Great, very easy. But, the restaurant's full. So, the AI system just hacks into the restaurant's code and gets you a spot. That's a way to complete the goal. It's definitely not what a human would have done. So, it's misaligned to how the human intern or assistant would have gone about doing that. And the reason an intern would know not to do that is cuz that's breaking the law. I did read your your approach to alignment, and it seems like it's much less of thinking about this issue as a technical problem and much more about this social scaffolding, and we should be thinking about giving agents access to the law. We should be thinking about giving agents access to human norms. What does that look like? How are we scaffolding agents with these ideas? Yeah, so it's both a technical question and what I would say is an institutional one or I really like your term social scaffolding. Um, it's a technical problem and and the the name I I give it is it's, you know, can you build normatively competent AI systems and agents? Ones that can, like a human, go into a new new situation and see, oh, here are the rules and the norms that people follow in this community because they also vary from place to place and like, you know, a workplace follows some rules and a different culture follows different rules. And to be normatively competent is to be able to go into that environment and read what is it okay to do around here? What would what would be acceptable behavior in this in this particular group? And that's a There's a technical challenge there about how do you build those kinds of agents. But the social scaffolding part of things is how do we build the institutions and the processes that AI agents and systems can basically look out to, can consult with, and that operate at the speed and scale of AI. I I sometimes say it's like, you know, if if we go back to imagining, you know, the AI agent that we've sent out to make a million bucks on the internet, if we hired a human to do that, the human would have various ways of participating in conversations with their friends and and other entrepreneurs, uh, and they could they could call a lawyer. They could say, "Hey, I'm I'm thinking about a product design like this. Um, would I face any liability risks if I did that?" Or, uh, here's here's the the I'm going to use this IP, am I infringing? Right, would I face any risks? I'm thinking about not paying my bills for a while with this supplier. Um, you know, what recourse might they have against me? So, it's you know, we we could call the lawyer, you know, you've got your in-house counsel, you could consult with your in-house counsel. So, what is the AI system going to do? What is the AI agent going to do? And remember, they're working, as you said, all through the night. Right? It's it's not just, you know, actually get them on a phone call with with a with a human lawyer, but rather how can we build the institutions that are providing that input for uh for our agents. Um and so, this is this is what I call building like the normative infrastructure. But, I like your term social scaffolding. That's um that's really capturing the idea. And is this what you mean when you say we need to think about digital institutions for these AI systems? I mean, yeah, let's talk about that for a second. A digital institution, because in a world where, yes, I've sent an intern to make me a million dollars, they could call a lawyer, they could check out if anyone's posted about this stuff on LinkedIn. In a world with AI agents, they're going to move at agentic speed, and they can read everything in an instant. The scaffolding needs to look entirely different. So, it's as if we have to invent the checks and balances that agents can, I guess, ensure that they're operating within the law. And this would actually, I think, bring us to regulation. So, when you think about the current regulatory approach to artificial intelligence, we see some countries with these sweeping AI acts, we see some countries with these AI executive orders, some countries feel just completely paralyzed, and they're like, "This This technology is moving too fast. You know what? It probably can't even be regulated." What are countries missing in how they are thinking about regulating this technology? Yeah, so there's a lot in there, and I'm let's focus on the regulatory question. The digital institutions piece is about how do we build that? How do you how does AI call a lawyer? And that's that's saying well, we've got human rules and laws out there and we need to make those legible. Mhm. In the right kind of time scale and scale for for AI systems. So let's let's call that digital institutions. And and now let's think about okay, what are the rules we should have in place? Like what kinds of laws and rules should we should we put in place? That's the regulation question. And um that's also as you're pointing out, that's something that we have not figured out. I mean again, it's when I feel nervous about how fast things are going, it's about the fact that we don't have this infrastructure for agents and our governments haven't figured out how does to to adjust and change the rules that we have in place for this very very different world. Like we talked right at the beginning about uh what does an economy look like with agents? Well, we have a lot of regulation of economies to make sure that we have stable markets and liquidity and and so on. Uh how are we going to adjust all of those rules? And then what kinds of rules do we want in place for these agents? Right? Like what kinds of agents do we want to allow to participate in the economy. So the thinking that I've been doing around regulation um and I've been uh working on this actually for for quite a long time even before I started thinking about AI, just thinking about the fact that our governments face the challenge with fast-moving technology and globalization to really keep up with the way the world is now. And you know, you add AI into the mix and it's a bigger problem. And I think the thing that uh governments are going to have to come to grips with is we can't rely entirely on what I call our existing regulatory technology of legislatures and courts articulate rules in in text and then we you know we use we use legal legal processes investigations litigation and so on to um to enforce those those processes are you know not producing the rules that we want and they're not they're not moving fast enough and if we look at where we are on the regulation of AI I I think just everybody agrees that we need governance of this massive change that's happening in our economies in our society but everybody also agrees and our governments just don't seem to know how to how to keep up so the EU which is um you know definitely um working harder than other governments I think to solve this problem um has enacted the EU AI Act but even in doing that it's had to rely very much on companies and industry to really supply the detail of what companies are allowed to do so that like what tests they need to run and what counts as a what counts as a risk and uh we're seeing a lot of pressure in in other uh other countries to say well don't you know don't regulate all the detail about what the companies can do because that's going to slow innovation so so what I think we're missing here is the idea that we could be using a different way of thinking about how we approach regulation and I call this originally called it um regulatory markets then it's the idea that um governments instead of saying here's how you need to specifically build your AI, here's the data you can train on, or here's the algorithms you can use, here's the specific red teaming tests you have to run, instead of governments establishing that detail, governments establish what the acceptable level of risk is. Say, what's the acceptable level of risk that a chatbot will talk to a kid about suicide? What's the acceptable level of risk that a system will um you know, assist assist somebody in in building a bomb or a biological weapon, right? So, government does what we call performance-based regulation. And this is something we've thought about in regulatory theory for many years, and we use in other different other settings. But, governments focus on setting those outcomes, which is I think fundamentally what governments should be doing and are not doing right now. But, that we push the question of but well, what do we have to do to get to that acceptable level of risk? What data should we train on? What test should we run? What algorithms should we use? What kinds of human input should we should we in include? Um that that's a technical question, and it's actually one that we really want to recruit markets to helping us to solve. Sometimes you can think of this as we're going to need more AI to regulate AI, because it's going to take AI systems that can figure out, oh, what's the likelihood that the chatbot is going to talk about suicide? Or what's the likelihood that the agent is going to collude on prices? Right? In in the market, or commit fraud in the market. So, that technical process is something that we really need to attract investment towards, and we need to attract the innovative engine of markets. So, the proposal of regulatory markets is that government uses those outcome criteria to license private actors, companies in the market that are saying, "Oh, I've got the technology that can do that, or I've got the processes that can do that. License me to be a provider of those regulatory services to the to the target entity, like the the AI developer." And we've we've got a proposal currently I'm working with uh the organization Fathom um to sort of do an entry-level version of this. It would be a voluntary regime to start building this ecosystem, which we call independent verification organizations. So, in the world of accounting, right, the government would say, "Okay, this is what we consider acceptable for a firm, and this is how we think of we set the tax rate, and we set all of the rules, but we're not KPMG or PwC. You can hire PwC to come in and make sure that you're following the rules." The government sets the rules, and then they hire a private actor to ensure that different companies are following the rules. So, are you saying that we need something similar for the world of artificial intelligence, where government just focus on the outcomes, focus on this is the acceptable level of harm or risk, and let the private market come in and innovate around the technology of how do you actually check that the company is delivering on that? So, is that kind of a summary of how we could think about regulatory markets? Yes, it's one step beyond where we currently are, say, with accounting, or just generally with standards that government set. And we have lots and lots of private actors, like accountants, like a lawyers, like private certifiers, that come in and check to make sure that you're following the rules, or um you know, doing things the way you're supposed to. Um then the the the only difference here is that we really push government to say, instead of saying what red teaming test you need to run, we say, you need to make sure that the risk of um uplift to somebody with a bio weapon is below this level or is at a reasonable level, acceptable level, judged by judged by government. I think this is something when we have our conversations about AI regulation we um often overlook about what the real world of regulation looks like, which is it's this complex ecosystem. Lots of private actors are playing a role Right. in determining. You know, if you've got your compliance department in your in your company, right? Their job is that's that those are private actors and you can hire the external private firm, like you're saying, the accounting firm in the assurance industry. Uh we have lots of private actors and what we're trying to do is say, let's um let's figure out how to harness that for the AI world, but still make sure that it's governments that are making and us collectively. That's all I mean by government is like we are deciding how much risk are do we think it's appropriate for us to be taking. Um and then saying we're we're probably going to need real technology to do this, not just come and check the books and make sure that we followed the processes, but I'm going to come test your system. I'm going to come test your system and see whether or not you're you know, I'm going to come test your agent and see how often they do something completely wild um when given these kinds of instructions. When I look at the different approaches to regulation and the different politicians that are supposed to be leading these charges or the different acts that have been passed or people are debating, none of them look like what you're describing at all. Everybody is writing these static rules and laws and and it's not passing or it is passing. Nobody is thinking it's essentially we have to think as innovatively as the technology is and it seems like everybody is grabbing these this 20th century infrastructure and trying to apply it. So do you think most of the approaches to regulation right now they are probably going to be insufficient or if not entirely fail if they don't understand we actually need to invent a new system doesn't even have to be entirely new system we can pull what works from finance and accounting but if we don't think about it that way all of the approaches that countries are taking they're entirely insufficient and that's how actually people get hurt in the end. Yeah, I think that's exactly right. Um and and that the the way you've expressed it the way I like to express it. We're going to need to be as innovative about our regulatory approaches as we are about this technology. Really take seriously this is transformative and so we should not expect that the systems and methods we invented in the 19th and 20th centuries are going to are going to work here. So two things happened. One is we get uh governments that write as you point out sort of static rules that are going to be outdated in 3 months. Mhm. Right? And that it's not wrong for people to say, "Oh goodness, you're really going to slow down valuable innovation if you put these wrong headed rules in place." And I think actually because so many governments recognize that that they're just stepping back and saying, "Well, we can't do anything." Or they look like they're doing something. So the EU AI Act definitely is doing something but it's it's it's still leaving a lot of the detail and a lot of the determination to industry and to developers. Our transparency laws are still saying it's great that we've got laws say now in California and New York that say you must you know companies must have a responsible scaling policy. And they need to follow their policy. Um but we we we haven't gone in and said, but here's the level of risk that we think as a collective is the appropriate level of risk. So, absolutely, I think we're facing such a transformative moment that we absolutely need to get really creative and to be bold experimenting with new approaches. Now, you know, are we there? I actually Sorry, I mentioned the um independent verification organization IDO effort which we have legislation um uh in proposing this as an approach and proposing it as a voluntary regime to get us started, narrowly scoped to start to build that ecosystem. Uh so, that's we have legislation in front of the California, Ohio uh legislatures, Connecticut now. Uh Virginia has has actually passed and the governor has signed an um legislation to study this method. So, we are starting to see some uptake and some willingness to consider um something um that might get us to being as innovative on behalf of the public, basically, Mhm. as the companies cuz we are so rapidly moving to a world I like to say we have lots of AI governance happening, but it's happening inside of the private technology companies. You know, and I'm I'm glad to know that, you know, almost all of them are doing things to think about safety and to think about reliability. Um they do, you know, to think about alignment. But as a matter of governance, right? That's not what's supposed to be happening inside corporations, those kinds of choices. Those are the choices that are supposed to be happening in the you know, publicly through our collective processes of government. And this is a piece of what we have to change our thinking about, right? Like, leave the corporations alone. I mean, markets only work cuz they're well regulated. >> Mhm. Right? They only work cuz people are confident about, you know, we have antitrust law, we have fraud law, contract law, we have IP law, we have all of this law in there, which is regulation that makes our markets work well. Um and and so I think, you know, the choices about what the world is going to look like, those are ones we collectively need to be making. And right now, um those decisions are being made exclusively inside private technology companies that we can't even peer into. Mhm. Because of, you know, the that the the legal boundary of the firm that says, you know, we we we you know, we we can't all just go in and say, "Hey, tell me what tests you're running. Tell me what data you're training on. Tell me what algorithms you're using." So we don't have visibility into that. So part of building this ecosystem is we absolutely need that independent sector of expertise that can act on our behalf to decide how much of this do we want? What do we want it to look like? Do we want people to be able to just release a thousand agents out into the world with a hundred thousand dollars each to go make money. Do we want that? That's for us to decide. Right, cuz right now essentially the founder of the AI company is deciding what the AI governance should look like for their company in the world. And what we are actually all currently relying on, we are making judgment calls based on how we see these founders operate in the world. Oh, I think I can trust that one a little bit more and that one not so much. You see what that one just posted on Twitter? Not subscribing over there. That's not an actual sustainable method to think about governing the most transformative technology in history. And I've also personally started to reject when I hear policy makers say, you know what? This technology is just moving too quickly. It's too overwhelming. I don't know if we have the faculties to figure this out. I've actually started to personally reject that because that paradigm and way of thinking is only true if you are trying to drag the 20th century approaches to regulation to this technology. But if you are thinking much more innovatively, that is not a problem anymore. So now every time I hear that I I think, well, that's actually a design it's a design challenge. It's not a a pace challenge um or it's not a a fundamental challenge of physics where this technology is just impossible to regulate. That's not true. There are methods such as yours and and and new frameworks and approaches. We just have to be much more innovative. Um and if somebody's struggling personally with the innovation side, then that's um an entirely separate thing for that person to recognize that maybe they just aren't able to think about the moment that we're in, but you can't just drag the 20th century and then say it's not working and then say we're behind and then say it's too quick. It's not. And that's the only thing we can guarantee about this technology in the future. It's not going to slow down. So the mechanism and the institution has to change because the technology isn't. No one's really pressing pause and doing a a snack break um so we can get our thoughts together for artificial intelligence. Um and there was something that you said at a economics conference we were both at about 3 years ago and I've been thinking about it ever since and it was what really inspired me to go follow your work and start reading about it. You said, you know, we have to think about where does the boundary of an organization stop >> agents. What even becomes of the idea of the firm in a world in which one or two people have 10,000 agents, and that is the company? How do we think about the nature of the organization in a world with these systems? Yeah, basically AI is very rapidly putting us to the test on just about everything we take for granted about the way the world is structured. Um and so uh you know, that that point about do we even have firms? So I'm I'm an institutional economist, organizational economist. That's my original training. And you know, say, well, you know, the theory we have of the boundary of the firm, like why we don't do everything in through markets, like why why isn't everything just a one-on-one transaction? Why do we have these aggregations so that we've got something we we eventually give legal personhood to that operates as a uh an independent entity. And that story is is again, it's about transaction costs. It's about governance costs. And it basically says, well, you should make the choice between having decisions made through a hierarchy, right, with a CEO and managers and employees that you instruct, and decisions made across, you know, like through the market, where you have to rely on contracts and property and so on to to govern your your relationship. So if if if all of those governance costs, because they're now questions not of oh, how do we overcome the the slacking incentive of employees or the private information and the the conflict of interest that a manager might have. If now that's not the main thing that's driving the structure of of we're going back to those just fundamental decisions about allocation and distribution. If what's driving that now is like the alignment problem. And how well does an AI system actually effectuate the unarticulated goals and intents of human society of the of the person who sent it out there and then the world at large which has also has an interest in how that happens. Yeah, so so we just do do we have firms anymore? I think the other piece of the firm that it's important so when I talk about this boundary of the firm and do we have firms? We have this artificial boundary cuz remember we were talking about identity and we say well, law created this artificial structure. So how do you create a firm? You go and you you incorporate. And you file documents and you say here's here's this new thing this new thing that can sue and be sued. And a part of that boundary of the firm is this is this information boundary. Right? So uh there are trade secrets and inside the firm that the firm is is allowed to uh get help from the state to make sure other people don't don't come and take those trade secrets. And we we define what it means to be an employee of the firm to say you're loyal to the firm and you won't share the company's secrets outside. So the this is what's creating this now this real boundary between the private technology company and the public and making it very difficult for us to say this and I think this is totally new actually. I think this is what is very distinctive about AI because of this artificial boundary that we've created through law for lots of good reasons in the 20th century because it drives innovation and creates good incentives. But today it is significantly shifting the the locus of where decision-making is happening about oh my goodness, what are we building? Is this the direction we want it to go in? Um how do we want it you know do we want it to happen this fast? What do we need to do to be ready? Um with very you can't really regulate or do good policy about something you just can't see and then as you're pointing out you end up relying on like the goodwill and and and uh and good intentions of you know frankly ordinary people who just happen to be the the the heads of of these corporations and you know even if you think those people are really really well-intentioned and I think some of them are. Um you know that's just not a robust way of regulating and it's it's not really a legitimate way of regulating either. And it's really interesting that you pose the questions around intellectual property and these boundaries because let's say I wanted to make a competitor to Pepsi or to Coke in the AI age I could just build a bunch of AI agents and they are the sales reps and maybe one of them contracts an actual robotic lab to cook up the new formula that's going to compete with Pepsi. If that formula leaks or somebody copies it and there's nobody else at this firm but me what happens to the idea of intellectual property or if the agent was hacked and another agent goes in and builds that same can of pop who gets sued? Who had the intellectual property rights and if the agent came up with that formula is that my formula for my who's even at the firm. Um so these are all these profound questions that I don't know how we would think about in this era. Yeah, that's going back to this the the the infrastructure point that says we haven't even created the infrastructure of identity and registration for these new actors in the economy. These new participants in the economy and that means we haven't even I sometimes talk about like a Velcro theory. We we haven't like we haven't laid down the little strip with with you know the the the fuzzy surface that when we figure out what should be the rule about the intellectual property ownership for the new recipe for Pepsi or Coke, like the the strip that has the little hooks in it, right? Like when we figure that out, we should have that that first Velcro mat already in place so that we could say, "Oh, wait a second. Um we've been approaching this question of the IP ownership of AI from a perspective of, you know, our existing copyright law or patent law. You know, we've had these decisions out of these different these different courts and then we get bold and we get creative and we say, "Well, no, we actually need a different way of thinking about IP for artificial agents." Um you know, we we then then we will want it what we want to do is to be able to quickly put something in place like that. You know, economists like to get up there and say, "I've got all these answers and I've got all these predictions for you, but mostly I've got I've got questions and prods to my fellow economists to say, there's so much that we need to be thinking about what this new economy that's rapidly coming upon us, um how it will operate, how it will function, and what kinds of legal rules, regulatory structures, infrastructure will need in place for it all to go well. So, I want to ask your take on on jobs as an economist. If you have a take on what that may mean, if you think about that world where I've built a competitor to Pepsi with a team of 4,000 agents that may be contractor robotic lab, where do you see people in that future or work? Two thoughts. Um well, a few thoughts. So, I think when people think about the future of work, we should really be talking about the future of the economy. Right? Like, how is all of this going to change? And that's the that's the way you're asking the question, right? Like, we get we Now, the firm is changing. So, the very structure of markets, the way marketing works, like, are we going to have websites for online shopping at all or are we going to have prices, right? All of those really fundamental questions about the way the economy is going to work. Cuz I think we can't just continue to think about AI as just oh, really ramped-up automation, right? Like, really ramped-up machines. I think and And so, if we're making predictions about that substitutability between human labor and machine labor, um I I think we're we're we're not well grounded on making predictions about it linearly. Yeah, exactly. I like to always emphasize it's a complex adaptive system. We actually can't easily cuz so much is going to change. And economists know this because they know about complementarities. I like to say like one of the anecdotes I heard when I was thinking about as working on legal innovation many years ago and somebody who would who would sort of followed the deregulation of the airlines said, you know, right up until the night before they deregulated the airlines, no economist was predicting what would what actually happened with deregulation, which was not just changes in prices and routing and so on, but rather the emergence of the hub-and-spoke system. You know, a totally different way of organizing airline travel. And so I like say so so we don't actually know what this is going to look like. And so we should be thinking about well, how are we going to put ourselves in the best position to be able to respond to what's happening. That's why I talk about the the Velcro theory. Get the legal infrastructure down. Think about what the components are for that. Um but if we you know, if if if we focus in on the question of like what's the role of humans going to be because if I mean, that's the the the definition of that that open AI sort of put out there in 2015 or 2016. You know, artificial general intelligence is when machines AI can uh outperform humans at all economically valuable work. Right? So that's that's a vision of oh, all the economically valuable work is going to be done uh by machines. But I think this misses something really important and I suspect it can be and if we are intentional about it, it's more likely it will be that any economy is making judgments about value. I mean, we use markets to make judgments about what should where should we put our resources? And if it's a well-regulated economy, we say well, let the market decide that as long as we've made sure that we've provided housing subsidies or we've got taxes that redistribute income or we've got rules about how safe the skyscraper has to be or how much pollution you can put into the air. Right? We but we leave a lot of that determination of where is the value? How do we decide where the you know what what what it would be valuable to build? Mhm. There's not a fixed set of economic things to do. Right. And then there's actually executing on the task of of you know building those those phones or those cars or um accomplishing those financial trades. Right? Like we're constantly making judgments about what should we build? What direction we build? How should we build? Uh where where does our economy go? That's that's the fundamental question of uh of an economy. And there there will still be all those judgments to make. And so so one view of the future which is a little less dystopian about this is well that's what humans are doing. Humans are engaged in this process of making the decisions, collectively making the decisions, figuring out where the boundaries are, when you personally can make that decision and when it has to be a decision made in combination with others. I don't think any of us are going to want to say like let's go back to your you know your your question about I'm you know I've I've got a personal agent that's going to go do my shopping. Um and you know look for a skin care product. I don't think you actually want to sort of set that in motion and then you know all of the decisions no matter what you think of them are ones that are made by this by this AI system. >> Mhm. Um and you know if there were you know say ingredients you liked that caused you know toxic chemicals or they you know they created um they used too much energy, right? Like we as a collective have opinions about what we think is good and bad and where you should go and not go. We have personal opinions about how we want to live our lives. We have collective decisions about how we think our world should progress. And And so, I think one of the versions of where we head and it's like the flip side of the alignment problem is we're going to need humans to be fully engaged. And maybe now it will be possible for more humans to be engaged in that. I don't know exactly what it looks like, but I I definitely have ideas and start already started to work on, you know, what are the kinds of institutions you could build, what are the processes you could build um to have those values decided by um by humans. And And maybe that's what more of human life becomes about. It's Oh, it's the decision-making and the agency of what direction do we go. Right. And I think it that the phrasing you have to first understand where what the market's going to look like in the economy's going to look like before you can understand the jobs in it. And you said something quite radical that I think um you know, will will we still even have prices? And that's just an assumption that we take, okay, yeah, right now we price products and that's how we make the decisions to buy them. This is too expensive. I This is my budget. But how do you even think about, well, what do marketing jobs of the future look like? You have to first think, okay, well, there may not be prices in the future because AI agents could be looking at different types of different factors. There may not be a human viewing the product first. Uh that could be done by agents. So, you have to start there before you can think of, okay, this is what we need to preserve in marketing. Well, if there's no prices and there's nobody looking at the product, it's just a bunch of AI agents, and that's something different. And then you can think about the scaffolding of work around that. Uh but I I we tend to think of jobs as this linear it's continuity into the future. We just do the same thing a little bit faster or around a bunch of agents, but the entire structure of the market could change like a world without prices. That sounds so impossible to think about until you remember we invented prices. That wasn't a real thing. It wasn't a fact of nature and evolution. We just made up that system to begin with. If we were to think about timelines, where do you think we are? I know that we're probably going to need a few more breakthroughs to get to this chapter a bit more reliably, but where do you think we are on the path to the economy of agents? There's two parts to the way I think about that. One is because I think a lot about this invisible quality that we take for granted that the human agents we hire are normatively competent. Uh there's all kinds of things I don't need to say that a human I hired to go make a million dollars on the internet. Right? Because I I have confidence that they're they they are well embedded in the norms and rules of the world we live in. I don't need to instruct them on that. That's that's what I've called normative competence. Because I think we haven't figured that out. I mean, I'm working on it in my lab and one of the things we're trying to get a handle on is oh, like how normatively competent are our existing systems. And the initial answers are not very. I mean, there's lots that they can do in that if it's a novel and you have to think about novel environments, they're not doing so well. So in terms of predicting where we go, you know, the the focus is on especially right now the focus is on how capable are the systems and the agents at accomplishing in particular verifiable tasks. Mhm. Like basically we're building them to be really good at math and and science and language, of course, but we're not currently focusing on, but are we building them to be good at integrating into the complex cooperative normative structure that is our world. So, so I think we may hit a a technological limit that we've built things that nobody's actually going to want to send them out there. Like Mustafa can Suleyman can say, "Well, I think we might be a few years away from um the agent being able to go out and make a million dollars on the internet in a few months with a general instruction." And and is not thinking about but nobody's actually going to do that because there's we have no idea what the liability is and we have no idea what kinds of crazy things it it might do and how you control that. So, that that's one thing about about the timeline. The [snorts] other thing about the timeline is um my worry is that we will get a flood of agents because right now you can send agents out into the world to do things. So, I've just started work with a group that's trying to develop benchmarks for open world tasks, like how well does um an agent like like an open claw agent do at a real world task. And one of the first task was, "Well, let's try instructing it to in come up with an app and get it posted in the Apple App Store." And it succeeded at that, right? So, um we could have millions billions of agents that get released into the economy well before we've actually figured out all this stuff about, "Well, wait a second, who's responsible? Who's liable? How do you trace?" We don't have any law in place to trace it back to the human who released that thing. When when I get anxious and and actually when I the last few months getting up to give talks on oh, let's get this infrastructure in place. Let's get ID registration independent verification organizations in place is a little voice in the back of my head saying, "Oh, is it is it too late?" Because it is is it happening so fast that um that we're going to have all this these agents out there before we've built that infrastructure. Um I don't think it is too late. I still I get I I sort of quiet that voice and say we This is what we need to keep pushing forward to do. But the timeline and is it possible we could have an economy of AI agents? Um I think that could happen really quite fast. Um everybody said 2025 would be the year of agents and it it really wasn't, but 2026, 2027? Mhm. Yes. Um and that's actually my version of existential risk is if we release that really quickly without any of the infrastructure in place, then I just I worry that the the danger we face is we just crash the economy. We crash our systems. Like nobody wants to invest. Nobody, you know, nobody wants to sell products. Um because they don't know what they're interacting with on the other side. Right. So, everyone's talking about AGI or artificial super intelligence causing some existential crisis rising up against humans and you're saying, "Stop. This system's autonomous agents that could be live to air in 6 months. That's enough to crash the system in a different way, not because they are super intelligent and going to scheme up something, but because our infrastructure wasn't built for an environment where there's billions of autonomous operators that we can't oversee. And no matter how um extravagant we think we can build a system to to monitor them, we're not going to be monitoring billions of AI systems. So, we need an entirely new system to think about that. And that is the existential crisis. And it's sooner than we It's sooner than people are planning for. And even just that idea that AI could crash the economy and not because again it was a malicious scheme, but because our system just wasn't built to hold what's coming. Right. Exactly. Exact That That This is why I I think the um uh ID and registration for agents is such a critical We We could do that relatively quickly. I think that's you know, as a starting point, like no matter what you think you want to do later, you're going to need to be able to do that. And we didn't do that with other kinds of software agents. You can sort of think about what happened with social media, right? We We said, "No, we don't want to create any liability structures. Uh we're not going to create requirements of identification so that we can trace back who was actually accessing that system, who was actually uh posting. Um And it's a different problem social media. It's It's I'm not saying it's the same problem. I'm just saying, "Can we have learned Can we learn from that and say, 'We should at least This is not deeply regulatory. This should not be exciting, you know, this this fight between regulate, don't regulate, you know, shut it down, let it rip.'" It's like, just build some basic infrastructure. It's lightweight. It's not that hard to to create. It's not expensive to comply with. And it gives us the option and the the to respond cuz you know, maybe maybe I'm wrong. Maybe the agents that people will deploy will only be the ones that they have been able to verify, they have high confidence will do what it is they want them to do. That's another view of how that future uh evolves. But, I would really like us to be in a position to act if we need to act. And do you think we're going to run up against trickier challenges? Because you use the word norms a lot. There are laws and there are norms. And the law says you can't break the speed limit, and we know that. But, we've also all socially agreed and accepted that in the event of a medical emergency, you're going to break the law, and everyone is going to be okay with that when you need to get that patient to the hospital. How do you think about that in the world of AI systems? That sometimes we do break the rules, and we're we've all accepted when it's okay, but it's not written down anywhere that that is the case. Yeah. This goes back to uh I was saying the first paper I wrote when I started thinking about AI and taking the lessons from law and economics. And this is um the reason alignment is hard is because of the incompleteness problem. That we can't express what we want an agent to do or we want what we want to set as a rule on the highway, and we can't fully explicate that in in language. So, we can set a we can set a um we can say that the speed limit on this road is, you know, 55 mph, but as you just pointed out, the real rule says, "Unless you need to race to get somebody to the to the hospital." So, the real rule is actually filled in. And we have expectations about the fact that, "Oh, and everybody would agree that was appropriate." And even if the the cop you know, sees you and pulls you over when you say I'm racing it's going to say, "Oh, let me put my lights on and get in front of you and take you at that speed." because we have that capacity. We we predict that's the stable world that we live in where the norms are and the the alignment problem fundamentally is how do you um build AI systems that are able to have that kind of natural way of thinking about it. I call it again normative competence. Um uh saying, "Oh, well, the the the rule on paper says 55 miles an hour, but everybody knows that everybody knows that everybody knows that if you need to race to get your kid to the hospital go for it." Right? That that's that's that's considered an appropriate behavior. And this is why it's it's really challenging and this is why when we think about releasing AI agents into the economy and say, "Look, there's just going to be a constant stream of those kinds of circumstances that are not fully captured in the instructions you gave or the formal rules that will require this much more complex filling in." What will be the norms here? What would people accept? If I had to get up and you know, and and justify why I didn't follow this rule. What's you know, do I think that most people would say, "Oh, okay, yes, we all agree. That's a case in which it would be appropriate." And that's that's why I always emphasize that we don't have legal systems that just consist of written documents saying, "Here's what you can do." We have processes. We have courts. We have lawyers. We have judges, we have juries, we have legal argument, we have treatises, we have all this process to deal with all those cases where we say, "Well, you know, it's not quite clear what the law requires here, or maybe we should really be reinterpreting the way we've understood the law in the past." And it's it's only a robust system because we have those, you know, all those written documents, and we have those processes. So, so, you know, I'll say like, "Well, we have this idea of constitutions, writing constitutions for AI." And you know, that could be valuable information for the system to make judgments, but there's in the in the human world, there's no such thing as a constitution independent of constitutional courts and constitutional doctrine and constitutional lawyers and that all that process that we use to resolve all those places where it's ambiguous and it's incomplete and we we still are just figuring it all out together. Mhm. And that kind of goes back to and that's how we we stay anchored in ordinary people and you know, like our judges are generally trained and you know, we have juries that consist of just ordinary people that are brought in to decide even complex technical cases and at the end of the day, you have to kind of get them to say, "Oh, yeah, we agree, that's bad behavior." Or "No, we think that was fine behavior." Mhm. We make judgment calls and it yeah, it keeps coming back to the idea that this this institutions of the future are going to need to look as strange and emergent as the new technologies that we're building. And I think my final question, why do you think we are underestimating where where are with agents and what's about to happen and how quickly this is all all moving cuz you have people that are truly in doubt about where we are in this moment. Why do you think that is? I am regularly stunned when I talk to even pretty sophisticated audiences, academics, business people, um people in government. Uh because I've been immersed in this for 10 years. So, okay, I've been talking to these people for 10 years. I've been working in this for a long time. I'm just quite stunned by how little most people in our environments know about what is happening. Um most people know about chat GPT. And so they know about chatbots, but even then they may not have they may use them but not have interacted with them and certainly not exploring all the capabilities. But I I'm just really surprised by how many people just do not know and my colleagues at university in other disciplines just do not know what is an agent. And um yeah, so it's the one of the things I've been trying to figure out how to do is uh to spend more spend more time sort of generating some writing to say, "Okay, here's what you need to know." I mean, I've been talking to social science audiences in particular and business audiences for a number of years. It's just not getting through to enough people and we've got this uh narrative that well, it's just hype. Mhm. Companies are just trying to sell a product. And yeah, there's hype going on, but I I'll tell you most of the people I know in those companies, they truly believe. This is not like they're just saying something they think is a load so that they can sell product. Um there may be some people like that in the industry, but most of the people I know have been in it for a very long time and they complete is totally sincere. They could turn out to be wrong. This is what I was saying like you know, will we get to an economy of agents? I think there are still questions about that. Um technical questions about that. Uh economic questions about that. But um we we too easily discount and say, well, I don't need to you know, those people who are spinning those sci-fi stories. Now I think that's shifting, but um uh it it's still a tiny population of of people who are um kind of in the weeds of what's happening and it's another reason why we I think really need to to be talking to people outside of the industry, people without that technical background. And I will always say, look, spend a couple hours. I can explain to you everything you need to know about how the systems work. I can't code either. Um but you know, in terms of understanding what is happening and why why it is not so straightforward to just say, oh well, but you know, a little piece of software will do what I want it to do. What's what's the big deal? Um to to sort of really explain where that problem comes from and why things are moving in the direction that they have and what evidence we have about like the wild stuff that AI system like it really is alien intelligence. Um that that I think I think we need to get that conversation moving more into the public and and in particularly into our into our governments because uh governments really do need to be taking seriously that they have to figure out a way to act and there are things we can be doing right now, I think, and be doing. Yeah, and it and that that saying that's existed um for for over a decade, the future is here, just not evenly distributed. We can see the pillars of where the future is going. It doesn't have to be a surprise. I mean, as someone who studies for the the future for a living, I'm often not that surprised by what emerges because you can see it in the investments, you can see it in the patents, and in this case, you can see it in the the verbal declarations that companies are making. They're telling us what they're trying to build. And by simply just saying, "I'm not going to acknowledge it um or I'm not going to believe them," that doesn't mean that that outcome isn't going to arrive, and you certainly don't arrive at the future you want by just not acknowledging the futures that you don't want. Uh so, I think yes, on the one hand, it's an education problem, and on the other hand, it's recognizing this is the moment that we're in uh and leaders are telling us what they are trying to build, and we should take that seriously. Professor Hadfield, it's been a pleasure. Thank you so much for joining. We we look forward to to having you back on. What impact do you think AI will [music] have on the workforce, and do you think we're headed for an identity crisis? >> And this is the question that's fascinating about AI. What else can I become? Very few people have the courage to ask that question. Why? Because they look in the mirror in the morning and they see an engineer or a doctor. They don't see a person. >> If they're not looking at artificial intelligence and asking, "What are we going to become with this technology?"