DeepMind CEO Demis Hassabis：从聊天机器人到 AGI 之路｜Hard Fork

I actually don't think they fully understand the Monumental of what is being built AI designed drugs and cures for really terrible diseases we're only just a few years away from that energy becomes free or cheap right and then suddenly the nature of of of money even changes we don't want to have to wait till the eve before AGI happens and go you know what maybe we should have thought about this a bit harder okay no we should be preparing for that now this week Google Deep Mind CEO Demis sabis on Google's newest AI breakthroughs building artificial general intelligence and what happens next in a world where computers can do every job personally I take a nap well Kevin the gang over at Google is added again yeah what have they done this time well from the minds that brought us Gemini Advanced and Gemini Pro and Gemini Nano comes Gemma have you seen Gemma yeah I will admit that when I saw that Google had come out with a new AI model called Gemma I did momentarily think they're they're trolling Casey Newton personally they they want to kill this man uh by introducing the most complicated set of product names imaginable so are you alive and and how are you uh recovering from the news of Gemma well I'm coming to you now from the hospital where I was taken for observation following my attempt to process the latest Google AI models but I'm I'm I'm going to be back on my feet soon but look we like to have some fun talking about the various models which we struggle to keep track of and and understand what they're doing um but fortunately there is actually a person within the Google Organization Kevin who could explain this stuff to us yes we took your complaint about Google's AI naming conventions all the way to the top uh today on the show we are talking with Google deepmind CEO Demis hassabis uh Demis uh is the head of all of Google's AI programs and he is really uh the person who uh has has caused you so much pain and grief over the last few weeks with these names yeah but but we should say Kevin like Demus is a big deal in the world of AI I would consider him one of sort of the four or five most important people in the entire field of AI and so today we're going to just devote the whole episode to talking with Demis um and I think we should just give a little bit of background for people about who he is and why he's so influen within the field of AI yeah so what what are things that that Demus has done that make you say that he is such a leading figure in this world so I would say that Demis has been present for most of the big moments in AI over the last 10 or 20 years um in 2010 he and his two co-founders started deep mind uh which was this sort of research uh lab uh based in the UK that was doing all kinds of research into things like reinforcement learning and eventually they sold deep mind to Google for $650 million in 2014 and for much of the next decade they kind of existed as this quasi uh Elite research team within Google's AI division um they did a bunch of stuff that you and I have talked about on the show including Alpha go which was uh teaching an AI to play the ancient board game go at a superhuman level that was a very big deal when it came out in the world of AI and then they came out with Alpha fold which basically took some of the same AI principles and techniques that they had used to teach an AI to play go and used it to solve what is known as the protein folding problem basically this problem from biology where you needed to be able to predict the 3D structures of different kinds of proteins this was something that had given biologists a lot of trouble for many decades and deep mind and their Alpha fold project were able to essentially solve this entire problem uh more or less overnight yeah I mean back in those days when you would ask Kevin and I to predict a protein structure we'd just give you a Blank Stare we had no idea how to do it and then Along Comes Alpha fold all of a sudden we're cooking but anyways you know Demus has a fascinating backstory and if you're curious about how he got into uh artificial general intelligence our friend Ezra Klein did a great podcast on the subject with Demis last year so check that out if you want to know more about deus's past what we want to talk to him about today was what he is working on now yeah so last year after chat GPT had come out and Google had kind of like rushed to catch up it had released this thing barred which you know was supposed to be the chat GPT competitor um they also did a big reorg inside their AI division so Google had historically had two teams that were sort of working on cutting edge AI Google brain and deep mind and last year they merged those two into Google Deep Mind and they made Demis hassabis the head of all of it yeah there is no more Google brain if you're look if you're on the Google campus looking for Google brain it's not there anymore yeah it has lost its brain much like you yeah we're going to talk about Gemini and some of the research that went into it and what makes it different than or similar to other AI products on the market but I thought today we should really focus on what Demis sees when he looks out the windshield rather than the rearview mirror like where he thinks AI is now both at Google with Gemini and in the World At Large and what he thinks is coming next and the way we pitched this interview Tim by the way was deep mine meets shallow mines Demis aabis Welcome to Hard Fork thanks for having me so Demis over the past couple of weeks we have seen a bunch of new models in addition to Gemini 1.

5 Pro there are now two models called Gemma what the heck is going on over there well we've been busy busy cooking things very very busy I'm not sure we're going to have a release every week but uh it's what it seems like at the moment so let me just unpack that for you so obviously there's our Gemini models our main models um Gemini 1.0 you know launched the Gemini ERA last December and then of course last uh last week we announced 1.

5 so the new generation of Gemini uh and then finally um we have Gemma which we announced today which is the uh open source you know lightweight open Source best-in-class models for open models open weight models yeah I I think some people may still be catching up to why there are so many different models often when we read about AI I think we have a tendency to think about AI as one thing that is just sort of gradually getting a little bit better can you talk about why um Google and Deep Mind have been working on so many different models simultaneously yeah because I mean we've always had um you know the core of of our sort of groups have always been foundational research so we have a ton of uh uh uh fundamental research going on uh into all sorts of different Innovations all sorts of different directions and that means at all times there are the main tracks of the models we're building uh the kind of the core Gemini models but there are also many uh more exploratory projects going on and then um when those exploratory projects yield some results we fuse that into the main branch right into the next versions of Gemini and that's why you're seeing things like 1.

5 coming you know so hot on the heels of 1.0 right because and and we're already working on you know the next version because uh we have multiple teams each working at different time scales um sort of cycling uh uh with each other and that's how you get kind of Relentless progress and um I hope that's you know this is sort of the new normal for us really is um shipping at this high velocity of course whilst um still being you know very responsible and and and keeping safety in mind I want to ask about uh the the latest uh big release that you had which was Gemini 1.

5 Pro um the feature of that model that the people I follow and talk to are most excited about is a very long context window um the the sort of previous uh longest context window I had ever heard of was uh from Claude anthropics model which could handle uh up to 200,000 tokens essentially 200,000 words or fragments of words your new Gemini Pro 1.5 model can handle up to a million tokens so five times bigger um can you just explain what that means and why that's a big deal yeah it's super important that long context you can think of as the model's working memory you know how much uh uh data can it sort of keep in mind at at once and and process over um and you know the longer you have that and the more ACC accurate it is as well is also quite important the Precision of recalling things from that long context uh the the larger amounts of data and context you can take into account so um you know a million uh means that you can do you know massive uh books um entire like films lots of audio things where you know entire code bases so if you have a much shorter context window you know 100,000 that kind of level only uh then you can only have Snippets of that and you sort of can't reason have the model reasoning over or retrieving over the entire uh Corpus that you're interested in so it actually frees up um you know it actually allows uh for all sorts of new use cases uh that you can't do with the smaller contexts uh and actually we've tested it up to 10 million uh uh uh tokens so in in research tests one thing I've heard from AI researchers is that the problem with these big context Windows is that they're very computationally expensive like if you're uploading an entire movie or a biology textbook or something and you're asking questions about it it just takes a lot more processing power to uh to to go through all of that and respond and if a lot of people are doing that you the costs start adding up pretty quickly so did did Google Deep Mind come up with some clever Innovation to sort of make these huge context Windows more efficient or is Google just kind of bearing the cost of all of that additional computation yeah no it's a it's a new totally new innovation because you can't you can't have context that long without some new Innovations on top and um and and but it's still computationally quite expensive so we're working hard on optimizations the initial processing of the uploaded data takes you know couple can take a couple of minutes if you're using the whole context window but if you think about that that's like you know watching the whole film or or reading the entire war in peace in in a in a minute or two that's not too bad to then be able to answer any question about it and then what we want to make sure is that once you've uploaded it and it's processed you know it's it's sort of read the document or processed the video or audio then the subsequent questions and answering of those questions should be faster uh and that's what we're working on at the moment is optimizing that and we're we're very confident we can get that down to you know order of a few seconds and you said you've been testing up to 10 million tokens like how how well does that work does that feel like that's pretty close to becoming a reality too yeah it's very very good in our tests you know um it's not really practical to serve yet because because of these uh these um these uh computational costs but it but it it works beautifully in terms of precision of recall and uh what it's able to do yeah I mean chat GPT released this memory feature um last last week or or the week before that's essentially just a tiny scratch pad that can remember maybe a handful of facts about you but man if you were able to create a version of that that has 10 million tokens about you it could know your entire life it might be able to be a really good assistant for you I think a lot of people see all these new AI releases coming out practically every day and it just kind of all blurs together for them like what can this model do that the last one couldn't so I want to ask that question to you about Gemini what is a thing that uh the Gemini can do um that barred or or previous Google language models could not well I think the exciting thing is about about Gemini um and 1.

5 especially is um the multimodal nature of the sort of native multimodal nature of Gemini we built it from the ground up to cope with you know any types of inputs text image code video uh and then if you combine that with the long context I think you're seeing the potential of that like you could imagine you're listening to a whole lecture or you there's a lecture but there's an important concept that you want to know about and you want to just fast forward to that so um another interesting use case is now we can put entire code bases into the context window um it's actually very useful for onboarding new programmers so you come in let's say take the Gemini cbase you know new new engineer started on a Monday normally you'd have to go and talk to search through this you know hundreds of thousands of lines of code how do you sort of access this function and you need to go and ask an expert on the code base but now actually you can use sort of Gemini as a as a coding assistant uh in this interesting way as well like and it will just return you some summaries of where the important parts of the of the code are and just get you started it's just super helpful I think um uh to have those kinds of um capabilities um and makes you know your everyday worklow much more efficient and I'm very excited about seeing how Gemini will work when it's in orporated into things like workspace you know and and your general workflow what is the workflow of the future um I think we've only just barely scratched the surface of that so I want to turn now to Gemma the new family of lightweight open- Source models you just released it seems like uh maybe one of the most controversial subjects in AI today is whether to release foundational models through open source or whether to keep them closed uh to date Google has kept its foundational models closed source um why why go open source now and and what do you make of the criticism that making foundational models available through open source increases uh the risks and and their the ability for them to be used by Bad actors yeah well look I I've actually talked about that a lot publicly myself one of the main worries so in General open source and um open science open research is clearly beneficial right but there is this issue specifically with AGI and AI techn powerful General AI technology because it general purpose you once you put it out there Bad actors can potentially repurpose it for harmful ends but of course once you open source something you you have no real recourse to pull it back anymore right unlike an API access or something like that where look it turns out um Downstream there was this harmful use case no no one had considered before you can just cut that cut that access off I think that means that the threshold of for safety and um uh uh sort of robustness and um and responsibility for what putting those those kinds of things out has to be even higher and and my my view is that as we get closer to AGI then they'll have more more powerful capabilities so one must be more and more careful about what they could be repurposed for in the hands of a bad actor right and I and I haven't heard a good argument from let's say the open source Max Mist who are many of them out there some of who are my respected colleagues in Academia and um what is the answer to that question of proliferation and bad bad bad actor uh access and we and we you know we got to think about that more as these these systems get more and more powerful so so why was Gemma not one of those things that that created that concern for you yeah well of course because Gemma uh as you as you'll notice only comes you know it comes in lightweight flavors right so they're relatively small and actually the smaller sizes are more useful for developers because it's generally individual developers and academics that or small groups that want to like have things working very fast on their laptops and so on so it's sort of optimized for that and because they're not Frontier models right they're they're small uh Scale Models um we you know we feel comfortable that they've you know that the the capability been very stress tested we know very well what they're capable of and there aren't you know big risks associated with models of that size I want to ask about a subject that we've talked about on the show recently which is personality in Ai chatbots and how much personality chatbots should have or be allowed to have by their creators some models including the original you know Bing Sydney have been criticized for having too much personality for being creepy or threatening or or harassing users um other models have been criticized for being too boring and sort of giving you know trit answers and and not being very helpful so how did you calibrate the personality uh so to speak of Gemini and and where would you say it falls on that Spectrum yeah look it's it's a very interesting question I think a live ongoing debate in the whole industry and field my guess is that ultimately you're going to want personalization is going to be the answer here where people will will individually decide there's some base model you know with Bas behavior and then you uh you know would opt in and you have a kind of personal assistant that you want to behave in a certain way and I'm guessing that is what will actually ultimately happen because the problem with a general one is you can't satisfy all constraints right you know sometimes you know I want my I want Gemini to be very succinct right and just give me the bullet points give me the facts other times you want it to be uh very discoursive and and creative and at the moment I think we're still quite nent and we're still working on these based generic models well Kevin won't rest until you've given him a personal assistant who is completely insane so I look forward to seeing if you can satisfy that that condition um I I want to ask another sort of question in this vein around you know personality and and how the prompts respond to us there's been some people noticing online this week that Gemini doesn't seem to create a white man if you ask it to or if you try to depict uh figures from history it um it sort of doesn't want want to do that or we'll sort of do it in a historically inaccurate way I understand all the the sensitivities around this but I'm curious what you make of that criticism and how you're trying to balance uh sort of you know not doing something deeply offensive with also doing stuff that is historically accurate yeah look actually we just became aware just became aware of that uh you know yesterday when it started popping up on on social media and um you know I think with this this is a good example of these nuances right the historical accuracy absolutely we want that um so we need to fix that versus when you have a generic prompt obviously then there things are Universal so for example if you said you know get draw me create a picture of a of a person walking a dog uh or a picture of a nurse in a hospital or something like that you'd want a universal uh uh you know depiction um but then others you know historical events or historical figures then perhaps that should be narrower so I think that's an interesting you know piece of feedback and and and um this is why we also have to put some things tested out in the world is something that um becomes obvious actually once you have it tested out in the wild and uh yeah we you know we'll be we'll be looking at that and and and as I said we're continually improving our models um uh based on feedback deis I want to shift Focus away from Gemini and sort of broaden our view a little bit here um you are less than a year into your big new job as the CEO of the combined um former Google brain uh research lab that existed inside Google and deep mind which was the the company that you and your co-founders started uh all those years ago and last year when Google brain and Deep Mind were combined um some folks I know in the AI industry were concerned they worried that you know Google had historically given Deep Mind this pretty Long Leash to work on whatever kinds of research projects it deemed important um and that as these units got combined deep mind's priorities were going to be shifted toward the things that were sort of good and useful for Google in the short term rather than these kind of longer term uh you know foundational research projects um it's been almost a year since that uh since those units were combined um has that tension between sort of short-term benefit to Google and maybe long-term uh AI progress changed what you can work on at all yeah well I I would actually say that this sort of first year you're you're you're alluding to has gone fantastically well um you know one reason that we felt it was the right time to do that and I did from a from a researcher point of view is that um maybe let's let's wind back five years or six years back right when we were doing things like alpago you know we were very exploratory in in in AI as a field in terms of like what was the right way to get to AGI what breakthroughs were needed what sorts of things should be bet on and in that situation you know you want to do a broad portfolio of things right and and um so I think that was very exploratory phase I think in the last two three years it's become clear what are the some of the main components are going to be as I mentioned earlier we're still going to need new Innovations I think and you've just seen one with our 1.

5 with the uh models with with the long context I think there are lots of uh new innov like that that are going to be required so foundational research is I I would say still as important as ever um but there's also this big engineering track now uh of of um ex sort of scaling and exploiting known techniques right and pushing them to the Limit and there incredibly creative engineering at scale that has to be done there um all the way from the bare metal hardware work you know up to the data center size and and the efficiencies of all of that and one reason why that that that the timing is right now now is that if we were to say to me make some AI Power Products five six years ago we would have had to build quite different AI to the let's call it the AGI research track that the track of researching General AI techniques that will one day be useful for AGI right versus doing something special cased uh for a particular product um that would have needed a kind of bespoke AI handcrafted Ai and and so that effectively there would have been two different types of things you would have to do that's not true today the to do to do AI for products actually the best way to do that now is to use the general AI techniques and systems because they've got to a level of sophistication and capability where they're actually better now than doing any special case type of hardcoded type of approach so in fact that's converged so what you can see today is that the research tracks and the product tracks have converged right uh and so now there's no you know I don't have to have a split brain on like oh I'm working on products over here and so I have to do this type of AI and then I'm I'm you know like a like a handcrafted assistant right Siri like assistant versus oh a true chatbot that understands language that they're one and the same now right and so then so that's so first of all so there's no kind of dichotomy there or tension there the second thing is if that's true it's actually really good for research to have um tight feedback loops with grounded real applications because that is the way you really understand how your models are doing right you can have metrics academic metrics coming out of your ears but you you you the real test is when millions of users use your product and do they find it useful do they find it helpful is it beneficial to the world and you get obviously a ton of feedback that way and um and then that leads to very rapid improvements in the underlying models um and so that phase that I think we we're right in the middle of now is very very exciting yeah so in in San Francisco the mood around AI is very optimistic it's clearly very optimistic inside Google um but it's more pessimistic other places uh Pew Research Center that a survey last year found 52% of Americans said they feel more concerned than excited about the increase use of AI only 10% are more excited than concern what do you think explains that downturn in in public sentiment and what do you think you can do about it I think you know I don't know what to make necessarily of those kinds of I think if it depends how exactly the question you ask it if you ask in a very naive way I think people are always worried about change right or disruption and clearly AI is going to bring enormous change I've always believed that's why I worked my whole life my whole career on this 20 plus years I think the world is realizing what I felt people like myself and other researchers have been in for a long time have known for decades now like if this was to work it would be you know the most Monumental thing ever right so um so I think people it's Dawning on people um but they don't know quite they haven't interacted it with many different ways and so it's sort of very new and I think what we need to do actually as a field is present concrete use cases that are clearly incredibly beneficial right and for one of the things we've done historically I think like that I'd point to is Alpha fold but the average person in the street probably doesn't know about that yet what that impact will be but they will do if that leads to AI designed drugs and cures for really terrible disease right and I think we're only just a few years away from that right obviously we spun out isomorphic Labs assisted company of Google de M an alphabet company U which I also run which is to F take the alpha fold Technologies move them into chemistry and biochemistry to actually design drugs to buy to the right parts of the protein structures that of course Alpha fold has predicted and um and then you know I I we we we've just signed big deals with with big pharmer and on real drug programs and I expect in the next couple of years will have um AI design drugs in the clinic in clinical testing and that that's going to be an amazing time and that's when people will start to really feel the benefits in their in their daily lives in in really material and incredible ways yeah I I agree I mean Alpha fold is the thing that I hear Far and Away the most when it comes to sort of like the the best possible uses of AI technology U but it was also a somewhat unusual problem because uh it was sort of the right kind of problem for AI to solve it had these you know huge data sets and a bunch of different sort of solved examples that the model could use to sort of learn what a correctly shaped protein should look like uh that's the kind of thing that you can throw at a machine learning algorithm and it can do it quite well do you think there are other kind of similar shape problems out there um or have most of the sort of lwh hanging fruit already been picked no I think there's many problems of that type so you know what way the way I normally describe it and obviously protein folding is archetypal example of this is you know imagine any problem in science which is which is basically has a huge combinatorial search space huge number of possibilities way more than you could search by Brute Force right so you know um let's take chemistry space the space of possible compounds some people estimate that's 10 to the^ 50 in terms of the possible compounds one could create right so truly enormous number of of possibilities um so intractable to do that by hand but what you do is if can build a model of chemistry right that understands what's sort of feasible in chemistry you could use that to do a search where you search but you don't search every possibility you search just a tiny fraction of the possibilities that make the models telling you are kind of the highest value and I think there are a lot of things in science that fit that and and and I you know I give examples so protein folding is one but I think finding a a drug compound that has no side effects but binds exactly to the thing that you want in in you know on your protein or on the bacteria that's another one I think um finding new materials like I dream of a room temperature superconductor you know that's cheap to make right so so so so that's that's one of the things I'd love to turn our systems to or the ultimate you know optimal battery design and um I think all of those things can be can be re uh uh uh reimagined in a way where uh these types of tools and these types of methods will be uh very productive do you think we're close to seeing AI being able to cure a major disease like an Alzheimer's or a cancer I think we are very close I would say um you know we're a couple of years away from having the first AI design truly AI designed drugs um for major for a major disease cardiovascular cancer we're working on all of those things are isomorphic and um and then obviously there's still the clinical trials and that stuff has to happen and right now that would be the bottom neck but I think certainly getting it into the clinic the discovery phrase I would like to you know shrink that from years to months maybe even weeks at some point so I think in a couple is we you know I would be disappointed if we don't have some uh great candidates for drugs for very important diseases uh you know starting to go through clinical trials I want to uh shift our conversation a little bit more toward the long-term future of AI um and a term that has already come up a couple times in this conversation is Agi artificial general intelligence uh which is a term that gets thrown around a lot these days without a lot of specificity um so I thought we should just start by asking you in one sentence what does AGI mean to you well AGI is Means A A system that is generally capable so out of the box it should be able to do pretty much any uh cognitive task that humans can do this might be a stupid question but when AGI arrives assuming it does how will we know how will we recognize it like one of your engineers presumably if this all goes according to your plan will show up in your office one day and say Demis I've got this thing I think it's AGI how do you test that is there one sort of battery of tests you could put it through that would convince you like this is Agi is there one question you would ask it to determine whether it was truly AGI or not just how will we know when this thing shows up yeah well actually one of my co-founders Shane leg you know he did his whole PhD on the testing of and measuring of of of of these systems and I think the best uh IDE because it's so General that actually makes it quite difficult to test right you can't test it in one particular Dimension I think it's going to have to be a battery of thousands of tests um and Performing well across the the board you know covering all of the different spaces of things that uh we know uh the human brain can do and by by the way the only reason that's that's an important obviously Anchor Point is the human brain is the only existence proof we have in the universe as far as we know of general intelligence being possible so that's why I originally studied Neuroscience as well as computer science because clearly certainly in the early days of AI uh it was important to get Neuroscience inspiration too for how how are these you know intelligent phenomena how do they come about what do they look like and therefore what does these systems what do these systems need to be able to do in order to um to exhibit uh signs of general intelligence right and I think we're still quite a long way off of that actually with the current systems right there's a lot of things you know all of us who've interacted with it can see all the flaws in the systems even though they're impressive in many ways they're also um not very good in in many ways still so there's still a long way to go and as I said earlier a lot of breakthroughs still needed and in your best guess how far are we away from that kind of AGI well look I think uh we're making enormous progress as a field we're making enormous progress with Gemini and those types of systems which I think will be important components of an AGI system um probably not enough on their own but but certainly a key component um and I would not be surprised if we saw systems nearing that kind of capability within the next decade or sooner have your timelines shifted at all over the past year or two as things like language models have gotten out into the public it's funny actually cuz I was looking at our original business plan we wrote back in 2010 when we started Deep Mind and uh we we had all sorts of predictions in that business plan including compute and other things and other inventions that would be needed and we we stated in their 20-year time scale and I think I think we're actually pretty on track that gives us six more years if if I'm counting correctly roughly roughly speaking but that's not that's that's compatible with you know I wouldn't be surprised within the next decade that doesn't mean it's going to happen so I just wouldn't be surprised so you can sort of infer some probability Mass based on that but so you know I think there's a lot of uncertainty because you don't know if the current techniques are going to hit a brick wall if they do then you know then you would have to kind of invent some Nobel Prize level Innovation to get through that brick wall right right now we don't see one but I I you know and ones have been conjectured in the past and some of my colleagues in the field conjecture uh that there could be some brick walls um but I think it's an empirical question actually I think we've got to that's why we do push really hard on both things we want to scale the current ideas and um knowhow and techniques to the maximum and we want to double down on our foundational research and Innovative research and exploratory research to find improvements to the existing ideas and also think through like what could be the brick walls and what if they are end up being brick walls with the with the with the scaling system then what would be the answer right so hopefully we have the answer at the point where we hit a brick wall we already have some ideas of like how to get around it right that would be the ideal if if indeed there turns out to be a brick wall because there may there may not be do you think the world is ready for something like AGI to show up like if we only have six years to prepare for a computer that can do everyone's job like what what should we be doing now to to get ready for that well look I think the debates are happening so I think the the Silver Lining with I guess this craziness of the last couple of years on AI is that everyone's talking about it I think chatbots have been useful in that sensing that the average person uh can interact with a cutting AED AI in a way that's easy to understand right you know Alpha fold you need to be an expert in biology or proteins or medical research to really get what it is so you know language is different right we all we all we all use language every day and it's an easy thing for everyone to understand so I think those you know it's good that those debates are happening it's something that's going to affect everyone in society I think there are questions on International cooperation I would like to see a lot more of that unfortunately the geopolitical IC nature of the world right now is not very conducive to that so that's unfortunate timing um because I think some kind of international collaboration would be very uh important is going to be in my opinion very important here which is why I was you know I was pleased to see um how many International leaders engaged with the the the The Summit in the UK you know back in in in last Autumn um and then we need to of course accelerate our um Research into safety uh guard rails control mechanisms uh and I think actually we need to sort of um do more uh work a lot more work in that direction and also philosophy too like what do we want from our systems kind of and and ethics right and philosophy of like it's kind of deep philosophical issues like what do we want our systems to do how what values should they have it look talk a little bit imping it kind of intersects with the earlier discussion we had about personas you know it's actually comes down to values right what do you want your systems to represent uh and of course um uh it's important who makes those systems because and what's what cultural and societal background they're in uh and and Western systems and China's Building Systems I mean there's a lot of complications here what what role do you see humans playing in a world where AGI exists and can just sort of run everything on our behalf well I think um this is going to happen in many stages I think initially I'm seeing AI uh and and the next versions is these incredible assistive tools that's how I think we should design and make so there's sort of this debate about tools versus creatures you sometimes hear and I think that we should be firmly I'm in the account we should be making tools uh to assist uh you know human experts and and and so whether they're scientists or Medics or whatever it is to free them up to do the higher level uh conceptual work right so um you know today our systems maybe they can help you with data crunching or some sort of analysis of a medical image but they you know they're not good enough yet to do the diagnosis themselves in my opinion or to trust them with that there should be an expert human in the loop and I see that as the next phase and for however many you know uh years or decades that will be um and then maybe we'll understand these systems better in the course of doing that and we'll be able to figure out like what to build with them next right to to to allow them to go to the next stage uh and I think Society will have to adapt about what it is um that that uh uh um you know we want to do in a society where we have um AI systems they're able to do very useful things for us maybe we have abundance because of that because we crack things like energy problems things like physics and material design so there should be a huge plethora of amazing um benefits that we just have to make sure are kind of equally distributed you know so everyone in society gets the benefit of that um and then you know I think incredible things might be possible that sort of written in science fiction books books like the culture series by in Banks and so on it's always been my favorite since I was a teenager of a depiction of um you know maximum human flourishing across the cosmos uh you know helped by AI systems uh solving a lot of these these fundamental problems for us helping us solve a lot of these problems I think it could be an amazing amazing future um and with incredibly you know big challenges that are facing us today as Society climate uh disease poverty a lot of these things uh water access you know could be helped by um innovations that um uh you know come about through the use of these AI tools so that's the positive side of the the sort of approach of of AGI there's also a you know a side that worries a lot of people including AI safety people uh risk people you yourself have worried about existential threats to uh Humanity that could result from very powerful uh AI systems um I'm gonna ask you a question that we've asked a lot of people on this show which is what is your P Doom yeah I know that's people are fixat with that do you know my honest answer to these things first of all um I actually find a lot of the debate on the social media sphere a little bit ridiculous in this sense you know you can find people on both sides of the argument very eminent people you know just take like Jeff Hinton versus Yan Lon right I mean both are chewing Award winners I know them both very well yosha Benjo you know these are the some of the top uh uh uh uh uh people who originally were in the field and um you know the fact they can be completely in opposite camps to me suggests that actually the the we don't know right with this transformative technology it's so transformative um it's unknown so I I I don't think anyone can precisely I think it's kind of a nonsense to precisely put a probability on it um what I do know is it's nonzero that risk right it's Al so it's it's definitely worth debating and it's worth researching really carefully because um even if that probability turns out to be very small right let's say on the on The Optimist end of the scale then we want to still be prepared for that we don't want to know have to wait till the eve before AGI happens and go you know what maybe we should have thought about this a bit harder okay you no we should be preparing for that now right and and trying to ascertain more more accurately what the risk really is right what is that risk how would we mitigate it what risks are we worried about is it self-improvement is it uh controlability is it the value systems is it the goal specification you know all of these things are research questions and I think they're empirical questions so it's unlike a natural science like chemistry physics and biology the phenomena you're studying is already out there exists in nature so you go out there and you study and and you sort of try to take apart and and and deconstruct what's going on but with the engineering science the difference is you have to create the artifact of of worthy of study first and then you can deconstruct it and and only very recently I would say do we have ai systems that are even sort of interesting enough to be worthy of study but we have them now things like Gemini Alpha fold and so on and we should be um doubling down and we are obviously as as as as Google deep mine but the the field should be doubling down on analysis techniques and and figuring out uh understanding of these systems um way ahead of where you know we're on the kind of cusp of AI and that isn't a lot of time because if we're you know less than a decade away these problems are very hard research problems they're just they're probably harder or as hard as as the breakthroughs required to build the systems in the first place so we need to be working now yesterday on those problems no matter what the the the the probability is because it's not it's definitely uh nonzero right so I don't agree with the people that say there's nothing to see here I think that's ridiculous how on what basis are they making that that that assumption right just like in the past 10 15 years ago when we started out well I remember I was doing my postdoc in MIT and that was the home at the time of traditional AI methods logic systems and so on I won't name the professors but some of the big professors there were like learning systems these deep learning reinforc learning they'll never work you know 100 300 years for sure never work and I was just like how can you put z% on something in 300 years 300 years think 300 years back what happened what we've what society's done like I mean that's just not a scientific statement to say 0% we don't even understand the laws of physics well enough to say things are 0% let alone you know techn so it's clearly nonzero it's massively transformative we all agree hugely Monumental like impact hopefully for good obviously that's why we're working on it and I've worked my whole life on it we just talked about that science medicine Etc human flourishing but we got to make sure it goes well so if it's nonzero we should be investigating that empirically and doing everything we can to understand it better and actually be more precise then in future maybe in five years time i' better give you I would hope to better give you a much more precise answer with evidence to back it up rather than you know slanging matches on Twitter which I don't think are very useful to be honest right right you know I have a a shorter or more medium-term fear which is that you know before AGI gets to the point of enslaving us or whatever it just massively concentrates power and wealth in the hands of a very few companies and it and it doesn't feel like the benefits are evenly distributed so I wonder if you feel like in your role you are in a position to make sure that those benefits are more broadly distributed or if that that risk of this stuff just really concentrating a lot of power and money is is real look I think um there's there's there's several sort of nuanced answers to that right it's a complex question I think that right now a lot of resources are required to build the most Cutting Edge models but you're already seeing like open source systems you know including Gemma our contribution to that today as well is um are are are getting pretty powerful so for a lot of everyday use cases you you know that that might be already plenty good enough right for a particular product or an application or so on and I think the the the developer Community is going to create amazing things with these um uh uh models um even the proprietary ones have API access to them like Gemini you know 1.

5 is coming 1.0 is already out including Ultra so you can build on top of that Enterprise customers and so on so that's all all happening um there are multiple providers of these models right there is just one company there's there's several so they'll all be competing on price I me you're already seeing price of tokens is going to like you know is is is is is going down you know discounted every every every day it seems so you know I think uh I think all of that is good for the consumer good for everyday users and good for you know companies and others Enterprises that are building on this um and then ultimately this is the funny thing is um so I would couch myself as uh a cautious Optimist right but the thing is and I think that's the correct approach when you're talking about something as as transformative as AI right and and I've thought about it for many many you know decades right and the funny thing I see of some of the more I would say techno Optimist I think they sometimes call themselves you know on the Twitter crowd is I actually don't think they fully understand the Monumental of what is being built right because if they did I think it cautious optimism is the right is the right is the only reasonable approach I would say in the face of quite a lot of uncertainty obviously a lot of uh obvious amazing things that could happen which you know curing diseases Etc um and and but uh but uncertainty over how the technolog is going to develop for something that transformative then cautious optimism I think is the only reasonable approach um and and and so yeah I think I think there's going to be incredible things and I think one of those things is going to be I'm not even sure if you imagine a world where AGI arrived and it's solved a lot helped us solve a lot of big scientific problems I sometimes call them root node problems so if you think of a tree of knowledge and one of the core big problems that you want to unlock that unlock many new branches of research and I think Alpha fold again is one of those root note problems um you imagine you crack fusion with it or you know room temperature superconductors and you know batteries that are optimal um that opens up you know suddenly energy becomes free or cheap then that has huge consequences on you know resources like be you freeing up like you could do more space travel M asteroids maybe becomes feasible all of these things right and then suddenly um the nature of of of money even changes right so I'm not sure people are really understanding like I don't know if company constructs would even be the right thing to think about at that point I think you know again this is where International collaboration between governments and other things may be required right to make sure that it's um these systems are well if there's multiple ones are you know managed in the right way and and used for the benefit of everyone yeah um a lot of AI Labs have been grappling with governance and what is the best structure for something like AGI to emerge you just mentioned the possibility of some sort of international Collective or cooperative that would handle this but um you know across the industry like open AI has set itself up as a nonprofit with a for-profit subsidiary anthropic is a public benefit Corporation um you're making a a slightly different bet which is to try to get to AGI inside Google which is a big for-profit company um that has a fiduciary duty to make money for its shareholders does that worry you and as we get closer to AGI do you think Google will have to change its corporate structure somehow in order to prevent uh some of the bad outcomes that people at other AI labs are worried about look I I feel like um uh the current construct is is is is is um is uh good for where we are right now with the technology so you know um for example the one reason we teamed up with Google back in 2014 was Google's a kind of came out of a research project that Larry and Sergey were doing at their phds right so I felt they were already very scientific of all the big companies in their approach right so it's very good match for you know how I was running Deep Mind and how we run Google deep mind it's scientific approach scientific method that's the best method we've ever invented for understanding things in the world unbelievably powerful from the en days of the Enlightenment right that's what's C created the modern world and all the benefits of the modern world and um so we got to double down on that method and trust that method that's that would be my Approach versus alternative methods which are very effective as well but I think less correct for this type of Technology like a hacker growth mentality move fast and break things you know you sometimes hear as the valley Mantra obviously creative phenomenal products and and and advancers but I think not appropriate for uh the the type of monumental technology we're talking about with AI right there I think this the scientific method is is the better approach and I think Google um is the most scientific of the big companies I would say always had that in its DNA and we've tried to bring that more into that and also receptive to um these ethical concerns from the beginning one of the reasons we team up with them is we had our own ethics Charter as deep mind we had it from the beginning and you now see that embodied in the Google AI principles you know Google was the first of the big companies to put out their AI principles you know it's it's out there publicly and they were sort of evolved from Deep mind's original ones we've being incredibly responsible I think and thoughtful about uh about the um the way we're deploying these Technologies and how we're building it and and hopefully you can see that in the approach we've taken and sometimes that means we take a little bit longer with things before we put them out because we are trying to fully make sure we understand uh to the you know to the extent doesn't mean we won't ever make mistakes because this is new technologies and sometimes as we talked earlier you need the direct feedback it's useful from from users and from experimental releasers and that's why we do stage release right like with 1.

5 now Pro it's in experimental release so we can get early feedback um but look I think that's the that's the right approach and I I think uh I'm very comfortable with where we are now in five 10 years as we get closer to AI we'll have to see you know how the technology develops and also how um what state the world is in at that point and the institutions in the world like the UN and so on um which we engage with you know a lot and I think um you know we need to see how that goes and the how the engagement goes over the next few years I wonder if you have a thought on what the most AI proof job in the world is right now is there anything you see out there where you think yeah that's a safe bet for the next five or 10 years well look I I actually think that what's going to be you know and and and um you can just say podcast host if you want that would make us feel better podcast host obviously but but I I I I I uh I think um actually you know a lot of the jobs where and I talked to a lot of my creative friends in Creative industry like film music and games and stuff I used to do games design myself back in the day early in my career you know I think what what there are certain types of creatives who are also really love technology as well as the creative process and I think they're going to be like super powered up effectively by using their creativity on top of these tools you know whatever these generative AI tools do they still need the creative input to make them do interesting valuable things right otherwise they're just sort of uh uh doing fairly mundane things uh in the with the average user and I think that um there's it could be incredibly uh um incredible multiplier for those types of creatives you know I have a friend of mine who's uh a sort of film producer and um that in for indie films and and they're creating entire fully-fledged pitch decks now to get their fundraising at can's Film Festival whatever it is where before they would have had to they would have had to have like you know just a couple of little uh uh pieces of artwork and then the the the the funders would couldn't imagine what this film would be like right but now they can really go to town and kind of showing you what the feel look and feel be like and so on and it's just means that the whole process is accelerated for them in terms of actually getting to the Film Production um and then stuff on the science I dream of a science assistant that can just summarize a whole field area for you or here's a bunch of tell me what the best reviews are and the counterpoints you know obviously we need to fix factuality and other things a lot better before we have that and we're working on that but that would be incredible for like then giving me the information where I could then make a new connection or new hesis uh to then go and test out right um or to help a doctor on a complex diagnosis you know they doctors unbelievably busy can they keep up with the latest literature as well The Cutting Edge of of research you know a sort of science assistant or medical assistant tool could help them do that while they're treating patients you know uh 247 right so I just feel like there's a lot of things like that and then you know I think a lot of jobs that are may be somewhat undervalued today manual jobs you know manual labor jobs things like that I think or or caring jobs where you really want the human emotional empathy and touch I think are going to be much more valued in future and that's maybe a good thing right perhaps they're being undervalued today uh in our capitalist Society uh all right last question Demis um what is your personal plan for the Post AGI world you know when the AI come and they don't need humans to you know run companies or go on podcasts anymore or how will you spend uh all your copious free time are you going to apply for a job as a a plumber or a gardener or what what does what does the world look like for Demus in the post AGI world so so what I've always wanted to use my AGI tools for would be to really understand the deepest questions of of of of Nature and physics so the fundamental nature of reality I'd like to have the time to ponder that think that through perhaps traveling on a Starship to you know Alpha centur um thinking about that meditating on these on these ideas maybe doing some extreme sports um you know stuff like that I think there'll be plenty of very exciting things for us to do we're just going to have to be very creative about it and as I said there's many many amazing science fiction books that positive ones uh that talk about uh what such worlds might look like and I think they're very exciting if we if we get it right you mentioned you wanted to do some extreme sports in the post AGI world what extreme sports are you interested in uh well look I haven't done very many extreme sports today because you know have to be careful right to keep healthy and fit but um you know we we we we maybe there'll be some new ones we're able to do you know mountain climbing up Mount Olympus on Mars that might be quite fun hiking up there I think that's a good place to leave it deis habus thank you for coming on her fork thank you thanks so much for having me it's been great fun did you see that chat GPT apparently had a nervous breakdown last night no what happened so I was just following this on social media but all of a sudden it just started like spitting out nonsense uh sometimes it would uh start just speaking Spanish Hi Kuma and no one could quite explain why I think it's back to normal uh but it did appear that chat GPT for a night just went crazy um my favorite response was there was a user who was talking with chat GPT and it said are you having a stroke some of what you're saying makes no sense or aren't proper words and Chad GPT says whoops I really apologize if my last response came through as un unclear or Cente like it drifted into some nonsensical wording sometimes in the creative process of keeping the inner time Spanglish vibrant the cogs and latla might get a bit Whimsical mucha scr for your understanding I'll ensure we're being as Crystal Clear KO low from now on winky face then it said would it glad your click Le to Grape turn tooth over a mind ocean Jello type or submarine else KK Sierra's kid dive into please share with their fourth KO desire period uh yeah chbt really snapped with that one you know what Kevin for so long now you've been asking for a chat but that has personality well congratulations my friend finally gotten one it's completely insane and it's speak and Spanglish and it's available for $20 a month head on over to open.