welcome to Google Deep Mind to the podcast I'm Professor Hannah fry right agents they are here or almost here and they are probably all anyone is going to be talking about in 2025 but they are definitely not new my guest on today's episode is someone who last came on the podcast in 2019 to talk to me about the multi-agent system he was working on it could beat the professional Starcraft players at their own game and eventually went on to achieve Grandmaster status but how have agents evolved since then what can they do now how have the advances in language models and multimodal AI changed things and how do you possibly go about building something that can make autonomous decisions on behalf of its user now now I should tell you if you want a primer on agents you can watch our episode with Frederick best that we recorded over the summer but for now Oriel vanal is Vice President of drastic research and C- Tech lead of Gemini and it's fair to say we've got quite a lot to catch up on Oriel welcome back to the podcast hi thank you for having me what is drastic research well I keep telling my team they have to think drastic meaning don't just do the incremental stuff uh that everyone is thinking about try to to drastically think what will happen in a few years time and then try to backport those ideas and then execute today with that mindset in mind so that's what drastic means but yeah it's a word that I use a lot I think when I last got to see you you had been working on an agent that could use a keyboard and a mouse uh to do things like draw pictures in paint or uh play Starcraft and um well things have moved on quite a bit since then so those agents at the time you took sort of a very generic set of principles very simple principles um in the field of machine learning and you would basically specialize uh a model on one task and what we were were doing at the time is have a curriculum of tasks that were more and more and more difficult right so when we last spoke for instance in video games we were looking at Starcraft which is one of the most complex modern you know strategy games uh out there and of course de mine is notorious for having started the trend with Atari which is a fairly simple game of you know left right hit the paddle and hit the ball and you know off you go so that sort of is what the algorithms themselves we try to push for them to be very general so we can kep when keep climbing this ladder of difficulty curriculum of games and doing more and more complex things and right now what has happened is even the models we train are broadly applicable to many more things than the models we developed back then were right so think about the process of creating this digital brain hasn't changed that much but what that brain was able to do was reasonably narrow although very complex like playing Starcraft or playing go uh right now this models can do quite a lot more um broad uh applications um and of course talking to us chatbots etc etc so so back then like reinforcement learning was your your main kind of lever I guess how different are things now yeah so algorithmically actually the process of um you know alphago and actually alpast star those two had the same set of sequence of algorithms applied to creating this digital brain and it's not actually that different from how current large language models or multimodel models are created today there's two basic steps that have been pretty constant throughout many years um in many of the projects that we've worked on which we can call the first one pre-training or imitation learning that is you start with random weights you have an algorithm that will try to imitate lots of data that humans have created to either play game or in this case all of the internet all of the knowledge available to us and in that first stage you just adapt the weights to try to imitate that data as well as possible and these weights are essentially inside each of the neurons is like a series of numbers that kind of describes how it's connected to everything else yeah so these are basically there are units of computations that are neurons and the connections between neurons are what actually you have as weights so um you can imagine that there's a neuron there's a few neurons connected to it and you're basically adding all the activations from the incoming neurons there multiplied by the weights and those weights are the only things that move and the inputs excite the neurons it's pretty much how you know a brain works with some you know freedom of uh yeah creativity okay if we were to do an analogy um it's almost like you've got you've got the neurons and you're like water is flowing through it and the weight is like the width of the pipes between the neurons yeah that's right and then you can imagine having millions of neurons and billions or even trillions of pipes and that is what we spend most compute actually training these models especially language models is in this pre-training or imitating all the data that we have available to us okay so you've now got this gigantic network with loads of pipes going between all the neurons uh and that's your imitation phase done next bit if you were doing say uh you know Alpha go Alpha zero you would then get it to play itself yeah so this model now is reasonably good at playing moves that look humanlike so that means um of course it's it's the sentences are very plausible sentences in English or if it was playing a game it would sort of Click things reasonably to move you know pieces on the board and whatnot um but what this model hasn't done is learn that these actions yield reward right that's the bit of uh reinforcement learning or post trining which is the second phase of training so you can write a poem by just hey just how does a poem on the internet look like on average but then the question is well I want only the good poems right so how can I further adjust these pipes based on um sort of a signal that now having written now a whole poem would G give a score of Z or one let's say right and if if it's a mediocre poem you get a zero if it's a good poem you get a one again for a game analogy which is what we use reinforcement learning traditionally if you be win at the game you get a one if you lose you get a zero and then you further adjust the weights but now instead of imitating humans you're just saying forget I want to go beyond what humans could do and try to really get all my poems to be the perfect poem right or all my chess games to be the perfect game and in language models this second phase which is reinforcement learning post- trining tends to be fairly shortlived because we do not have access to super clean reward as you know you've won the game or or you lost the game when you do selfplay in traditional board games for example so once that's done right then you've like that's all the stuff that goes on behind the scenes and then you're like hold it right there yeah stay exactly where you are everybody we're going to take just basically a snapshot of this entire network and that is what you actually get to access as as as a user yeah so now this amazing process finished these are these weights are super precious right so this configuration you found um you've really spent months to finesse it to tweak everything and now you you sort of Will Never Move it anymore right so training is over you're not changing the configuration anymore you might want make it super efficient right so say like you find that oh look this neuron is not that useful it's not used for anything you remove it so everything becomes faster and cheaper to run it at scale and then as a user you just get the same weights everyone gets the same weights we've trained that's what we call like you know Gemini 1.5 flash that just means a set of Weights that are frozen will not change will not further train or anything so those two steps actually pretty much are identical from alphao um to alpast Star to like current large language models and of course there's details that matter and the field has evolved certainly but the principle is pretty much unchange actually because because under the hood as it were there are differences between I don't know I'm thinking like dqn here which was the Atari example or or you know the the types of algorithms that were used in in Alpha go or then again in the the large language models like the architecture is different right yeah so there's a few components that go into then what the digital brain is one is the architecture right so there are these neuron networks um now we have the Transformers which we certainly didn't have back in the dq1 days so there's always some sort of breakthroughs in architectures that are better at learning from the data um but then from Transformers to today it's almost all about little tweaks I mean even if you look at Alpha fold which also is fed by a Transformer um what that teams do for ears sometimes is just to find little Twigs on hey let's remove the set of neurons let's um add another layer let's make this a bit wider so you shape the brain shape changes a little bit and that makes it or breaks it sometimes in terms of the performance achieved so if these are all the things that have been achieved so far I mean the the goal as I understand it is to to create more agentic Behavior to kind of get these things to to to make autonomous decisions how did these help to achieve that end yeah so let's zoom in a little bit on the current Trend like we call it large language models but they're multimodal I think we had an episode earlier covering heavily the multimodality aspect how good it is to be able to add an image then ask something a follow-up question and so on so this core we we will still improve it right these these set of Weights that do these amazing sort of um inferences about the input right what's this image about what's the user asking can I write a better poem can I just make it longer whatever like all these interactions we have all like kind of get to play with these days um but this is just a component and we can think hey this is now our CPU and we can add more to it around it what if the model could go off and do research for you could um for example right one example we were already thinking about that back in the day I could ask a Model A language model or visual language model to learn to play the game at of Starcraft that's a very different approach to say create one agent that does play the game in this other example right it could go online watch videos about the game it could uh of course download the game to start interacting with it to learn oh yeah like I know how to you know I get it um do research online go to forums read the forums um go play and and figure out that it's weak at this thing and improve and so on and after literally it could be weeks it sends you an email it says I now know how to play the game Let's Play right that's not a reality that's that far away but these models all of a sudden actually do something take some actions and learn anything new that is available to them and that's pretty powerful to think about it's what pushes the generality the most and that's what makes kind of the AGI as as many people call it feel closer so if I understand it correctly then it's almost like the the stuff that we have at the moment the large language models the multimodal models whatever you want to call them that's like kind of the central core but the next step is that you build stuff on top of that Central core that it can go off and and you know take off the stabilizers and kind of go off and do its own thing yeah exactly if it has access to all the knowledge and it can sort of use its time to do some proper research I mean write hypothesis write some code and so on and take its time to really answer very very very complex questions then the yeah the possibilities now have broaden quite drastically although of course we're not going to need that for everything I mean if we ask a question like hey you know what I like rice I mean what should I prepare tonight probably no need to do a very deep dive into thinking or just going on off for three weeks then you'll probably like not be very happy about the waiting time right um but I think that's kind of for to push the frontier you're giving a digital body to the computer so it can not only just think and give you know an instruction or a word output but it can also go off and do things online or on documents that U you might upload or whatever and ask very very complex questions and personalize to you etc etc I like the idea this Central core and then you're giving it a digital body you've got the electric brain and now you're giving a digital body it's kind of okay so in terms of the electric brain then in terms of this core this processor um let me just ask you a bit about that I I guess we should be thinking um about Gemini here right which is which is essentially what we're talking about the multimodal model that that that you guys have I know that one of the big ideas for large models was just to scale it up right to get them bigger and bigger and bigger and bigger and bigger do you think that the results that we've seen from scaling have sort of plateaued by now yeah it's a very important question right like we've we have studied how as you make the models larger that is how many neurons literal these models have how do they become better at certain you know task that we have clear metrics from the whole machine Learning Community for example one that is very simple to understand is machine translation right so how good how good the models are at translating between two languages as you scale as you go from Millions to billions to potentially trillions of neur neurons um you can see the performance keep improving now even when you do those studies one trick is that you it looks linear but you have to plot logarithmic axis right what that means in late terms is that let's say from the last three years we had some improvement you shouldn't expect the same Improvement in the next three years it's actually exponentially hard to get there right so um that means the investment which of course also advances at a super linear rate but perhaps not as good as this know sort of trend suggest you would you would just see some diminishing returns because simply scaling the x-axis right the number of parameters you need to go 10x to see the same Improvement and that just creates some pressure to hey maybe we can't scale as much and we need to think about other ways to scale to make the models better the examp I give to my students is like if you've got a room that's really messy the first 10 minutes that you spend tidying it's going to make a massive difference you pick up all the DAT plates put away all the dirty washing fine but once you're like seven hours in that 10 minutes extra 10 minutes is it's it's not going to make any difference at all and that's essentially where we are right yeah that's exactly a very good analogy and in fact that analogy can even apply to then the performance of the models even if you have extremely good performance if you want these models to be 100% factual right will never make something up we know that if you probe them you can make them say something that are not real that even that last mile also is super hard uh which creates some interesting challenges to deploy then at scale so okay I hear what you're saying about how there's diminishing returns in all of this right but in terms of how you make these things better how you make these models better is it just data computational power and size are those the only things that the levers that you have to pull yeah so certainly if you froze the architecture let's say for the next year no innovation we just scale because there's better Hardware coming up make it bigger um that certainly would have a trend that would look okay but what's happened and um certainly in Gemini we have other Innovations like other you know tricks techniques um details about how to from how to order the data that you present the model with to the details of the architecture to how to run the training process how long to run it for um what kind of data do we actually present the model how do we filter do we present more data that's high quality less data that's low quality all sorts of different what we call hyper parameters we and and of course other algorithmic advances we also investigate fairly carefully because the process of training model is expensive so we need to be extremely care ful with piling up Innovation so that eventually when we are ready we have enough Innovation and also probably we have a better scale to to run for the next iteration of models we run it and then we get algorithmic not only breakthroughs through data and and compute I guess the other thing about this this scaling stuff is that you've got there's there's no limit really to the number of nodes that you can put in maybe there's sort of no limit in theory to the computational power that you put in but there is a limit to the data that you can put there's a limit to the number of human words that are out there good point so I think I mean there is a limit on the nodes because how you scale these models is well they don't fit on one single chip Hardware chip so now you have a mesh of chips they're communicating you know there's certain limits like speed of light etc etc so there starts to be a time where the efficiency of training such a big model also just not worth it even from a utilization of the hardware at your disposal but very good point right the other bit that is critical on this pre-training imitate all the data is that we do not have what we call infinite data regime um there's finite data and so as soon as the models need to you know you can think well let's train on all the data if you want to everything humans have ever read everything all of the internet so we're just starting to think okay we're running out of data there are techniques like synthetic data can we write or rewrite existing data in many different ways I mean languages would be obvious Ways to Think hey you you could write the internet I mean it's mostly in English I mean 60% I don't know what's the exact percentage but um there are ways to rewrite the same knowledge in different ways we're exploring those that's kind of a research area that many people are starting to invest because if you run out of data the scaling laws punish you even more so for I mean for example then you could get Gemini to write its own version of the internet and then use that to train a new version of Gemini yes is there a danger though that if you start feeding in the output of the same model that you can end up creating these little well unhelpful feedback loops they certainly can you know do some interesting experiments to test ideas like this one you just mentioned and indeed that is not you know on the surface not a good idea like the model um suffers uh if you just ask ask it to recreate all of the internet and indeed a prior from a like information content point of view look this data set has the information that it has how could you create new information right I don't know like these ideas um might help a little bit because there's machine learning deficiencies that are not we're not at that fundamental ability to extract all the information truly from the internet I mean we have good algorithms but they're not perfect so we'll see yeah I mean I guess I just want to think about that a little bit more because it's really interesting idea because of course naively if you did it without thinking then it's like the new version would be sort of have the biases in it the you know and and then the the new version on top of that would be more bias and you'd end up sort of spiraling away from from the original human one but then what you're saying is as though in the original human internet is sort of embedded these conceptual connections and if you can extract those I'm sort of thinking almost like E equals MC squ right if you can sort of find the E equals MC squ for human Concepts and then just generate new data using that alone then that seems more realistic yeah exactly right and that's I think that's where you start hitting um I mean are these language models just repeating what's online and not being able to create anything new or are they learning a world model truly that you can then from the principles it extracts possibly generalize beyond what the data has and under the more optimistic version which I tend to believe more we can push the limits of data a little bit more than the current limits that we have that being said I mean there are some data sources that we haven't quite seen a breakthrough like video data there's a lot of it and we haven't quite seen a moment of take all the video data where you probably can derive a lot of knowledge a lot of lots of physics a lot of how the world works even if there are no words associated with the videos necessarily and extract that knowledge even that I don't think we tapped into that source and it doesn't work that way it I mean right or you don't know yeah I mean it it it feels like it should I mean even how we learn um I mean we learn there's some language learning in the early days but we learn by also observing uh three dimensions and so on so forth right so there probably is more knowledge that we haven't extracted um what obviously we've gotten pretty well at and you can see by testing the models is connecting the concepts present in the video by and then you can do amazing things like hey take this full hour video and just extract me three interesting moments right but the model itself has it actually used that information directly probably not oh I like this so much we were talking to Jeff about with multimodal models you know if if you get these models to just watch all of the videos that have ever been created can it quite literally extra what gravity means as the concept so but what but what you're describing here if I understand it is that at the moment it can tell you what in the video that it's seen but it can't then say and you know E equals MC squ or if you showed it pictures of the the night sky it wouldn't suddenly be able to predict the the planetary motion in the way same way that that human astronomers did yeah exactly I mean the shortcut we're taking here is that the data we train when we train on um images and or videos we always almost always have a text representation associated with that modality right so it could be a caption explaining what this image has or this video has and so on so forth and that makes you know of course then it's incredible right you can put a picture of a homework and with a little drawing conceptual and it will it will connect and do quite a lot of good logic just based on that but what I'm saying here is could I just take videos no language and then train a model to then understand what's happening maybe even in a way derive a language maybe it's obviously not going to be our language um and extract those Concepts and that has not happened and I mean it probably will um just going back to what you said at the beginning about there's two phases to to basically all the models the Deep Mind have built yeah the the imitation phase which is what we've been talking about right here but then the the reinforcement learning phasee on top and I know that alphao and Alpha zero and and and and many more got better by playing themselves does that apply here as well yeah that's that's one of the main open challenges like scaling not only pre-training but post-training or reinforcement learning right so the beauty about reinforcement learning in games is that there is a set of rules that are coded and if you've won you you know you've won there is kind of a program that if you play chess and you've won it will check everything okay that's a Checkmate congratulations you've won the a clear metric of success clear metric um now in language much trickier right like is this a better poem than this I mean good luck discussing this even amongst us right so it's it's the generality again makes Computing exactness very hard right how I mean is this a better summary of the movie is this the most interesting bit of this video it's very hard to quantify but we can try and we do try it's you train a model and based on some human preferences roughly you just say okay try to now generalize so I ask a model to criticize its own output is not going to do that bad it's going to be good maybe I don't know 80% of the time which is not terrible it can give you some signal but at the point you start saying well now you climb this metric you have this imperfect way to assess performance but you now we're going to start training against this reward that is not perfect what the model is going to do is exploit the weaknesses of the reward right is you know maybe using the chess example imagine that I did I I had a back and if a pawn is in a certain position you always win and it's it's a position that no one would play ever so maybe no one knows this exists but but now you ask a algorithm to please explore everything and try to discover how to win at this game all of a sudden you you're going to find hey oh if I move the First Pawn just to this position no one plays this opening you've won the game certainly you have the algorithm has nailed the game and then a researcher goes and sees how you play chess and it's just terrible NY AI basically that's what we talking so we that's the challenge right basically you're finding exploits rather than really learning what a good poem means uh truthfully right can you not just add in another player right so add in another model which is like the the kind of ultimate Arbiter I mean good good good suggestion but then the problem is how do you train that model right like we have only a finite notion of what's a good poem by by some experts that we might ask hey compare these two poems and so on so there's just a limited amount of um data we have to train these Arbiter so the ground truth might be to ask someone that is the expert of course and if we could we would but that's not scalable right then imagine how slow it would be to say okay like I have a a parameter update um that in three seconds I found now please review these 10,000 Things by an expert because that's a source of truth right so and we don't have enough data to train a good enough reward model so again there's some ideas but uh but I I mean intuitively I mean you clearly understand but the problem is we don't have access to to the ground truth I mean that's it she like it's like you're feeling around in the dark with oven gloves on right yeah you're not even completely sure that there are solid objects to go on to um okay so so if that's the core then right that's the like electronic brain um and now we're building the digital body so so how do you what kind of capabilities do you want that digital body to have like reasoning for example because because there's been quite a lot of work on that too hasn't there yeah so you start thinking well what are the main sort of surfaces that we would be able to give these models limited access to so they can see beyond what's in their weights which are frozen uh to be able to gather knowledge or or do maybe something a bit more complex than just predicting the next word from just what they have in context plus what they have in their weights and so you know obvious things that come to mind is giving them access to a search engine that's that's what we do very well at Google um another one is to give them the ability to run the code they write right and then of course maybe even more broadly um that could be more General is giving them the ability to maybe interact with a browser that has access to the internet right with all of these you you always have to be careful to sandbox that just means protect prot these environments so that the models even if they're not that advanced wouldn't do something that is unintended right so there's the whole like safety aspect of this that as you move beyond the model it starts to be quite interesting but if we just in kind of dream what would be possible Right by having these tools available to the models all of a sudden they can start doing much more advanced things beyond what what the training Corpus that we used at the time right they can be more they can rely on the latest news to explain us or to summarize what was what was the main thing yesterday that happened all these kind of things you need to give them these tools okay so how does reasoning fit into all of this then yeah reasoning is interesting right so so what I described um could be summarized as hey like I mean I want to know what happened yesterday um then I could just say look take maybe personalize it a little bit right so I could describe inwards I could say hey hey model I'm orial I'm interested in this and that my political views are this are that give me like a positive take on the news yesterday right and the model would probably search right retrieve all the news and then given what I asked it to do just do it in a way that I like and I find it enjoyable maybe if I don't like it I can even go and then say I didn't like this or this is not a good joke and then we could iterate a little bit in a conversation now reasoning is a bit of a different axis of scaling right and so you could imagine um the model deciding what kind of intermediate steps to do to give me a better answer right so imagine there's like 100 news outlets that Google search retrieves maybe the model decides hey I'm not just not going to read this and just try to summarize it all at once I'm going to Summarize each of the 100 articles first right so that that means the model decided I'm gonna write a summary for each of the 100 Pages I'm going to write it not to the user to myself and then then now it has a 100 summaries and maybe the next step it decides to do is I'm going to group this by topics then it decides one of the Articles looks suspicious so maybe it goes online and checks if in any forums someone discusses like oh this might be like sort of not truthful because of the author and so on so forth so it can do a lot of steps to do research and you know it could do this for quite a while and only when the model says well I think now I have a much better quality answer then it will give you the you know the few word summary but now it had all this time to do much more processing on the information that was available to to it and that inference time compute we hope that the more time we give to the model the better is going to summarize the news the better is going to write a poem the better is going to of course do math but that's certainly another axis of scaling which we're starting to quite unlock um we hope to unlock and again we'll break a bit of the SC scaling LW and the limits that we see in pre-training uh pure pre-training does this also include planning like uh could it look at your calendar work out um when your payday was maybe know that the January sales are coming up soon and uh tell you to to postpone booking or holiday for a few days I mean that can be get very complex but of course when you factor in things like personalization and like when to do things because of all the other things that are ongoing right you have more sources of information you need to collect them and then give the best answer and it stops being what color is the sky which is is not that simple to answer I was thinking about that example I mean we had a very early paper where we have that example as oh something that language mods can do amazing right you don't program them to answer but they answer but then actually even the answer is quite Nuance if you start thinking oh yeah you know planets and where what time of the day I mean is it cloudy or not right so so the thinking and the planning yeah it can definitely be um that's that's something that these models can do I I'm reminded by I had a conversation with Demis probably back in 2019 and he was talking about um the caraman tki idea is of how the human brain has almost two systems of thinking the sort of quick instinctive intuition based and then the much slower calculated sort of the way that you do maths and chess and and and Demis was saying that that kind of that that second one has been traditionally easier for us to do with computers um but that now we're seeing the much quicker instinctive stuff but but I mean you're sort of talking about putting the two together right yeah right I mean the probably what Demis was talking about these systems too which is indeed one that you you reflect a bit more and in games it's very clear right you just you just could say oh this move feels right you just move but if you think and Ponder um you might get to a better move um the challenge is that now because we are in such general direction right these malls can do anything I mean anything literally you just do whatever you want I mean upload an image talk about the news so what it means to to have this deeper thinking is so domain specific that how are you going to do that and I mean the there's a few answers but the one I like is like well these mods are very general to add the ability to think on top of a very general set of capabilities you probably need a general way to think and so you use the model itself um to generate how it should think about anything and the model will come up with oh like I'm going to Summarize each article I'm going to do this and that and than that and it is not us programming it that's a very deep Insight now is it the only way to do it and is it the best way to do it early days five years we'll see exactly I'll talk to you in 2029 okay I'm I'm thinking now though also about lots of the things that that felt very important back um you know five years ago um and a lot of it was about inspiration from from Neuroscience so I suppose in a way here you're talking about planning and reasoning but memory was the other really big one and I has that kind of come through people talk about long context and short context a lot I suppose that sort of is working memory in a way isn't it yeah I mean there's techniques that are out there that you can apply to a language model there are at the very least three and they're reasonably simple to explain right the first way in which we have a system that memorizes all of the internet is by literally doing the pre-training step right that that's literally a memorization step in a particular format which is we have these weights they're random and then we assemble them in these amazing architectures now the second level is maybe I explained a little bit how you would give the tool of a search engine such as Google to the model that you could claim is a sort of um what neuroscientists would call episodic memory which you know as a human maybe it's like you know we have this memories from the long you know a long time ago they're not very precise so they tend to be a bit more fuzzy right like if I have to think oh what was the my first day at Google I remember bits and pieces or being in a room or someone I met or whatnot the gist yeah the gist right now interestingly these models maybe don't have that limitation right you can literally get an article written many years ago on online and it's going to have all the images everything will be perfect reconstructed perfectly so that second mode um called episodic memory clearly we're seeing that when you integrate especially powerful search engines into our models and then the third one is what you could call working memory right which actually the whole of thinking that I describe is one of right like if we take every news article but then we want to create summaries find how they relate to each other criticize some of them this starts to combine working memory meaning I'm going to have a scratch P of you know the the summaries the issues that I think I'm finding and that when we call short or long context generally we mean this last bit like the working memory whether you have a thousand tokens which means I couldn't possibly do much right I can retrieve articles it's already over a thousand words not much I can do to summarize them or it can be massive in which case you have many more possibilities to do reasoning on top of that and so on so forth and so one of the breakthroughs of of the Year actually I still we still in 2024 it was yes but was just to enable millions of tokens in context which enables many things you can retrieve something from the past but then bring it forward and then do a very detailed analysis that's a bit of the examples of we have a movie we can upload like a movie or like some very long video and start doing summarization the fact we kind of upload it is more episodic memory but then now we have it in memory it all fits in memory we can do quite a lot of associations within each frame each you know object in the movie and so on so forth is a longer context window better always I mean because I'm just thinking about I don't know how much you guys are still using Neuroscience as an inspiration for what you're doing but I mean the human memory like there's a limit to the working memory right like there's certainly sometimes you're like my brain is full and I'm done yeah sometimes brain is an inspiration but computers certainly have advantages we should build on its strengths right so perhaps the fact that they can have literally in memory like know every like Wikipedia article whatever it is we can't but if the model can well there there you go you have new capabilities but also it might be too confusing to have too much information even for these neural networks so it might be a good idea to compress so that's where you probably want to push for getting some inspiration for how we might do what we we do which is quite amazing right in terms of memory retrieval and so on yeah this is why you're leading up drastic research that's I mean what we want to do with the models um should be definitely inspiring and forward-looking and then you're what are the main limits of the technology and then try to of course place the beds and Inspire the teams to finding Solutions around the the critical components but some of the bets that you've already made have come off I I know there's there's been a big announcement of I mean a dizzying number of new features that have just come out can we talk through some of them and then but then maybe also talk to me about the different skills that we've already spoken about and how they they appear in each of these yeah so we have quite a few sort of systems around our best Gemini models um so one of the things that that we've done is update to 2.0 we seeing a generational leap even if you say look let's not scale anymore or can we get better quality so we we've done it sort of again these models are faster they're cheaper um and they're actually better basically Geminis got better yeah Geminis got better but not only because we skilled I guess that's that's kind of one of the one of the main messages tell me more about the the agentic uh capabilities that you've brought to Gemini yeah so we're releasing a companion in Chrome where you can to type to do a task uh that maybe you know some of these task are tricky because I partly enjoy them but also partly don't like them so I'm thinking now very clearly about trips okay so you travel and you look for hotels or flights or whatnot and a lot of it feels like oh this I wish you know this could be automated but at the same time I wouldn't just want to not be part of this journey right so so I guess the kind of thing we're we're releasing is something that hopefully will automate parts of the more trivial steps or repat or things that need automation because I can't be bothered to click everything right so we're adding sort of an agent that you can ask it to do something for you and then it's going to um again through thinking and through acting on like kind of the basic clicking on links and so on um try to solve the task for you right and that's quite an exciting you know both research challenge opportunity because it's very general environment for a very General um agent and and model ultimately and some examples that we had again in the early prototypes is I mean we can ask to play a game which of course goes back to the roots of of uh de mine um on the browser and it kind of did okay right it finds a website it starts playing the game it's kind of a cool connection to the more General you you are then the more you can treat environments where you had specialization in the past as now oh I just can type it and it just goes and learn to play this game I mean we're not quite there but this is a glimpse of maybe where we could go with this kind of technology I mean you're right then it does it does really bring us back to that that that thing that you were doing so many years ago which was something that can use a keyboard and a mouse right it's like really similar thing yeah even the actions are very similar right the understand the screen and given what you ask I mean where would you click and so on that's kind of the same the same sort of actions even that very general games would would you would have to interact with the the difference is the goal that is narrow it's just one game and the same kind of screens whereas here is the whole which yeah is pretty best well okay but then so I'm sort of imagining what you could do now I mean could it look in your calendar could you say I want to go on holiday next year and it could look in your calendar and work out when the best week was you know know your budget etc etc etc yeah so these models are not far from being able to automate this right so now it's a matter of making it better right making it safe there's a lot of steps but if you just fast forward anything a human can do on a browser like these these things can do in principle and then if you make them really understand what you want and really good and through thinking and other techniques um they'll get better and better and they'll be probably faster and maybe in some cases much better than you at doing that though so that's kind of the dream um and this is super early stages but is also super EX exciting and I think certainly next year we're going to see a lot of experimentation around this idea of intersecting you know language models identically with you know browser or computer more generally how about coding yeah coding is a great one as well we are also releasing tools for software engineering um which of course they generally require not only hey here is a perfect description of a puzzle about coding and please write me the code and by the way I know how to test it it's more iterative right you have to write code um run the code and so on so forth so we're putting that capability as well forward um with you know from an agentic point of view um we also I mean games are very important um and of course that was means to an end to develop powerful algorithms but it's also very interesting to think about how these very powerful multimodal models start to understand games and can Aid users to you know entertain doing like a game session um give them advice or you know tell a joke about the game or whatnot right so we're we're also experimenting with the sort of um game companion okay all of these things that you're talking about I mean this is sounding very close to uh intelligence that is quite General I mean are we getting close to AGI yeah there's a good question look I was I was thinking about this um earlier this week if 10 years ago five years ago even I would have been given the models today right and I I would say look there's a secret lab this is a model um play with it and tell me if you think this is actually like close to a general intelligence I would have claim oh yeah that comes from a future where AGI is basically either has happened or I can see that this is very close to it right so the closer you are the more you find oh but it hallucinates of course that's very important right but I think just zooming out I it just feels like okay it's getting pretty pretty close but then deep Minds mission statement solve intelligence that sort of intelligence like super intelligence something that is surpass is human intelligence do you think that scaling is enough to get us there or do you think that we need something else yeah I mean Google deep mine has this mission to obviously intersect intelligence with science to push the boundaries and we SE a good example very recently of course with with Alpha fault so in that sense right from a domain's perspective we honestly have seen some examples already of narrow but super intelligent system I mean Alpha was only doing that and I think probably that's the domains to think about where we're going to start seeing super intelligence even from the general sort of capabilities these models have you might need to do some specialization and again it might be worth it I mean was it worth it to solve protein folding of course absolutely right um but I think that's a good test to use and we are very well positioned because we of course have like the the whole of science team and so on working on very interesting problems um now if you take the language models and you start thinking about agents putting them in environments that could be more about science simulation um theor and imprs and so on will something very discrete be needed to enable other breakthroughs I would say probably not without another Transformer like Breakthrough perhaps feels like we're going to start seeing more examples of oh my God like yeah like in math now it just discovers new theorems that mathematicians find interesting and it happened by a just of course very good execution plus scaling up of some of the ideas and so on so forth it is it is interesting though that the the the first dominoes to fall are the ones which have a ground truth right like science as you as you described yeah although I mean yeah science it depends which Sciences might have ground truth I I suppose protein folding definitely yeah yeah it's true um I'm hoping we we also see some other ways to advance in a superum way like you could imagine having a brainstorming science Scientific Advisor that is powered by one of these powerful models and more than it discovers something or it proves something new it just challenges your assumptions and it makes you think out of the box in a way that then my creativity sort of gets me to a place I couldn't have gone then you would call that super human in some ways as well right so I think those are definitely not out of scope and much harder to also of course think of what how do you reward that behavior absolutely fascinating I mean there was there was definitely a lot of drastics stuff in there yeah yes thank you so much for joining me yeah likewise thanks pleasure see you in 5 years I think there was this real theme that emerged from that conversation at least for me anyway which was this idea of generality and if you think about it there is this generality in the way that intelligence advances knowledge like those old astronomers like kernus they were assessing lots of data from observing the sky and using that to extract a a model of the solar system but in the case of alpha go it was observing games of go to extract a model for the best possible way to play and now somewhere embedded in everything that has ever been created by humans is this this model this underlying truth of how we experience reality and the model that we're looking for of course it's never going to be as neat as heliocentrism but that model does seem to be in there his hidden among the Frozen weights of Gemini now if that's what we've done so far the next phase is to try and use those General ideas to extract a model of human preferences too and that is of course a lot lot harder but if we succeed it might just get us to a more general form of intelligence to AGI now if you found this conversation interesting I think it's worth also checking out the episodes I did with Jeff Dean on among other things scaling and Yas on Gabriel on the ethics of AI agents or if you want to dig a bit deeper into the development of Gemini 2.0 then you can check out the latest episode of the new Google AI releas notes podcast with host Logan Kilpatrick this and other episodes can be found wherever you get your podcasts until next time
Oriol Vinyals · Google DeepMind 研究副总裁、Gemini 技术联合负责人
Oriol Vinyals 谈 Gemini 2.0 与智能体 AI 的演进
→ 在 AI 访谈库中阅读（可切换中英、记录进度）与 Hannah Fry 谈智能体如何从单任务模型演化为 Gemini 这样的通用模型，结尾巡礼 DeepMind 新发布的智能体能力。最有价值的是他给出的极简框架：多模态模型=两步训练——预训练是模仿学习，后训练是强化学习。