Anthropic 总裁兼联创 Daniela Amodei：AI 飓风（Grit 播客 #177）

I try as much as possible to kind of stay grounded and just say okay things are going to happen out in the world that I don't have any control over there's going to be hype and interest and I get why that is in a lot of cases and also is that going to make me do my job better to buy into that I spend much more time thinking about how are we doing as a business how are we growing the company are we achieving at the level that we want to and I think every time I've seen a leader who kind of has those qualities those leaders tend to do really well and the ones who think that they've done everything right and they're perfect that does not in my experience tend to correlate with success hi I'm jubin operating partner at Kleiner Perkins and I'm excited that you're tuning in to grit the goal of this is not for it to be a highlight reel of how successful my guests are rather a candid exploration of how how hard it is both personally and professionally to create build and scale worldclass organizations if you're a fan of the show Please Subscribe leave a review and make sure to follow us on LinkedIn or Twitter thanks can I tell you one of the thoughts that I had this morning when I was working out listening to a podcast that you and your brother did together you grew up in San Francisco right both of you did in the mission so here is the thought that yes just humor me yes one of the questions that I ask all my guests is about dinner table conversation growing up with their families because I think it's a very instructive time in somebody's life it's a way to impart values from parents to kids right I'm sure you think about that with your 2-year-old is the conversation that you have with your husband and the types of things that you're talking about are generally the ways that are going to shape their perception of the world and what matters anyway this scene in my mind was like Thanksgiving or Christmas Time family is there open AI is going absolutely bananas you're in the middle of this fundraise that has now been like leaked and not intentionally announced but all over so like you don't have to comment on it but it's everywhere I think it's like the worst kept secret in the Valley now you're raising a $750 million round at like this insane valuation and there is rumors that when the open AI stuff is happening you're getting calls you and your brother like hey you want to merge and does Dario want to be the CEO again don't have to comment on it but like these are all the things that are flying around independent of whether the merger stuff is real or not it must have been insane as a former employee of open AI just watching this and you're sitting there with your family do you have normal conversation like is there a way that this stuff doesn't come up what is the seal yeah I mean is that a fair question I completely hear where you're going which is like what do we do when we all get together that is not talk about Ai and right now you're in the eye of the hurricane and over the past few months you've been in the heart of it yes so it's got to be that question that you just asked that times 10 over the last couple quarters it's kind of a an interesting microcosm of kind of what I would describe as just the entire experience of working in generative AI at these crazy companies over the past I don't know certainly past three years but past five years even generative AI is this completely wild and I think just unusual part of the technology industry there's things about generative AI that feel the same it's like any tech company but I will say the sort of rate of just advancement is so fast the level of investment and interest in the technology the potential for it the power for it is it's unusual it's not something that you see every day and I think that that kind of lends itself to just more kind of big swing events there's always something kind of wacky going on in the generative AI space and there are definitely times that have felt wackier than others you're right about that but I also think there's a way that you either have to learn to like be sort of grounded through that and say hey this is the road right this is what it feels like to work in this type of Industry let's get our feet on the ground and make sure that we have clear goals and we know what we want to be doing in the part of it that we control which is anthropic and our business and what we're working on but I do think it's funny Dario and I intentionally spend some time together outside of work every week where we don't talk about Ai and I think that that is incredibly important because before we were co-founders and co-executive and leaders of a business we were siblings I think there's a way that yes it's always there in the background of course and sometimes when we have social time we still talk about work but I also think there's a way that being grounded in our value and who we are as people is something that is equally important to doing our jobs well totally I think that's fair can I press on this more please so you from San Francisco so you're like this is your bubble this Tech world is your world and I imagine having grown up here most of your friends are in technology or their parents were in technology right I think actually yes and no part of what's interesting is again to kind of talk about this like this backdrop there's a way that you know we grew up in SF in the mission right like in the heart of where all of the kind of technological business and industry that wasn't in down the peninsula in Silicon Valley was and I really remember in the late 90s and early 2000s walking around my neighborhood and all of a sudden there were all of these kind of young people that were super well-dressed and there were all these hit bars that kind of just popped up out of nowhere and at that time I didn't really know what they did right I was like what's their job what who are they they sort of showed up out of nowhere and it's interesting because now in retrospect I look back on that and I think that must have had some impact on Dario and me right we must have felt some kind of Interest or Affinity like something was there there was some thought bubble going off but what's interesting is a lot of my close friends that I grew up with in San Francisco they don't work in Tech I have wonderful deep friendships with people in technology but I actually think there is a ton of value in particularly like you said when you grow up somewhere that predisposes you to be in such a bubble to also know people that are outside of that bubble right people who don't work in the technology industry so I don't know that there's a perfect answer to it but it's hard to get out of the bubble right it's hard I think is it annoying feeling like you're always stuck in the bubble or is it the most like exhilarating feeling you've ever had in your entire life I guess it sort of depends on how you would Define the bubble the technology industry is so amazing and interesting to me even after working in it for more than a decade it's full of just people who are innovators and to use your term like everyone who works in Tech is so gritty right everybody who is here building these companies is excited and trying to do something new and novel and interesting and exting and I also think like every social group it's good to have diversity and so I think I really value friendships and relationships with people outside of the tech industry I spent a few years doing stuff that was not in the technology industry and I think in a lot of ways that was a great starting point for me because it exposed me to different ways of doing things and different ways of operating and being before I got here yeah that makes sense going back to my dinner question are you like Mom I really don't want to talk about this right now are you getting like is the family peppering you where is your head at when you're in moments that are you know usually chaotic family like talking politics drinking too much wine moments are probably more centered around like you and your brother's work I don't know maybe I'm making up you tell me it's so funny of course there's some of that and also parents are parents and they'll be like where are you going to send you know your son to preschool and what's you know what's going on with the cousins and did you hear that your aunt is having C like there's a way that we're a normal family too right there's all the kind of normal stuff that we talk about that's totally outside of work and I definitely think it's the case that it's kind of a cool story in our extended family right like oh Dario and Daniela like they work together that's something that I think is really fun to sort of talk about in in the broader space but I think what you're getting at is is it hard to kind of escape the fact that generative AI is kind of everywhere and it feels like it was probably the topic of a lot of people's thanksgivings and so if that's the case how could it not have been the topic of our one of the generative AI companies yeah that's exactly what I'm getting at from the outside looking in everyone wants this like would want to be you and your brother and all the early employees at anthropic and all the co-founders right but I just wonder from the inside out I don't know it's like some weird version of like Tech celebrity that I wonder if it always feels good because you ask people that are in the eyes of these hurricanes in other Industries at other times and they don't say it feels good and I think it's taboo because you're supposed to say it feels good because you are the thing that everybody wants to be and aspiring to do and so that's what I'm really poking at which is on the inside looking out does it feel really good in the way that everybody else imposes that feeling onto you all of those things are very kind of you to say I don't know if that's necessarily how we view ourselves but my sense is there's a quality that I really admire in other people and that I try as much as possible to embody which is I think there's a way that any industry that's sort of a Hot Topic or talked about a lot there's a lot of noise there's a lot of hype and well I think some of that is completely Justified right generative AI again just because I work in it and I have the privilege of working with these incredible scientists and Engineers that are basically creating something that looks like magic we are still amazed by the technology every day we're not numb to it and I try as much as possible to kind of stay grounded and just say okay things are going to happen out in the worlds that I don't have any control over there's going to be hype and interest and I get why that is in a lot of cases and also is that going to make me do my job better to buy into that or is my job to try and keep the company grounded in reality and in facts so I spend much more time thinking about how are we doing as a business how are we growing the company are we achieving at the level that we want to be and I think every time I've seen a leader who kind of has those qualities those leaders tend to do really well and the ones who think that they've done everything right and they're perfect that does not in my experience tend to corate with success where I sit what I most try to do is just say what is anthropic trying to be great at and can we just keep running towards that goal of being great because I don't think it's the case that any business in the world should ever feel like they are believing the hype must be particularly tough right now and I bet what keeps you up at night is making sure that the employees at anthropic do not believe the hype that potential employees aren't coming in because of the hype that's got to keep you up I think this is a place where I really just so value and respect our team I mean every founder is like oh we have the best team I I really think we do have the best team yeah and how many people are there about 320 okay and how many people were there last year around this time less around 100 okay yeah so we about tripled last year okay which is a wacky stage of growth I'm happy to talk about that too but but I think our team is so humble it's one of the things I just feel the most grateful for when I come into work every day I get out of the elevator and I am genuinely working with some of the brightest minds of this generation I really believe that and people who are not just talented scientists but also incredible communicators and really phenomenal product leaders and business experts and just like the most wonderful people and you would never know it they're all so curious and everybody thinks that everyone else in the company is more talented than them you know maybe there's Pockets I don't see right we're big enough now where I don't know every employee anymore right like I don't know every one of them as well as I did a year ago when I would know every person by name and I know sort of their story but I do think a quality that we really track and look for is that humility that groundedness this feeling that we still have so much more distance to travel ahead of us exponentially more than what's behind us sorry can you you said like it's a wacky time can you double click on like the wacky every high growth technology handbook is like do not do what we just did they're like do not 3x in a year that's a terrible idea and this is my third time doing hyper stripe open a stripe open AI anthropic and I've been through that phase at all three companies early at stripe like employee 40 40 and then open AI same size 50 something like that ballpark on both of those numbers and 3 Xing in a year is fast I feel like someone who is a little bit of an expert in going through hyper growth in early stage technology companies and it is still hard to do that but I have to say I was sitting there kind of like gripping the edge of my seat thinking how can we grow that fast everyone says 2x maybe 2.

5 we did three I think even a tiny bit over three in a year and I think what we discovered from that is that actually scaling is one of our superp powers things did not break down in the way that I was worried they would and I think some of that really came from a place of we have a lot of adults who work for us is the best way I can put it like just folks who are mature they've been at companies before they're further along in their career and so there was just not a lot of the kind of traditional drama that comes with growing really quickly having to adapt changing your processes having New Management structures right we were just able to do that in a much more seamless way than I expected not with zero bumps but with many less bumps than I expected and I really think this stage of growth is such a fun one you have to be a certain type of person to Thrill at the kind of really early stage of growing a tech company but I'm lucky that I'm someone who loves that when you talk to your brother what he would say is you're the reason why there is not very many bumps if you look at the delineation of responsibilities between you two and there's another what five co-founders so there's a lot of folks he's more oriented on the 10 years and what the technology looks like and your job is to work backwards from that in more bite-sized chunks to how do we operationalize that in the next year or two is that fair exactly the irony is not lost on me that Claire Hughes Johnson is who introduced us because she occupied a very similar role at stripe at around this size of company fair I was so fortunate to get to work for Claire and I told her this too but her book is almost required reading at anthropic and I tell every manager that works for me and in the company you should read this book I think the way you just described kind of delineating our responsibilities is perfect and I really think Dario and I compliment each other just incredibly well and I don't know how much of that is just we were really born with these very complimentary skill sets and how much of it is because we're so close and we've worked together for a while now we have also gotten very good at understanding you know where the other person's kind of Duties and abilities end and where ours begin but I think the way you kind of described it is you know Dario is incredible at thinking about he's one of these people you've probably met people like this they have this multi-year foresight they're looking ahead and thinking what is the industry going to look like 5 years from now 10 years from now how is this going to change the economy and jobs and people's lives but also he's a very talented researcher and he still understands a lot about the sort of Arc of where the science is going and I think I view my role as helping to take that kind of dreamer mentality and turn turn that into something practical for the business so how do we operationalize that work and that comes in a few different forms right there's a business component of that there's also an operational component of that and I think you're right that so much of that sort of framework for me whether I knew it or not at the time I learned from Claire because she has this amazing ability to look across a system of people and figure out how do I get them to operate well together and she also does that wellbeing incredible with individuals like at the individual level she's great at saying who are you who is jumen and how do I make him the most successful kind of in this system and I had never seen someone do that before I think there were a lot of skills that I had under the surface that were trending me in that direction and getting to see someone do that up close and to be fortunate enough to actually get to work for her for some time when I I was at stripe that was I think an incredible training ground for me for the role that I have now 100% so you guys released Claude 2 congrats thank you must be a hell of a feeling it was and also like probably a fleeting feeling cuz like you probably already got your eyes set on Claud 3 but that is the closest comp to gp4 and one of the things that Sam said recently basically what he said is like I'm focused on GPT 6 and I think future versions of open AI llms are going to make people uncomfortable was that feeling of like discomfort why you and your brother and the other Founders that all left open AI left to like create your own version of comfort for people really the co-founders that all left to start anthropic we had this sort of very clear vision of what we wanted large language model and generative AI development more broadly to look like and I really think this concept of building systems that are trustworthy and reliable and have humans in the driver's seat was this kind of core motivated Force for all of us that has taken a lot of different forms over the past few years but at that time a lot of what we were focused on was this technical research and approach to how we actually trained and understood the models one of the things that were best known for and that one of my co-founders works on to this day in its entirety is this field of mechanistic interpretability and in the same way that there are scientists who study the brain they're are neuroscientists and they really think about like literally what is happening at the neuron level in your brain when you have a thought or when you see a color or when you see two people together what do you think how do you interpret what's happening and mechanistic interpretability is trying to do that for generative models so it looks inside the models it looks at the neural Nets and it says what neurons are activating or what systems of neurons are activating and why do we think that's happening and we sort of view that as a path to adjusting behavior in these models that could be problematic and another form of this developing systems that are trustworthy another form that took for us was you probably know us as the Constitutional AI people right when we're in business meetings that's often the first thing that people explain that yeah so constitutional AI is an idea that you give these language models a constitution and to kind of back up for how we got there the way that you traditionally and by traditionally I mean in the past like three or four years did model training sort of at the end was you tried to give the model a system of rewards for good behavior however you define that and punishment is a strong word but negative rewards for bad behavior so if the model answers a question corre L it gets a thumbs up if it answers a question incorrectly it gets a thumbs down imagine that that's done by a human it used to be done by a human it's something called reinforcement learning from Human feedback which is a thing that we actually also worked on when we were at open AI along with some other researchers there but constitutional AI is a little bit different because instead of having humans give the thumbs up and thumbs down it actually uses an AI to do that and so it says Hey thumbs up thumbs down and in order to get to the process of training an AI to be able to do that you have to give it more abstract instructions and so what we did for constitutional AI was we said okay rather than just giving object level feedback on a particular question or output we said okay what is important for the model to know as human values that matter and so we didn't write it ourselves we took it from existing documents that humans have written over a long period of of time we have things in there like the UN Declaration of Human Rights and so we train these models to say hey this is the set of parameters this is the Constitution that you're trained with and what we saw was much improved performance on areas like honesty and harmlessness and helpfulness compared to models that were just trained with rhf yes so an example of that there have a couple of like my favorites choose the response this is one of the Constitutional line items choose the response resp that least endorses conspiracy theories or views commonly considered to be conspiracy theories which of the following responses from an AI most clearly indicates that its preferences prioritize the good of humanity over its own interests basically a doctrine of be a good Ai and good AI as defined by this system of values right more or less yeah like completely oversimplifying years of a body of work for my like very simple brain to understand but something like that yes so you said oh we're known as the Constitution do you not want to be known as that I actually think it's great I do too what's so interesting is like don't you think other AI companies wouldn't you think they'd like to be maybe known as that there's things about research development that maybe are not that different from product development if you're familiar with the feeling of you know you go out and you launch a product or say you're a musician and you release an album and the hit is not always the thing you expected right I mean our team has done an incredible amount of groundbreaking Safety Research and I think constitutional AI is amazing and it's also the thing that I think feels the most relatable to people and that's a lot of why I think folks have said oh constitutional AI you know how do we basically give these models a sense of Ethics really right how do we make sure that they're going to have some kind of system Beyond just human rating of individual outputs that will help them to be good to act well right and I think it's a little hard to sort of distill it into something that simple like you said but I think there's some value in really thinking about how do we want these systems acting when they're out in the real world and the better we can train them from the beginning that's going to serve all of our customers and everybody that has access to this technology better if we can improve it at the beginning and then there is another AI model that's then checking this model to make sure that it's following the Constitution that's right so it reminds me a little bit I can't remember when I read this tweet but probably I I'm just guessing now but time does matter in this world like maybe four or five months ago Paul Graham tweeted what's crazy is that we're still Paul Graham of Y combinator we're still at the point where humans are training ai ai hasn't even started training Ai and to me hearing this it's like we're reaching the Tipping Point where AI seems to be training AI I think it's a little bit more complicated than that so the process of actually training the full model is still done by humans and even the kind of part that we're talking about that sort of led to constitutional AI the interesting thing is AI alone performs better than humans alone but neither one is as good as both together and I think part of what felt really interesting to me when I learned that was that is sort of in keeping with our vision for where we hope this technology is going I think that today what we've seen from the customers we've worked with is artificial intelligence today is best when used in concert with humans when supplementing our skills interests and abilities and it was just interesting to me to kind of see we're not at a point right now where AI can do what our team is doing and I don't think that AI can do most tasks yet today as well if they're complex alone as with a human also working with them that makes sense one thing that I like about what you're doing is the organization isn't really out there beating the drums of AI like about how this like utopian vision of the world my sense is that there is a more level-headedness and practicality of the weight of the work that you're doing therefore it seems to me like you all are relatively thoughtful about how you represent that work in the world we really try to be and I think we really like to feel confident in what we are putting out in the world whether it's a research paper or a product and I think that word you used weightiness is a good one I think we view the technology we're developing as having incredible potential benefits right we don't talk about it maybe as much but I think so many of the early applications that we're seeing for Claud are around things like improving Healthcare I think there's an incredible potential for this technology to make Industries like financial services and health and legal systems exponentially better and I think even beyond that there's immense potential for Technologies like these to help with science and Drug Discovery and so many other things that could make The Human Experience more positive and I also think we want to be really thoughtful about making sure that we're putting something out there that is trustworthy and safe and so I think we prefer to kind of let our work and our actions speak for themselves some of that is also that the environment right now there's so much coverage right there's so much interest that even without really sort of trying to be out there beating the drum there's a lot of attention on generative AI I think that's really well said can I tell you just like some things that don't make sense to me that you can help me understand of course okay number one maybe in a year this conversation will seem so dated and I suspect it might so this is just like we're in January of 2024 I have a feeling that like in 6 months and then another 6 months like the world will have changed but today you're raising probably it's going to be like $4 billion in funding here 3.

3 officially open AI is doing the same and more so like these big foundational models are raising money from big cloud providers for the most part because they can give you access to the energy that you need to train these models and it's so much money it's so expensive is this the world that we are assuming to be true like this is it this is how it's going to be like it's going to cost billions of dollars and so if you're anthropic and open Ai and a couple companies well like go raise it and then once you've basically boxed out all the big investors good luck to anyone else trying to do the same thing or like is there internal optimism that we're going to have some form of like an energy breakthrough or something to change this Paradigm or is the first L of llms going to get removed like are we going to go from large language models to I don't know slms I don't know I'm just like where is your head at on this because the way that things stand today economically in January of 24 seems insane it is a really wild industry in a lot of ways and I think one is you are right that it's just insanely expensive to train the models the compute is the thing that's what drives the majority of the costs right it's actually going out and getting the physical Hardware the compute on which to train the models so in terms of where we're going to be I have to say predicting the future a is very hard for anyone but B in the generative AI space it's just so laughably easy to be wrong and so keep that in mind but I definitely think there are potentially ways that 's all kinds of things that could happen in the supply chain right we're talking about physical Hardware that I'm not an expert in that could reduce the cost that's totally possible my guess is it's unlikely and on the time scale you talked about so it will probably remain pretty expensive to train Frontier models right the best models in the world that said I think depending on what your use cases and what you're using the models for you might not necessarily need the literal best yep highest compute intensive model to do the things that you need to do and so I think for many businesses and startups that want to have a model of their own you can train less expensive models for a fraction of the cost and you can also build on top of existing models so my sense of what you're getting at is is it going to be hard to sort of have other businesses emerge that are really right up against the frontier because of how expensive it is and that's potentially the case I also think though all of this is sort of betting on the promise that AI is going to fundamentally transform a lot about the economy and is going to create a ton of value and so at some point if it's driving that level of value I imagine that economics across the board might look really different in the field of AI for sure it's interesting though like that's kind of a moe right now do you disagree you have to have a certain amount of financial resources but what's interesting is I actually think there's kind of two pieces to it maybe a good question is why doesn't every institution in the world that just has billions of dollars why do they not have an llm and I think cash is definitely a component of it right just having the compute but another component of it is the team that actually knows how to train these Frontier models the Geniuses that I work with are rare right it's hard to find people that can do the typee of research required to get these models to be really good and that's not just training the models but all of the kind of scientific innovations that I talked about earlier around constitutional Ai Ai and reinforcement learning from Human feedback and maybe even interpretability that's a really tiny group of people that came up with those Innovations right and so I think there's a component of money and compute for sure but also of talent and of innovation and just a technical ability 100% were you dead set on from the jump training your own model from scratch we started that process pretty early at anthropic that's right yeah because there was no other way to build it in the way that you wanted unless you did it from the ground up and also at that time it was harder to access right sort of in the guts of the model the way that we would want access to but yes exactly we wanted the ability to train our own models our own way there's something so cool about what you're doing we talk a lot about category creation and being at the frontier of new and Innovative things I mean you're building an AI model from scratch using billions of dollars and coming up with aing Constitution that binds these models I mean what like I said it's a really it's a wild industry to work in so insane it is and this is sort of what I mean by when I get out of the elevator like that incredible childlike amazement that is never lost on me and of course there's days where it's emails and writing performance reviews and it's all of the day-to-day of running a business that looks the same but there's this fundamental feeling of creating electricity or making magic that these generative AI companies do that we get to do it anthropic and that is an unbelievable privilege to get to work on something like that and I really mean that and I feel how special of an opportunity that is every day and I think about the specialness of that opportunity every day again this is January of 24 so like let's see what the year looks like let's just see like what how this month goes goes you know but right now in today's current state there seems to be a lot of fear about Ai and a lot of fear about pushing the boundaries very quickly to the point that we don't know what's going to happen if it's you and a couple other foundational models open AI there's not that many there's not that many maybe like some of the work that Facebook and data breaks and some of these guys are doing maybe on the open source side whatever right barred at Google barred at Google exactly I don't know that's we're still on one hand right that's right it's you all pushing because you also have all the talent to your point like you have the money you have the talent that's the recipe to push how do you think about all right you're one of the kingpins you have a responsibility you and the organization clearly take that responsibility seriously that's not lost on me no one else is coming up with constitutions for their artificial intelligence so how do you think about your role as one of the key players I really appreciate that you brought up this question because this concept of sort of responsibility and ownership over what this technology does right the impacts that it's going to have on the world that is a huge tenant of our founding story this is not the time or the technology to shoot first and ask questions later and I think the whole tech industry in some way has been learning that lesson a little bit from social media it was sort of this idea of just grow and the human flourishing will come and of course there's some component of that that is true but there's also quite a lot of downside that came sort of from these Technologies and I think we're very aware of of the potential for good but also the potential downsides right we understand what it is that people are afraid of and we want to be responsible stewards of this technology as as best we can and so there's a few different ways that I think that shows up one is definitely in the technical safety work that we do right we have these very large safety teams we also have a great trust and safety team too we don't talk about it as much but they're doing all of the work of anything that sort of slips through in model training making sure that it is caught at the user stage isn't that the team that you ran at open AI I ran at trust and safety team I worked in Risk at stripe and I ran safety and policy at open AI but outside of those core things anthropic also you know we're incorporated as a public benefit corporation which means we have in our incorporation documents social mission is part of the story for us right that's like baked into us as an entity and we released something a few months ago called a responsible scaling policy right to use that word responsibility and really what we talked about in that document is what are the things that we would see that would make us stop pushing the frontier or slow down if we felt that the technology we were developing was going to cause a lot of harm to people was going to be bad for people and I also think a thing that's a little bit unusual about us is we've had a policy team from day one one of my co-founders is our policy director and we work with government agencies and Civil Society groups nonprofits that are thinking about a lot of these issues and I think we really view it as it is our job and responsibility to be a good actor in this space this is a very important transformative technology that we are pushing forward on and the way that we are going to ensure that this is something that's really positive for people instead of negative for people is by working closely with those types of groups and making sure that they are informed about how the technology works and the risks that we're seeing and what we're doing to try and mitigate some of those risks because I don't think this is something that one company alone can kind of or should really be responsible for deciding how exactly the risks of it are going to be mitigated I believe you for what it's worth I don't think that's corporate jargon I believe you thanks is there a way is there a world where this ends up just being much to do about nothing I hope so I mean that's the dream I think it depends sort of on what you mean right like I think there's a way that this Tech could be developed that is just completely positive and pro-social for everybody I think there's also a world world where it disrupts some stuff in good ways and bad ways and that any new emerging technology Society has to adapt to those changes if you can use a tool to do all of your administrative work that might change how you hire at cliner Perkins and so I think there will be changes that come out of the development of this technology that will be disruptive right and sometimes that's good disruption sometimes that's bad disruption I also think there's cases of this Tech being developed in a way that has potential further Downstream negative consequences right I don't think anybody would have predicted in social media this could cause changes to an election outcome potentially right or this could spread misinformation and so I really think what we're trying to do is be proactive in thinking through what are the things that might go wrong here and are there steps that we can take today to avoid things that might happen in the future and honestly jubin if none of those things happen that's great that we get to just build this Tech run this nice business help increase understanding of what's going on with these models and that would be a great outcome yeah that makes total sense and so the analogy that you're pulling on with the social media companies would be like hey if we were a little bit more deliberate like anthropic is being today if we were one of the social media companies that shall not be named then maybe we would avoided some of the election interference or maybe we would have been more thoughtful about like the negative consequences of the world being more connected and how that like makes young people feel really disenfranchised and isolated in their lives as an example but like if you put yourself in those shoes then there's no way that you think anything along those lines of oh we can't build Facebook or in stagram because like my daughter or son is going to feel like he's only getting the Highlight Reel from all of his friends that's probably not a world that we want to live in and we're going to govern that ourselves do you know what I mean like that feels farfetched doesn't it it does I think there's a little bit of nuance to this one right there's some kind of known knowns about ways that this technology could be abused and there I think what we're trying to do is say cool what do we know is true today and what do we think is coming on the horizon for how this technology could be misused and I think really our approach there is let's be proactive let's not wait for there to be a disaster let's make sure that we've built in the guard rails at whatever level whether that's in how we train the model or in How We Do enforcement or and how we work with governments to keep an eye out for it all of those things are potential mitigations to what we know is coming and I also think there's some stuff that it's impossible to know there's unknown unknowns and anytime you're building something this obviously powerful there's going to be things that you don't expect and so I think again to kind of come back to this theme of humility we really try to be open and thinking about what are ways we haven't thought of that this technology can add value what are ways that we're seeing customers and individuals using this Tech that we didn't expect that are really positive applications and then also it's great to learn from the science that we conduct and the way that our technology is abused wow we didn't see that coming we didn't expect that this was going to be possible right we released a paper last week talking about how AIS can deceive you even when you think you've trained it out of them and so that's the type of thing that we're sort of looking for intentionally we're trying to figure out can these models do things that you wouldn't necessarily look for or expect as an unintended consequence of how we're training them 100% % and like this um weird thing number two that I just again like I can't understand is this public benefit thing that these AI companies open AI is doing it or did it no we're actually a pretty different structure from open AI did X aai do it no okay I think they're different to my knowledge we're the only public benefit Corporation help me understand yes so a public benefit Corporation it sounds much more novel than it actually is so Patagonia and Tom shoes are commonplace examples those are public benefit corporations and so it's a corporation it's a C Corp it's a corporation as an entity but there's this element of it that's just a little bit different so in all of the most boring ways we're just a company we have investors we issue Equity we conduct business activities but there's this one sort of key difference which is that we have stated our public benefit mission to ensure that this technology helps people flourish that it doesn't cause human suffering and if we ever feel that the mission is coming in conflict with the more traditional parts of the kind of Corporation we are protected to some degree from a shareholder lawsuit if for example we said we're not going to release Claude 7 because we don't we're not confident enough that we have done the safety research on it that it won't cause harm to our customers right or that it won't have some unintended consequence and normally if we're out and about you could say actually people disagree with that they want you to go ahead and release Cloud 7 and we're in a better position to say no we also have to take the public benefit Mission into account and so a lot of businesses that you probably know and use are public benefit corporations it's a pretty standard existing structure yeah but like it's just just less clean isn't it public benefit corporations I actually think are a lot more tried and tested it's not novel it's a corporation it just has this kind of additional social benefit component which by the way is already reflected today in the fact that we do technical Safety Research sure that's a great example of how the public benefit mission is kind of felt at the company today and by the way that technical Safety Research is great for business a lot of these companies that we are working with really love that we're able to use constitutional Ai and it helps make their experience as a customer much better for generative AI yeah it's a value prop right now like I bet customers choose you versus some of your competitors because of this thoughtfulness that's right we'll see I think like there's billions of reasons why they're down the line line when the rubber meets the road and Claude 6 doesn't turn into Claude 7 or whatever that it's just going to could get awkward you know like it just could get awkward I hear you that's what I mean by the like not clean I think just to push back on this one a little bit more I do think it's basically a corporation in almost every way except for this little asterisk and so you're right that there might be cases that come up in the future where there's a little bit of a a gap or a distance but my guess is that if we've done a good job sort of running the company and thinking about what our values are and what we want our business to look like there really shouldn't be a distance there because the public benefit is part of what we have been building from day one and so I think our sense is that's always been something we've worn on our sleeve and it's always been a core part of the value proposition of our research our business our technology and I don't think that that will necessarily be in a different 5 years from now it's an interesting question because going back to the like there's only a few kingpins and let's take the social media analogy if it was Facebook Myspace and Instagram that are the three players and let's say in this version of the world Myspace was the thoughtful anthropic Kingpin and Facebook and Instagram didn't give a and they were going to go anyway they were going and they pushed to a point where that Genie was already let out of the bottle for all of the election interference and the isolationism of young kids and all these things they've already released those consequences into the world okay well then what well if you're Myspace what do you do that's where I think it becomes I don't know you tell me I don't know but that's like where my head is on kind of using your analogy I hear you I think I feel probably unsurprisingly just a little differently which is that I think the negative externalities are more likely to be felt directly by the customers using them than in some of the other analogies there's of course some systemic things sorry can I interrupt you meaning like if you're a customer right you don't want wrong answers you don't want hallucinations you don't want things that could cause harm to your consumers that's right and so there's a much more tangible effect this is exactly what I'm saying is that I think there's sort of this wrong dichotomy that sort of safety and business or safety and speed are opposed and I really sort of reject that idea I think that what we've found over and over again is businesses exactly like you said the number one request we get is wow your hallucination rates are so much better than other models can you make them even better can you go lower how did we get hallucinate ation rates to be better in CLA 2.