The Wu Tapes: Q&A with Cognition's Scott Wu

First, Wu predicted earlier than most that the AI industry would converge on agents that run in the background 24/7, taking handed-off tasks like coworkers rather than autocomplete tools. Second, he saw that GPT-4-era models were already capable enough to start building those agents when the consensus was that they were still several years away. Third, when others were dubious, he believed that product-market fit for AI agents was a non-question, considering the world already spends trillions of dollars a year on “meat computers” (human software engineers).

Last, he was willing to ship early and absorb the backlash. Wu launched Devin in March 2024 at 13% on SWE-Bench (a test for AI coding agents that measures the percentage of software bugs they can fix on their own), took intense public criticism for it, then used his head start to compound into the lead Cognition now holds, with the relevant benchmarks sitting around 90% on the original SWE-Bench and 80% on SWE-Bench Pro.

Devin is now the agent of choice for enterprises from the U.S. Army to Goldman Sachs and Mercedes-Benz. Cognition, whose customer base a year and a half ago was zero, is raising at a valuation around $25 billion.

When Wu talks about his company, and AI in general, he muses not about zapping the proles into very small office supplies, but about “making it easy for everyone to build really good software,” by which he means making the DMV, IRS, hospitals, airlines, and the website of your child’s elementary school work as smoothly and intuitively as Instagram, and other such things that he hopes will help eliminate the infuriating sludge from modern life. “So that we can stop living in Minecraft survival mode and start living in Minecraft creative mode,” as he likes to say—a refreshingly normal sentiment from an AI mogul, and the emperor of the nerds.

So will it work? Perhaps not. Perhaps AI is a zero-sum market, and starting an agent lab in November 2023 means Wu was too late. Perhaps it is a competitive market, yet Cognition will nevertheless get crushed by OpenAI’s Codex, or Claude Code, or the xAI-Cursor hippogriff. Perhaps it’s unwise to try to win a war like that with a superteam of nerds and no jocks. Perhaps it matters that early reviews of Devin were not as good as they are now. Or maybe it doesn’t, but Cognition will eventually concede its independence, like Cursor did. Or maybe we’re just talking NFTs again, and none of this matters in the first place.

In my capacity as a writer and collector of rare human butterflies, I recently traveled to Cognition’s San Francisco headquarters to speak with Wu, who handed me a plush stuffy of an otter, the zoomorphic representation of Devin (a sort of portmanteau of dev-elop-in’). I remarked that on my way in, I noticed precisely zero employees making use of the office’s gym, perhaps because it’s also where the company stores its wet garbage, to which he responded that they store their wet garbage in the gym because they are olympians in math.

I set out the tools of my trade on the table and waited for Kevin Liu, the founder and CEO of Metronome, to complete his delivery to Wu of a punnet of strawberries he claimed “will change your life,” before hitting record. The following is a transcript of that conversation, which lasted approximately three hours. It has been edited and condensed for your reading pleasure.

THE MARTIAN Jeremy Stern: Scott, you are an AI tycoon with a robust lore that suggests you are more computer than meat and were raised in a swamp. So let’s back up.

Scott Wu: My parents were both from Shanghai, they grew up in China during the Cultural Revolution. There was a long period where the typical exam in China that allowed you to go to college just didn't happen. That lasted for a period of about 10 years. Both my parents were in the unfortunate time period where they didn't get to go to middle or high school, either. Then there was a point where exams started opening up again.

Naturally what happened was there were 10 years’ worth of students applying for the same number of colleges. Since they hadn’t really gone to school, my mom was working in a textile factory at the time. They worked and studied for the exam in the evenings and eventually went to university. They both did very well as students.

My dad was a very avid Go player, and he had a professor he played Go with. That professor came to the U.S. sometime later and wrote my dad saying, you know, you should come study in the U.S. It's much better here and there's a lot more opportunity. He helped him apply for it and everything. And so he came to the U.S., and six months later my mom came. They both studied chemical engineering at Colorado State.

Did they ever speak about the culture shock?

It was certainly different for them, but it was just so much better in terms of having an apartment and food, which were still scarce in China. They were very grateful. My dad has always said the best decision he ever made in his life was coming to America. Which I think is true.

They had never been outside of China. And even still—my mom passed away a few years ago—she had only ever been to China and the U.S. Same with my dad, until I took him to Japan last year. So it was a big culture shock, for sure.

There were lots of little things. China was so poor at the time, and they had saved up for months to be able to do this. When they came, my dad had about $80 total. He told me this story about riding with other graduate students in a cab from the airport, and naturally they all pitched in a little to tip. When he put in one or two dollars for the tip, he realized that was literally about three percent of his net worth. That was a very shocking and kind of embarrassing moment for him.

So quite a time to be a student in America and not in China. How do we end up in Baton Rouge, which is not exactly the Ellis Island experience, either?

They both studied chemical engineering, and naturally, because Louisiana has so much of the oil and gas industry, they both ended up finding jobs there. My dad worked in consulting for air quality permits, my mom worked in the government Department of Environmental Quality. I have one older brother, Neal, who’s four years older than me. I was born in 1996. By then we had a small house. We had a reasonable middle-class home in suburban Baton Rouge.

I was always definitely different from the people around me there, who were probably quite a bit more normal. People would play soccer at recess and stuff, and I just was never really into it. I loved math and I was extremely competitive, always. I remember learning times tables when I was, I think, three or four. A lot of the math I learned initially was kind of organically at home. It helped that my brother was four years older than me. We had a lot of the same interests and I really looked up to him, because there were not a lot of other people like us in Baton Rouge, Louisiana.

No particular reason I would think this, but were you bullied?

For sure, yeah. I was bullied a lot. I was just very different. I was quite a loner up until the point when I started meeting people similar to me in competitions. I had a couple close friends from Baton Rouge, but honestly I kind of think of myself as having grown up instead with a lot of kids from around the country and around the world who did these math and programming competitions, because they were by far much more similar to me.

I want to get to the competitions. But first, how did you express your love of math beyond challenging other children to times tables at recess?

When my brother was in sixth grade, I was in first grade. That's when he started learning about these middle-school math competitions. I naturally tagged along and wanted to do math too. That was how things really started for me.

I was in the gifted program at Buchanan Elementary, our local public school, and we had a middle and high school next door called McKinley. I started taking high school math there in third grade. For a couple years, a teacher would come over to the elementary school every day to walk me over to McKinley, then in fifth grade I started walking there by myself. One of the great things about the school system there was that they were very accommodating to me in terms of figuring out how to make it work.

My brother taught me programming when I think I was in fourth grade. I played Pokémon on my computer on an emulator all the time, which is how I learned to program more. I played Tetris. There's this card game called 24. Various little games like that where it's just logic or puzzles or strategy and you can calculate options. Those were always my biggest hobbies.

My parents definitely pushed me a decent bit to study and practice, but pretty quickly I was at the point where I just wanted to. I was fully self-propelled. It's very hard to do if you're not deeply interested in it. I had always loved math because it’s just so elegant. But it was helpful to have the push from my parents.

How does a small child experience “elegance”?

I remember my dad explaining the quadratic formula to me. He showed me on a piece of paper: you have Ax2 + Bx + C = 0. Here's how you do this. First you divide the A out, then you do a square thing, you solve the equation, and you find that it's negative B plus or minus whatever. I really liked that. I've always just really liked logic and the ability to draw things out to their natural conclusions. The cool thing with math is that if you really deeply understand all these things, it all just makes logical sense. It also didn't hurt that I was pretty good at it and therefore could beat people, and I obviously love beating people.

You say “obviously,” but where does that come from?

I’m still the most, but my parents and my brother were all like 99th percentile competitive. Maybe my mom was the most of us, actually, if I had to guess.

The trope about parents is that they’re always comparing you to other kids—like, “so-and-so's kid is doing even better.” But my mom would always talk about it the other way around—like, “that smart kid’s actually really not that good. This person's kid got into Princeton, and he’s not even impressive.” She always strongly instilled in me that you have the potential to be the best. It was never like, “All these other people are so amazing and you should be learning from them.” It was more like, “You can do any of the things they do.”

OK, so James Jordan has built the basketball hoop in the family’s backyard in Wilmington. Michael’s first love is baseball, but he starts hooping one-on-one in the yard with his older brother Larry, whom he idolizes, but also wants to beat. Then there’s the hinge moment of the rest of his life, when Michael expects to make the varsity team but tries out and doesn’t make it. Tell me about that part.

I’m not sure I accept the premise. The best competitive programmer ever is Gennady Korotkevich, who works here [at Cognition] now.

Sure, but being like Mike isn’t just the rings. It’s also the multibillion-dollar shoe business and turning every banality into a competitive game, and the 1000-watt smile and the hot Cuban wife. Who by the way I think was a model for Alexander Wang. The other one.

Okay. So the first math competition I entered was at Southern University in Baton Rouge. They were hosting it as a thing for middle schoolers and high schoolers—sixth-grade math for sixth graders, seventh-grade math for seventh graders, etc. As a second grader I entered the seventh-grade math division. At the award ceremony they were calling out the names, and I was expecting to hear my name at some point. And then it didn't come. I was so salty. I was just… man. I still remember this. I was just so, so, so salty. That was one of my earliest memories.

The same competition, I went back in third grade, at which point I was taking Algebra 1. I competed in the ninth-grade division and I think I got a perfect score and got first place. From a very young age I always wanted to beat everybody. It didn't matter that they were older than me.

I definitely didn't have that many friends in second grade. But around sixth grade, middle school, is when it started. In middle school, there's a competition called MathCounts. The top kids in school go to the city competition, the top kids in city go to state, state goes to nationals. They fly everyone to Disney World and it's a three- or four-day trip. The competition starts with a big written round during the morning—everyone's in a massive room taking the test. Then the top 12 kids get selected for what's called the countdown round, which is head-to-head.

This is the viral video of you and that poor girl.

That’s right. So I went to nationals and met a lot of the other kids there, and we’d keep in touch over Google Hangouts. That continued on in high school. There were summer camps for top students in math and programming. All of those folks were a lot of my closest friends. I started Cognition with many of them.

Most of them as you can imagine are from Cupertino, or around the D.C. area and went to Thomas Jefferson, or they went to Stuyvesant in New York. I was from Louisiana, but I still felt much more similar to them. We were always very tight. That’s how I met Jesse Zhang, Johnny Ho, Jeffrey Yan from Hyperliquid, Alexandr Wang from Scale. Jeff was one of my best friends in high school and college. Alexandr was like my best friend from middle school. He was from New Mexico, we were kind of similar in that way.

Third-grade Wu wins the ninth-grade math division. (2006)

Why’d you switch from baseball to basketball?

I always did both math and programming tournaments, but I was better at programming. I loved the programming competitions more for a few reasons. The actual topics were much more up my alley. In math, I was very good at probability, combinatorics, counting problems, basically a lot of the same things as strategy games: computing the different options, understanding what leads to what. Versus something like geometry, which I was fine at but didn't enjoy as much. Programming is much more like combinatorics in all of the problems.

It’s also…there's all this terminology in math. Like, oh, here's where you use Pascal's principle, or here's where you use the Brahmagupta Theorem, or whatever. Funnily enough, one of the things that's been fairly consistent for me is I've never learned the names of any of these things. I've just always thought about it at an intuitive level—like, when this is true, then therefore it has to be that. I often don't know what something's called or what theorem it is. It just feels like one case of a broader, universal truth.

This happened literally yesterday, when we had a research presentation about quantization-aware training. They were going through the terminology and I said, “I'm sorry, I don't know what these terms mean. But you're just talking about the part where you do this and then that, right?” I guess I've always just thought of things in somewhat more intuitive terms. I think it is a bit unusual.

And the thing about programming obviously is just that it’s so nicely practical. I remember training for the MathCounts national competition and making myself a program that would feed me different mental math problems. I would play my own game that I created and use that to train. Realizing you could actually go and build things, that was very satisfying.

So you win gold at IOI three times, you become a coach and mentor to the youngins, you’ve got your distributed friend network of math and programming geniuses, then from what I understand you dropped out of high school. Is that legal?

I technically graduated, but I left high school after junior year. The child labor laws in California say you cannot work full-time in the state until you're at least 18 years old or you have a high school diploma. I realized that because I wanted to work in the Bay, and I basically figured out what I had to do to get a diploma. And I did that.

So there’s this question of how it is that many of the greatest mathematicians and physicists of the 20th century, who ended up working together at Los Alamos or Met Lab or came up with the models that underpin modern math and physics and economics, were all middle-class Hungarian Jews born in or around Budapest around 1900. I think it’s Szilard who coins them all the “Martians.”

In his book on the atomic bomb, Richard Rhodes speculates that the answer is something like: rapidly modernizing Budapest, plus assimilated Jewish bourgeois still excluded from older status systems, plus excellent schools and math competitions, plus a culture that valued intellectual distinction, plus forced migration to Germany and then America, plus wartime scientific opportunity. Which is about as unsatisfactory an answer as one can imagine, but I guess it might be accurate.

Having just analogized you to His Airness, I’m not about to compare you and your friends to the Martians, at least not to your face. But I do wonder what explains this founder cluster of people who do MathCounts and IOI together and keep up on Google Hangouts, then drop out of Harvard and end up in quant-firm training, then master the LLM opportunity shock, all of whom are Chinese-American and born around the turn of the millennium.

I wonder about this too. First, I think we're very fortunate. For all of us, one of the coolest things is getting to be here at the dawn of AGI. I think it is truly the most transformative event perhaps in all of human history. Certainly of our generation.

I have a few thoughts. I think one is that everything basically becomes Moneyball at some point. In poker, in the ’70s, ’80s, and ’90s, you had all these very raw figures, kind of the Hollywood trope of the troubled past. They played with very deep intuition for the game. They definitely thought about it analytically, but they wouldn't be out there nerding out about probabilities. But over time, a lot of what poker evolved into is actually just that: It’s a lot of math, and a lot of the best poker players now are very much math nerds. The same was true with chess, and the same is true with a lot of these games.

I kind of think startup founding has a similar arc. There was this Steve Jobs-type profile of the past. But now I think AI in particular, and to some extent crypto as well, has really led to this very technical profile. Of all the big companies people talk about in AI, so many are run by people from a fairly similar community. OpenAI comes to mind. Greg Brockman, for example, was older than me, but he did all the same competitions.

He was top 24 in the U.S. in math. He was on the U.S. Chemistry team, he got a silver at the Chemistry Olympiad. Mark Chen, who runs the research team at OpenAI, we coached the IOI U.S. team together. Jakub [Pachoki], who's their chief scientist, I used to compete against him all the time. He was from Poland but we did the same international competitions. Dario from Anthropic was on the U.S. Physics team.

Long story short: One, there's obviously a deep analytical element that carries over. Two, in AI especially, having a very strong technical background has, for better or worse, shown to be extremely valuable, because the problems you're solving are very deeply technical. But for our vintage in particular, there's also an element of infectiousness. I give Alexandr [Wang] a lot of credit for that, because he was really the first of our group to say, “Okay, I'm going to go start a company.

” Since we were all so tight and knew each other, it was very much like, we should all go and do this. Alexandr and I had this doc in middle school that I still remember of startup ideas. I'm sure all the ideas were really bad, but we were already thinking we should start a company at some point.

OK, so that explains the timing part and the IQ part. Not every period in technology history has been this IQ-pilled; there is something about AI where a disproportionate number of people who’ve made important advances in AI, like Noam Shazeer, are also math olympiad champions. Maybe the IQ bit also explains the first principles reasoning ability, like seeing further down the tree to, you know, this is how good the models will or won’t get, this is how scaling and RL [reinforcement learning] are gonna work, this is what will happen with the data and processing power, etc. I get that.

But it still doesn’t explain the other piece, which is you also have to be a leader. You have to be able to attract other very smart people. On a stage full of Muppets, you have to clearly be the main Muppet. In group pictures you have to seem to be lit differently. You have to have product intuition and people skills. If there are 25 of these gold medalists a year or whatever, and let’s say a 10-year period when this is relevant, you’re talking at most about a pool of a few hundred people out of the billions of people on Earth. And the vast majority of them do not have a single ingredient of founder energy, let alone several.

Which is why I’m still curious about the Chinese-American part, which wasn’t the case until extremely recently. I’m trying to avoid the obvious answer here, which I’d obliquely analogize to Jews and Hollywood. Or banking.

It feels like you’re setting me up to come off as a smug asshole.

Those are my shortcomings, not yours. No one will be confused about that, I promise.

I guess my take on this is that the two qualities that really matter in our fields are intelligence and hunger. The other stuff you mentioned flows downstream from the two fundamental things that the competition community selects for, which are a) you’re super smart, and b) you’re salty and want to beat everybody up. If you're smart enough to always analyze and adjust and think about what you did wrong and do it better next time, and you're motivated enough to just do that nonstop, day in and out for several years, I think usually you can get pretty good at any of these other downstream things.

As far as the Chinese-American part, there’s what I mentioned before about it just being kind of infectious. We were all very, very lucky to have each other. A lot of people don't realize that entrepreneurship is an option that’s available to them. I just think we were lucky to go through this together and watch each other bloom.

I’ve taken us off track, so let’s speed up a bit. You go to California at 17, what year is this?

OK, so you’re a child slave under California law, which makes an exemption for the diploma you’ve finagled, which allows Addepar to hire you. You do that for a bit, you do the HRT [Hudson River Trading] thing with the other Martians, you go to Harvard and drop out, you start LunchClub, then Cognition. What stands out from that almost-decade?

Yeah. So I got to know Vlad [Novakovski] through Johnny [Ho]. Vlad was VP of engineering at Addepar. I worked as a software engineer, I did performance engineering and optimization. Addepar had a lot of financial analytics that had to be calculated for customers, wealth management firms. A lot of what I worked on was just making it run really fast. Over the course of my year and change there, some of the particular factors got around 100x faster.

I also got to spend a lot of time on recruiting, I think I was maybe the number two or three interviewer at the company that year, going to MIT campus events and stuff. All the recruits were older than me, but I'm just explaining, guys, you've got to come work here at Addepar, here’s why.

What’s your business acumen at this point?

Zero. I’d spent so much of my life doing math and programming. I could make stuff, but in terms of learning how the world works, I had no exposure to it. It was a lot of learning from scratch.

So then I left Addepar to go to Harvard for about two years. I was an Econ major, which is kind of hilarious. I kind of knew I wasn’t going to graduate. I took writing classes, public speaking classes, philosophy classes, one CS class. But I didn't really care about the school part of it. Harvard would always do this reading week before finals, like a week where everybody would be finishing their essays and studying. I just didn't care at all. My entire week was just hanging out with different people. Many of them were studying the whole week and didn’t see other people. For a lot of them I was their only break from studying.

Then I dropped out and moved to San Francisco to start LunchClub with Vlad. This was summer 2017. We raised our pre-seed rounds, we were in the first year of this SPC [South Park Commons] incubator, working out of there. The first idea was called Elliot Technologies—it was this app for doing scheduling and figuring out who you should catch up and connect with. Over time there were lots of incremental pivots. Instead of personal catch-ups, it was a bit more professionally focused.

Instead of just scheduling, it was about figuring out which new people you should meet. That's how things became LunchClub, which was a fun experience. We ran that company for about five years together. Made millions of meetings, hired about 30 people. There was a niche of folks who really loved LunchClub and just enjoyed taking the meetings.

Then various things happened. COVID, obviously, which was interesting for a product that made in-person meetings. And it was also just difficult to grow and monetize at some point. My mom also started getting quite sick in 2020. I left LunchClub in June 2022.

When I ask people about you, besides the competitions and card tricks and Cognition, this tends to be the first thing they mention.

She was diagnosed with stage-four lung cancer. She had a targeted medication, which she took for a few years. Then there were other complications and other things that came up.

The thing people mention though is not just that she died, but that you moved back home to care for her. You nursed her.

It was probably… It was one of the bigger events of my life. [Pause]

I ended up moving to Louisiana, I was there through the majority of COVID to spend time with her. I spent a little more than a year at home. In that way, COVID was honestly… For me and my brother it was a big blessing, because it was a chance to just spend time with our parents during that period.

In 2021, when things started opening up, I started spending more time elsewhere. I was in Miami for some time and then New York. Then around 2023, when my mom started getting a lot sicker, I just came back home again.

It was an interesting time overall. A lot of it for me was a period of figuring out what I wanted out of life. I had done my first company, which went okay but wasn't a smashing success. I had taken a break to explore various other ideas. I was taking care of my mother. It was a very emotional experience going through it all. But it was also an interesting time to think about what I valued, what I cared about. What I wanted to be my life's work.

Me and Steven [Hao] and Andrew [He] messed around with a lot and just learned about the world. There were some fun things in crypto, just learning about markets. There were some interesting things around exploring security, learning about zero-knowledge, for example. And then ChatGPT came out in November of 2022. Toward the second half of that year, a lot of what we spent our time on naturally was getting deeper into AI and really thinking about what would happen next with the technology. But I think of that as almost like a sabbatical, because I was also spending a lot of time with my mother.

My mom passed away on October 6, 2023. We started Cognition in November.

The Cognition house in Burlingame, CA. (Nov 25, 2025)

On the one hand, given your whole life, training our AI overlords how to program seems like the thing you were put on this earth to do. On the other, it seems like it was all pretty contingent. I guess to some extent it’s the same way for any of us, but how do you square that?

I can tell the retroactive version of the story where it was all on purpose, but I don't think that's quite the truth. What happened was we were exploring generative AI. Soon after ChatGPT came out, everyone was talking about all the different applications. At the time, most of what people cared about was text completion—the natural mental model being, this thing is trained on all the internet and is supposed to complete what somebody on the internet would say. ChatGPT is just a Q&A, like this.

But naturally, one of the things we were extremely interested in from the beginning was coding, because we're all coders. And teaching AI how to code is the coolest thing you could possibly try to do. There was a point in late 2023 where people were really starting to see that RL was working. I think of that as the start of actually the second era of generative AI. ChatGPT was the first era, but was a bit more basic.

How do you know that RL is working, besides knowing a lot of the people working in the labs?

There was that, but I’d also always kept up with the research, and you could see it in papers. It's an interesting phenomenon in AI. People used to publish all their results, and then that immediately stopped in November 2022, when they realized this was probably pretty commercially valuable. But you could still see the work that was done on training the model on math questions and code and so on.

There were some pretty interesting papers. A simple way to put it is that the logical next step was actually kind of obvious, but the next step wasn't published. We started to talk to people and learn more to understand this. We got to the point of thinking, okay, AI is going to get extremely good at this kind of logical reasoning. So what happens next? That was the seed of Cognition.

The actual jumpstart though was November 17, 2023, which was the day Sam [Altman] got fired.

I was having lunch that day with a few folks in New York. I was pitching this idea of: Reasoning is getting way better, RL is starting to work, now's probably the right time to start a new lab. At the time, this was very researchy—we didn't have a clear stance yet on what the product or business model would be.

Then that afternoon, Sam got fired. Naturally, we were just like, okay. If there's ever a time to do it, we should probably just do this right now. We’d already planned to and probably were going to. But it was a good impetus, a nice forcing function for us to try to go fast and to actually take it seriously. Which is sometimes what you need in a startup. The moment of just actually committing yourself and saying okay, we’ve all started companies before, we’ve all been founders before, but this one is the big one. This is the one for life.

So we flew to the Bay, we organized a hacker house. We emailed and messaged a bunch of the folks we knew who were working in AI. We said hey, we're putting this together, just come hack with us, let's explore and see what we come up with. We did that house in Burlingame, California.

Burlingame being the town named for Abraham Lincoln’s ambassador to China, who negotiated the treaty that lifted restrictions on Chinese immigration to America.

Right, that’s interesting. One of our conference rooms here is still named after the address of that house. So we did that for about two weeks. It was during Thanksgiving break. I was going to spend Thanksgiving with my dad, it was just a few weeks after my mom passed. But I was like, I have to do this now. We're going to make this happen. So I canceled my plans. It turned out to be nice to be with friends just hacking on that Thanksgiving.

Then we did another hacker house in December. We kind of just kept going with that until we had an actual company. And way past when we had an actual company, actually. We kept going with that house until January of this year.

At what point do you decide what form Devin will take?

We've always thought about it in the coworker form. Obviously there are some things that are different about an AI coworker versus a human coworker. But even two years ago, when arguably we were too early to the concept, that was always our view. It would be a full agent, it would have its own machine, it would go do its own work, it would work in all the same systems, it would be with you in Slack, with you in Jira or whatever, and just work with you.

Even down to the name—there's a reason we decided to call it Devin as opposed to naming it like a tool. We've always thought of it as its own being that can go and do stuff. As things have continued, I'd say that's more and more been the case for us. Two years ago it was kind of an aspiration; day-to-day you obviously still needed to do a lot of handholding with Devin. We're honestly now at the point where you really can just work with it as a coworker. One concrete example would be that at this point, a decent fraction of all of our Devin sessions—or all of the Devin sessions that our customers run—are not run by humans anymore. They're run automatically.

There are specific triggers that come up, or there are Devins running on loops that are looking for certain behaviors or things they find in the products that don't seem right, and then going and fixing those. But not only are the Devins able to do the tasks you give them, they're actually going out and finding the tasks to go do.

How do you know when an agent or model is becoming better and more useful in reality rather than just being optimized for the evaluations and benchmarks, which get saturated as the models get smart enough to game them? We’ve all met people of exquisitely poor judgment and incompetence who also aced the SAT, after all. As a competitive test-taker you’ve probably thought about this.

Once the language models started beating us at the AMC [American Mathematics Competition], the rest was kind of obvious. The AMC is a really hard high-school math competition. To your point, there's obviously a huge difference between working in sandbox environments versus going out in the real world.

However, I think of it less as a direct application and more as a proof of concept of what's possible. In order to do some of these really hard problems, you need to develop a lot of fundamental logical reasoning, a lot of creativity, and the ability to string together very, very long chains of reasoning and make sure they all line up. Knowing that models can do this with sufficient reinforcement learning and the right training data does suggest, pretty clearly, that a lot of these practical tasks can be done.

That first happened around early 2024, when the models started getting good at these kinds of math problems. In the last two years, you've largely seen the same techniques applied broadly. I'll call out something: I don't think there's been an enormous breakthrough in the last two years on the size of attention or even RL itself. If anything, what's happened over the last two years has been a couple of other things.

One is just scale: more compute, more data. But besides that, a lot of it is actually figuring out how to tackle the practical problems that people around the world are dealing with every day, figuring out how you train the models against those rubrics, helping the models understand what is good or not good for any particular task, and then deploying in the real world and building the product experience that actually gets you there. It's actually this messy, practical work over the last two years that has led to the explosion of AI usage and the value that people are getting from AI. More so than anything from a pure researcher technique side.

When will model intelligence stop being jagged? When will they stop making really dumb mistakes?

Maybe one way to put it is: I think they'll always be jagged, but there's a level at which the jaggedness intersects with humans. It makes sense, because if you ask what a human is trained to do, it's very different, and naturally you should expect different capabilities to come up as a result. Maybe the most obvious example is working in the physical world versus doing knowledge work and thinking.

Models are trained on all the tokens out there on the internet, and of course it makes sense that they'd be better at knowledge work first. Whereas humans, the first thing you have to learn how to do is walk and talk, and more of our neurons are devoted to those kinds of problems. So I think that jaggedness will always be there. There's an interesting question of whether it could be jagged relative to humans but still be the case that, across all these different capabilities where humans are near-perfect, the models will be able to do it just as well or very slightly better.

We're already close to the theoretical limit there, whereas on some of these other things where humans are just not wired to do it at all, computers that are optimized and trained for it will do very well.

Funnily enough, with coding and math… It's in some ways a wonder that humans can do math problems at all. I don't know which part of the caveman survival experience taught us that it was important to do math. But it's there. In that sense, it's maybe not so surprising that models really focused on coding—which is literally the art of talking to a computer and telling it what to do—can train to do better on that than a human can. From that perspective, it's not so crazy that this happened to be the first great AI use case. But here's my point: It almost says more about human intelligence than it does about computer intelligence.

I hear people talk about how everything in the industry is going to converge, in the sense that the model companies get into the application layer and the app layer gets into building frontier models. What’s stopping you from becoming a model company?

I think focus is really important for a company. You can have different functions, different verticals. But I think you can only have one goal. You can't be a company that's trying to build this and that while also trying to solve this fundamental research problem. If I ask what Cognition's one goal is, I would say it's to make it much easier for everyone around the world to build software. That is the goal we go in pursuit of, which I think is different from the goal of a foundation lab.

In pursuit of that goal, there's work to do in model training, and there's also work to do in products. But that is our fundamental goal. I think the DNA of a company is in some ways kind of rigid, more so than people usually expect. The mission can change over time, but not as much as you think. At some point, everyone at the company is only thinking about one thing: the customers you serve, the products you build, the business model you operate. All of those are rooted in one particular thing.

So yes, I think both sides are going to continue moving and branching out as you say. But the core DNA of a company like ours, which really exists to bring the benefits of AI coding to people around the world, is going to be pretty different from a company that is really much more focused on advancing the state of research itself.

We really like being Switzerland. There are a couple reasons. First, we see what we do as building the necessary infrastructure to be the trusted partner that companies can work with to figure out how they can go much faster and ship much more. So I think there's a lot of value in being neutral, honestly. You kind of saw this with Databricks and all the clouds, but here I think it's even more extreme, because signing away our trust to a specific model would be a scary thing to do.

Nobody even knows who's going to have the best model in 12 months. And so I think that by getting to work with all the different models, using each whenever they're best for specific use cases, we can quickly adapt and help our partners understand how they should be using different models for their work. I think that's very important.

Beyond that though, I would also just call out that there's a real discourse that's happening in tech right now along the lines of, oh, why even start anything new? Is it really time to build something from scratch? As if there can only be so many companies, because the existing ones are going to go eat everything. There’s a certain cynicism which has gone too far in Silicon Valley about whether or not it's possible to build meaningful new things from the ground up, which I think is totally misguided. So we also just enjoy the challenge and ambition of getting in a room together and making something great ourselves. It’s just more fun this way.

What does human genius look like in a world of AGI?

I think for everyone in AI, one of the biggest guiding principles is the fact that the human brain exists. We don't know everything about the human brain, but I think we know it well enough to say that ultimately it's just circuits—basically a computer made of meat. There's obviously a lot of detail and a lot of interesting questions about how this came to be. But in AI, we literally call our circuits “neural networks” because they're named after the neurons in the brain.

You can see in the way human brains are wired, they can learn and solve some incredibly difficult problems. The next realization is that with enough time, enough scale, and enough energy, we should be able to make electric brains that operate in much the same way as human brains do, except we can make even more of them or add more circuits. People tend to ask, well, what is something that humans will always be able to do that AI will not? It's kind of funny, because the last eight things people proposed as answers have since been proven wrong in the last few years.

I remember when people used to say, sure, maybe computers can do chess, but not Go—no computer will ever be able to play Go, because it requires some deep intuition and deep understanding of the world that a computer just can't grasp. Obviously, that didn't turn out to be true. Same thing with solving math questions, producing code, taking novel information and proving new theorems. We've already crossed those points with AI. To your question about what the role of the human mind will be, I don't think there will be any one capability that in the long-enough term AI will just never be able to do that humans can.

However, I think there's a lot of intrinsic value in the human experience itself. My co-founder Walden says, “We've always lived in Minecraft survival mode, and now we're going to be living in creative mode.” I think that's right. We all have desires, things we want to build, ways to express ourselves, ways that we find joy and meaning. We will still want to go do those things. And now we will have AI with us that can solve extremely difficult and powerful problems.

But humans are not in fact computers made of meat. We have things like dignity, which in the tradition we both belong to is bestowed on every person equally regardless of their capabilities, and therefore cannot be outcompeted. We’re born from particular parents at a particular time into particular places, with names given to us. We come from somewhere. We’re made in large part through the things we remember, which we don’t choose, and it shapes things central to our understanding of consciousness, like the idea of “home.”

If this is all just reproducible circuits, then isn’t referring to “intrinsic value” just a sedative? Or a cop-out?

The version we just talked about is much more the long-term end game of what happens with AI. In the short and medium term, AGI is not really binary—it's much more of an ascent. For the next while, there will be incremental steps of, well, it can do all these things but it can't quite do that. Then we'll solve that one. For those reasons, I think a lot of these high-context, general-purpose tasks—where you have to know a lot about what's going on across the world, take in a ton of different information, and come to the right intuitive conclusion—those are things where in the medium term humans have the edge.

As time goes on, if AI has the same context a human has and is working with the same goals, then eventually it should get there.

Now, I think one of the biggest advantages that I or any human has is just the wealth of soft context that we all have access to. If I think about the work we do at Cognition every day, or the work at any company, there's so much there that's not just solving a fundamental logic problem from scratch. On the pure logic, the AI is already basically as good as any human at most of it.

On the other hand, if I think about all of the little decisions that we make day to day, or the principles of how we've chosen to do things, or the history of, “here's why we architected the system this way initially, but ever since this new technology came out we've decided to move toward this other framework,” or, “here's how we specifically do things at Cognition that are different from most other companies,” all of these things are where the human ability to do context and retrieval, and to have access to all this knowledge, is really, really powerful.

I kind of like to say that retrieval is one of the things we really do best, which is actually kind of to your point about memory. If you think about models compared to the human brain, the model has a context window, it has tokens it can see and read, tools it can call on, and perhaps one of those tools is retrieval. Our context window is way worse. My joke on this is the six-digit codes you make for two-factor authentication. If those were 10-digit codes, I don't know if we could handle that. That's how short our context window is.

On the other hand, you have this really beautiful way as a human of pulling up one file of code and suddenly getting hit with this memory of, oh yeah, four months ago I was working on this and we had this bug, and here's what happened. And that's exactly what you needed to know for this particular task—

The programming nerd’s madeleine dipped in tea.

—or you'll be talking to someone and go, oh yeah, didn't you say something about this seven months ago? That's different from raw context. It's really accumulating and understanding all the soft context of things we've experienced. I think that is the main thing that humans have over AI right now.

Now, there are a few categories of problems in AI that folks are working on around this. One is embedding search, which is in some ways the literal parallel: from a massive database of all previous conversations, all the different files of code, or whatever data you might be searching, how do you retrieve exactly which pieces are relevant to what you're searching for right now?

But there's another one actively in progress, which most people in the space call continual learning. What that means—and I think it's a little more akin to how the human mind works—is, how does the AI update its own weights based on what it learns? You have this general idea of the neurons that fire together wire together, where you want to be able to update the weights of the AI itself as it's going and having these conversations. In theory, that should allow it to recall for the next thing. These are all topics that are actively being explored. I think there will be a point, maybe not so long from now, where AI can do this. But for now, humans have the edge there, too.

In some sense, solving AI really does mean solving cognition. It's a crazy thing to really wrestle with in your head, because for all of human history, we have always lived with the fact that we can speed up this thing or that thing, but there will still need to be a human that has to go solve the problem itself and figure out how to do that. There will still have to be a human that goes and invents the thing.

Up until now, what we've taken as the most core part of what it means to be human is the ability to think and solve novel problems. And a change there doesn’t mean it’s a bad thing—I think it'll be a very good thing to have AI that can help us with that. The poetic way to put it is that AI is the lever that will ensure our dreams don’t have to forever remain dreams. But to your point, it’s also time to actually really rethink and recenter on what makes us human.

Here’s the analogy I would give. If you think about all of our ancestors from hundreds or thousands of years ago—what would they think of us? They see us pushing buttons on a computer, then getting in a room with other people and just talking, and we call that a meeting, and that's what we call work. Meanwhile, they're out in the field seven days a week making sure they have enough food to eat.

What changed to get us here is basically that we figured out how to automate raw physical strength, meaning humans get to do work that is primarily based on their own knowledge and thinking. That's what the last 200 years have done for humanity in one line. And now the thing we're looking at is being on the verge of solving raw mental strength. What does that bring? The silly examples are things like, you want perfectly nutritious delicious pizza? You want a Ferrari? Want to dunk a basketball? Well, now you can.

More seriously, I think this shift from survival mode to creative mode is the right one. It’s not going to be in the next two or four years, but it’s coming. At some point you really will only be capped by what you can imagine and the ideas you have for how you want to spend your life, which will be a very different life. To your question, I think a lot of it will be pure self-expression. People will get to live the lives they want to live. The bottleneck to turning our dreams into reality will dramatically narrow. And the way we live now will seem crazy.

When I first hear this question, again my immediate thought is about a lot of the technology revolutions in the past. In Silicon Valley, the things that come to mind are the mobile phone, the internet, the cloud, the personal computer. But you can go all the way back to electricity or the Industrial Revolution. While I think AI is bigger and maybe a little different in form than a lot of those, I think actually a lot of the high-level is quite similar.

In the long run, it's an extremely powerful capability that can change all of our lives and allow us to do much more. I think we all agree that being able to live the lives we do right now instead of having to spend 90% of human labor on farming has been a really great innovation.

On the other hand, in the short and medium term, I think there are real frictions. If anything, the biggest thing I worry about is not even AI itself, but the frictions that will come with how fast all this moves. A lot of what that points to is really a lot of emphasis on good education of folks around the world—how should people use this technology, how can it make people's lives better, and getting everyone on it at the same time.

I think the bad outcome is a scenario where there's a substantial period of time where it's just the elites in SF or whatever that have access to this, and everyone else does not. Or to call out a specific one we're all experiencing these last couple weeks: If there are very good cyber capabilities that models have, but a lot of the world is not using those same capabilities for defense in the way that intelligent hackers out there are using them for offense, then you get to a world where the balance gets pretty tricky.

So it's actually really important that the whole world understands this technology, is able to use it, and is able to work with it. A lot of it is just how fast we can get humans to prepare for it. In the Industrial Revolution, it took place over the course of multiple generations. It was a very gradual shift where people would learn over time, and these changes became more and more a part of people's worlds incrementally. What we're seeing now is going faster than that.

I think people will be able to adapt and we will find all sorts of things where having new technologies is going to make our lives better—for drug discovery, for making pizza taste perfect and also perfectly nutritious, for all the technology we use every day. But it really does mean that folks need to be on top of what's going on. In a lot of previous revolutions, if you adopted two or five years later, you were still mostly in time. In this case, it has to happen faster.

It’s hard to contemplate when you’re not one of the people actually working on AI, which is partially why those of us on the outside tend to experience a lot of the major AI figures as full of shit, at least when they speculate about the future.

One of the things I think about is the METR study, which shows that now agents can do hours of human-equivalent work before they need to be interrupted, and that has continued to double every few months. That's how agents have gotten from seconds of work to hours in the last couple of years. The obvious question is, okay, what happens if that keeps doubling? Instead of hours, we're talking about days and weeks, and then a year.

It's funny because we've seen it happen already along the curve from seconds to minutes to hours, and yet it still feels almost foreign. Speaking of things the human mind isn't so good at—we really have a tough time thinking about exponential curves. What does it mean to have an agent that can do a year of work? You just give it a task and it says, I'm on it, and comes back and it's done the next year. That sounds crazy, but I think that is where we will get to.

I think a lot of folks outside of AI always have the sense of, okay, wow, the curve spiked, but here's where it stops and plateaus. And then it goes up again, and they're like, okay, well, but that's the ceiling. I think part of the reason it's different for folks in AI who are seeing it is that you see every little data point on the curve. You know how it got there. And it wasn't 14 breakthroughs in the last three years, it was maybe one. But we see what we can do if we continue down the path and push the capabilities further. And that's still something that, even for everybody deep in AI, is really hard to grapple with.

How much of the drudgery of daily life can you automate away for me, so that I can get back to my preferred pastime of smoking weed and looking at grass?

A simple way to put this is: How many ideas does one person have in a day, and how many of those things do they actually get to do? Until that proportion is 100%, you know there is a pretty meaningful bottleneck in terms of the drudgery of execution. I don't think it’s anywhere close to 100%—it's probably like 5%. You have all sorts of different ideas of things you'd want to do or build or try, and in practice it's much harder to actually do them.

I think of that as one of the big things AI will unlock. You see this especially in software. I have never met an engineering team that thought about their work as, alright, we're shipping this project this month, and then next month we're all done, no more software, we've built everything we wanted to build. It's always the opposite—you have 85 projects and you have to pick six because that's how much bandwidth you have. What we'll get to see is that so many more people will just get to do all of the things they want to go do.

When people say they love building software, what is it that they love? In reality, there's this 10% of the job which is really just getting to express yourself—thinking about the trade-offs you want to make for every problem, what architectures you want to use, what the specific product is you want to build. And then there's this other 90%, which is code monkeying and execution. When people say they really love building software, it's usually the former that they gravitate towards.

It’s the same in every field, and we'll get to a point where you can just do 10 times more of the 10% you actually love.

Final question. I am interested in people who in one way or another were blessed or burdened by the gods. They have a gift. They might also work hard, have been born to good parents, had access to good schools, found great friends and mentors, gotten lucky, been born at a propitious time, done whatever Malcolm Gladwell commands them to do in order to achieve what they’ve achieved, etc. But at the end of the day, there is the gift. And that’s the knife’s edge in their life. It can be used for good. It can be abused. It can be squandered or exploited. It can do the thing you say AI will do, make all your dreams come true. Or it can be run into the ground. It can undo you.

So I’d like to know: As a man who was given a gift, which is some combination of a 16-cylinder math brain with salt sprinkled on top, what is it that you believe you are choosing to do with that gift, and are you choosing wisely?

The simple way to put it is that… There’s a chance AI is the most important event in human history, and coding is how it will learn to act and build things in the real world. If I was meant to do something, if I’m here for a reason, I think it’s to be the guy who teaches AI how to code.

I could fail. If it turns out I wasn’t the one meant for the role, I’d be sad, obviously, because I’m super salty and competitive. But the world where I actually couldn’t live with myself is the one where I took this thing that was a perfect match for everything I’ve always naturally loved and cared about, and been fortunate to have some aptitude for, and just didn’t give it my all. It’s the outcome where I didn’t try hard enough, or didn’t have enough motivation to do it, where I don’t think I could live with myself anymore.

Yep. Here, we should try these strawberries.

Correction: A previous version of this article did not make it sufficiently clear that an accusation made against Mew2King was false and later retracted. That sentence has been revised.

Jeremy Stern is the editor-in-chief of Colossus.