John Schulman 谈死胡同、扩展 RL 与建设研究机构

OpenAI, middle to late OpenAI, Anthropic, Thinking Machines, Google all. Um maybe some subset of these like how would you characterize their differences? And what types of problems are those environments best best suited to solve? Um and like, you know, maybe the popular conception of each of these places is uh Google's a place with great engineering, but maybe it's historically been a little slow moving. Um uh Anthropic's more safety focused.

I imagine maybe Thinking Machines is a bit more product focused than some of these other environments.

well? Yeah, that's a um like that's a very pretty broad question and it's hard to like talk about all these different places. Also, they they've all changed over time. Perhaps uh early OpenAI versus Thinking Machines. Actually, I do see a lot of similarities because I think uh there is a lot of um like there are there are several different things that people are working on in parallel and we're still sort of shaping the vision of the company and uh the vision is going to sort of emerge out of um like uh seeing these different projects take shape. It's also a different point in the history of the field. So, um like the field is moving

really fast now and there are other uh like companies that are moving quickly. So, uh there's some like pressure to catch up to the um like current state of the art alongside whatever um new things we want to do at Thinking Machines. Whereas at like in the early days of OpenAI, the field was uh like there was a um I mean, obviously there was DeepMind, but it wasn't like everyone was uh competing in some coherent uh like direction. There wasn't like a clear direction to go in. Um maybe there was uh like some idea that um like you wanted to um like scale up RL and make RL work better and discover the uh like

better architectures and and so forth. But I'd say it wasn't like there was a um there was some axis that everyone was trying to scale on. So so to some extent, it was more like peacetime uh in the early days of OpenAI and that led to a lot more exploratory work. And uh whereas I think a lot of companies that have started uh more recently are uh like more compelled to be in catch-up mode for a while and and first to sort of uh replicate the state of the art. Um so, actually I've been try um like I've been uh pretty aware of this and try and and trying to um I've definitely tried to

make sure we're not just in catch-up mode and we're also like building up a lot of muscle around doing exploratory research and exploring new ideas that aren't necessarily along the main path of uh that the rest of the field is going in because um I think it's easy to uh if you just if you're just in catch-up mode, it's harder to build up that like exploratory research muscle later. In the culture, it it takes a lot of uh like building the right culture is is hard to do later. Why aren't you value functions popular in RL? Right now. Yeah, I'd say they don't seem to help very much uh in the the settings where um

the peop- people are doing RL on right now. So, for example, doing um uh like RL from human feedback and RL on uh on these verifiable rewards with a fairly short um like time horizon. Um or or actually um I don't want to just say um we're working on short time horizons now because if you're sampling tens of thousands of tokens, that's a pretty long uh time horizon, but uh yeah, the current um the current set of um tasks that people are doing RL on um for some reason, value functions don't seem to help very much. And um it's a little um yeah, it's hard to it's hard to know why. Um I mean, I'd

say uh I'd say value functions help a lot. Value functions um give you variance reduction. That's the their main purpose and uh so uh so for some reason, you don't get that much variance reduction on this current uh like set of tasks. I don't think uh whereas you do get a lot better uh variance reduction on some other tasks that people have used um for RL research. So, uh why that's true, I I couldn't say. Um I would expect value functions to make a comeback at some point, though. What's your best guess at how we solve continual learning? Um and do you think Laura will play a part in it? Yeah, I mean, I'd say

continual learning uh might mean several different things. Like there I mean, there's several different kinds of learning. Uh there's um sort of like uh like to use more of uh psychological analogy, there's like motor learning and there's like episodic memory and and learning new knowledge like uh procedural uh memory. Um so I'd say there's different uh kinds of learning that might benefit from different things. I'd say um I'd expect um I'd expect the uh like in-context um methods or context management to go to continue to get better and long-context uh abilities to continue being important. I'd say Laura um I would expect Laura to stack um stack on top of that or basically like parameter

uh fine-tuning to stack on top of that and uh be better for some kinds of memory than others, especially ones that require a lot of capacity and absorbing a lot of knowledge and maybe like it's hard to say exactly uh Do you think we will need ideas other than putting the right things in the context window and then a bit of uh parameter fine-tuning on top to solve the problem of we deploy these systems into the world, we want them to learn new things on the fly. It's hard to say what we need to what we need because um I'd say if we keep scaling models and making them better, like all of whatever metrics

we write down will continue to improve. On the other hand, um Yes, so so it could be that we'll get uh even if we don't change any of our methodologies and even if we don't do parameter fine-tuning, uh we'll we'll eventually solve all these problems. Though it's also likely that um there's some new ideas that'll solve those same problems faster and might uh give you a different scaling law where it's either like a different uh um like an effect an increase in an effective compute by either a fixed multiplier or maybe you have like a different slope of the scaling law um if you use a different method. So, I could see um Yeah, I

could see other methods um giving you a much uh better uh scaling law or faster like yeah, more effective uh continual continual learning. Um It's hard to say exactly which um tasks will benefit the most from parameter fine-tuning, but I would expect um like it's a I would expect it to help in a certain interme- intermediate regime where uh like um you have more I would expect like in-context learning to help in the in a very short horizon regime and that it's to be like really hard to beat in the uh over a short time horizon, but I would expect um weight updates to win over a longer time horizon. How worried are you about

brittle generalization being a blocker to having uh general AI um working in useful ways across many areas of knowledge work? Um are you worried that we're going to um do good pre-training work and that will get us only so far uh and then we will use RL. Uh RL will um work for the domains and distributions in which we're RLing things on. There won't be great transfer. Yeah, I'd say it's hard to speak clearly about how well models generalize uh and like what their sample efficient what what the um how their sample efficiency compares to humans because uh like in within-context learning, models can have very good sample efficiency that's uh on par with uh

humans or or better. Um And then um but then it seems like um certain kinds of training requires a lot more uh data than um who uh like what it takes humans to learn the same thing. So, um I'd say um Yeah, I'd say there's there are some ways in which um models are a lot more brittle than humans, but it's uh yeah, it's hard to give a really clean description of uh what those are. Um and I think it's sort of um I think it's a human start to um do better on longer time scales and uh Yeah, because we have been optimized by evolution to operate over an 80-year like time horizon. So,

we have there are a lot of sort of self-correction mechanisms that that occur. Um I mean, people aren't perfect at uh self at uh like correcting their errors, but they're pretty good at it and people um if you give people a goal and uh motivation uh to pursue that goal, they will uh like real they will be um very uh like resourceful and try a lot of different things. So, I'd say um uh And like uh and models will tend to um models can also be uh very persistent and uh uh in some cases more persistent, but uh like also tend to get stuck um more easily over when doing larger chunks of work. So,

to what it's hard to say um whether this is just a temporary phenomenon and uh like the time horizons of models are about to increase like drastically or um or whether this is just like some fundamental weakness or or rather this this will take a really long time to get up to the human time horizon. Um and it's actually hard to figure that out because if if we're talking about like uh a time horizon of decades, then obviously it takes decades to test out um like uh to evaluate the models for this. In a world where it becomes much more popular to co-train models together. So, maybe just you have a general setup where you

have a generator um trying to solve RL problems and maybe the way you assess rewards starts to involve models, too. You're maybe co-training judges and generators together. Do you think any ideas from 2010's GANs or like any old ideas that have been forgotten about training models together will be useful or important um or things we we should take inspiration from or or not sure? Yeah, I think um like co-training generators and verifiers makes a lot of sense at uh because you can uh like in in theory you get some kind of self-improvement because uh if you have a model that's um like doing reasoning and instruction following as part of the verification process and you're

using that to um provide learning signal to the generative model, then uh like as the model gets better at reasoning and following instructions, it also becomes a better verifier and you have uh somewhat of a virtuous cycle there. So, uh I think that makes a lot of sense. I also think um I'm pretty uh fond of ideas around like multi-agent training or games, uh like setting up uh games where you have either a zero-sum game or a multiplayer game uh where um uh yeah, where you can sort of design the game so that um the equilibrium is is something very interesting. And like games give you a lot of uh give you a lot of

nice properties like they give you an automatic curriculum uh like because uh like the um if you're a player in a game uh like if you're playing against copies of yourself, uh your opponents are getting better at the same as you get better. Um they're also sort of um they're sort of theoretical CS uh flavored reasons why um why sort of uh setting up games uh uh like games might be uh a good idea like uh it turns like this these ideas around um uh like if you like there's these different complexity classes uh that are defined in terms of uh like two-player zero-sum games where uh you have like a uh polynomial time uh

judge and uh like two players and the equilibrium of this game like solves a really hard problem. So, like using an inexpensive uh uh like a computationally cheap process, you can create an incentive such that the equilibrium involves solving a very hard problem and uh like there is some uh alignment uh literature about this idea. I mean, in particular, the idea of the debate game uh which uh I think was pretty compelling and I I haven't seen um Yeah, there's there's been some uh some work on that idea, but I I expect that to be more and more important that kind of idea to become more and more important. How do you personally use AI?

I use AI to for coding a lot. I mean, I use Yeah, I use cursor a lot and uh Claude Code and some other tools. Um I uh I mean, I spend a lot of time I have like uh chat windows open with different uh like uh with different models a lot and I ask them a lot of questions um multiple times a day. Like the types of questions you'd go to Wikipedia for or um do you actually involve them in the research process at all? I definitely use models for research a lot. I mean, I'd say uh I'd say if I have an idea now, I'll just um I'll just like fire off a

bunch of questions to GPT-5 Pro and uh have it do a bunch of literature searches for me or sometimes if I have a vague idea, I'll like write a paragraph or two and uh just tell like tell the model to flesh it out a bit more. Um and I'd say um Yeah, I find this quite helpful and uh Yeah, definitely the different definitely the literature search ability is is extremely useful cuz uh it used to take a lot longer to to find relevant literature. And um I mean, this this extends to also things like finding open-source libraries uh that like that's also much easier now. Um So, yeah, I definitely use models a lot for

uh like yeah, finding relevant literature and sort of iterating on ideas uh Yeah, I also use them for um like writing, giving feedback on writing a lot. Like yeah, I'll often have to um do most of the um like uh most of the thinking myself, uh but then I'll I'll usually um the I'll use the chat models as like my uh like first uh first round of feedback. What does a day in your life look like when you're doing research? I can imagine some people would guess that you know um to be as productive as you, you know, maybe working wall-to-wall, staring at data, talking with colleagues. Uh there's also another version of the world

where uh you know, maybe you're someone who uh actually uh is working in very short concentrated bursts, but taking lots of time to walk around and and think and generate ideas. Um but yeah, what what is a day in the life of of John Schulman look like when when doing research? Actually, I come to coffee shops a lot. I like thinking at coffee shops where there's sort of a uh there's just a buzz of activity around and I can just uh like sit with my coffee and a notebook and just kind of jot down some ideas and remove uh distractions. So, I definitely do uh some of that where when I'm in more of the

idea formation phase of projects or when I'm thinking more about what's the right thing to work on. And then at a certain point it becomes about execution and at those those times and uh like if uh if we're in a project that's more in execution mode um I'll I'll just I'll be spending more time either coding if I'm working on it myself or um I mean I spend a lot of time actually just like reading docs and uh like messages that other people have written or looking at their looking at their plots and and code. So I mean I'd say I spend a lot of time doing research advising now and and there I'm just

uh usually like reviewing other people's work. Do you think the skills someone would need to do effective research in 2019 or 2020 are the same as one needs now? Uh and in particular you wrote this blog post in 2020 on how to do effective research um and I'm curious if you have any updated recommendations for folks who are broadly you think it stands the test of time. Yeah um So thinking back to that blog post I'd say um a few Yeah I made a few points about different kinds of research uh like there's like you have goal-directed research and or idea-driven research and I talked about keeping a research notebook and building up your taste

by reading a lot of papers. I think all of that advice is mostly uh mostly still holds. Um so yeah I I would still endorse that advice. I'd I'd say um Yeah what's what's changed? I mean I'd say Actually I'd say uh keeping a lab notebook is probably even more useful now now that we have LLMs like context is so important. So if you want to get good feedback on what you're doing you can probably uh like paste your notebook into the LLM get some get some uh some feedback. Um yeah I'd say probably um the biggest change I can think of is just that uh like now it is really useful to figure out

how to incorporate like uh LLMs into your work and uh I mean that uh I So actually I haven't thought um I haven't thought hard enough about what's the best way to like accelerate your research other than things that just generally accelerate your engineering. And it's actually a little bit um Yeah it's a little bit not obvious. The The advice might be different for research than other areas of software engineering because I do think uh there's a lot of value in like understanding every line of code and uh like having having something very simple where you understand every line of code rather than just sort of uh like having the having the model just like

uh write large amounts of code that you've never read. So I I think like this style of AI-assisted coding where you just have a model like write a big code write it the whole implementation for you might work well if in in other domains where you kind of just want to define the spec and have the the model just write something that seems to satisfy that spec. But I think for research there is a lot of value in like knowing about exactly exactly what's going on in every line of code and the people I think the people who have done the best work really have have that understanding of the whole uh like uh down

all the way to the nuts and bolts. Probably since 2012 but uh it feels like especially since you know 2020 and uh uh the advent of scaling laws uh lots more researchers have entered the field of ML both in academia but then also in industry. Um but and and please uh disagree if you do uh it seems like the rate of consequential big idea generation has been fairly constant. Like the story of ML feels like the actual progress has been so big in building useful systems. But uh to build these systems there's uh the you know underlying few big ideas and then lots of details filled in to make them work and then a whole

graveyard of lost ideas that just haven't worked. Um but how would you explain consequential idea generation being constant even though number of researchers in the field has 10x or 100x? I feel like these questions about like quantifying the rate of scientific progress are always a bit tricky because you have to think about low-hanging fruit being picked and uh and there's also it's also very hard to um to really uh like uh measure the rate of progress in the recent past because I you sort of don't know yet which ideas are important. Um so I um yeah I would be a little hesitant to conclude that like the progress the rate of progress is um constant

and hasn't really accelerated even with the large number of people. I mean I think uh I think if you look back to um papers in uh like the like 70s 80s 90s um I think like the um the experimental rigor is lower. I think the standards have increased in certain uh certain ways for like uh definitely for the level of experimental rigor in terms of like uh trying out baseline methods and doing a lot of uh a lot of different experiments on like different tasks. Like you might like in the old days you might have an RL paper that had like um like some very elaborate set of ideas and just one experiment on a

toy task that like uh was like very questionable um and uh like this could be a seminal paper. Um so uh it and and a lot of the math like the mathematical uh ideas weren't very sophisticated either. I wouldn't be surprised if actually the rate of idea generation has increased a lot and like standards and sort of the level of um like actually quality uh has increased in in certain ways as more people have entered the field. Does that match your intuition? That's your intuition?

of my intuition. Yeah. Yeah. And I also think um like there are a lot of problems with uh with the like academic like publishing system and reviewing and everything and it's it's definitely like frustrating um to a lot of people who are part of this system but I also think um I also think it's it's not terrible because I think the uh like uh like the field is driven by a lot of uh like objective improvements uh that people are seeing in reality. Like like it is um the field is driven by real goals and real problems. So I I think that grounds to a great extent. So even though you have a lot of

problems and there is a lot of fake research I think overall the field seems to make progress. And actually as an aside how do you think the academic publishing system compares to the internal coordination system of these large AI companies uh in their Slack channels around ideas uh and do you think that there's anything to borrow from how a thousand-person research organization one of these companies works that can then be transported into open academia? Oh yeah that's uh that's really interesting. Um yeah I'd say the um if you look at the internal uh like the presentation of internal results at one of these big research labs I'd say I'd say it's better in some ways

and worse in other ways than the the publishing world. I'd say uh um it's better in terms of having a higher accuracy of like drawing real conclusions about things like what what improves pre-training. I'd say much better. Like people have much better methodologies. And uh like uh and like these experiments are much more driven in like real consequences as opposed to just getting a paper published. So So the successful companies have gotten better at at sort of drawing accurate conclusions. Um I'd say overall like no one uh is going to take the time to write a tech report in anything near the uh level of detail of anything published externally. Like no one writes really

detailed tech reports. I'd say even though the like the overall level of accuracy is is higher in terms of uh claims. Um I'd say the um just the thoroughness of experiments is usually less um in sort of internal research. Like people aren't going to uh try the baselines try as many baselines or um I mean they'll uh I mean a lot of academic papers have sort of uh have baselines that are nerfed in some way that you can't and you can't really trust the results. But at least at least someone the best work actually does is actually quite thorough and and does a lot of good baseline uh comparison. So yeah I'd say um I'd

say overall there's like uh some yeah the write-ups are much more detailed in the outside world and uh the and there's more thoroughness in certain ways but also less accuracy. Um and I would say overall um it it would be nice um I'd say that um I've been interested in like uh sort of at at these institutions trying to uh improve the like research writing culture and uh and try to get people to write like how to get people to write uh more detailed tech reports that really like uh that go deeply into the science as opposed to just doing like the minimal thing needed to like find the recipe improvement that's shippable and uh

it's it's definitely been a bit of a challenge because it like the incentives of the companies usually aren't to do like really uh to build up a nice theory and and do really thorough science. Yep. How have the characters entering the field shifted in distribution um from what things looked like in you know 2015 2016 2017? Like were people back then you know you on average more or less skilled um you know more you know worse or better engineers like uh creative um was there a difference in any way of like, you know, people attracted to doing this work back then uh to the set of people uh entering the field now? I mean, I'd

say uh You mean weirder? Yeah, I'd say maybe people were a bit weirder back then because I think now um it's sort of more obvious uh it's yeah, more obvious and um like conventional wisdom that uh like AI is uh AI is the most important thing going on and um so it's going to attract a lot of people who have more uh like conventional career paths and um are sort of uh less risk-tolerant. Um so I say um yeah, I'd say the previously the people were a bit weirder. Now there's more conventional um like conventionally minded people um I'd say overall it's it's hard to compare the talent distributions, but just given the numbers I

would say the uh yeah, I'd say the bar is high has gotten higher just because so many people are trying to get into the field and applying for all these roles. Yeah, I'd say probably um like engineering skill matters more now that uh it is more important now than it was before as opposed to like research uh taste and um like the ability to do uh like exploratory research because I think now scaling things has driven some like so many improvements and there's so much low-hanging fruit just from sort of scaling uh the simple ideas and executing on them well that I think people and also because the field has gotten more mature, so you're

usually building on you're not like writing a bunch uh some code from scratch in a Jupiter notebook anymore. You're uh like you're building on someone else's code base and there's a lot of like good uh like uh infrastructure to build on. So yeah, since um since you're integrating a lot with other people's code and um and tools uh like people who have more of a software engineering background have more of an advantage now. What do you think the future of RL research looks like broadly? Um and uh over the last 10 years it looks like a bunch of topics came in and out of focus. Um and then it looks like the thing that's publicly

worked well on LLMs has actually been fairly simple and it was pretty close to set of ideas that worked in other domains. Um do you think there's much more to do in RL research? Um and do you think that uh you know, the most capable RL LLM systems in a couple of years look look very different than some of the old ideas? Yeah, like you said, I think some ideas go in and out of fashion um and uh sometimes they're uh they um they become fashionable too early and then they they don't live up to their promises, but then they come back later. So I expect that to happen a bit more. Um so uh

yeah, it's hard to say exactly what what the important ideas will be, but I mean, I think uh I think like offline RL is a pretty interesting idea and set of ideas and and it's sort of um as a yeah, to some extent um what we're doing in the LLM world now is like uh sim-to-real or what in robotics they call sim-to-real where you build a bunch of uh like simulated environments and you try to do uh you try to do RL on them at at scale and then like you try to randomi- uh have enough diversity that you'll generalize to the real world and uh I think uh like actually I think sim-to-real is

still yielding uh yielding good results in robotics. So it's not like sim-to-real uh has been sort of uh discredited there. I think I think it is a very effective technique, but I think there's also a lot of value in learning from the real world. So I expect that to um that's to some uh like at some point come back to the LLM world where uh you figure out how to learn from real like uh sort of deployment. If some of the biggest AI labs um did develop really really really powerful AI systems to the point where it would be pretty important to coordinate with uh both with each other and also with you know, other

societally important institutions like governments how confident are you that they would coordinate well if that were going to be necessary for you know, the future of AI to go well and how worried are you that at that they wouldn't get along and that they wouldn't coordinate well? I feel sort of medium worried or confident. Uh I'd say um I'd say there is like a reasonable amount of uh commonality of viewpoint and vision between like the leading AI labs. Um so um I could see them uh and also there's been uh some collaboration between the labs recently. Um on safety things. Um I'd say uh I'd say there's definitely some uh like bad blood. There's a

little bit of bad blood between uh like the personalities involved, so that that might uh make it a little harder, but um coo yeah, I could see it um I could see it working out if if um if it became sort of more uh clear that this is the the right thing to do. Two, with uh this being a time where technology is getting better fast, um there's lots of talk about predictions of how fast the technology uh is going to get better uh and in particular lots of people talk about, you know, what's the estimate uh uh what when do they think AGI is going to happen? It's, you know, common topic. Um uh

where most people when they're talking about AGI are talking about something like um all knowledge work on computers being done by AIs instead of humans. Um and one way in which um I've tried to reason about this in the past is AGI seems like a big complicated engineering and research project. And at least in my personal experience, engineers and researchers are kind of abysmal at estimating uh for much smaller projects when they're going to finish them. Um and you know, I think uh in particular like the the systematic bias I've seen from from engineers is they will uh always assume that they're going to finish something uh much earlier than they actually do and you

know, maybe like uh the rule I apply is like apply like a constant 3x factor on top of their their predictions to get something closer to the actual uh time it's going to take them to finish something. Uh do you think that this is uh like a reasonable critique of most people's AGI timelines um and they will, you know, underestimate how long it's going to take because researchers and engineers generally underestimate how long things are going to take. Uh and then in your personal experience, you know, the researchers that you've been around, have they and engineers you've been around, have they been good about estimating project timelines? Yeah, I agree that there's a consistent bias

to underestimate timelines and yeah, maybe it's uh two or three x in the good in the good case. Uh I'd say um and um yeah, using that uh sort of heuristic um I think it is reasonable to uh to predict that uh like AGI will be a little further out than um people's uh timelines predict when they're using um sort of uh they have like a definite um prediction for how it'll play out. Um And um I guess we've seen um this in other the most analogous um problem is maybe self-driving cars and uh we've seen um we've seen this take longer than people expected uh to sort of get to the um yeah, to

to get to full autonomy and get to like uh robo taxis and everything. Um so yeah, I think that's a reasonable hypothesis. I think um I mean, the whole on the other hand, there is this positive feedback loop where AI accelerates its own development that's also also probably going to defy intuition. So I think uh people people who are incorporating that a factor um coming up with um like pretty short timelines and I also think that is somewhat compelling of a line of reasoning. So um I I think yeah, there's a lot of uncertainty about like how much uplift people get from AI and if there's like bottlenecks around humans understanding what's going on and

so forth. Um so yeah, I wouldn't make a really confident prediction either way. Two, you and Thinking Machines have released Tinker. Uh what is it and who's it for? Tinker is a um it's a low-level fine-tuning API that basically uh uh um it it gives you a small set of um low-level primitives to do training and sampling uh that lets you express uh like um almost all um post-training algorithms that you might want. Um while not having to worry about um like GPUs or or accelerators um and not having to worry about like a lot of um like distributed um like distributed systems issues you otherwise would. Uh so it's sort of trying to um

it's finding something that we think is like a good uh layer to abstract away and and building that as a service. Like people don't usually use um like services for ML training uh and the services that exist are much more high-level um and so it's kind of novel in in that it's this uh yeah, it's a service that uh is around this lower-level primitive. Like the closest analogy would would actually be uh like the sampling APIs that you can uh get from OpenAI and Anthropic and so forth where you uh like you don't have to spin up your own um like GPU box to do sampling. You you just uh you just do an API

call from uh Python or JavaScript or whatever. Tinker lets you basically write a lot of um like training code um by writing uh just writing some Python scripts and and having that just work and not and not having to worry about installing a bunch of uh installing a bunch of stuff to run on GPUs. Is your ambition that the next Thinking Machines that get started by a group of researchers would would actually just build on top of Tinker. Yeah, I would hope that a lot of companies would be able to just build on Tinker instead of developing their own infrastructure, build really sophisticated custom models just building on top of Tinker. And the the scale

Oh yeah, and to answer the question you asked before like who's it for? I'd say like right now it's for people who are are pretty sophisticated in like in terms of their their knowledge of ML and like who want to use these low-level primitives. I mean we we ship a lot of open source code that goes along with Tinker, so you don't have to write all the training algorithms yourself, but I think I think Tinker is best for people who do want to look at the details and dig into those details. But I'd say over time we're going to make it more and more user-friendly and build a lot of like tooling and sort of

higher-level components on top of Tinker so that you so that it becomes more of a like a like a full full stack thing that you don't have to be an expert to use it and you can you can just come up in with your understanding of the business problem you want to solve or the like the the spec of the model you want to build and and like the software we we ship will do that for you. And then what should we expect from Thinking Machines in the next in the next year or so? Anything you can share publicly? You'll see some things coming out with our own models. That's uh um sometime like next

year and also Tinker. Expect us to keep improving Tinker to add a lot more models functionality like multimodal train like various kinds of multimodal input and output like really scaling up the sizes of jobs you can do with Tinker. Thank you, John. It's been fun. Thanks for having me.