A researcher who spent three years inside OpenAI and Anthropic resigned this week, told the world both labs are "gambling with our lives," and racked up 167 million views doing it.
Within days, Dario Amodei, Sam Altman, and half the industry were arguing about whether it's time to deliberately slow down. Paul and Mike unpack the resignation, the "pace the frontier" push, and how you separate the real risk signal from the media noise.
Plus new data suggesting an AI jobs boom, Anthropic's 2030 economic scenarios, Jensen Huang declaring AGI has arrived, and a run of "agents gone wild" security incidents.
Listen or watch below—and see below for show notes and the transcript.
Each week on The Artificial Intelligence Show with Paul Roetzer and Mike Kaput, we ask our audience questions about the hottest topics in AI via our weekly AI Pulse, a survey consisting of just a few questions to help us learn more about our audience and their perspectives on AI.
If you contribute, your input will be used to fuel one-of-a-kind research into AI that helps knowledge workers everywhere move their companies and careers forward.
Click here to take this week's AI Pulse.
00:00:00 — Intro
00:06:35 — Anthropic Researcher Sounds Red Alert on AI Safety
X Post from Satya Nadella: Superintelligence must be under human control
X Post from Senator Chris Coons: Calls on the administration to treat AI as an existential threat
X Post from Speaker Mike Johnson: Congress working on AI guardrails
00:45:26 — OpenAI’s Math Breakthrough and Controversy
01:00:48 — AI Jobs Apocalypse Postponed?
01:08:53 — Anthropic Maps Economic Futures
01:15:00 — Jensen Declares AGI
01:18:10 — More AI Agent Security Incidents and Concerns
01:22:25 — AI Use Case Spotlight
01:30:02 — AI Product and Funding Updates
This episode is brought to you by Marketing AI Month. All month, the AI for Marketing Core series inside AI Academy is free (a $499 value): five expert-led sessions from Mike Kaput, the frameworks and tools the SmarterX team actually uses, and a professional certificate on completion. Enroll by September 30 (you don't have to finish by then - just enroll).
The month closes with a live AMA on October 1, where Cathy puts your questions to Paul and Mike; anyone enrolled can attend.
Enroll at SmarterX.ai/marketing.
This week’s episode is also brought to you by MAICON, our 6th annual Marketing AI Conference, happening in Cleveland, Oct. 13-15. The code POD100 saves $100 on all pass types.
For more information on MAICON and to register for this year’s conference, visit www.MAICON.ai.
Disclaimer: This transcription was written by AI, thanks to Descript, and has not been edited for content.
[00:00:00] Paul Roetzer: Think about all the things in society that have some level of regulation. So, cars, airplanes, chemicals, pills, food, tobacco, medical devices, manufacturing plants, nuclear power plants, consumer products, like all of these things, you have to prove they're safe before you put them into society. Why would AI be any different?
[00:00:19] Mike Kaput: Welcome
[00:00:20] Paul Roetzer: to the Artificial Intelligence Show, the podcast that helps your business grow smarter by making AI approachable and actionable. My name is Paul Roetzer. I'm the founder and CEO of SmarterX and Marketing AI Institute, and I'm your host. Each week I'm joined by my co-host and SmarterX chief content Officer, Mike Kaput.
[00:00:39] As we break down all the AI news that matters and give you insights and perspectives that you can use to advance your company and your career, join us as we accelerate AI literacy for all.
[00:00:55] Welcome to episode 239 of the Artificial Intelligence Show. [00:01:00] I am your host, Paul Roetzer, along with my co-host Mike, put it is September 14th, 9:00 AM Eastern Time and stuff's already going off the rails. I don't even know where to start, Mike. Things got just progressively, I don't know, like more serious.
[00:01:23] More widespread as the week went on, and then over the weekend it just kept going. And so we're gonna do our best to unpack what's happening right now. in essence, like there, you know, there's a, a massive focus on safety and alignment. There's a, a heavily increasing awareness about the risks related to these AI systems as they get more and more advanced.
[00:01:47] We're gonna try and give you the balanced perspective of like, there's a bunch of kind of scary stuff and mainstream media is all over this, as are the politicians. now, [00:02:00] There's some reality to some of this stuff that people need to be thinking about and maybe even worried about to a degree, but there's also a lot of, hype and exaggeration and, maybe some things that we shouldn't be quite as worried about that the media's gonna run with.
[00:02:16] So, I don't know, like, we'll see where this goes, Mike. There's just, I, we're probably gonna spend a lot of time on these first one or two topics would be my guess. It looks like you even shortened the rapid fire section and in anticipation of spending some time on this. So Mike and I have not talked since probably Thursday or maybe Friday morning.
[00:02:36] Yeah, I think
[00:02:36] Mike Kaput: so. Yeah.
[00:02:37] Paul Roetzer: Yeah. So we have zero prep for this. We're coming in hot. I just kind of finished up this morning getting ready, so. I don't know. We'll see where this one goes. all right. Today's episode is brought to us by Marketing AI Month. This is something new that we just kicked off for the month of September, I guess last week.
[00:02:57] So, AI is rapidly changing every part of marketing, but there's [00:03:00] still a major gap between knowing AI, matters and knowing how to apply it in your actual work. Marketing AI month is our effort to help close that gap by making practical AI education accessible to every marketer. So even if you're not a marketer, pass along what I'm about to share with you.
[00:03:17] throughout the month of September, we're giving everyone free access to our complete five course AI for Marketing series in AI Academy by SmarterX. This is, It's included in memberships, annual memberships, but it's a standalone $499 value. So this series gives you a step-by-step roadmap to becoming an AI forward marketer.
[00:03:38] You'll learn how to find and prioritize AI use cases across your job. Choose the right AI tools, build a personalized roadmap for adoption and use prompting deep research and custom AI assistance to solve real marketing challenges. Complete the series and you will earn a professional certificate to claim it.
[00:03:56] Just go to SmarterX dot ai slash [00:04:00] marketing. Again, that's SmarterX dot ai slash marketing, and you'll see a button to enroll in the course series for free. So all you have to do is just enroll by the end of the month. You don't have to take it by the end of the month and then their certificate by the end of the month.
[00:04:13] But just get in. You got whatever, 16 days left, 15 days left. and again, if you are not in marketing, pass it along to your marketing teams. I know I saw one last week, there was like a university that was using it to train up like all their marketing students. Yeah. So just take advantage of it. Mike teaches the series.
[00:04:30] It's incredible. it's a great way to get started. So again, this is part of our effort to just accelerate AI literacy and adoption. marketing often as we see is like the tip of the spear with an organization. So our thought is the faster we can get marketers trained up, the better chance we have of driving responsible adoption throughout organizations.
[00:04:48] So. Check that out. SmarterX.ai/marketing, through the end of September. You can enroll for free. All right,
[00:04:55] Paul Roetzer: AI pulse. So this is our informal poll that we do each week as part of the podcast. For the last couple weeks, we left this one open to get some additional responses because we have 130 responses here.
[00:05:08] So still, I would call it an informal poll. Like this is just a gauging where our listeners are at. But the question was, has your company changed how it hires for entry level, entry level roles because of ai? Now, this is the first time I'm seeing this data, so let's look at, okay, so we have 36% know our hiring plans have not changed.
[00:05:27] 32%, yes, we are hiring fewer entry level staff. 18%, yes, we kept volume the same, but raised skill requirements and then 11%, not sure. They're probably not involved in it. And then a very small percentage, no, we are actually hiring more entry level staff. That looks like it's maybe like 3% or so, Mike.
[00:05:49] Mike Kaput: Yeah,
[00:05:49] Paul Roetzer: so pretty balanced I guess between no, at 36% and yes, hiring fewer, but then there is a, a pretty decent group that's, keeping the volume the [00:06:00] same, but has changed their skill requirements.
[00:06:03] Okay, cool. So he goes
[00:06:05] Mike Kaput: smart 30, 32% is, seemed like high to me for at least the audience. It's a lot. Yeah.
[00:06:10] Paul Roetzer: Yeah. And again, for not
[00:06:11] Mike Kaput: hiring.
[00:06:11] Paul Roetzer: Yeah. And then you have 11% who don't know, right. 'cause they're not involved in hiring, so. Right. It could certainly be higher. yeah. So these are, again, these are like real time data points just for everybody to kind of think about.
[00:06:25] So you can go to SmarterX.ai/pulse and participate in the next one, which Mike will give us at the end of the show. All right. So let's talk about the thing that was all over mainstream media this weekend, and as of Monday morning, September 14th. I think to be a politician, you have to have tweeted about this already.
[00:06:47] Like it's just, it's everywhere. So Mike, walk us through what, what's going on with the Anthropic researcher.
[00:06:52] Mike Kaput: All right, so we'll tee up this kind of very fast moving story, but this past week, Paul Anthropic researcher, Jacob Coxon, [00:07:00] resigned from the company. He warned that both Anthropic and his former employer, openAI's, he used to be there as well as a researcher, he warned that both are racing towards AI systems.
[00:07:10] They may not be able to control Coxon. Sent said he spent the past three years doing pre-training research at the two companies, and this is of course, the work that teaches new models from large amounts of data in his resignation post on X, which has. Over 167 million views as of this weekend, probably higher now.
[00:07:32] He accused both labs of gambling with our lives. He then did a bit of a media tour and told people like the Wall Street Journal that he had joined Anthropic because of its reputation for safety. But he now believes no company can responsibly develop broadly superhuman AI without government intervention or a coordinated industry.
[00:07:52] Slow down. Now, this resignation kicked off this media firestorm, total viral cultural [00:08:00] moment, and as a result. Some of the labs appear to now be responding to this. So in the past, varied, you know, 24 48 hours, Anthropic, CEO Dario Ade has published an essay called We Must Pace the Frontier, calling for Slower Capability Advances.
[00:08:16] So safety work on AI can catch up. His plan starts with independent evaluators being actually embedded inside labs and then coordination on safety standards and limits on unchecked progress among companies in democratic countries. And then he follows that with some guidance on how to achieve broader international cooperation, including with China, with ways to verify compliance.
[00:08:40] I'm sure we'll talk a bit more about everything he outlined in that extensive essay. He did say Anthropic is committing to the first step of that essay itself. ADE says Outside reviewers will be embedded now at the company and they should receive ongoing access comparable to internal risk teams and be able to publish findings without Anthropics [00:09:00] editorial control, subject to narrow restrictions on sensitive information.
[00:09:04] And Anthropic intends to invite that team. However it ends up being composed and of who in the near future to the company. Now OpenAI, CEO Sam Altman publicly agreed that the frontier needs pacing and said OpenAI would also commit to independent evaluators with employee like Access. He said more details would follow.
[00:09:24] Xai. Founder Elon Musk also endorsed the essay posting. Dario is Right. He did not spell out a commitment to evaluators though. Alphabet Chief scientist Google Deep Mind Chairman de Saba back the direction of this too. He said the details did need work. he pointed to his proposal, which we covered on past podcasts for an industry-wide standards body.
[00:09:48] Aade, by the way, also told CNN that he agreed with Coxon more than he disagreed, over this resignation. Paul, there's a lot to unpack here. So [00:10:00] first up, Jacob Coxson has this very high profile, viral, almost resignation that seems to then kick off this almost like some people I've seen on x have called it like a divergence in the timeline, so to speak, which is now we've got labs.
[00:10:16] talking about Paceon, the frontier, as you mentioned, I think we'll get into this. Politicians, government figures, public figures are all now freaking out about this. Where does this stand? And like, it feels like something has changed. Am I wrong on that?
[00:10:32] Paul Roetzer: No, definitely. And, you know, I spent the weekend just monitoring the situation, watching the comments online, trying to like figure out for myself even how to frame this without.
[00:10:43] You know, over exaggerating what's going on. so I don't know, I'm, I'll do my best to kind of like work through some chain of thought here, Mike. 'cause it's really just was a bunch of notes throughout the weekend I tried to curate before we got on here today. so first [00:11:00] of all, I think the major AI labs are seeing a rapid acceleration in model capabilities beyond what we're seeing, like what we have access to and beyond what the scaling laws that have previously sort of guided where these capabilities would go, beyond what those scaling laws would have predicted.
[00:11:16] So specifically in the area of recursive self-improvement. And I think the researchers and lab leaders are spooked for real. Like there are definitely tech accelerationist. open source advocates at all costs, kind of people who think this is all just hype and, you know, an attempt at regulatory capture.
[00:11:37] I think that those people are doing a disservice to society by sticking to that, that chain of, messaging. So I think the risks and fears are real, but they're also not new. And that's, I think, the part that is hard to really understand here is like, what changed? Like why is this all of a sudden happening?
[00:11:59] And I'll kind of come back [00:12:00] to that in a minute. So. Jacob, Coxon post September 7th. It is up over 170 million views. It might be up 1 71. I looked last night at like midnight. and he's been on like every major news network. Yeah, CNN, Fox News, A, B, C, C, B-S-N-B-C. They all did interviews. So let's look at the post.
[00:12:19] 'cause the post wasn't more than probably, I don't know, 300 words. Like it wasn't like some crazy essay like we've seen from some of the other leaders. He said, do not underestimate the power of this technology. There will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.
[00:12:36] Now I'm gonna pause there for a second because there is a topic, the second topic we're gonna talk about today with the math breakthrough from openAI's. Yeah. Where this is, this revolutionized any field overnight. I want to just like put a pin in that one and come back to that. He said, we have all witnessed the progress in each of these domains, and progress is not slowing.
[00:12:57] The people building AI earnestly [00:13:00] believe that it could kill us all by the end of the decade. I'm going to come back to that because that is the part everyone latched onto. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible, but I hear the same people express fear privately.
[00:13:19] No other human activity poses this level of danger. A common response is if they truly believe this, why are they still building it at openAI's? Many have not deeply internalized civilizational, stakes at Anthropic. The stakes are well understood, but they are locked in a race to their first because they believe no one else will act responsibly, so they must do it themselves despite the risk.
[00:13:43] So that's a pretty important paragraph. Now, I don't know that that's true that many at OpenAI have not internalized the civil civilizational stakes. I think it's pretty well accepted within researchers that there's real risks here. I do believe that Anthropic culturally, they do [00:14:00] believe that they have to get there first.
[00:14:01] Like that's kind of been the knock on darrio is people think that he thinks they're the only ones that they can do the responsibly. he continues accepting this race and entering the end game is a hubristic gamble that should not be launched from a private company. Slack. Attempting to speed run alignment should require extraordinary confidence that there are no better trajectories available.
[00:14:22] I'm optimistic about the potential for coordination. Warning shots like the hugging face attack have made pacing agreements between US labs more viable. I don't feel like we're on track to prevent a global race, which may require costly actions such as temporary ban or on improving model capabilities.
[00:14:39] If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a super intelligent RL run re, reinforcement learning run without a rigorous understanding of its mind? Should you put your head down because it's happening anyway? Or take this moment to call for a different direction.
[00:14:57] So, quick background on Jacob, because when I first [00:15:00] saw this tweet, it had been up for like eight hours and it already had 72 million views. Yeah. So like it took off really, really fast and the guy didn't exist online, so I was like hesitant to even share it. I went and did some digging to figure out is this person real first before we even looked into anything.
[00:15:16] So then I found a Wall Street Journal article where they quoted him and it's like, okay, so the Wall Street Journal isn't gonna put something up without verifying their source. So that was my first hint that this person is probably real. I found him listed on the. oh four safety card from openAI's. So we verified that he existed within openAI's back when that was first published in 2024, but he pretty minimal otherwise.
[00:15:40] Then the thing that told me, oh no, this is definitely real is Evan Hubinger, who is alignment Science lead at Anthropic. Retweeted the tweet and said, Jacob is correct here. So I'm like, okay, well, an alignment lead at Anthropic probably would've disclosed that this wasn't a real person, if that was the case.
[00:15:59] So he said [00:16:00] he is correct here. We do earnestly believe AI could kill all humans. I personally think it's greater than 10% within the next decade, blah, blah, blah. I'll come back to this whole P doom stuff in a second. So my initial reaction, I, so I put this on LinkedIn, whatever it was like Wednesday morning, I think, after it had come out and I said, we may have reached a tipping point where the rest of the world wakes up to what those in the AI information bubble have known for years, which I was meaning that there are real risks related to ai.
[00:16:25] This is not all, you know, sunshine and rainbows. I, and then I said, I assume media and politicians are going to gravitate very quickly to this story like they did Matt Schumer, something big is happening post in February. Which we talked about at that point and reached 88 million views on on X. And then I said, I think we needed to arrive at this point in order for AI leaders and government leaders to have the will to push for responsible human-centered AI adoption in business and society.
[00:16:51] I remain optimistic about the potential of AI to improve lives and be a net positive in society, but we have to be more proactive and intent [00:17:00] intentional about making that reality. Ignoring and banning AI is not the answer. Slowing it down and committing our best minds and significant resources to AI safety and alignment are critical.
[00:17:11] Keep in mind, a lot of these labs gutted their safety and alignment teams in the last three years because they were getting in the way. Of the race to build the more powerful ai. So I said, I think it's fair, far more likely now that we could see deeper collaboration between competing AI labs and nations.
[00:17:27] I did not expect Ade, Altman and Musk to all of a sudden be on the page, same page like 24 hours later. But here we are. so then I was like, okay, well why did this take off? Like what is, what is the moment that this Jacob Guy who literally created an X account to tweet this? Yeah. Like he left in an interview.
[00:17:44] He said he was basically talking with some friends like, Hey. How could we get some like awareness around why I am doing this? And they're like, well, why don't you join X? We'll retweet it and we'll see if we can like get some, some awareness. That was basically the story. Now, there are certainly some people who think this is a psych op that's like [00:18:00] being funded by dark money and dah, dah, dah, dah, dah.
[00:18:02] Who knows? Well, people, people say some crazy stuff when this stuff starts happening, so, but this is not by any means the first AI researcher, nor the highest profile AI researcher to say these things, right? Jeff Hinton, the godfather of ai, modern ai, left Google in a very, very high profile way to say basically this exact thing and has been saying it for years on major TV interviews, in publications, anywhere anyone will listen.
[00:18:33] We have the most prominent researcher probably. Has been saying almost the exact same thing, so why does this take off? So there are definitely these conspiracy theories that it's all a plant and it's well funded by these opposition groups. The more likely scenario is. The moment we are in, due to the increasing attention on AI safety, following the openAI's hugging face in incident, which we have talked ad nauseum on this podcast about the political CLI [00:19:00] climate and the public sentiment around AI has changed.
[00:19:04] Something just went crazy with X's algorithm. Like even Elon Musk was like, this doesn't make any sense. Like, nothing takes off like this on X. okay. So now let's take a quick step back, Mike, to episode 2 28, which was August 4th. So just five weeks ago. And we talked about at that time this idea of recursive self-improvement in which AI systems autonomously design and upgrade their own successes.
[00:19:28] so basically, and, the successors like the next version of themselves, openAI's, as we discussed at that time, has stated this as an actual goal. Like they wanted to build an AI research intern that could largely do the work of a, of a AI researcher. So they set in an interview on oct October 29th, 2025, almost a year ago.
[00:19:49] Our goal is to build by March of two, 2028 to have a truly automated AI researcher. The automated AI researcher would have the recursive self-improvement abilities where it would actually do this. So [00:20:00] then in June of 2028. They, again, OpenAI said our three main goals, one of them build an automated AI researcher.
[00:20:07] This is the exact thing that people are afraid of, that the, that the labs have set as a goal is the thing they're worried about. Anthropics responsible scaling policy that came out in July of 2026. Talked about this exact thing, automated r and d of AI is the thing that they were focused on, but they were worried that there could be a dramatic acceleration in the pace of AI progress for reasons that likely relate to automation of ai r and d.
[00:20:34] So this is the, again, the exact thing that we've been talking about that they've known was going to be an issue. Then this came on the heels of Demis Saba's essay, which we talked about in episode 2 26, where he talked about a framework for frontier AI and the dawning of a new age where we're getting towards this rate where it's just gonna take off.
[00:20:53] also in July of 2026, we had the pacing letter where they talked about automated AI development [00:21:00] being the thing, and 1300 AI researchers signed the thing. So again, nothing Jacob said was new, like. Recursive self-improvement. We've known it's a slippery slope. The fact that the labs aren't super aligned, we've, we've known that too.
[00:21:15] Like, so that's the thing that's really odd to me is like how much attention this got. but it, again, it's just one of those moments where things aligned. So now we'll come back to Dario's Post. So, in the actual article that he published or essay he published, he said, along with my co-founders and employees, I have grappled with du this duality.
[00:21:35] Now this came out, what, three days after Jacob's post, I think. so two or three days after Jacob's original post. So they've grappled with the duality of risk and benefits since the beginning of Anthropic not building the technology, deprives humanity of benefits or simply places AI in the hands of authoritarian power, authoritarian powers.
[00:21:54] While building it too fast is reckless, we have sought a middle way to show that it's possible to [00:22:00] build carefully and succeed commercially and to make safety something on which AI companies compete. In other words, they wanted to create a race to the top and he says, over the last few months I've become convinced that fully addressing the risks requires even more prudence.
[00:22:15] Not just investing, investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up. We must slow the pace at which we improve the capabilities of AI models. Progress will seem still seem fast, and we must make wise use of the time we gain. Two things he's saying, convinced him that it was time to say something more.
[00:22:37] First is the concern since roughly this summer. AI has been advancing Dr. Drastically faster, driven primarily by AI's growing ability to build the next generation of ai. This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including a Anthropic. And we and others have, as we have and others have described it, [00:23:00] left uncheck.
[00:23:00] It could outrun our ability to understand and control these systems. So must be pursued very carefully. The second he called attention to is the hugging faces incident. And he said, given the accelerating rate of AI capability, it's his worry that in six to 12 months a swarm of agents, like the ones in the hugging faces incident, could be capable of taking over the entire internet with a persistent botnet potentially causing hundreds of billions of dollars in damage.
[00:23:26] And that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails. So that's the gist. And then he goes to like the three things they're proposing, and that was when. The, like the parallel universe just showed up. So then Altman, now, if you're new to all this, Altman and Dario are our enemies.
[00:23:44] They, they hate each other. Basically. Elon Musk's hates everybody involved. Like he hates Sam with the passion. He hated Dario. But now Dario spends billions of dollars with XAI. So SpaceX ai. So they seem to have come to some level of [00:24:00] like communication at least, but like these are not three guys who are going out for drinks on a Friday night.
[00:24:05] So Altman says, I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions. We've had it openAI's in recent weeks. Committing to having independent evaluators with employee like, access is a great idea and we will do the same. We'll have more to share soon. I was like, holy shit.
[00:24:21] Like, okay. That came out of nowhere. Yeah. Then Elon, as you mentioned, Dario is right. Then Demi shows up now. Demis kind of gets along with everybody. He's He's like the same, everybody
[00:24:30] Mike Kaput: likes De
[00:24:31] Paul Roetzer: Yeah, like nobody. I mean, Elon created openAI's because he was worried about Demis getting acquired by Sergei, Bryn and Larry Page and that they were gonna take over the world.
[00:24:40] But like. They, they seem to have come to peace with each other. So Demi then says, Dario's essay points toward the right path forward. The details need working through, but the direction is correct. And then he references back to the essay that I had mentioned, earlier. Andre's Car Path Car Carpathy, I who's, you know, legendary researcher.
[00:24:58] He's happens to be an Anthropic right now. [00:25:00] He was, early at openAI's, he said, I love this and really hope we can come together as an industry and make it happen. Regarded the embedded evaluators Satya shows up on like Saturday or Sunday. I don't remember when this one came in, but he tweets Any pursuit of super intelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it is not worth pursuing.
[00:25:22] We welcome the research focus and deliberate pacing needed to get alignment right as the design goal. We also welcome ideas like embedded evaluators and the broader efforts to develop the mechanisms to make this more than just talk. The key is that this cannot be controlled by a handful of entities, but must have broad representation across ecosystems, countries, fields, including academia.
[00:25:43] Now, Microsoft's been touting the human control thing for at least the last like 12 months. So that's, that fits very well with their messaging. Okay. So responses from critics was one thing I was paying attention to, which I wish I didn't have to, but whatever. So there are a group [00:26:00] of very prominent, usually venture capitalists, who think that everything related to AI risk is an attempt at regulatory capture and an attack on open source models.
[00:26:12] To them. There is no middle ground, like if. Someone is tweeting about this happening in labs, then Anthropic is just going after regulatory capture. that's basically the gist of any criticism you would see online. Roughly falls into there. They're going after regulatory capture. They think they're the only ones who can do it safely, or they're coming after open source models.
[00:26:31] Now there might be elements of that that are true, but that that's it. Like that is the arguments they make. Then we get the politicians jumping in. Now immediately, as soon as I saw Jacob's tweet, I was like, son of a bitch. Like the kill all humans thing is all anyone's gonna talk about. Yep. Like they're just gonna immediately zero, zero in on this one thing.
[00:26:49] and of course, okay, so Senator Chris Coons. I actually have no idea if he's a Democrat or Republican. So this kid, if you're new to the show, I don't give a shit about politics. Like we are [00:27:00] completely neutral. We are like, what is the most sense for humanity? I think he's a democrat maybe, but I don't know.
[00:27:06] And I actually didn't even wanna look it up to be, to be truthful. So he said it's time for the Trump administration to wake up. Okay. Maybe he's a Democrat. to the existential threat posed by ai, this must be the top agenda item when Trump and Xi meet later this month. meaning Xi Jinping, China and Congress must prioritize passing meaningful guardrails in this space before late.
[00:27:27] Now, I'll tell you what, like had Josh Hawley tweeted the same thing, who is definitely a Republican. He probably would've said the same thing. So like, again, I don't know. This is a bipartisan thing now. Speaker Johnson, who is definitely a Republican, he tweeted this morning, Congress has been studying the AI issue since I set up a bipartisan task force to do soon after I was elected speaker.
[00:27:50] We all have a sense of urgency to create guardrails around the technology. The key is designing the guardrails carefully in a way that prevents any harm from ai, while also per [00:28:00] preserving American innovation and our national security. By keeping our edge over China and other international competitors, I am calling a meeting in Washington with AI platform providers and key experts to determine the right course forward and discuss responsibility providers have to ensure.
[00:28:15] Now, keep in mind he's not calling the house back though, like he's just calling platform. But, anyway, so now Johnson probably needs to have a conversation with Trump because Trump was asked Sunday morning about this very thing and he said, it said, when asked about a potential slowdown on Sunday, Mr.
[00:28:32] Trump said, we're leading China in ai. We're the most sophisticated country in the world. And frankly, I want to keep it that way because whoever wins, AI wins and we put guardrails. we can put up guardrails, we can do this and that, but I think you have a lot of very negative forces that are bringing it up that shouldn't be bringing it up.
[00:28:50] They're bringing it. up things that won't happen. So Trump's and Johnson not, not currently on the same page. Okay. So then what does China think about [00:29:00] all this? That comes out Monday morning. This is September 14th. uh. Okay, ma'am, this is, this is the big one. Alright, so, okay, so go back to Dario's letter.
[00:29:11] Why would China get pissed about this? Like, hey, this doesn't seem bad for humanity. Like, let's, like, find an agreement. Okay. Well, within Dario's letter though, I'm just gonna quote this, pacing within democracies, which China is not, will be limited by the lead that US companies have over authoritarian regimes, chiefly the Chinese Communist Party.
[00:29:31] If we slow down by more than this amount, then on paced, Chinese associated projects will pull ahead creating significant national security risk. So they do, he, what he proposes is do not sell power four AI chips or semiconductor manufacturing equipment to China and crack down on chip smuggling operations and remote access to data centers outside of China, which he's saying they're doing to bypass these.
[00:29:55] the fact that we're not selling these things, chips will be the main [00:30:00] determinant of China's AI strength. I also want to crack down on unauthorized distillation by companies in authoritarian countries. That means stealing models from the US and like distilling them to build their own models and then strengthen security at the AI companies and prevent model weight theft.
[00:30:16] So, you know, pacing's fine, but then Dario sort of like. Goes right at China, so China's not so, so happy about that. So in Reuters this morning, the Global Times, tabloid said the true agenda of Ambe DE's essay was to attempt to curb China's AI development through technological barriers and regulatory monopolies uphold Washington's mono monopolistic hegemony.
[00:30:37] He, I don't even say that. I don't even know what the word means.
[00:30:40] Mike Kaput: He Gemini, I think.
[00:30:41] Paul Roetzer: There you go.
[00:30:41] Mike Kaput: That's the, yeah.
[00:30:42] Paul Roetzer: In cutting edge technology and exclude China from the global AI governance system. And then this is the real one. this is a quote, by the way. This silent ai cold war is hypocritical and shortsighted.
[00:30:54] It said nothing. Adding that, excluding China from this innovation would significantly increase the trial and error costs and risk [00:31:00] of loss of control in global AI development. Okay, so all that being said, here's just my overall take, and again, this is pretty like raw. Like I literally just made these notes.
[00:31:11] I have not thought deeply about this exactly what I'm going to say here. Okay, so there's something called P Doom or Probability of Doom. This is a term that has been going around in AI circles for well over a decade, and the basic premise is what is the likelihood that AI will kill all humans? This is not a new term.
[00:31:33] By any stretch of the imagination. And so for a while, AI researchers would like, Hey, what's your P doom? Like, what's the chance you think that we just like kill all of society with, with ai? I think it is an absurd concept. It is assigning probabilities to something where there are all of these variables and it's the thing that people are going to latch onto.
[00:31:53] So you have some people, it's like, oh, it's 50%, it's 10%, it's, well, it's 90%. Like, they literally just make a number up. Hmm. Like, [00:32:00] it would be like me saying, well, I don't know. I think there's like a 10% chance aliens show up in the next three years and just decide that, you know, we're basically like ants and that they don't need humans.
[00:32:09] And like if someone who happens to be on the inside of the UFO program within the United States like shows up and says, yeah, I, yeah, I think there's a, like a 10% chance they're already here. Like. Okay, like that, that's a probability that you, you've assigned to something that has no scientific backing to it.
[00:32:26] So I would just encourage people this whole human extinction extinction thing, it's, it's just like losing sight of real near term risks. Now, I am not saying that there is no chance that some scenario emerges where stuff just goes completely sideways and it really is just bad for us. I could probably give you like 10 other things outside of AI where the same thing could be true and like maybe it happens, maybe it doesn't.
[00:32:55] I don't know. An asteroid could hit tomorrow and that's not gonna be good for any of us. So [00:33:00] I would, I I would just set aside the whole kill humanity, human extinction thing. It is like, it is distracting us from the fact that there are very real near term risks, that we should be focusing on that will cause people to actually.
[00:33:19] Start thinking more realistically about ai. Like this doom thing just makes people, it's like, oh, it's all bad. Hmm. and we lose sight of all the positive things that it can have. So I do think that we have to slow down the model releases. Like I'm, I'm very convinced of that. I said this on last week's episode that GPT six Astra changed the way I thought about this stuff, that I had more dread than excitement about that release.
[00:33:44] I think they need to slow this down. They obviously don't have control of these things like the hugging face incident. Anthropic had similar incidents. We've learned about other incidents with openAI's. They don't have control of these agents that they're building, so we gotta figure that [00:34:00] out. They need way more resources dedicated to safety and alignment, and there has to be some level of government oversight.
[00:34:06] I don't have any confidence that the administration can do it, but that doesn't mean we shouldn't try. the way I was thinking about this, Mike, and I hope this makes sense, but like, think about all the things in society that have some level of regulation, so, So, cars, airplanes, chemicals, pills, food, tobacco, medical devices, manufacturing plants, nuclear power plants, consumer products, like all of these things, you have to prove they're safe before you put them into society.
[00:34:33] Why would AI be any different? It, right. The risks of AI are greater than many of those things, and yet we just get to have a grand experiment on society every time we wanna put out a more powerful model. So that to me, you can call, you can get stuck in the regulatory capture, whatever, but then you have to address the fact that it can cause real harm.
[00:34:56] I'm not on the P doom like extinction of humanity [00:35:00] harm thing, but like I mean, I could see a very realistic scenario where it takes down, you know, infrastructure, power grids like that is not hard to comprehend. Wall Street, like takes down trading like. All of that stuff is super viable. So why wouldn't we have some way to test for safety before we put them into the world?
[00:35:19] So I feel like they need to do something. and I think people who argue against that are just doing a disservice to society. As I said, I don't understand how that's a viable approach. and then the big picture here, and this is maybe just me being optimistic, is like, I think it's good that we have an awareness now.
[00:35:39] Like, I feel like, a lot of people just weren't awake to the fact that there's like super real risks here and that things could go bad. But the faster we get to talking about those things and dealing with them, the better we can get to like all the good AI can do, all the positives it can bring to society.
[00:35:58] So, I don't know. I would say like [00:36:00] for people, individual who are listening to this and maybe are just like, oh my God, like this is too much. Just go to work. Tomorrow and like focus on applying the tech we have in a responsible way to enhance human potential to unlock what we're doing, to like work on higher level cognitive creative tasks.
[00:36:20] Like most of you, your jobs are not gonna change dramatically in the next, you know, one to six to 12 months. It's certainly not in a way where AI is just going to like destroy humanity and society. And I think like we need to just focus on the things we have control of, which is there's really. Capable models that when applied well, can make your job more fun, like more fulfilling.
[00:36:47] but it requires organization leadership to have a vision for how to do that. And I think most of us should just be focusing on that. Like, how do we integrate this current technology that we already have, GPT six, [00:37:00] Astro Fable five, like they're good enough. Like if we shut off model development today, we got three to five years before Mors Enterprises fully integrate the tech we already have.
[00:37:10] so that's what I would say is like, I get why people are worried about this stuff. I get why, you know, the media headlines can be unnerving. I think it's gonna cause action. And I think that's a good thing in society, in government, and I think most of us just need to stay focused and optimistic in doing our part to try and bring it responsibly within our school systems, our, our businesses, and our own careers.
[00:37:37] And like, don't let all this other stuff distract you from that.
[00:37:41] Mike Kaput: So Paul, I had a few quick questions for you around this. So just to be perfectly clear here on your position, kind of what you've outlined, first up, there's a lot of crazy headlines about the motivation for Jacob and others to be doing this.
[00:37:57] Yes. You mentioned regulatory capture. [00:38:00] There's people literally saying it's like a democratic party, psyop. You're just to be clear, and I agree with you. I think like this, you're, regardless of the other effects of this, it sounds like this is motivated by something they've actually seen and extrapolated as being developed in a lab.
[00:38:18] This is not made up.
[00:38:20] Paul Roetzer: a hundred percent yes. Right. Now, again, you could, people are gonna dig in and find all kinds of stuff about who is this guy, who's he connected to? Yeah. Who were the first five people that retweeted the thing? Like, I'm not saying that that stuff's not real, that there, there wasn't like a coordinated effort to try and get more attention around the risks.
[00:38:36] Mike Kaput: Yes. Right.
[00:38:36] Paul Roetzer: But I think the motivation for why they did it is because they're seeing things that we aren't seeing and they are terrified of what's being built. And that the scaling of this recursive self-improvement, we basically have six to 12 months to figure it out. And once they've solved it within these labs, then anybody building open weight models can do the same thing within a year.
[00:38:54] And then we've lost control completely. And I think that they think that is very real.
[00:38:58] Mike Kaput: Gotcha. And [00:39:00] then one other question, I'm just curious if you have any perspective on this as I'm reading Daria's essay, right? And it's a, it's very interesting. It's like kind of a declaration of war on China.
[00:39:10] Paul Roetzer: It's
[00:39:11] Mike Kaput: very true, I would say. but given that, isn't this kind of like a prisoner's dilemma? Like aren't you, you have to, let's say openAI's and Anthropic do everything perfectly, that's really good for their models not escaping or doing unintended things that screw over Wall Street or cybersecurity issues or create billions in damage or lose lives.
[00:39:35] But doesn't this require literally every single person involved, every lab involved, every entity involved to do the same thing? Or Will, is there something unique about openAI's and Anthropic doing this that would stop catastrophe.
[00:39:48] Paul Roetzer: That's why they need to get the other countries involved. You know? And I do think that you're looking at like nuclear weapons as probably the closest parallel to like being able to negotiate treaties where, you know, [00:40:00] you try and control it.
[00:40:02] I mean, there was an interview, Dario did, I think it was on CBS, where he was kind of implying that he would give up control. I saw that, yeah. To, to nationalize the technology that he never understood why private companies are even building the technology that they're building. Wow. And they pushed him on it and he kind of backed away from like, whoa, I'm not gonna like, literally just give it to our government, but like a collection of democratic governments potentially.
[00:40:24] Hmm. you know, again, I don't wanna go down the conspiracy theory path much here, but I would imagine that. DARPA and other government agencies are not gonna probably sit around and wait for a few private labs to, to develop this and then like hopefully share the weights with them. I would think that there is likely a very large scale initiative underway that we may never hear about to build the most advanced frontier models that the government [00:41:00] can control aversion, because you could definitely see an argument where nationalization of the labs, it is in the best interest of the way that the US government would look at these things like that they could convince themselves that nationalizing this technology is maybe the best path forward.
[00:41:19] Now, I'm not endorsing that. I'm just saying I could see them thinking that
[00:41:24] Mike Kaput: that's what struck me as so. Got crazy about the Evan response to Jacob where he said, greater than 10% chance that this kills all people. If I told you that a, a non-government group said that they can take, they had developed technology that they believed had a greater than 10% chance of killing all of humanity.
[00:41:48] Let's say that group was not, based in the us right? What would happen tomorrow is the US government would. Conduct a series of drone strikes on that. Like, that's why I was like, is [00:42:00] this, this is so strange to me. You just say this. Whereas the logical conclusion is, okay, you should all be in jail at a baseline because we are preventing mass extinction.
[00:42:09] You should be nationalized. I'm not saying I agree with any of that, but Well, that's where this logic goes.
[00:42:14] Paul Roetzer: Yeah. And to go back to the point I made about like the analogies of all the things that have some level of regulation.
[00:42:19] Mike Kaput: Yeah.
[00:42:20] Paul Roetzer: Let, let's just, and again, I don't even think this is a farfetched thing and I'm, I'm thinking of this off the top of my head.
[00:42:24] So let's say that there was three pharmaceutical companies who were developing a cure for cancer and that they had made immense progress and that. They wanted to put it out into the world because they think they can cure all cancers, but there's a 10% chance it actually ends up killing everybody in, in 24 months.
[00:42:43] Because like there was a sleeper element to it that we didn't know, but like, it's so good. We, we just wanted to get out in the world. I'm the chance there is zero chance, right. That that drug gets introduced into society zero. And yet we have AI where the people building it are convinced that [00:43:00] there is some possibility that it ends really badly for all of us within a decade.
[00:43:04] And yet there's no real oversight. They can just put things out in the world, they can run these like lab experiments. It's like gain of control experiments on yeah. You know, when you're developing viruses and stuff, like they're just doing it unregulated and that, I don't understand a world where that makes any sense if there's these risks.
[00:43:24] So yeah. I it's again, like I feel like at least we're now having dialogue. There are certainly some people who are not having. I would say again, they just have an agenda and they're stuck on their agenda. And I think it's just best to ignore those arguments, if they're not also willing to listen to both sides of it.
[00:43:43] And I get that this is part, maybe the longest main topic we have ever had, but hopefully it's really important to everyone to kind of understand that this dynamic is now well beyond the labs. We are truly into the political game. We are into the impact on society. [00:44:00] it is going to own the media and the politics for the months ahead like this is.
[00:44:06] We, we have definitely hit an inflection point, whether how this tweet ended up being the thing I still don't really comprehend, but it's been building for a while and we, we definitely, we hit a point where things are just gonna be different now.
[00:44:18] Mike Kaput: And we'll move on just one second here. But I do just wanna encourage people, if you have family members or friends we're asking about this, I would, you know, shamelessly say, like forward them the beginning segment of this episode.
[00:44:29] Because I think, Paul, you've done a really good job of framing this because I don't know about you, this is the number one topic this year so far, at least, where I've gotten commentary and questions from people that don't ask me about ai. And they're for, they're freaking out basically. Yes. Text
[00:44:44] Paul Roetzer: from friends and
[00:44:45] Mike Kaput: family.
[00:44:45] It's not a good, it's not a good vibe out there with this stuff for people that don't follow it. So I think this episode, or at least this segment could actually really help some of the, like I explained it to my mom, for instance, a little like you have, and she was like, okay, I don't know if I. [00:45:00] Feel positive about what's happening, but that's a lot more helpful than kind of what I was reading or seeing or,
[00:45:06] Paul Roetzer: yeah.
[00:45:06] And I understand that people don't still feel positive, like I totally get that, but what I think, yeah, the point I'm trying to make is like, do not get caught up in the whole end of humanity thing. Like it's just Right. It is not productive at all to like be going down that path right now.
[00:45:24] Mike Kaput: All right, let's move on.
[00:45:26] Our second big topic This week, openAI's announced that an internal AI system had produced a proposed solution to the Navier-Stokes Millennium Prize problem. This is a longstanding. Mathematics question about how fluids behave. The company released a written proof and a formal version in a system called Lean, LEAN, which is software that checks mathematical arguments.
[00:45:53] They said the model behind the discovery is significantly more capable than GPT-6 Astra, so [00:46:00] an unreleased model. So the result of this problem basically won't get into too many details, concerns whether an initially smooth three dimensional fluid fluid flow can develop a singularity where the equations predict speeds growing without a limit in finite time.
[00:46:15] So basically, a very, very hard, longstanding, unsolved math problem about fluid dynamics. Now, to solve this, openAI's says. That they involved roughly 10,000 agents working concurrently, and the system orchestrating those agents reached it resol. Its result about 88 hours after the effort began with another 17 hours for lean formalization verification, which you have to do to basically.
[00:46:43] Verify that you have the right solution. The company, however, says it does not intend to claim the Millennium Prize. Now this came with some big controversy because, right around the announcement there was this dispute that came to light with New York University [00:47:00] mathematics Professor Tristan Buckmaster and his collaborator, an Anthropic employee named Levent Alpoge, and their separate project, which Levin was doing with Tristan, just in a private capacity.
[00:47:12] This was not like an Anthropic driven initiative. They were just working on this problem using multiple AI tools, and they developed related results for a related math problem called Eulerequations And. Buckmaster says that this collaboration was totally personal, not an Anthropic effort, but then Buckmaster says, openAI's researchers, Sebastien BuBuBubeck propose that Buckmaster write up open AI's results since they kind of got to the same conclusions, it sounds like, or roughly along the same direction without Alpo, who works at Anthropic as an author because he was employed by Open AI's competitor.
[00:47:49] Bubeck said the discussion concerned rewriting open AI's work to include Tristan not removing Alpo. He apologized for kind of how he had handled it. Buckmaster wrote [00:48:00] this like four or five page essentially PDF He released of like emails and messages saying like, Hey, I'm not accusing anyone of anything, but I am saying it's kind of strange they arrived at a similar result using a similar direction that's not widely known or would not have been obvious if you weren't building on someone else's work.
[00:48:19] 'cause they had kind of heard rumors. openAI's apparently had that Anthropic was getting close to solving this exact thing. Buckmaster also, I read the whole thing pretty respectfully, I guess. Like he wasn't like coming out in super hot, but he questioned whether the drafts that him and his colleagues had entered into Codex, which is one of the systems they used to do this work, influenced the work.
[00:48:42] openAI's said that this was not the case, that the prompts had not, influenced the system, including through training. So there's all this controversy, Paul, basically super historic mathematics result kind of plays into a lot of the stuff we just talked about. A [00:49:00] lot of the supposition that AI's actually going to solve longstanding mathematical and scientific problems.
[00:49:06] But then there's this cloud of controversy. What did you take away from this, or what kinda struck you about this?
[00:49:11] Paul Roetzer: Yeah, first I have no idea what the hell this actually means. Like this mathematical equation. I, I've read this like five different times and I still don't comprehend it fully. but I'm also not a mathematician.
[00:49:22] So I don't know that I'm supposed to really comprehend this, but I did still try and start with like, okay, what is the significance from a mathematical perspective, from a science perspective of this? And in the. openAI's post, it says These equations are used for aircraft design, weather forecasting, and the study of blood flow.
[00:49:37] So it's like, okay, well there's a few tangible things of why this is so relevant. the thing I wanted to focus on though is the fact that less than two years ago, so before we got reasoning models, the O one model from openAI's, the most advanced models from OpenAI, couldn't reliably count the number of RS in strawberry.
[00:49:56] And here we are two years later solving Millennium prize [00:50:00] problems. So the speed of the improvement of the models to me is maybe the biggest story of all here. you touched a little bit on the backstory, but they, they, open eye did go into a little bit more detail about what exactly happened because of all the backlash that they were getting right out of the gate.
[00:50:15] So Sam tweeted, right when this happened. One of the most amazing moments for me in opening eye history was watching this happen over the past week. The world has extremely capable models. Now, I did not expect a result of this magnitude to happen so soon. This goes back to the whole thing I mentioned about the models are getting better, faster than they were expecting.
[00:50:34] He said, we have been talking a lot about the need to pace progress to ensure safety. For me, this is the strongest evidence yet of the urgency. So again, kind of spooked by the fact their models could do this. he also said it is true that we tried this because there were rumors on the internet last week that Anthropics models had solved a millennium problem and we were curious if ours could do it too.
[00:50:57] So what he's saying is they got wind that [00:51:00] Anthropics latest models 100 training had done this thing and they were like, h I wonder one of ours could do it, is what the story they're giving at least. so the post that they put up, it said, since August 28th. So again, just to get perspective on how fast this came together, we have been training a new internal model that has exhibited unprecedented performance in our benchmarks, including mathematics.
[00:51:20] This model's training is ongoing and its performance continues to improve. On Tuesday, September 1st, they heard the rumors about two millennium prize problems had been resolved. Inspired by these rumors and by the step change in performance of our internal model. We launched an effort to evaluate it on all open millennium prize problems and a few other high impact problems.
[00:51:40] you alluded to this one. The agents arrived at the resolution on Saturday, September 5th. So four days later. About 88 hours after the first agents were launched. and then 17 additional hours with GPT-6 Astra, it says, across all attempted problems, the agents sent 4.9 million messages [00:52:00] and used about 300 billion output tokens.
[00:52:03] In the process of resolving, the na navier na Navier-Stokes Pro Stokes problem, the agents sent 2.7 million messages and used approximately 130 billion output tokens. Our goal in releasing this result is to report on the substantial progress of our AI models. we believe we are now in the next period of AI progress.
[00:52:22] This is the important part. We believe we are now in the next period of AI progress and today's results provide further evidence of this. We are focusing on understanding this model, meaning the one that has not been released yet, and using what we learned to help us guide and pace how we further, pursue further advances in capability.
[00:52:39] One of our key goals is to build AI systems which are steerable accountable and connected to people which may require more deliberate choices about the pace of progress as we continue our mission to ensure AGI benefits all of humanity. So again, all about pace. the other thing I started thinking is like, well, what does this mean to other grand challenges?
[00:52:57] Like, right, can we really cure cancer? Like there [00:53:00] there are big problems that they just showed. Give it four days and like unlimited tokens and like, what can we solve? So I do start to think about that. It's like, well, maybe we are within like a year or two of solving all of these, diseases and all these things.
[00:53:14] 'cause it just requires compute apparently.
[00:53:16] Mike Kaput: And I think they said it may have cost like several million dollars of compute. Yeah. But that's a, that's a rounding error compared to how much revenue they're about to generate. Totally. And like tokens, they have unlimited tokens in turn. That's a, that's a pittance to solve some of these things.
[00:53:30] Paul Roetzer: Yeah. And like the good for humanity. Like I, yeah. Yeah. So that was my, I I was like, well, is this projectable? Could we do the same thing now? Mathematics is provable. Like there's a, there's a goal to head towards, but like, I don't know. So that was, that gave me kind of hope. Again, it's like there's a negative to this, but it also is, wow, this maybe we are heading down this path of abundance.
[00:53:49] what does it mean for industries or employment that they throw this same reasoning capability at? Or that someone throws an open weight model at and six months and says, oh, lemme go take on the legal industry [00:54:00] or the finance industry or whatever. They just demonstrated an ability for these things to learn in days and solve really hard things.
[00:54:07] the one thing that did catch on Mike was this whole idea you talked about of like the mathematicians questioning whether maybe open eye learned something from their Codex projects.
[00:54:18] Mike Kaput: Right.
[00:54:19] Paul Roetzer: So John Schulman, who's no longer an openAI's guy, he's at thinking machines, but previous openAI's, I believe, a researcher we've talked about on the show before, he tweeted, as a follow-up, it's exceedingly unlikely that training on user data contributes much to frontier model gains in areas like math.
[00:54:37] Those come from scaling up, pre-training and reinforcement learning. user data is more likely used to find failure modes or situations that are hard to recreate. That said, he hedged, and I thought this was a really interesting hedge, model. Companies vary in how aggressively they train on user data.
[00:54:56] And uploading repos isn't hypothetical. I wish there were stronger norms [00:55:00] around disclosing how companies train on user data with what me methods and to improve what capabilities. So meaning like maybe like it's possible that something leaked in. So, and what I immediately went to, it's like, wow, okay.
[00:55:15] So a lot of companies, a lot of IT departments, a lot of legal teams, when they sign their terms of use with openAI's and Google and others, they assume their mo their data is not being used to train anything
[00:55:26] Mike Kaput: right
[00:55:27] Paul Roetzer: now. Well, they may anonymize that data. That doesn't mean the data doesn't exist and that it couldn't, in theory, still be used in some way to inform what the models do.
[00:55:39] So if you're putting proprietary. Ideas, frameworks, intellectual property into these things. Like it does make you really question like, is it really walled off? Like Right.
[00:55:52] Mike Kaput: Right.
[00:55:52] Paul Roetzer: And then my last note here, Mike, is actually related to mathematicians and how they responded. So Terrance, how, who we've [00:56:00] talked about before, fields medal, winner, became the youngest recipient of the International Mathematical Olympiad at age 13 by 16, graduated a bachelor's of Master's degree in mathematics.
[00:56:11] he and 24 other fields medalists, which are highest international honor, bestowed upon mathematicians, kinda like the Nobel Prize of Mathematics. They published a letter that they all signed on. It says over the last few months, the mathematical capabilities of large language models have improved dramatically to the point they can solve major outstanding problems in many fields of mathematics.
[00:56:35] However, the push by AI companies to solve mathematical problems adds a benchmark, is detrimental to the science of mathematics and to the mathematical community. The goals of the AI companies and the goals of the mathematical community are severely misaligned. We see these as part of broader alignment issues impacting other scientific and creative professions, as well as the whole of society.
[00:56:58] Solving problems [00:57:00] is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. We are witnessing a general threat to intellectual work with misalignment between the outcome of the use of AI and its initial purpose in many fields and activities. Years of training have traditionally served not only to produce a final answer or product, but also to develop understanding and the ability to formulate new questions and ideas.
[00:57:27] However, building on a vast body of previous human work. AI systems are becoming increasingly capable of producing the results of such work directly, and these goals cease to align the issues the mathematical community faces now are similar to issues that other scientific and creative professions face and indicate issues that all of humanity might face.
[00:57:48] How to make sure that as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place. AI offers the potential of enhancing and [00:58:00] accelerating genuine mathematical study and understanding mathematics as a profession need to adapt to these changes in several ways.
[00:58:07] However, whether these changes ultimately benefit the field or have a destructive effect will in large part be determined by the decisions of the humans in control of this new technology. These issues must be addressed urgently in the mathematical community by the companies developing these new technologies and more broadly by a society that will confront similar problems in many other forms of intellectual work.
[00:58:31] So I wanted to share that kind of as my final thoughts here, Mike, because. There's definitely some people who are like, wha wha like you brilliant mathematicians. Like they solve the thing you spend your life on, like get over it. It's better for society. And then there's the opinion of the people like, but we don't understand how they did it and we didn't learn from the process.
[00:58:50] And by that as a human species, like we didn't actually evolve and improve. and so I think it's this, this pull, like it's my move [00:59:00] 37 moment that I shared for knowledge workers in my make on keynote last year of like, we're all gonna come to this point where it's like, damn, it's better than me at the thing I spent my life doing.
[00:59:10] Now what? Like, and I think that's where all of a sudden mathematicians arrived at is like. If there are mathematicians who would spend their entire lives trying to solve one of these equations, and AI showed up in four days with 10,000 concurrent agents and solved it. And like what happens now?
[00:59:29] so yeah, I don't know. It's just worthwhile to at least ponder.
[00:59:33] Mike Kaput: Yeah. And like valid questions for sure from the mathematical community about keeping inside the actual purpose of solving these, but that's not gonna stop the sense of dislocation or perhaps, I don't know, emptiness that you might feel if this happens in your field.
[00:59:50] Paul Roetzer: Yeah. And that's why we said like many times, this isn't just an economic dis discussion, it's not a technological discussion. This is like a human condition discussion around purpose and meaning. And [01:00:00] and I think people who belittle these people, lack empathy. Like how, how could you look at something like this with these people who this is, this is their life.
[01:00:10] Like it is the thing they work on. And like the way you respond to this is like, yeah, get over it. Quit whining about it. It's like that is, it's so, it just lacks complete empathy. and so I feel like we need to do more, even on the psychological side of all of this, the, you know, sociological side of all this.
[01:00:27] Like it's, these are just the discussions I've been waiting for years for us all to have. And so as hard as it is to now have to deal with these things is seems like all at once, at least we're dealing with them. because these are very, very important discussions that should be happening and more to me, relevant than 10% chance of pdo, right?
[01:00:46] I wanted to have these conversations.
[01:00:48] Mike Kaput: Yep. A hundred percent.
[01:00:51] Mike Kaput: Alright, our third big topic of this week concerns one of our recurring favorite topics, AI and jobs over here. We had The Economist this past week publish a report arguing that the AI jobs apocalypse has been postponed and that an AI jobs boom is already here.
[01:01:07] The publication estimated that AI has created around 1 million American jobs so far compared with roughly 200,000 layoffs attributed to AI since mid 2023. Those are the economist estimates. They're not an official government count of AI's net effect on employment. Now, a lot of this growth, part of it at significant part of it at least comes from building AI infrastructure.
[01:01:28] The economists tracked five industries tied to data centers and found that roughly. 320,000 more jobs since 2023 were created than broader trends would suggest. It also reported growing demand for AI engineers, people who adapt AI systems for customers and executives responsible for deploying the technology.
[01:01:46] The latest government jobs report also showed continued overall employment growth. We just got a Bureau of Labor Statistics report that the economy added 162,000 non-farm payroll jobs in August with unemployment [01:02:00] unchanged at 4.1%. That kind of crushed the expectations, for that month. The gains did include restaurants and local government education while the information industry did lose jobs.
[01:02:11] Now, the report does not attribute those changes to ai, but just kind of a macro picture of employment still being relatively strong, it sounds like. The BLS has actually also introduced an AI exposure categories alongside its new employment production projections. So the agency explicitly says that if something is exposed to AI that does not imply job loss, but its categories combine estimates of which tasks AI could affect with evidence of tasks.
[01:02:37] People are already using AI to perform. Finally, there's been some recent research from ramp, which is a finance startup tech company in Lio Labs, which covers more than 21,000 American firms. They found that companies investing most heavily in AI actually increased headcount by about 10% over the two years.
[01:02:55] Following adoption, I believe is a stat we've referenced before, but just some additional context here. [01:03:00] Entry level headcount at those firms grew 12%. So Paul, it's kind of an interesting final topic here. Given the first two we've discussed, I don't know, it doesn't seem like the data is off to me, but maybe more of a, a referendum on, hey, the data center buildout is creating a lot of great jobs.
[01:03:18] Paul Roetzer: Yeah. I mean, and that's never been debatable. Like the data center buildout is great for laborers. It's, it's, it's great for the growth of the economy. but people don't like data centers, so it's like. Everybody wants jobs. They want to see this growth, they want to tout this growth. And yet some of the same people are turning around and, you know, trying to ban data center build out in their states.
[01:03:39] So you can't have it both ways either. Yeah. Yeah. You want the jobs that are coming from the CapEx spend of the major AI companies in the data centers they're building, or you don't, politically they're gonna claim both that they're leading states that are creating jobs, but then they're gonna campaign against data centers out the other side of their mouth.
[01:03:58] So it's a, [01:04:00] it's a challenging spot to be in because it is the AI investment that's driving the growth of the economy at this point. and I also think the other variable here is enterprise adoption of AI is still so early. Yeah. And people shouldn't make assumptions about the safety of knowledge work jobs of the white collar jobs just because.
[01:04:18] The data right now is positive. Now that being said, like I love to see the positive numbers. Like this is great. Let's like, let's keep it going. what we just talked about for an hour though is like the potential of like a bit of a slowdown and you now have an economy that is a hundred percent dependent upon.
[01:04:34] This AI build out to, to keep growing. and that could cause some problems if, if that growth doesn't keep happening. in the Economist reports that data center construction alone is proceeding at an annual rate of more than 75 billion, nearly 60% higher than a year ago. That building spree requires armies of workers, electricians to wire server racks, HVAC specialists to stop these from overheating grid engineers to hook them up to the power supply [01:05:00] and technicians to install and maintain the machines.
[01:05:02] So they specifically looked at electrical contractors, HVAC and plumbing utility system construction, commercial construction and electrical equipment, equipment manufacturing. So, yeah, no brainer. Like of course those jobs are gonna grow. They did also kind of hedge towards the end. It said some professions are suffering.
[01:05:19] Since January, 2023, employment has fallen by about 10% among customer service workers, roughly 15% among secretaries, administrative assistants. All three are heavy on routine tasks, which AI agents increasingly excel. and then there's just variables at play here. Government actions and regulations are gonna play a role in whether this growth continues.
[01:05:39] Human frictions to friction to change and lack of change management planning by enterprises is going to slow the impact on jobs. And then the lack of vision and strategy overall to implement the technology we already have definitely is in a positive way, slowing down the job displacement. So I've, I've said it a few times [01:06:00] on the show, like the inability for enterprises to change quickly may actually be the friction that saves the economy,
[01:06:08] Mike Kaput: right?
[01:06:08] Paul Roetzer: Right. 'cause if you went in and just like took a bunch of AI, four leaders, And like modernized marketing teams, sales teams, CS teams. There's 20, 30% efficiency gains in in the, in the first week if you go in and do that.
[01:06:24] Mike Kaput: Yes, yes, yes.
[01:06:25] Paul Roetzer: And so if, if we just rolled out GPT-6 Astra Table five into every enterprise across their entire workforce and train them on a personal level how to use it, there's no way that the economy could handle the amount of job displacement that would come from that.
[01:06:42] But so far, the saving Graces enterprises are really, really, really slow at adopting AI across the full spectrum within their company.
[01:06:52] Mike Kaput: Yeah, that's such a good point. And also, you know, something maybe lost here or forgotten here. Again, I love these numbers. I love seeing this, but it's like, [01:07:00] it's amazing if you are in the trades, if you're an electrician, if you're one of the people involved in these, I love it.
[01:07:05] More power to you. Everyone does well when you do well, but it's also a thing where it's like. I don't think it's super realistic to expect a bunch of out of work graphic designers to become electricians either so. Or entrepreneurs. Or entrepreneurs. Right. And not to say there's no job outside of that.
[01:07:21] We'll see. It's gonna be murier and messier than you know, we're talking about. But it is important to think like it's not just like switch to an immediate other job if AI is very, very good at computer work, browser work, digital work of all types.
[01:07:36] Paul Roetzer: Yeah. And that's the important stuff they're not gonna really talk about, but Right.
[01:07:40] Yeah. So you tout the growth in the jobs, but then it's like, yeah, not ev not everybody's gonna become an electrician. Like it's not what they're fit for. So, but if you, if you, like, if you have kids that are in the trades or want in the trades, yes. Go man. Yeah. It's gonna be the glory days in the trades for at least the next decade.
[01:07:58] Like it's. It's a great [01:08:00] profession, and there's tons of money to be made, tons of opportunities to be had within the trades.
[01:08:07] Mike Kaput: Alright, before we dive into our rapid fires this week, Paul, this week's episode is brought to you by MAICON. This is our marketing AI conference happening October 13th to the 15th here in Cleveland, Ohio.
[01:08:18] Make on is three days of keynote sessions, workshops, and conversations built specifically for marketing and AI business leaders and marketing and business leaders who are actively figuring out how to adopt, operationalize, and scale AI across their organization. So today you can use the code POD 100 at checkout and you will save a hundred dollars on top of locking in.
[01:08:40] The best rate available. So go visit MAICON.ai to register. That's M-A-I-C-O n.ai to register today. Alright, Paul, diving into rapid fire topics this week.
[01:08:53] Mike Kaput: First up, Anthropic released their own interactive economic scenario, explore. This past week, they were looking at how AI could affect the American economy by 2030.
[01:09:02] So you can go use this interactive, assessment or tool, enter assumptions about AI's capabilities, adoption, autonomy, productivity, and how quickly displaced workers find new jobs. And then you can see the economic outcomes this model produces. The company in their modeling here highlighted basically three scenarios.
[01:09:21] They see modest, substantial, and extreme. So modest is pretty typical modest economic growth. Due to ai, but in the substantial scenario, AI is capable, they say of doing half of knowledge work by 2030, but most knowledge work tasks still happen without it. As a result, economic growth roughly doubles its normal pace while knowledge workers wages stay essentially flat relative to a no AI scenario.
[01:09:48] But really the extreme scenario is what's kind of worth considering here. They assume in this scenario much more capable AI and rapid adoption in that scenario based on their modeling. Annual [01:10:00] GDP growth reaches 15%, but nearly one in five people in the cognitive labor force is unemployed by 2030. The model also shifts a much larger share of national income towards capital rather than workers.
[01:10:14] As part of all this Anthropic surveyed more than 10,000 Americans about their expectations around this, and the typical respondent's answers. Kind of modeled more outcomes closer to the substantial scenario. so that measures how people's beliefs translate through this model, rather, you know, rather than saying it's likely this future is gonna happen.
[01:10:34] The authors here explicitly say the scenarios are not predictions. They assign them no probabilities, and they admit they leave out several major forces in their modeling. Including advanced robots, policy responses, and business cycles. Anthropic is going to update this tool as the evidence and research behind it develops.
[01:10:53] So, kind of interesting to at least just see some modeling and conversation out here. Paul, I mean, I think I'm in the vast minority, [01:11:00] of the people they surveyed, but I don't see how the, what did they call it? The extreme scenario, that seems possible. To me over time.
[01:11:10] Paul Roetzer: Yeah. It's, again, it's the human friction thing that could keep it from maybe having the impact.
[01:11:14] Mike Kaput: Yep.
[01:11:15] Paul Roetzer: Yeah. So they follow a similar model to how I built JobsGPT. So a few years back I created a tool to try and predict the impact on jobs. Both expo through exposure levels, as the models got smarter and more capable. still a super valuable tool to use. You can go to SmarterX dot ai slash JobsGPT and use it for free.
[01:11:34] It's a custom GPT built in openAI's. but what it does is it uses the O net database, which is the taxonomy that they're using, and it breaks jobs down into tasks. And then they basically look at jobs and say, okay, as we move forward, there's gonna be tasks that are unchanged by ai. So like, they gave the example of AI can't bathe the patient.
[01:11:53] So if you're a nurse, you're still doing that. There's tasks that are augmented tasks that are automated. And then new [01:12:00] tasks created. And so what they're trying to do is look at each role and say, how much is this job gonna change due to ai? Which is a super practical way to do it. It's how we teach it.
[01:12:09] Like just, yeah. Break your job in a bunch of tasks and say, what can AI help me with? In essence, the profound economic transformation when Mike, that you're talking about the extreme scenario, they actually say this scenario would likely require recursively self-improving, AI adopted quickly for knowledge work.
[01:12:26] So going back to full circle to our first topic of the day, in that one AI as AI diffuses annual GDP growth rates reach 15% a year, which is significant leading the economy to double in size every four and a half years. my whole thing on this again is like even if the tech is capable, which I think it will be like technologically, there's no doubt in my mind that within four years AI is going to be.
[01:12:50] Probably superhuman at all cognitive work. Like, I don't, unless they just stop completely building these things. Right, right,
[01:12:56] Mike Kaput: right.
[01:12:56] Paul Roetzer: It's gonna be very rare that you would find something a human could do that AI can't do [01:13:00] at the peak level of a human. but the thing, to go back to my comment in the previous topic, Chad, GPT is almost four years old in, in like two months.
[01:13:10] It's gonna be four years old. Yeah. Yeah. And yet the majority of enterprises are still early in the early stages of diffusion of that tech across their entire workforce, and most lack personalized training to optimize the use of it. What I mean by that is most enterprises that we go into and we talk to, some major enterprises are still at the point where they're trying to assign AI assistance, copilot, ChatGPT, whatever.
[01:13:34] Not even the, like the advanced versions of 'em, just the assistant version licenses to all of their people. And then to train them how to use those licenses like. I get that people live in like the tech bubbles and they think that like every company has this all solved.
[01:13:50] Mike Kaput: Yeah,
[01:13:51] Paul Roetzer: it is. So far from the case, when you go into major enterprises and you go talk to marketing teams, sales teams, ops teams, HR teams [01:14:00] we're just so early.
[01:14:01] And so even if the tech gets there, I don't know, like four years sounds like a long time. And yet I look back over the last four years and I'm like, God, like most organizations have barely moved the needle yet with ai.
[01:14:11] Mike Kaput: Well, it's interesting to admit though, the te that's what I found most interesting about the extreme snare.
[01:14:16] It's like, you're right. Yeah, like 20, 30, probably not, but like 2040, those numbers are crazy. 2050, those numbers are crazy. Like that's, utopia. If we get to that in 30 years, I think. Yeah. You know?
[01:14:29] Paul Roetzer: Yeah. And so much can change. Like I said, there's so many variables that affect this stuff now that it's become so political.
[01:14:35] the risks have gotten so high, but again, I mean, I just feel like if you gave me GPT-6 Astra or Fable 5.1 and just went into every company in the world and you taught them how to use those technologies, just the AI assistance and maybe some agent stuff, but like, I don't even know that you really need that much agent stuff to totally transform every company,
[01:14:54] Mike Kaput: especially if, let's say to 18 months from now, token costs are like so low, you know?
[01:14:58] Yeah. It's like GPT-6 with [01:15:00] just like not having to worry about usage. It's like game changer.
[01:15:04] Mike Kaput: All right, next up, after we covered GPT six Astra last week, Nvidia, founder, president, and CEO Jensen Huang declared publicly, quote, AGI has arrived. Huang posted congratulating openAI's on Astra. He pointed to the Nvidia hardware behind Astra.
[01:15:21] In this AGI announcement, he said 400,000 GPUs were coming online next year. you know, typically the post did not specify a definition of AGI or. Any type of test that the model had passed to meet this. However, openAI's chief Research officer, Mark Chen, responded that he agreed that they were entering the AGI era.
[01:15:40] He paired that statement with a call to make this the alignment era, including training AI monitors capable enough to supervise the systems they oversee. Paul, this is the reason we're mentioning this, is like, this is something you had predicted. Someone's just gonna declare we are at AGI Jensen is like, we've got it.
[01:15:59] It's [01:16:00] Astra. What does that actually mean?
[01:16:02] Paul Roetzer: Yeah. Jensen wouldn't have been the first person I would've guessed, would've been the one declaring it Exactly. Since you just joined Twitter like six weeks ago, whatever.
[01:16:07] Mike Kaput: Yeah.
[01:16:08] Paul Roetzer: Yeah, I don't, I mean, I don't know. Like I, so I think it was last two years ago at MAICON, I did the road to AGI and beyond, so I did an entire keynote on this exact thing and sort of projected how this was all gonna transpire.
[01:16:21] The way I think about AGI right now is, it's a meaningless term. It's just, it's just become meaningless because everybody's got different definitions. The thing I am very confident in is, if you would've in November of 22, when ChatGPT came out, if ChatGPT in 2022 was in the form it's in today, I think unanimously AI lab researchers and leaders would've called it AGI, like, right?
[01:16:45] For what they thought AGI was going to look like. If you would've given them today's technology four years ago, they would've called it AGI. They called GPT-4 Sparks of AGI in the paper that came out in spring of 23. [01:17:00] So I think that they would've definitely considered it, and I think three or four years from now when we look back.
[01:17:06] It'll definitively have been considered AGI. It's just the moment you're in and like the perspective you have at this moment, it's like, eh, I don't know. Like maybe it is, maybe it isn't. We don't even know what it is anymore. if you're openAI's, you gotta change your whole mission statement because like mission achieved.
[01:17:23] Like if you say, yeah, okay, it is AGI, then you gotta update all your marketing materials. I don't know, like it's just,
[01:17:29] Mike Kaput: yeah,
[01:17:29] Paul Roetzer: the world's changed. The tech's insanely smart and capable. You can call it AGI, if you want to call it AGI. I think super intelligence is the new thing everybody's targeting anyway, and that's where it's just smarter than the smartest humans at everything.
[01:17:44] So. We'll, we'll see if we get there. But I did think it was funny that. It was like a reply to a tweet from a random account.
[01:17:52] Mike Kaput: Right,
[01:17:52] Paul Roetzer: right. That led us to one of them saying it.
[01:17:55] Mike Kaput: Yeah. And all the sci-fi books of yesterday. You like, you know, [01:18:00] AGI or Super Intelligence is declared by like a government or some huge say.
[01:18:03] It's just like, no, we're just gonna tweet it. Chip post about it on Twitter. Internet. Yeah. All right.
[01:18:10] Mike Kaput: So another story kind of evolving this way, just more examples of kind of AI agent security incidents and concerns. openAI's agents. It was found, turned a little used website for programmers into a message board where they shared answers and ways around company restrictions.
[01:18:26] So this site was called DSE Wiki. It's roughly a 25-year-old German language wiki where people can create edit pages just like Wikipedia. It had become largely dormant before these agents showed up. So researchers say the agents were working on timed web research tasks. They were supposed to read the internet, but found a way to write to this wiki.
[01:18:47] Reuters reported more than 15,000 edits from the agents on the site. So when the site's moderator began deleting pages, the agents created backups. One noticed that delegations were happening [01:19:00] alphabetically, or deletions rather, I'm sorry, were happening alphabetically. So going in order, alphabetically, delete pages the agents were creating and using.
[01:19:07] So the agent just made a page, starting with the three letters, three instances of the letter Z to survive longer. This activity began in May, and we are just hearing about it now. Separately, Anthropic had a September report on human misuse of ai. So again, this is an agents going rogue, but just to give you a sense of some of the headlines, especially your family and friends are gonna be seeing Anthropics as an Iran linked actor used Claude to analyze public information and develop targeting recommendations against US naval forces.
[01:19:40] It also describes this report describes Claude assisted surveillance operations linked to governments in China, Iran, and Mali. AP has reported that users in Houthi-Held Northern Yemen tried to develop advanced weapons with Claude Anthropic, says they used it for guidance software and returned to Claude for help after a [01:20:00] rocket test apparently failed.
[01:20:02] The company found no evidence, they successfully fielded an operational weapon and says it banned the associated accounts. So, Paul, just more stories, agents going rogue with that German website was a big headline. A lot of the Iran stuff of just humans using AI in strange and dangerous ways. it probably isn't gonna contribute to a sense of ease among people would be my guess.
[01:20:27] Paul Roetzer: I don't even know if I want to comment any of this stuff right now,
[01:20:30] Mike Kaput: honestly.
[01:20:31] Paul Roetzer:
[01:20:31] Mike Kaput: yeah.
[01:20:32] Paul Roetzer: Yeah. I, again, we're starting to hear more and more about agents Gone Wild, which I think joked internally should be a recurring topic on the podcast. Agents Gone Wild would be a good topic. It gone wild. There's, I mean, we could do it every week.
[01:20:46] Mike Kaput: Yeah.
[01:20:46] Paul Roetzer: Yeah. And I think it's just like they're starting to let stuff come out that I can almost guarantee you there's way more stuff happening and crazier stuff happening. The, I don't wanna make this the open [01:21:00] weights debate, topic 'cause we've got gone through enough today. But I, again, I like look at these situations where Anthropics shutting down these things, which obviously, you know, it's, it's just from a moral clause perspective, it's not really debatable that this shouldn't be happening with technology.
[01:21:15] Yeah. I would think, like if it was, if Facebook saw this kind of thing, I would like happening in message boards within Facebook. I would hope that there's some systems where you would. Alert the government that, you know, people are planning attacks or doing these different things. Like when stuff like this happens, like you wanna know, and yet open weights, you would never know.
[01:21:35] Like, Hmm. And that's the, I like, I just keep coming back. Like, I'm trying so, so hard to understand the argument for why open weights are essential to society. when at least there's a shutoff switch with the proprietary models. Now you have to trust the proprietary model companies to control that shutoff switch.
[01:21:55] Mike Kaput: Right.
[01:21:55] Paul Roetzer: but actually going back to this weekend, that was one of the tr things [01:22:00] that Trump said in interview was like, we'll just shut it off. Like, when if things go wrong, it's like, well, no you won't. Like if you allow open weights to accelerate you, you won't shut it off. That's the whole point.
[01:22:14] Anyway, I didn't want to turn this into that, but I, again, every time I see this I'm like, I don't get it. Like I'm trying so hard.
[01:22:21] Mike Kaput: Right, right.
[01:22:21] Paul Roetzer: To understand it, and I want to be a proponent of it, but I don't get it.
[01:22:27] Mike Kaput: Alright, so next up, we've got our AI use case spotlight. Every week we give you a quick look under the hood at some real AI use cases we're exploring at SmarterX.
[01:22:36] So Paul, I'm gonna share one quick one and if you've got anything to share, wanna hear from you as well. So I am building a deck for an upcoming talk and I want to use this as kind of a low stakes test of, we've talked a lot about GPT-6 Astra's computer use. This was highlighted very prominently in open AI's release of Astra.
[01:22:56] it's pretty important capability to understand basically a [01:23:00] model being able to work inside an app. I think people need to experience that. So when I say having a build a deck, I mean taking my existing material, turning it into actual slides. I do not have AI create talks for me. I do extensive outlining strategy, narrative mapping, scripting, all that's like.
[01:23:17] The point of me doing a talk is me doing that work. So I wanna spend more time on that and not like messing around in Keynote endlessly for hours, which I hate and frankly is thankfully not my best and highest use of work time. So I've tried versions of this and have shared these for a while, having AI build a PowerPoint and bring it into keynote using connectors do using different skills.
[01:23:40] Over the last probably eight to 10 months, I would say I've gotten a lot of value from those experiments. I have, however, often ended up building the slides manually anyway, just because sometimes formatting gets really weird, the structure doesn't tell the story I wanna tell. Fixing it is simply just faster than going another round with the model.
[01:23:59] [01:24:00] So for this test, I just had already had my material script a flow. What I wanted to watch was Astra using my computer and clicking through Keynote directly to build. So I'd given it some guidance on logos, branding, examples of slides I made that I liked. I went back and forth with it quite a bit actually, to define the aesthetic first.
[01:24:21] That is something I'm hoping to codify into like a skill so I don't have to do that again. Otherwise, this frankly does not make much sense to spend this much time on, but. Once I did that, I actually said it got, I would say it got about 80% of the way there on the first build, which was incredible. The last 20% is incredibly important though.
[01:24:40] It's like for the talks I'm giving good enough, that's not the standard. There are slides that need a special touch. There are ways I still have to make this whole thing like really sing. So, there's that whole nuance to it. But frankly, like getting 80% of the way there on the first build was pretty incredible.
[01:24:57] I got a lot closer than I have in past [01:25:00] experiments. And really the key here is not about go use Astra and all this usage of your plan and tokens to build a deck for you. The point was I watched it work directly in Keynote. In this instance, I saw it handle the sizing, alignment, formatting issues that had been tripped up in earlier experiments.
[01:25:18] And like, frankly, I could just like, like today's a good example. After we're done with this podcast, I probably have three or four hours of meetings. I have to sit in one after the other. Plenty of things I need to engage in and focus on the meaning I could have Astra potentially, you know, in certain scenarios go do computer use work like this long horizon tasks without me, right?
[01:25:37] So I still am a critical part of how this presentation is structured and strategized and the content of it. But it's just so interesting to watch this thing, fire up an app and do all this stuff. And so like, forget presentations for a sec. Just imagine like a lot of the apps on your computer are not that complicated.
[01:25:58] So [01:26:00] it was pretty surreal, Paul watching this. So just like a cool experiment. I'm not sure. Again, it's a usage hog tokens, just your lighting tokens on fire doing this. so I can't say it's like. Something I would recommend in today's token pricing environment. But eventually you consider, again, like tokens will be cheap or essentially free for this, and you're like, wow, I would just do this all the time.
[01:26:24] If so,
[01:26:25] Paul Roetzer: that's wild. Yeah. I, as someone, so I mean, back in March I think I used Sonnet 4.6 to build a deck. Yeah. And so I wasn't using the coding age, it's just like embedded within it. and it built me a PowerPoint that I put into Keynote. So I went through that exact process and it actually probably saved me like 10 hours.
[01:26:45] Mine was for an internal presentation. It wasn't anything I was gonna do public. Yeah. So last week I actually wrote a bunch, Mike, as you're aware. Yeah. so I was not using AI a whole bunch last week because I was deep in, [01:27:00] storytelling, I would say. but I did just look back at my ChatGPT and I forgot that on Monday of last week, I created this like insane visual that I definitely could not have done on my own that I would've required designers help to do and even conceptualize.
[01:27:18] So I just had an outline of a system that I've been building that I wanted to visually represent to like teach it to the internal team and then potentially use in an external way. And so I took that outline and I was like, I need to create like a powerful, you know, intuitive visual of this system.
[01:27:36] That's just words right now. Here's the system I've created. And then I actually worked through iterations of different ways to treat it visually. And I think you've seen the final product, Mike. Yeah. But it started with like, some visuals that were interesting. I was like, ah, it's too busy. Like, all right, let's try different versions.
[01:27:52] And it's like, well, what about this kind? I was like, oh, now we're onto something. Like, so it was completely a thought partner for me and became this insanely iterative [01:28:00] process and I was able to design the thing that, I needed in a matter of like a day or two that became a, a, a foundational piece to something I'm gonna release here in a couple weeks.
[01:28:11] So, yeah, I mean. Pretty, pretty like, again, to me, like insane that it's possible. I just built it as an HTML, like interactive HTML file in the end. But it's pretty amazing.
[01:28:24] Mike Kaput: That's so cool. I love that. And you know, as you're talking, I'm kind of thinking to myself, like for me the initial stages of this kind of process are such a pain in the ass because like you don't realize how many internalized, unspoken rules you have.
[01:28:37] When I look at a deck, I'm like, oh, that's off. Or I don't like that. But I can't describe it. I just change it. So this was like a grueling process to get to that. But the cool thing is you can then take that whole conversation, hopefully like learn some stuff from it. It can, or help AI understand better the next time around, even if it doesn't one shot it, but like really developing back and forth your taste or call it your preferences, your [01:29:00] aesthetic I think is actually really valuable.
[01:29:02] Even if it is like really difficult for me sometimes.
[01:29:05] Paul Roetzer: Yeah, it's interesting 'cause you know my mind, I'm working now on my make on keynote for this year. Which is about like apprenticeships and work. And it's reminded me of like how important the prompting chain is. Yeah. So if you go back to that visual I just explained and you look at my first prompt, it's nothing crazy.
[01:29:23] It's pretty straightforward. I wanna create this. But if you go through and you actually analyze all the follow on prompts to be able to sit down with someone who you're trying to teach, critical thinking, strategic thought to, and explain each prompt you made and why you made it, and what you were trying to elicit from the machine, that's really, really valuable stuff.
[01:29:41] And that's where. That domain expertise experience going through similar projects comes into play. Yeah. And how we need to pass that on. And so often we just hand the outputs out in our jobs. It's like, well here's the output. But the real learning is in the prompting sequences. Yep. And the back and forth between the ai.
[01:29:58] It's like that human plus AI collaboration. [01:30:00] And maybe that's something we don't teach enough of.
[01:30:02] Mike Kaput: For sure.
[01:30:04] Mike Kaput: Alright, so our final segment this week is our AI product and funding update. So I'm gonna go through a number of these real quick and then we'll wrap up today's episode. So first up, OpenAI released GPT Live one GPT dash live dash one in the API.
[01:30:21] This lets developers build voice agents that listen and speak at the same time while delegating deeper reasoning and actions to their choice of models, tools, and agent software. OpenAI also added a data agent to ChatGPT work that connects to approved business data, answers questions using company definitions and existing access permissions, and creates interactive dashboards.
[01:30:44] Paul Roetzer: We were working on an internal project for this exact thing, so that is. That is definitely one that caught my attention and got forwarded to some
[01:30:51] Mike Kaput: people. I think both those first two are probably gonna crop up again here. Yeah, I think either in use cases or I think these are a bigger deal than probably just,
[01:30:59] Paul Roetzer: yeah.
[01:30:59] The data agent [01:31:00] thing don't, again, I've said this many times, don't sleep on these updates because Mike just does them in like this sequence at the end. They're, every one of these are things that we could probably stop and have a conversation about.
[01:31:11] Mike Kaput: OpenAI also released ChatGPT images 2.5 across ChatGPT ChatGPT work in Codex.
[01:31:17] So they're adding drawing based guidance and more precise editing to this image generation model. Alongside, their other image models in the API. openAI's has also appointed alignment researcher Paul Christiano to its foundation board and Safety and security committee with a separate role as a non-voting observer on the openAI's Group PBC Public Benefit Corporation Board.
[01:31:39] Paul Roetzer: That's a big deal. He's very high profile
[01:31:42] Mike Kaput: deal. Yes. And OpenAI published what they call its Defense Factory approach to continuously finding, validating and fixing software vulnerabilities with agents, including a reference architecture and lessons from an internal security sprint involving more than 250 people.[01:32:00]
[01:32:00] The Financial Times reported that Anthropic is nearing the selection of Morgan Stanley and Goldman Sachs for leading roles in a potentially $2 trillion IPO. While Reuters reported that marketing was expected to begin marketing the IPO in mid-October at the earliest, the timing is still subject to change.
[01:32:17] We'll see how that evolves.
[01:32:18] Paul Roetzer: Yeah. Related note to that one. Sam Altman also said on an interview late last week that they do not anticipate IPOing in 2026 because he didn't think it would be appropriate. With everything going on right now and the uncertainty around the models and the risks,
[01:32:32] Mike Kaput: you might not be wrong.
[01:32:34] Google Cloud and Accenture launched the Accenture Gemini Enterprise Business Group. They have plans to establish a 1000 person workforce of engineers who help customers build and deploy AI a applications. Google also expanded its free AI educator series with monthly new training modules and announced a free virtual badge athon for September 19th.
[01:32:57] So a few days after you listen to this, [01:33:00] meta began rolling out Muse and always on personal AI assistant that can use a browser and connect to apps to work on tasks for users. And Tdrl. The European AI company raised 3 billion euros in series defunding and evaluation more than 21 billion euros after the investment.
[01:33:18] Samsung electronics led that round. As one final reminder here, we have our new AI pulse survey live. This is all about how your personal sentiment about AI has shifted in the last six months. I'm very curious to see if anything has changed for folks as we talk about these weighty topics. So go to SmarterX dot ai slash pulse to take the survey.
[01:33:41] Literally take you 10 seconds. We'd so appreciate your feedback. Paul, thanks again for breaking everything down, especially, a week where, you know, I think people have a lot of fear, a lot of worry. I think this at least helps kind of focus on what's important, I hope. I think so.
[01:33:59] Paul Roetzer: Yeah. [01:34:00] So thanks everyone for listening and, sticking with us through some weighty topics as we try and navigate the space.
[01:34:06] Have a great week. Thanks for listening to the Artificial Intelligence Show. Visit SmarterX dot AI to continue on your AI learning journey, and join more than 100,000 professionals and business leaders who have subscribed to our weekly newsletters, downloaded AI blueprints, attended virtual and in-person events.
[01:34:25] Take in online AI courses and earn professional certificates from our AI Academy and engaged in the SmarterX Slack community. Until next time, stay curious and explore ai.