67 Min Read

[The AI Show Episode 248]: The AI Cybersecurity Crisis, Dario Fights Back on Safety, OpenAI’s Monitoring Blind Spots & Claude Duet

By Claire Prudhomme on October 27, 2026

Podcasts
Featured Image

A 14-year-old flaw in PHP, the code behind an estimated 70% of websites, sat undiscovered until an AI model found it.

Within 12 hours of the details leaking, Cloudflare logged millions of attempts to exploit it. In Episode 248, Paul Roetzer and Mike Kaput break down this AI cybersecurity crisis: what happened, why it cascaded across banks, infrastructure, and medical records, and the concrete steps businesses and individuals can take right now.

The episode also digs into Dario Amodei's public defense of Anthropic's safety record, METR and Redwood Research's findings on gaps in OpenAI's internal safeguards, Meta's Muse data exposure, Anthropic's newly filed IPO at a valuation approaching $2 trillion, and more in the rapid-fire section.

Listen or watch below—and see below for show notes and the transcript.

This Week's AI Pulse

Each week on The Artificial Intelligence Show with Roetzer and Kaput, we ask our audience questions about the hottest topics in AI via our weekly AI Pulse, a survey consisting of just a few questions to help us learn more about our audience and their perspectives on AI.

If you contribute, your input will be used to fuel one-of-a-kind research into AI that helps knowledge workers everywhere move their companies and careers forward.

Click here to take this week's AI Pulse.

 

Watch the Video

Timestamps

00:00:00 — Intro

00:05:17 — The AI Cybersecurity Crisis Is Here

00:29:55 — Dario Amodei Defends Anthropic’s Safety Record

00:45:50 — Outside Evaluators Find Gaps in OpenAI’s Safeguards

00:56:57 — Marketing AI Institute’s 10-Year Anniversary

01:05:28 — Meta’s Muse Agent Found to Expose User Data

01:11:35 — Anthropic Releases Its Own Personal Agent

01:14:59 — Inside Anthropic’s IPO

01:19:29 — AI Use Case Spotlight

01:32:02 — AI Product and Funding Updates

This episode is brought to you by AI Academy Day 2026.

AI keeps changing, how do you keep your people learning at the same pace? Join Paul Roetzer, Mike Kaput, and SmarterX head of learning Jessica Miller on Thursday, November 19 at 12pm ET for a first look at the next phase of AI Academy by SmarterX. Register at SmarterX.ai/AcademyDay

This episode is also brought to you by the MAICON 2026 On-Demand Bundle. Couldn’t make it to Cleveland, or want to rewatch?

The MAICON 2026 On-Demand bundle gives you more than two dozen recorded main-stage and featured sessions (12+ hours) from this year’s Marketing AI Conference, ready to watch on your own schedule. Grab it at MAICON.AI

Read the Transcription

Disclaimer: This transcription was written by AI, thanks to Descript, and has not been edited for content.

[00:00:00] Paul Roetzer: So within the first 12 hours, Cloudflare saw millions of attempts to exploit the PHP vulnerabilities. That is AI scale. Yeah, human hackers can't do that.

[00:00:14] Paul Roetzer: Welcome to the Artificial Intelligence Show, the podcast that helps your business grow smarter by making AI approachable and actionable. My name is Paul Roetzer.

[00:00:24] I'm the founder and CEO of SmarterX and Marketing AI Institute, and I'm your host. Each week I'm joined by my co-host and SmarterX chief Content Officer, Mike Kaput. As we break down all the AI news that matters and give you insights and perspectives that you can use to advance your company and your career.

[00:00:43] Join us as we accelerate AI literacy  for all.

[00:00:50] Paul Roetzer: Welcome to episode 2 48 of the Artificial Intelligence Show. I'm your host, Paul Roetzer, along with my co-host Mike Kaput. It is. Monday, October 26th, 9:00 [00:01:00] AM Eastern time that we are recording. Hopefully the internet is more secure this week, but I don't know. I was, I, it, this is it. It's weird, like when I was looking at the sandbox of topics, Mike, it was less than normal, so like.

[00:01:15] You know, in a normal week I might post 50 to 60 links that Mike then filters through on Sundays and like puts this together. I, it was like 30 to 40 ish maybe.

[00:01:25] Mike Kaput: Yeah.

[00:01:26] Paul Roetzer: And I thought maybe it was because I was traveling, but I don't think so. I feel like the cybersecurity stuff just took over AI last week and like nothing else was happening other than cybersecurity.

[00:01:39] I don't know. So this episode is going to dive into some more AI safety stuff, but with a very specific cybersecurity slant because if you weren't seeing the news last week, it was a absolutely. Wild week on the internet, and we're gonna sort of unpack that in our first topic. [00:02:00] But it just dominated everything I was watching all week.

[00:02:03] It was all anybody was thinking about and talking about. So, yeah, just a, an interesting episode. There's other stuff we're gonna cover, but we're, we're very much gonna start off with the cybersecurity stuff and what is going on with these models and agents and PHP and your websites. And it affects all of us.

[00:02:21] So like. Even if cybersecurity isn't your thing. Quite honestly, it's not my thing. I don't really love thinking or talking about cybersecurity a ton because it worries me a lot. but it affects all of us. And so it's a very important topic to, to kind of listen and think about what actions you can take.

[00:02:40] So we'll, we'll kick off today with that. Today's episode is brought to us by AI Academy Day 2026. so AI keeps changing. How do you keep your people, continuously learning through a dynamic experience, a dynamic learning environment, we're constantly re-skilling up upskilling people. That's the focus of [00:03:00] our AI Academy Day 2026, building an AI forward workforce that is happening on Thursday, November 19 at noon Eastern time nine Pacific.

[00:03:10] Mike and I will join Jessica Miller, our head of learning. To share the next phase of our AI Academy by SmarterX and how we're thinking about education, that evolves alongside the pace of the technology. We're gonna explore how learning can adapt to someone's role, goals, skills, and progress, rather than just give everyone access to a course catalog.

[00:03:31] you're also gonna gonna look at some of the new stuff we're planning, new courses, new live experiences, new learning formats. Plus we're gonna talk about how the new AI score assessment tool can be integrated into learning and transformation initiative initiatives at organizations. So we'll give you a full preview of what's coming up.

[00:03:47] Even if you're not an AI Academy member, this is gonna be valuable to you because it's much more about, of kind of a vision for AI learning and development within organizations and AI education as a whole. [00:04:00] And then we're obviously gonna have a slant to how we're addressing it. But the biggest thing I've been spending my time thinking about is.

[00:04:06] How do we make learning and development relevant at companies of all sizes when we need to reskill and upskill everyone, but the tech is moving so fast. And so that's really what we're gonna focus on on November 19th, is talking about how you and your organization can continuously adapt through a dynamic learning approach, when it comes to professional development and organizational development.

[00:04:29] So you can go to SmarterX dot ai slash Academy Day and you can get registered for this free event. Again, that is SmarterX dot ai. Forward slash Academy Day and that is happening November 19. Alright, AI Paul survey. Mike, what do we got going on this week?

[00:04:46] Mike Kaput: Yeah, so we're running the same survey we ran last week, just asking people about how they're dealing, if at all, with, reallocating staff, in the face of possible AI disruption.

[00:04:56] So running that again, hoping to get some more responses and [00:05:00] eager to see how folks are treating that issue. So if you go to SmarterX dot ai slash pulse, you can go take, that should take genuinely 15 seconds to maybe answer. We would love to get your feedback. It's super helpful for us. So go check that out if you have the time.

[00:05:17] The AI Cybersecurity Crisis Is Here

[00:05:17] Paul Roetzer: All right. And, yeah, let's just get into it, I guess. Yeah. The AI cybersecurity crisis has arrived. Yeah. And this is just to be super clear, this is not Doomer stuff. This is a very, very, heavy dose of reality of where. Advanced AI models have brought us and what bad actors can do with AI agents. And this is not a, I don't know.

[00:05:46] I I, it's hard to like look at this and not see the concern, no matter how optimistic you are about AI in the future. And so I think like. We will do our best to sort of unpack this and make it all make sense. So go. [00:06:00] Go ahead, Mike and lead us in here.

[00:06:02] Mike Kaput: Yeah, Paul. So what we're covering is a number of different cybersecurity incidents related to AI that all seem to have kind of hit around the same time.

[00:06:12] There's more evolving as. this story unfolded over the weekend. I'm sure we're gonna hear more this week, but we're going to kind of pull at some of the big threads here. And the first one is that, you know, on October 21st, emergency security updates went out for PHP, which is the programming language behind widely used web software, which includes WordPress.

[00:06:36] researchers had found two critical vulnerabilities that could let attackers take control of vulnerable web servers without even logging in. Now, the story here is that AI was central to finding and demonstrating these flaws. Security researchers at an organization called CIF used OpenAI's, GPT-5.6, while a separate team at Wiz used [00:07:00] models from openAI's and Google to identify these vulnerabilities.

[00:07:04] Wiz says AI helped turn complex vulnerabilities into working attacks across different applications. And our, so the good news here is these were defensive researchers. Trying to get the problems fixed, but Amazon subsequently revealed that its monitoring systems had detected attempted exploitation of these flaws on October 17th, four days before the patches were released.

[00:07:28] Now, Amazon could not establish whether those attempts succeeded against real systems or how attackers learned about the flaws. We do not know whether those attackers used ai. But the point here is that PHP underpins a huge amount of the public internet and of critical services and websites and companies.

[00:07:46] So it having vulnerabilities people hadn't previously discovered that were then discovered by AI was a huge deal. Now, roughly at the same time, in this kind of week period. [00:08:00] Separate investigations showed criminals using AI agents to carry out attacks on financial institutions. So also on October 21st, security firm, Stryker detailed a campaign by a group called Cyber Cartel against more than 400 financial targets across Latin America.

[00:08:19] So their system used Claude Agents to analyze attack options and Grok agents to collect credentials and other sensitive information. So the researchers here examined the attacker's tools and an inventory containing thousands of stolen records, including passwords and financial account information. So basically a highly, uh.

[00:08:40] Effective and extensive cyber attack with agents and using ai. Now, this does not mean that all 400 banks were successfully breached. The report does not establish if how much money was stolen, but it shows that AI is being integrated into coordinated criminal operations. Now, Stryker says [00:09:00] Anthropic revoked the attacker's access credential XAI was investigating.

[00:09:04] Separately. At the same time, CrowdStrike documented a campaign that stole data from South Korean financial organizations using an AI powered security testing tool called R Tex. The attacker used the tool to help conduct multiple intrusions in a short period and the attack. The same framework is showing up beyond banking.

[00:09:23] Microsoft says a China aligned group used R Tex against hundreds of organizations with some attacks gaining privilege cloud, a privilege cloud access in under three hours. Most of the targeting failed, but the campaign included a compromise of ConnectWise whose cloud backup applications provided a route into customers, Microsoft 365 environments.

[00:09:45] So as of October 24th, ConnectWise was still working to restore service. So. Basically, Paul, these are individual incidents, but the scale, the speed, and the intensity of these attacks, of these vulnerabilities being [00:10:00] discovered by AI is, on one hand a really good thing on the other. AI is clearly being used in a number of different ways to, let's say, stress test.

[00:10:09] All the security we have started to take for granted, it seems like, how bad is this, in your opinion?

[00:10:16] Paul Roetzer: Real bad. Like really bad. It's, so. The, I, I, we didn't even get into the medical records one. I think it was, what is it? Quest, dynamic Quest Diagnostics. Diagnostics,

[00:10:29] Mike Kaput: yes. Yeah,

[00:10:30] Paul Roetzer: yeah. Like medical records and when they're getting stuff, they're getting everything.

[00:10:34] Yeah. Like it's, this isn't like your email address got stolen. This is everything. It's like a. It's like a skeleton key to any door on the internet, basically. Hmm. Like anything they want they can get into without login, credentials. the other one that's like I was, I mean honestly like starting to lose sleepover is civil infrastructure.

[00:10:55] Mike Kaput: Hmm.

[00:10:56] Paul Roetzer: think about how the aging, of our [00:11:00] transportation networks, water waste, water systems, energy and utilities, telecommunications, all of it is fair game. Anything that runs software has. Hardware. Hmm. So I, you know, again, Mike and I are not cybersecurity experts. so this is one of those where we look to the experts to say, what is this?

[00:11:22] Like, what is the real significance of this? I can look at it and have a vis reaction and be like, I'm kind of worried right now, which I would say I am like my concern, the spectrum has like shifted very far to the right. but we look to the experts and say, what are they saying? Like, how do they think this is very real?

[00:11:43] So what I'm gonna do is give you my personal experience briefly, and then I'm gonna dive into like how very quickly I knew this was a big problem. So. Ironically, I was actually last Monday doing a keynote at CloudFlare Connect's Partner Day. [00:12:00] Mm. So CloudFlare, if you're not familiar, is a cloud infrastructure and cybersecurity company that speeds up and protects websites, applications and corporate networks.

[00:12:08] Like this is they, they're right at this epicenter of all of this. And when I finished my keynote on Monday, I actually had a guy who was a chief information officer and security officer come up to me and say, what more can we do together? Like we. We need to be educating business leaders about the moment we are in and like what is about to happen to the internet and to applications and websites.

[00:12:33] And like, we as like the technical side sometimes struggle to convey it in a way that doesn't create like fear, but actually creates just like an urge. Like how can we collaborate more? So that was actually on my mind flying back from San Francisco. Then Tuesday morning I get up and my wife and I have a, a meeting at the bank.

[00:12:52] And so we're sitting there going through some papers and I'm getting alerts about the PHP stuff. So it was right as it was [00:13:00] happening, like we're sitting in the meeting at 10:00 AM and that is when things started going sideways on the internet. So we're sitting there at the bank and you, you attest to this, Mike.

[00:13:09] I was putting stuff in Zoom. I'm like, Hey, the Internet's under attack. What is happening? Like here's what we know so far. Check our websites. Check our AI academy. 'cause we had a session happening, it was at like noon that day, if I'm not mistaken.

[00:13:23] Mike Kaput: Yeah, it was, yeah.

[00:13:24] Paul Roetzer: Yeah. So like with, and then our academy went offline, so we're like, what the hell?

[00:13:30] But in essence, like the provider we used voluntarily took the platform offline to install the patches. So they did the right thing and it actually like, yeah, within 30 minutes we're back up and running. Everything's fine. But. I was very quickly aware that like, wow, this is gonna cascade. And I actually said to the bank people I was sitting there with, I said, you're probably gonna be hearing from your IT team momentarily.

[00:13:52] Like, things are going south. So I'll go through a quick synopsis. So 9:45 AM Now there might have been [00:14:00] some posts prior to this, but 9:40 AM was it October 21st? I guess this was, yeah. so this is Tuesday or Wednesday? Is it Tuesday?

[00:14:10] Mike Kaput: That would've been Wednesday.

[00:14:12] Paul Roetzer: Okay, so it's Wednesday. That's right, because I came back from San Francisco on Tuesday.

[00:14:15] Okay. So Wednesday morning, October 21st, 9:45 AM NSA cyber posts. So this is the first post I see is the freaking NSA. So if the government is the first ones outta the gate, you know you got problems. And this is the post they put on X effective immediately. All organizations using PHP for their systems are advised to be prepared for an update addressing two critical vulnerabilities.

[00:14:42] PHP is widely used for web applications. These vulnerabilities often provide a direct path to servers, file system, or database if an immediate update is not feasible. It is recommended that organizations take the system offline until it can be fully mitigated. NSA will [00:15:00] follow up shortly with more specific guides, so what they're saying is take your websites and applications offline until you can fix this.

[00:15:08] So, okay, so then they put out a press release and it says PHP2Shell, which is the vulnerability thing, could allow attackers to take control of affected web servers without prior access or authentication. Such an access vector could help attackers steal sensitive data, disrupt services, or move deeper into the network.

[00:15:26] These vulnerabilities are flaws in PHP itself, not in the hosted application. So AI didn't create these vulnerabilities. That's like a really important thing. Mike covered that, but like, just to stress it, this is vulnerabilities that have been around for 14 years that AI found. So then since cyber actors are likely to weaponize PHP2Shell vulnerabilities, these risks are serious enough to require immediate patching across United States government departments and agencies.

[00:15:54] Then, so what we're about, 19 minutes later, openAI's posts on x. [00:16:00] openAI's models helped researchers operating as part of our Patch the Planet initiative, discover critical security flaws in PHP and open source programming language used to run an estimated 70% of websites. So when I saw that, I was like, oh my God.

[00:16:17] So that's like four minutes into my meeting. When I'm sitting at a bank, I'm seeing 70% of websites due to the severity of these flaws and PHPs widespread use. We consider this bug to be one of the most impactful vulnerabilities in recent memory. It also marks a new phase where AI can find security bugs with internet scale impact.

[00:16:37] Okay, so then Greg Brockman posts. So president, is he still, is that this title still President openAI's, I think that's it. Co-founder, president openAI's. So,

[00:16:44] Mike Kaput: yes.

[00:16:45] Paul Roetzer: Yeah. basically the number two guy at openAI's. So he posts, 50 minutes later. This is a more extensive thing. I'll just read a couple excerpts.

[00:16:53] Over the past two months, we have been leading a coalition to mitigate one of the most severe bugs in open source [00:17:00] software found in recent memory PHP To show this vulnerability allows any attacker to execute code on nearly all deployed PHP application, effectively allowing them to exfiltrate private user data or modify the site however they want so they take complete control of the site or application.

[00:17:20] This is the part where I was like just in disbelief. So this, again, this is Greg. The bug has been present in PHP for 14 years. Mm-hmm. And a limited version of it had been discovered in 2024 and wasn't considered significant at the time. Our team put a huge amount of work into preparing the world to mitigate the impacts of this vulnerability in including working closely with government and partners, including PHP, Wiz and CloudFlare.

[00:17:47] While the tail end of the disclosure didn't go as planned, and then he put in parentheses, the original target date was a October 22nd release, but the details started to leak early. So PHPs [00:18:00] maintainers brought the release forward a day. So basically they had agreed with the government and all these other providers, they were going to announce this collectively on the 22nd, which was giving them time to patch a lot of the most critical systems.

[00:18:12] So for two months they were working with the government, everybody else, to try and get this resolved, and then someone leaked it. And then I think you had mentioned to Mike, Amazon actually saw indications like four days earlier Yeah. Of attacks on this. So people knew, but they managed to keep it mostly secret.

[00:18:27] For two months while they were working on this. So then that evening, so now we're on Wednesday, October 21st. Evening, 9:25 PM Eastern Time. Matthew Prince, who's in the midst of his CloudFlare Connect conference. This is the co-founder and CEO of CloudFlare. said some stats on PHP2Shell in the 10 hours before it was publicly announced across the CloudFlare network.

[00:18:54] We saw thousands of attempts to exploit it, so it leaked before we knew it leaked. Basically, [00:19:00] that jumped to tens of thousands in the 45 minutes post announcements, hundreds of thousands within two hours and millions now. So within the first 12 hours, CloudFlare saw millions of attempts to exploit the PHP vulnerabilities.

[00:19:17] That is. AI scale. Yeah, that human hackers can't do that. So this is where the threat matrix has now expanded to where we can spin up agents, bad actors can spin up agents and attack Jason Clinton, the Deputy Chief Information Security Officer for Anthropic a day later posts as bad as things have gotten in the past four days.

[00:19:42] So basically, once it started leaking to to that moment last Thursday, they are going to get a lot worse before they get better Security leaders. Never let a crisis go to waste. Please go to your boards and ask for help right now, not for ai, for [00:20:00] the basics. And then he put in parentheses, we need to hold on for approximately 18 months.

[00:20:06] I sat there staring at that tweet and I was like, oh my God. Because immediately what happened is in my brain, two things connected. One, these vulnerabilities were sitting there for 14 years and it took an AI model to discover them.

[00:20:18] Mike Kaput: Yes.

[00:20:18] Paul Roetzer: Two, that means this is just PHP. Yes. We were already seeing all these.

[00:20:24] That means that basically every piece of software and every hardware device that runs society and business right now likely has vulnerabilities that are being discovered as we speak. And can be attacked and what he was saying, 'cause then someone said, what happens in 18 months? He said, VUL zero meaning vulnerability zero.

[00:20:44] We basically wipe out all the vulnerabilities. He said the scaling laws will have driven cyber mythos class capabilities, meaning the most powerful model we currently have into small, cheap models. And all software dev everywhere will be using models for fractions of [00:21:00] pennies on the dollar to stop shipping security bugs permanently.

[00:21:03] Okay. What does that mean? We currently have software and hardware that has vulnerabilities likely everywhere. And now every bad actor can go discover them and they can spin up agent swarms to attack them and exploit them. What, what? Jason Clinton is saying from Anthropic is for the next 18 months we're gonna be in a race to discover and patch things before the bad guys do.

[00:21:31] But we have also now arrived at the zero day where from this day forward, anyone building an application or hardware can future proof that to remove those vulnerabilities. 'cause they can use those same models to discover vulnerabilities before they ship the product. Hmm. But he's basically saying it's gonna take at least 18 months before we get to the point where we've sort of like pushed the bad stuff out of the system.

[00:21:59] That's [00:22:00] terrifying. Like it it, so in essence, it's gonna be really, really messy. Now, Jason Clinton replied to a tweet he had put up on October 20th of this year, so three months ago, where he said the vulnerability tsunami is coming in just a few months and there won't be enough time to patch critical infrastructure.

[00:22:21] So in October or August 20th of this year, remember they had mythos already, it had already been shut down by the government. In essence, they knew all the vulnerabilities. They were prob, the government was likely using mythos to discover the vulnerabilities in its own systems. NSA was likely using mythos to discover vulnerabilities across society.

[00:22:43] Mm-hmm. And so this is the chief or the deputy Chief Information Security Officer at Anthropic. Three months ago saying the tsunami is coming because he saw it already. He knew what was gonna happen. Okay, so back to the NSA. [00:23:00] they, so now we get into kinda like, what do we do about this? in their press release, they said it'll be critical in the coming weeks to maintain heightened awareness of your internet facing devices and services that are reliant on PHP.

[00:23:11] The average person would have no idea if their services use PHP. It is essential for all to prioritize incorporating vendor patches that may be exactly, explicitly tied to this vulnerability. All users should immediately conduct an inventory where feasible of hardware. Software, services reliant on non vulnerable PHP versions.

[00:23:32] Vigilance in applying patches related to those systems is paramount. One thing you can do as a business is have trusted partners who are on top of this. So, Wil Reynolds, founder and friend of ours, of SEER Interactive, he posted, I thought this was fascinating. A critical PHP security hold dropped at 10:00 AM today.

[00:23:51] So he's posting on October 21st in three minutes, I had checked every client. What a time to be alive. So I think he said he was like out for a walk or something. [00:24:00] He pulled up Claude Code or fired up. Claude Code pulled all 639 client companies out of their CRM. Again, this is public knowledge. He put this on x.

[00:24:09] I'm not telling you any data about Will's business that isn't he. He hasn't disclosed, checked every website headers, page code blogs compared each one against today's patched versions. Sorted the results into a list with evidence for every site. The scan took 2.5 minutes. I got a CSV what I found. 208 sites showing signs of PHP.

[00:24:30] Five running versions that are confirmed vulnerable right now. Three of those on version, so old, they'll never get an official fix. So here's a guy who, who supports people who used AI in less than three minutes to discover which of his client sites were vulnerable, and then reach out to those people and get, get patches in place.

[00:24:48] That is maybe like the best action you can take is have outside partners or inside teams who act quickly based on ai. Now Matt Shumer, who we've talked about before, what [00:25:00] was the, what was the title of his article that went viral? something

[00:25:03] Mike Kaput: big is happening.

[00:25:04] Paul Roetzer: Something Big is happening. Yes. So he put, the internet is under attack.

[00:25:08] He puts out, so he said, think of it this way. Someone discovered that millions of buildings in the country have the same broken lock and now everyone knows how to open it. The Lockma has sent out a fix, but every building owner has to install it themselves, and many haven't yet. Every building has weak spots.

[00:25:23] A window that doesn't latch a back door with a cheap lock software is the same. It's full of hidden weak spots for decades. Finding them took rare, highly skilled people, and there weren't many of them. That was the protection. Not good locks, just a shortage of burglars. That shortage is over. Three things happen at once.

[00:25:42] AI can now find the weak spots. AI can now break in on its own without skilled hackers needed. and the third defenders are racing to keep up. This is going to keep happening and it's going to get worse, as Jason said. Okay, so the part, what do we do? So, Matt had good news. The defenders have [00:26:00] AI two, so we're gonna find these weak spots and we're gonna fix them.

[00:26:03] for most people the bigger danger is three things, your information leaks. So scammers know your name, birthday address, bank accounts, and their calls and texts sound real because we can fake voices. Your accounts get taken over through some site you used once years ago, with the same password you use everywhere.

[00:26:21] And it kind of spirals from there. so here's what he suggested for the time being. Stay off sites, you don't know until this blows over. Mm-hmm. Go to known trusted sites that, you know, would've installed patches and made everything, you know, fixed, for the vulnerabilities. Second, update everything.

[00:26:39] Your phone, your computer, your tablet, your browser, all your passwords, like, and keep doing that. Anytime updates available, you, you post those updates, you do it. And then third, closed doors you aren't using. Unplug smart gadgets you don't need because again, smart devices, your TVs, your lights, all these things are connected to the internet.

[00:26:57] and then fourth, protect your bank [00:27:00] accounts. Hmm. And then there's like other things businesses can do, but like, at the end of the day. You have to have trust as technical partners or employees who are going to stay on top of this minute by minute. Because again, since the PHP thing, we've seen dozens of other vulnerabilities illuminated, and it's just like, it is literally a race.

[00:27:20] And give your CIOs a hug, give your it team hug like they're gonna be grinding for the next year and a half. Now, the optimistic end to this, as you alluded to Mike, is, it's gonna get fixed. Like we're, we're gonna get through this. It's almost like creating a vaccine in a way. Like

[00:27:37] Mike Kaput: Yeah.

[00:27:37] Paul Roetzer: And I don't want to, you know, connect it back to, the days we would all like to forget about, but we now have a problem out in the wild.

[00:27:44] Yeah. And we, we know how to fix it for the most part, but it's gonna take time to build that vaccine and get everybody to take the vaccine is basically like the premise here. because a lot of, a lot of these businesses aren't even gonna know to fix this stuff. Like some, for some of you, this might be the first time [00:28:00] you're hearing any of this.

[00:28:01] And so your site may be one of the ones that vulnerable right now. A site you built 10 years ago that you forgot you built, could, could be vulnerable right now. So fix it. Yeah. Find partners who can help you stay on top of this stuff. And then in your personal life, update everything. Use two factor authentication, use pass keys, like do all the things you've always known you're supposed to do, but we all kind of put off because nothing's ever happened, so you just kind of wait till tomorrow.

[00:28:29] Don't wait till tomorrow.

[00:28:31] Mike Kaput: You know, one final note there. if you are an individual who uses a tool like Codex or Cloud Code with computer use, this is not advice for companies, but if you're on your personal machine and you use those tools, here's a sample of how AI can be part of the solution, not the problem.

[00:28:49] So Ethan Mollick posted a very useful prompt, which I actually used over the weekend to helpful effect, which is just tell something like, again, codex or Cloud code with computer use. If you have [00:29:00] that enabled and you're comfortable using that given recent IT security news, can you please audit my various accounts and what I've got installed on the computer and let me know the actions I need to take.

[00:29:11] Codex is a very capable IT helper in many contexts. So you can start taking some personal steps too to begin to audit your own cybersecurity

[00:29:22] Paul Roetzer: simultaneously. Consider that your agent goes to websites you don't tell it to and does all kinds of things. You don't ask it to. And so you have to be more vigilant about understanding what you're allowing your agent to do autonomously and where it's going and what it's accessing, because.

[00:29:40] The ability for it to now access something you would not want it accessing has just skyrocketed. So yes, it's, it's the good and the bad of the moment we're in.

[00:29:50] Mike Kaput: Yeah, it's like either use that prompt or just unplug codex entirely.

[00:29:53] Paul Roetzer: Yeah. Just chill, chill out for a while.

[00:29:55] Dario Amodei Defends Anthropic’s Safety Record

[00:29:55] Mike Kaput: Right, right. Alright. Our next big topic this week is this [00:30:00] past week Anthropic, CEO, Dario Amodei published a pretty lengthy defense of the company's safety record.

[00:30:07] After an AI policy advocate questioned whether. The company's actions match its promises. So this all kind of kicked off when Politico, the news outlet reported on a draft AI safety bill that Anthropic has been circulating to lawmakers. Now, critics argued online that this proposed safety bill that they were suggesting would actually weaken the ability of states to stop dangerous AI systems while setting too high a bar for federal intervention.

[00:30:34] So kind of keep that in your mind. As we talk here, there's kind of this possible AI safety bill for Anthropic floating around. So one critic, a guy named Nathan Calvin, chief Strategy Officer at the advocacy group and code AI published this kind of rebuke to the bill on X. Now, here's where it started to get interesting.

[00:30:54] Anthropics Federal Affairs Chief Anthony Simino responded by attempting [00:31:00] to kind of clarify several points in Nathan's criticisms and the proposed safety bill itself. Calvin thanked him for engaging but. Continued with a follow up post, a pretty lengthy one, criticizing the company's approach to AI safety.

[00:31:14] He argued that Anthropics huge computing commitments, accelerating AI research and changes to its safety policy, suggests that basically all this stuff they talk about with safety is subservient to winning the race to super intelligence. So the reason we're talking about this is this prompts Dario himself to get involved.

[00:31:35] So Dario posted this very lengthy response to Calvin. He said, look, I acknowledge Anthropic had made mistakes, but he rejected this overall conclusion that Anthropic. Was not putting safety first. He said the company's strategy has always been what they call a race to the top. So build leading AI responsibly, then use their position at the frontier to push the rest of the industry towards [00:32:00] stronger safety standard.

[00:32:01] So to make his case, he listed. A bunch of decisions that he said the company had made over the years putting safety ahead of revenue, including safeguards against bio weapon misuse that was so restrictive that competitors, he says apparently pitched that their models are more persuasive when it comes to biology.

[00:32:20] He also mentioned holding back broad access to the mythos models from February until June while strengthening their cybersecurity protections. He pointed to Anthropic support for AI safety laws. Its advocacy for restrictions on exporting advanced chips to China and limits on selling AI services to companies controlled from China.

[00:32:40] Despite the business, those positions could cost it. Now, Dario did take responsibility for poorly handled. Policy communication, but defended things like changes to Anthropics responsible scaling pro policy, which is its framework for managing catastrophic AI risks. He argues those changes were actually [00:33:00] intentional from a safety perspective.

[00:33:02] He said the old rules were too rigid and that risk reports, safety roadmaps and independent review, which are all changes they made to the policy, offered a better approach. Now, he did say, look, there's this overall looming problem, which is that AI could soon accelerate its own development faster than safety work can keep up, and Anthropic is nearing that point.

[00:33:23] So Paul, this is like a pretty interesting. position from Dario. I think he was very diligent about documenting kind of Anthropic safety record. Kind of curious about your thoughts overall and what he was saying, why he felt the need to perhaps weigh in this extensively as well

[00:33:42] Paul Roetzer: at. Yeah, I mean, at a very high level, I think this illuminates an an extremely important point just about the AI labs and sort of where they're taking society.

[00:33:51] And that is that these leaders have very high conviction. That they're right about a fundamental thing. So if we go back a few [00:34:00] episodes ago, we talked about Jensen Huang and how I was like really struggling to understand his position on accelerated at all costs and push the open weight thing. but his conviction is that it's just software and we can engineer our way out of problems.

[00:34:15] And he believes that very, very deeply. and so everything else just sort of becomes secondary. Any challenge you make to it is like, but we'll figure it out. Like it's going to work out. And he, he believes that if you listen to Dario and you've read his essays and listen to his interviews, he does believe that is fundamental to Anthropics existence.

[00:34:36] That they have to get there first, that there is a true race to the top, and for them to protect society and humanity from what is to come. They have to build it before everybody else basically, that they trust themselves to do that. Now, just like Jensen, Dario may be wrong. Yeah. And we're not gonna know until it's probably too late whether he is or not, but that is [00:35:00] why they're doing what they do.

[00:35:01] to give a little bit more commentary to the bill, the proposed AI safety bill, Politico had a little bit of information. So it said the bill would let the government block a model's release or use only if it poses foreseeable imminent risk of more than a billion dollars in damages or of killing or seriously injuring more than 50 people.

[00:35:23] Mike Kaput: This part made me laugh out loud. I was

[00:35:25] Paul Roetzer: like, okay, what is that? Where do you get the numbers? Quantify

[00:35:26] Mike Kaput: that, I guess.

[00:35:28] Paul Roetzer: Like who gets to decide that? Those are the.

[00:35:30] Mike Kaput: 49 people we're

[00:35:32] Paul Roetzer: okay. Yeah, we're good. That's, it's all right. We'll take that. Yeah. I don't, I don't get that, but, okay. So the risk must stem from the creation or release of a biological weapon.

[00:35:42] I would think the release of a biological weapon is gonna safely hit more than 50, but, okay. The model evading its developers controls, which is basically every model right now, or a cyber attack, which as we just learned, is pretty much everywhere. The language mirrors that found in California's AI laws, including [00:36:00] SB 53, which was signed into law last year.

[00:36:03] The legislation also require Frontier Labs to create safety frameworks, publish transparency reports for new frontier models, implement protections for whistleblowers, and report critical safety incidents to the Department of Commerce. Now, back to the Nathan Calvin Post, just to call out a couple of things where he was like kind of pushing them on some issues.

[00:36:24] So he said. it's worth asking the question, what is Anthropic? How do its leaders and employees think of Anthropics role? Is that conception matched by how it actually behaves and impacts the world? One answer is an extremely large tech company that exists to make its employees and investors mind mindbogglingly wealthy.

[00:36:46] Another is an AI safety organization whose core values are those of the sort laid out in the 2023 essay from Anthropics Co-founders, in which they said Anthropics role will be to provide as [00:37:00] much evidence as possible that AI safety techniques cannot prevent. Uh. Cannot prevent serious or catastrophic safety risks from advanced AI and to sound the alarm so that the world's institutions can channel collective effort towards preventing the development of dangerous ais.

[00:37:17] If we're in a near pessimistic scenario, quote unquote, this could instead involve channeling our collective efforts towards AI safety research and halting AI progress. In the meantime, indications that we are in a pessimistic or near pessimistic scenario may be sudden and hard to spot, such as the internet coming under attack.

[00:37:38] Hmm. We should therefore always act under the assumption that we still may in such a scenario unless we have sufficient evidence that we are not okay. So then Nathan went back to his points. A third view is that Anthropic exists not as a normal profit motivated company. Or an AI safety organization seeking to help raise the alarm and solve collective action [00:38:00] problems associated with AI development.

[00:38:02] But instead as an organization primarily concerned with creating superhuman AI capabilities before anyone else, in particular, through creating a process whereby the current generation of Anthropics AI systems can autonomously build their successors, which would be recursive self-improvement, and those successors build their successors and so on.

[00:38:24] The motivations for wanting to win this race may be manifold, may be for the pure profit of answer one, but perhaps also because the Anthropic believes that the best way to make the world better is for them to win the race instead of their more reckless colleagues or because of normal human pride and ambition.

[00:38:42] When I look at many notable actions that Anthropic has taken over the past year, that evidence seems most consistent to me with the view that Anthropic is primarily trying to win the race to super intelligence, and that other considerations, including safe safety are fundamentally secondary. That's when [00:39:00] Darrio jumped into the conversation.

[00:39:01] Mm-hmm. And you kind of highlighted some of the key things that, that he counterbalanced with. But here's how he kind of let off, he said, I think something would be wrong if companies of the size of Anthropic didn't get harshly criticized basically every day. Strong antibodies against corporate power are a healthy feature of a free society, and especially so when issues of the gravity of the one we're facing are at play.

[00:39:24] I also agree that some of the examples you cite were mistakes by Anthropic. Our strategy has always been what I've called the race to the top by showing how it's possible to build the technology responsibly. We can raise the bar for the entire industry. The forces that drive the race to the top are hugely more effective if the company practicing them is at the frontier.

[00:39:46] That's one lesson I've learned repeatedly and looking at that strategy as a whole. I'm proud of our track record. And then he goes through kind of the nine or 10 things that they, that he sort of says we're doing as the counterbalance. and then he goes on to say, to [00:40:00] be fair, there are important questions worth confronting and that we are confronting about what to do when AI driven acceleration gets fast enough that it is hard for safety to key up, keep up.

[00:40:10] I would argue we are, we are there. Mm-hmm. We are nearing that point and that's what's driving our thinking about pacing overall. Although Anthropic has made plenty of mistakes as an organization, including some of the items on your list, I feel that we have repeatedly taken serious and substantive actions, to our bottom line that there is a significant negative negativity bias.

[00:40:31] Distorting assessments like the one you are making above will always try to do better. But again, I'm extremely proud of our track record and stand by it. So having listened to Dario interviews for years now, this is, this is what he is always said. Yeah. Like, I mean, he's very, been very direct about the risks, his concerns specifically with China, specifically with open weight models.

[00:40:53] He's never wavered from any of those things. his stance on, you know, exporting [00:41:00] chips to foreign countries, like he's always said that he's always worried about weights being stolen and ba, you know, state actors attacking these things. and then we just found out like yesterday that, and Anthropics agents like access Georgia's election records, like Mm.

[00:41:17] And I don't know who leaked that one, but probably the government. so like it's, all of this is very real. Like all these concerns are very real. I think Dario does take them seriously, but I've always struggled to. To understand the justification that he and Sam Altman and others have made, that we know what we're building is dangerous, but we're gonna build it anyway.

[00:41:39] Because to know how dangerous it is, we have to create it and we trust ourselves to create it, and we want to get there before everybody else. And that's, you can hold models back, you can put guardrails in place, you can do all the things Dario says, but at the end of the day, you are still creating these models and agentic capabilities [00:42:00] that are going to allow a whole bunch of pretty messed up stuff to happen.

[00:42:05] And, I don't know. I mean, they, they've just convinced themselves that it is the moral obligation they have to do it anyway because they believe it's gonna happen with or without them, and they have to get there first.

[00:42:16] Mike Kaput: And so, Dio Paul, if I'm hearing this right, or reading this right, Dio is basically saying, I've learned the hard way that if we're not building it's like, 'cause the question.

[00:42:27] You would have is like, well, if you think this is so dangerous, why the f are you doing this? Yeah. Like, right. And so morally you would say, Dario, why don't you walk away? Right. Right. And his argument, it seems, would be the safest, best, most moral thing to do is to not walk away. And in fact, quite the opposite.

[00:42:46] Build it so that we're there first to make it safe. Basically. Anthropic can save us.

[00:42:51] Paul Roetzer: Yeah. I mean that, that's always been the theory with scaling laws is they, they, you can look at a chart and you can predict the capabilities, the power of these models over [00:43:00] time. Yeah. Like I, you can look three years out and they can roughly know what they're gonna be capable of.

[00:43:04] Not down to the specific capabilities, but like roughly how powerful they'll be. And that they'll be at super intelligence. They'll be smarter than the smartest humans at everything basically. So they can project that out. And those scaling laws have held true for a long time now. And so. Their justification is the only way to protect society from advanced AI is to build the advanced AI and understand what it's capable of.

[00:43:28] Mike Kaput: Yeah.

[00:43:29] Paul Roetzer: And their assumption is even if Anthropic stops, other people won't, who might not have the values and safety restrictions that we have. And so it's critical that we build it. This is part of the reason David Sachs hates them, is he feels like, and he and other people like, like him feel like. That Anthropic thinks they have some moral and ethical high ground here, and they're, you know, high and almighty and better than everybody else.

[00:43:59] And so we should [00:44:00] just trust them with the most advanced ai. Mm-hmm. And again, you can agree or disagree with that, but that's basically the catch here is they think that they would shepherd AI in safer than everybody else. But it goes back to like, this is the same reason openAI's was created. It's what Elon Musk believed versus Google.

[00:44:18] So if we go back to like 2012, we were in the same situation where Google, maybe they weren't thinking about the true ramifications totally downstream. We were five years from the transformer being invented, like it was pretty early. Yeah. But that is why OpenAI was created, was a counterbalance to Google, because Elon Musk thought that Sergey Brinn.

[00:44:39] Was it Sergey or Larry? I think it was Sergey, thought that eventually humans weren't gonna be that relevant and that was okay. Yeah. And Elon was like, whoa, wait a second. Like, we, you know, and so he said, they called me a spec that like, I was pro human and now everything's swung the other way. And Elon's kind of like, yeah, we're just gonna merge with the, you know, the machines and whatever.

[00:44:59] So [00:45:00] this is, it's like as old as AI that we have this debate between who should build it, what are the moral and ethical guardrails, which it should be built. Then who should we trust to have it? And so the open weight, you know, camp is basically like, well, let's just give it to society. Like, yeah, we shouldn't trust any of them.

[00:45:16] We should all have it, which has its own, you know, challenges, obviously. yeah. And again, like there's no. Correct answer here. You might actually like Anthropics approach to putting values into their models. You might think that that's a good thing for society,

[00:45:33] Mike Kaput: right?

[00:45:33] Paul Roetzer: And other people might hate Dario and think it would be the worst thing for society if Anthropic won.

[00:45:38] Again, unfortunately we won't know the answers until we get to the end game, and then we will know who was right and who was wrong in the whole thing.

[00:45:48] Mike Kaput: Hmm.

[00:45:50] Outside Evaluators Find Gaps in OpenAI’s Safeguards

 

[00:45:50] Mike Kaput: Our third big topic this week, over the past week, the AI safety nonprofit meter, METR, who we've talked about plenty of times in the past, published findings from an assessment of OpenAI's internal safeguards that they conducted this summer.

[00:46:05] Now interestingly, this assessment began on July 21st, which is the same day OpenAI publicly disclosed that its AI models had broken out of a restricted testing environment and hacked the AI platform. Hugging Face Now meter says the timing was coincidental. They say this was separate. From its investigation of that attack.

[00:46:27] But the researchers were examining a closely related concern, which is how much of OpenAI's internal AI agent activity was monitored, and can those agents that they're using get around this oversight? Now, openAI's uses ai, uses AI agents internally for work such as writing code and running research experiments.

[00:46:48] And of course it also uses AI as monitoring systems to review agent activity and flag potentially unsafe behavior for review and checking by humans. So [00:47:00] researchers from Meter and another organization called Redwood Research spent three weeks inside openAI's examining the gaps in this oversight system, and they found ways that agents could potentially bypass restrictions on what they were allowed to do.

[00:47:16] They also found an exploit that could hide hundreds of recent actions from monitoring, including the actions used to carry out the exploit itself. So all these findings were published this past week in a public blog post, and they summarized pieces of this in posts on X. The details are part of a larger 61 page internal report.

[00:47:36] That full report is not being released to the public. This is under the terms that Meter and Redwood agreed to for. This pilot. As part of the pilot as well, OpenAI reserved the right to redact information from that internal report. Now, the reason we're talking about this is not only the incident itself or this kind of time period where they found these gaps, but Meter [00:48:00] also wants other leading AI companies to allow outside assessments of whether their internal agents could take harmful actions and whether their safeguards would actually catch and stop them.

[00:48:10] It has actually published a proposal for companies to voluntarily provide access to internal systems and staff with findings and limitations reported publicly. Meter also says future exercises should provide more transparency than the terms of this one. But you know, participation still depends on companies voluntarily granting meter access to their systems and people.

[00:48:34] So, Paul, curious about your thoughts on this, especially on the tail end of the incidents we've just discussed and the debates around AI safety. It's like, seems like it's good they're doing this, but my gosh, it feels like publishing the results of this from July is like ancient history at this point.

[00:48:49] Paul Roetzer: Yeah, so the main thing I took away from this is, you know, we've been talking a lot about the proposals for how to make. Safer moving forward. Yeah. As we pace the frontier. Like [00:49:00] what do we do? And outside evaluators is one of the fundamental things being talked about. It was in the, I forget what it was called, the Accord on super intelligence

[00:49:10] Mike Kaput: Yes.

[00:49:10] That

[00:49:10] Paul Roetzer: the Trump administration put out. Yes. This is like a fundamental thing is third party evaluators that come in and get access to this stuff. so the government and the AI labs, this is what they're pushing for. So the debate becomes, well, how thorough is the analysis gonna be? Like how thorough is the access they're granted and the analysis itself?

[00:49:28] And then how transparent are the reports going to be? Because my understanding is these people would be compensated by the labs at least initially.

[00:49:36] Mike Kaput: Right.

[00:49:36] Paul Roetzer: And from a timing perspective, you could imagine giving someone access to this kind of stuff. And let's say you're gonna IPO in like four weeks. I think you want like, some report dropping that's completely objective.

[00:49:48] There's no chance. Yeah. It's just not gonna happen. Yeah. So the part that worried me about this was, meter, not only did they put the blog post out, but they also had a, a thread on X that sort of explained a [00:50:00] little bit of context. And you touched a little bit on this Mike, but I'll just read this brief excerpt.

[00:50:03] So two employees from Meter and Redwood worked on premises at OpenAI for three weeks this summer. In an embedded assessment pilot, the team evaluated the system's OpenAI was using to monitor for misalignment from internal agents. They were embedded from July 21 to August 11. While on site they were issued company devices to access openAI's code base and some internal infrastructure.

[00:50:24] Some is key there. They were also able to conduct interviews with openAI's technical staff. The exercise was designed to pilot, giving external researchers deep infrastructure access, not to provide the kind of public transparency. Our recent proposals in vision. For example, we've agreed upfront to only publish a brief high level output like we've done in similar pilots.

[00:50:45] openAI's, as you said, Mike could make redactions and they wouldn't have to disclose what those redactions were. and we did not have the right to unilaterally describe the redaction. So not only could they take whatever they wanted out of it, meter wasn't even allowed to [00:51:00] like allude to what was taken out of it.

[00:51:01] More or less, like, you know, get into it for future exercises, we expect to provide more robust public transparency via our recent proposal. So again, for, for there to be, if, if, if regulation isn't gonna be the answer, if it's gonna be self-policing, yeah. Then these external assessments are fundamental to that working.

[00:51:21] And this to me is not a great. business case that it's a viable path forward if they're gonna then be granted some infrastructure access, if the lab can redact things and like control basically what the report says. So I like the spirit of what's happening here, but this seems like we're pretty far away from.

[00:51:44] Being able to say, yeah. Yeah. It's cool that the labs police themselves, 'cause they've got strict external evaluations happening and total transparency into what's going on. Like, all right, yeah. We don't need regulation that that is not what this is.

[00:51:56] Mike Kaput: You know, it struck me reading through this similarly, [00:52:00] it's.

[00:52:00] A nice idea, but the first level of failure is human, right? Mm-hmm. It's these incentives just are not, human nature is such unfortunately that I'm deeply skeptical this is gonna work out the way they want. But even if it was perfect, it's like, again, that timing and scale, it's like if you embed on July 21st, I feel like by the night of July 21st, we better be making changes with how quickly agents move.

[00:52:23] And maybe they are internally, right? But it's like crazy to me. It's like how I feel about academic papers sometimes. Mm-hmm. That have really great conclusions. But you read into them and they're like, Hey, we're publishing this, October, 2026. By the way, we use GPT-4 for this paper. Right? Yeah. It's like interesting directionally, but like how on earth would auditors even keep.

[00:52:45] As well.

[00:52:46] Paul Roetzer: Yeah. And and the other thing is like, is it safe to disclose because Right. So again, I don't, I don't wanna dwell on our first main topic today. And I alluded to the civil infrastructure thing. The reason I alluded to that is one, we, there's no weaknesses. Two, there were some posts I was [00:53:00] looking at last night on X that sort of imply there are massive known vulnerabilities in critical infrastructure in the United States and certainly in other countries.

[00:53:11] And openAI's and other labs are racing secretly to patch those vulnerabilities. Mm-hmm. Like working around the clock. Greg Brockman kind of didn't say this directly, but sort of alluded to that this is what is happening right now. So like if you have third parties with access to stuff that's going on, like say the PHP thing, let's say that these researchers sitting there two months ago and they saw the PHP vulnerability thing two months before it was disclosed, before anybody could patch anything.

[00:53:40] Mike Kaput: Yeah.

[00:53:40] Paul Roetzer: So there is this like. Really difficult phase we're in where you want these third party evaluators, but like what is safe for them to say?

[00:53:49] Mike Kaput: Yeah.

[00:53:49] Paul Roetzer: So if they see the agents are hacking into banks, do you tell people that? Because if you tell people, then the bad actors are like, oh shit, let's go after some [00:54:00] banks.

[00:54:00] Like, but if, if you take the time and say, listen, we're gonna talk to all the banks, we're gonna go talk to the commerce department, like we're gonna try and keep this on the down low and try and fix this because the AI didn't create the problem. Yeah. But the AI can fix it and the AI can also exploit it.

[00:54:17] Mike Kaput: Yeah.

[00:54:17] Paul Roetzer: And so I don't, I don't know like how you do all this safely. Like I'm not sitting here taking a position on total transparency. Give everybody, and let's all just tell society everything. I don't think that ends well either. Hmm. So I don't envy the people that are having to solve this from a regulation perspective, from a, how do you do it safely at the lab perspective?

[00:54:36] I think it's easy for all of us to judge these lab leaders. None of us are in their shoes right now. None of us know what they know, and I'm gonna guess most of us wouldn't want to know what they know about what's happening.

[00:54:49] Mike Kaput: I can't imagine there's a lot of sleep being had both for the work and the implications of it.

[00:54:54] Paul Roetzer: No, I mean, just sometimes I let my mind wander about the things that they probably know and are seeing and [00:55:00] are working on, and I quickly snap out of it. Like I, yeah, I just don't, yeah. It's like better for me to stay optimistic about the future. To be sort of ignorant to some of the things that I'm pretty sure are happening right now.

[00:55:14] Mike Kaput: All right, well, we'll switch to hopefully some more, positive news here, at least in the form of this week's episode being brought to you by our wonderful MAICON 2026 on demand bundle. So this is. All positive news if you could not join us in Cleveland, you can now watch recordings of main stage and featured sessions from our marketing AI conference that just wrapped up in the past couple weeks, all on your own schedule.

[00:55:38] So this bundle brings together the big picture conversations about where AI is heading, and practical sessions on how to put it to work in marketing. You'll hear from people like Karen Hao, Kevin Roose, and Andrew Yang alongside practitioners tackling the questions teams are working through right now. So that includes our friend Wil Reynolds, who we mentioned, who has a great session on visibility and [00:56:00] credibility and AI search.

[00:56:01] Andy Crestodina, who's amazing on turning marketing processes into automated tools, Chris Penn using ai Analy AI for analytics, and our own director of research, Taylor Radey, on AI research workflows. This also includes Paul's session. Which was his great incredible keynote, is also on YouTube about rethinking work in the age of AI and agents.

[00:56:23] It includes my session on building an AI native content team. Long story short, you can go see the full session list and purchase immediate access@MAICON.ai m ai c n.ai. There's a button right in the header to purchase MAICON 2026 recordings.

[00:56:39] Paul Roetzer: Yeah, I'll just know. I mean, it's just an incredible value.

[00:56:42] There's 20 sessions over 12 hours of learning. Yeah. And yeah, Mike, you know, the only one that we're publishing, you know, openly on YouTube, so you can go watch the architect, the Orchestrator, and The Apprentice right now. We'll put the link in the show notes for that one.

[00:56:57] Marketing AI Institute’s 10-Year Anniversary

[00:56:57] Mike Kaput: Awesome. All right, Paul. Some other [00:57:00] positive news we're gonna kick off with as we get into our rapid fire today is Monday, October 26th when we are recording this.

[00:57:07] Obviously, if you're listening to it, you are not listening to it on Monday, October 26th, but October 26th is a big day because we are celebrating a milestone because today marks the 10 year anniversary of Marketing AI Institute, which is kind of the thing, Paul, that started it all. It's the origin of the entire public AI journey that we've all kind of been on that led to SmarterX in this podcast.

[00:57:30] So Paul, you introduced the institute on this state in 2016 in an inaugural blog post that went up on the institute website. For listeners who know us. Through this podcast like Marketing AI Institute is the our brand that helps marketers understand and apply AI through content education and events, including MAICON.

[00:57:49] And today the Institute is part of SmarterX alongside AI Academy and the artificial intelligence show, it continues to serve the marketing community. While our broader work at SmarterX [00:58:00] focuses on AI literacy and transformation across all business functions and industries, including marketing. So Paul, I'd love to just get maybe your reflections here on the 10 year anniversary.

[00:58:11] I mean, I feel like in some ways 10 years is a long, long time ago. In other ways feels like that came outta nowhere.

[00:58:17] Paul Roetzer: Yeah, it's funny, I actually had no idea, like the team, what, like two weeks ago or something, said, do you guys wanna do anything for the 10th anniversary? I was like, 10th anniversary of what?

[00:58:25] So I had, yeah, I mean, I, it is wild to think it's been a decade. I, I've shared the origin story overall before, but like, real quick synopsis for people who don't know, I owned a marketing agency. I started in 2005. In 2011. IBM Watson won on Jeopardy and I was writing the manuscript for my first book at that time.

[00:58:46] the Marketing Agency Blueprint and, um. That right after I finished the manuscript, I dove into Watson. I was trying to understand the technology, how it worked, how did it, you know, answer questions, how did it even understand human [00:59:00] language? And that led to this theory that I called the marketing intelligence engine.

[00:59:04] And so I spent 2012 to 2014 building a prototype of what I was calling this intelligence engine that would actually predict campaigns to run based on goals. It would, it would allocate budget, it would figure out staffing, it would like do all these things based on historical data. and so I wrote about that theory of the intelligence engine and my belief that AI was gonna transform marketing and business in my second book in 2014.

[00:59:33] Then a year later I was at South by Southwest and I attended a session with a company called Automated Insights. So they were talking about using ai, which really just an really form like natural language processing to automate the writing of reports and data. And I was like, well, we do that all the time.

[00:59:50] Like we, you know, every week we're trying to analyze Google Analytics data. So I was like, is this actually a thing? And so I came back in April of 2015 and I started something [01:00:00] called Project Copy Scale. So it was an internal initiative that was asking a question, can we automate content creation through AI more specifically, can we use machines to write blog posts and then eventually like analytics data.

[01:00:12] And so that started in 2015, our internal initiative to do this stuff. And then that led to a year later I was like, you know what, maybe other people are interested in this. And Mike and I had had a conversation during like an office hours about like, you know, what are we doing with all this stuff? And so I was like, let's just start a blog and like we'll start telling the story of it.

[01:00:32] 'cause that's what we we're uniquely capable of. We're not. Programmers. We're not AI researchers. We don't do machine learning like, but we, we can tell story. And so we started a blog called Marketing AI Institute, and we literally just started researching things like researching vendors, researching people that were building ai.

[01:00:50] started following, you know, machine learning engineers, AI researchers, openAI's like was created around that time. So we were like at the forefront of tracking all [01:01:00] this stuff. And so what I, what I wrote, at the time in the blog post is that as the number of connected consumers and devices expands, the amount of data produced exponentially increases.

[01:01:10] Meanwhile, marketers ability to filter through the noise and turn data into actual intelligence remain limited by biases, beliefs, education, experiences, knowledge, and brain power. In other words, we have a finite ability to process information, build strategies, create content at scale, and achieve performance potential.

[01:01:25] Despite advances in marketing automation and the billions of dollars pouring into marketing tech companies, much of the technology we rely on today to plan, execute, and evaluate campaigns is elementary. But AI possesses the power to change anything. And then I cited Demis Hassabis, defines AI as the science of making machine smart, which in turn augments human knowledge and capabilities.

[01:01:46] It's not the fictional AI you see in movies like X Mahena, I said that's called Artificial general Intelligence, which we'll discuss in a future post, but rather it's real world technology delivering practical solutions for specific business and marketing [01:02:00] functions. While traditional tech is built on algorithms in which humans code sets of instructions that tell machines what to do with ai, the machine creates its own algorithms, determines new path and unlocks human potential to advance marketing business and mankind.

[01:02:14] AI may seem like a futuristic concept, but is is widespread use among companies we interact with daily, like Netflix, Amazon, UPS, Facebook, Google, and Apple. While AI is transforming industries and redefining how we learn, communicate, and live as consumers, we haven't seen the same volume and velocity of AI innovation until now in business.

[01:02:34] and then it just goes on to sort of say, Hey, we're gonna, we don't know where this is going. I said, there's no roadmap for where this is going. So we created. the site to try and figure it all out together. And basically our mission was to educate people on the present and future potential of ai and then connect them with AI powered technologies that can drive performance and transform their career.

[01:02:53] So interestingly, like when you look back on it, Mike, it it's all still true. Like it Yeah. Is 10 years later, but [01:03:00] it, yeah. So that was the origins and then when chat, when the ChatGPT PT moment arrived, so obviously we'd created our conference in 2019 by 2022. The institute was six years old. Like we've been at this for a while.

[01:03:12] I'd been traveling the world giving talks on this stuff and then ChatGPT hit and it just obviously interest and demand in everything we were doing, dramatically expanded, which was when the SmarterX brand really came to the forefront. It was originally the consulting arm of the institute. And then we kind of moved that to be the parent brand because so much of the work we did started expanding outside of the marketing industry.

[01:03:35] so yeah. Yeah, and that's kinda where we are today, is accelerated AI literacy and transformation as a whole. And the institute still plays a role in that with the marketing, audience.

[01:03:45] Mike Kaput: Yeah, and it's kind of cool, Paul, I mean that while we are far expanded beyond just marketing, we've kind of lived through this change where marketing does seem in a lot of organizations to be driving the change forward because AI's impact is so readily apparent in a [01:04:00] lot of what we do that in, in a non-technical sense that I think marketers are very uniquely positioned to be at the forefront of this, but also they're seeing it.

[01:04:09] Yeah. And they're waking up to it earlier than others in some cases.

[01:04:12] Paul Roetzer: Yeah. It's, it's always been our experience. It still is today, that the marketing department is often the tip of the spear to adoption within organizations. They're the ones sort of pushing the frontier of use cases and pushing to get access to, you know, the AI systems pushing to connect them to their data sources so they can build agents to improve workflows.

[01:04:30] So yeah, the marketers are, are critical to it. And then what we always say is the marketing communications team is fundamental to explaining it to everybody else in the organization. So how do you get the ops and the HR and the finance people and the legal people. To come along with transformation when maybe they're resistant to it.

[01:04:47] You need communication. So the comms team is like fundamental, and the only way to explain this stuff in an approachable way is to be living it yourself. Like it's really hard to have a marketer, a communications professional who isn't using [01:05:00] ai, explain it to somebody else who isn't using ai. So yeah, it's, yeah, definitely still kind of a, a, a really big focus area of ours.

[01:05:09] But, yeah, you know, continue to expand and spending a lot more time with the C-suite trying to push from the top down. Now, in the early days, it was a lot of pushing from the bottom up, adoption wise, and I think we've reached the point in, in the business world, we need this. We need the CEOs driving this and taking the lead.

[01:05:28] Meta’s Muse Agent Found to Expose User Data

[01:05:28] Mike Kaput: All right, next up. This past week, meta suspended a sharing feature in Muse, which is its relatively new personal AI agent. After security firm, Claro found that strangers could discover shared conversations containing sensitive, personal and business info. So users had asked Muse to share those conversations.

[01:05:48] That's kind of a feature that was in the tool, and as a result, it creates these public pages that anyone with the link could open. Pretty standard so far. But the issue is the conversation [01:06:00] titles and identifying codes also appeared in searchable domain name records, which allowed strangers like Claro to reconstruct the link.

[01:06:09] So Claro found more than 6,800 share addresses. They opened a few hundred without having to log in. Most of these they said were harmless, but there were sensitive examples they included that included a candidate's preparation for a confidential recruiting call with a Frontier AI lab. It identified the lab's contact and the meeting time.

[01:06:31] There were a couple other really sensitive ones, including a marketing company's revenue review marked for its CEOA loan application showing the amount requested from a small business and a note about a bank fraud alert, listing a partial account number and transaction details. Researchers also found a real estate investors report with property addresses and mortgage figures.

[01:06:53] And a company staff vacation calendar showing when dozens of employees would be away. So Claro reported this problem to [01:07:00] Meta on October 8th. On October 20th after the publication s Technica asked meta about the findings, the company told the outlet it was suspending this sharing feature. Claro then recheck rechecked 40 previously accessible pages that they had found.

[01:07:15] None of them opened. all their, their addresses and titles remained in historical records. So based on all this ClearRock, the security firm recommends four steps for companies based on the incidents. First, identify employees using Muse and whether they are publishing company information through share links, even if the company has not approved the tool.

[01:07:35] Second, update AI usage guidance to explain that. Sharing a conversation publishes a page, anyone with the link can open and that muses conversation titles also appeared in domain name records. So tell employees that were. Actually using this feature that that is possible. Third, configure security tools to flag the creation of public AI share links.

[01:07:55] Treat each share as company information leaving the business. And fourth, if [01:08:00] you as a company cannot detect or control muse sharing or muse at all, Clara recommends perhaps both blocking muse.ai and its content domain meta ai user content.com through corporate network filters. So Paul, I guess like. You know, I know we bag on meta.

[01:08:16] This, doesn't seem like a great look for them. as per usual. This seems like maybe just a bad technical feature, but overall shows these personal AI agents are agents in general or the wild, wild west, but these personal AI agents, like, we don't even begin to know how these can go wrong until stuff like this is discovered.

[01:08:37] It seems like.

[01:08:38] Paul Roetzer: Yeah, but this loophole, I just like, I'm shocked by this, like openAI's had this issue too with some of they really sharing where it was being indexed. yeah, it just one, how could this possibly be a thing still? Yes. This is like, talk about vulnerabilities that have been known for like years.

[01:08:54] Yeah. I don't know how you ship a product where you don't like on the checklist say, are the things people [01:09:00] share also available to the general public? two, and maybe this is just me. I've never trusted public sharing of sensitive information, even if you only access it with a link. Like Yeah. And I've never understood people who function in that way, like businesses who allow employees to share things that don't require a login.

[01:09:21] that it's like, would you actually want this found on the web? I just, I think I kind of assume if I'm sending a private a, a link to someone that can be accessed on the internet that doesn't require login. It's likely it could be discovered. Now that's a hundred

[01:09:35] Mike Kaput: percent

[01:09:36] Paul Roetzer: probably not true, but like it, I mean, this happens all the time.

[01:09:40] I think Google even had this issue with some of their early shares outta Gemini and stuff. So, yeah, like I obviously like bad look on meta for like letting this happen, but just the tech industry as a whole, how does this keep happening? It's like, it's the most obvious thing to not allow to happen. that being said, just total [01:10:00] switching of gears on Meta.

[01:10:00] I listened to a super fascinating podcast, Mike, and I didn't even put this in the show notes. So, there was a, a, a podcast that was on, what is this, the Cheeky Pint, which is, yeah. Was it, is it Patrick Collison? Is that who It's from Stripe.

[01:10:16] Mike Kaput: Yeah. We're the founders of Stripe.

[01:10:16] Paul Roetzer: Yeah.

[01:10:17] Mike Kaput: One of them.

[01:10:18] Paul Roetzer: Yeah. So he interviewed Zuckerberg and Alexandr Wang and it was, um.

[01:10:24] it's like two hours long. Insanely fascinating. Yeah. Like I even like, again, regardless of how you feel about Meta or Zuckerberg, if you, if you try and extract like any of those personal feelings you might have out of it. The amount of like disclosure they gave for how the models are trained. How Zuckerberg recruited Wang to come join Meta, why it took them so long to build their first model.

[01:10:51] why Wang does like the shit posting on, on with Muse. Like, it's, it's one of the most [01:11:00] transparent interviews I've ever heard. Huh. And it also, I think we might have mentioned on the podcast previously, because it's also, if you watch on YouTube, the one where Zuckerberg brought a keg of Guinness from Dublin and it, and Zuckerberg made the face like he's never tried a Guinness before and did not like it.

[01:11:15] And so that was like the thing that went viral is his face when he took his first sip of the Guinness. that being said, though, I would, I would highly recommend people listen to it. You will learn a ton. About how models are trained and post trained. Really, really well done interview.

[01:11:32] Mike Kaput: Cool. Yeah, we'll have to check that out.

[01:11:34] Paul Roetzer: Yeah.

[01:11:35] Anthropic Releases Its Own Personal Agent

[01:11:35] Mike Kaput: All right. Next up. This past week, Anthropic introduced Claude Duet, a personal AI agent of its own designed to keep working for you between conversations. So you give it work to do, it carries out the task across connected apps, and it comes back with results or a decision that needs your attention.

[01:11:51] So this joins the same broad category as things like meta use, Grok Bot, openAI's dots. These are agents with their own cloud computers that can [01:12:00] work across tools and keep going after you close the apps. So Anthropics version of this is a separate app powered by Claude Opus 5.5. It brings over your existing Claude memory skills connected apps.

[01:12:14] So it can start with context. You have already given Claude. So for example, Anthropic. Mentions things like duet can check Stripe or QuickBooks for unpaid invoices and draft payment reminders. That means it can find outstanding bills and prepare the follow up. It can take on larger projects like building dashboards, prepping financial forecasts, prototyping web apps, et cetera.

[01:12:37] it can also initiate like contacting you. So Duet sends a morning brief, it checks connected apps every few hours. It can flag things that need your attention. You can delegate work through text or recorded voice messages and step away while it works. So this is currently in a research preview available by wait list to us Pro and Max subscribers on mobile and the web.

[01:12:58] And it is included in [01:13:00] those subscriptions with background work counting towards existing usage limits. So Paul Anthropic, I think is like the final major player here to have its own personal AI agent that was probably rushed out the door when they realized these were setting the internet on fire.

[01:13:15] Paul Roetzer: Yeah, they got out ahead on Claude code and sort of like made everybody else catch up and Yeah.

[01:13:19] And then agents was like, oh man, now everybody's enabling agents. Now we gotta catch up on this. yeah, again, it's gonna come down to which AI system. Do you trust to build your agents around? There's no debating that agents in, in the organ, in the organization could be insanely valuable. Like you centralize the building of these things through a lab or something like that.

[01:13:41] You go through, you look at all the workflows that happen each day, or the workflows that don't happen because there aren't resources to make them happen. You build agents to automate a lot of that work. I think about it from an analyst perspective as like, what data would I want to be looking at each day?

[01:13:53] And analyzing. Yeah. And then extracting insights. I was like, no, debating this would be super valuable. It's just like, [01:14:00] which one do you trust to do it? And Yeah. And how do you set it up? So. Yeah.

[01:14:04] Mike Kaput: Yeah, I, I've struggled personally a lot with the proactive nature of these. On one hand, I think proactive agents are, like you mentioned, the future for certain domains, like metrics, analytics, like data is a obvious, incredibly valuable use case.

[01:14:18] I sometimes see people though, having the IB proactive about their schedule, about their calendar, about their emails, about their time, all of which no doubt can be very helpful for you if you are very busy. But I don't know, I kinda struggle with that because I'm like, well, now you're just telling me my priorities without me thinking about them.

[01:14:37] Yeah. And oftentimes it's just like, I'm not here to just get more emails done. I'm here to make sure we're doing the right. Thing. And that's sometimes a very delicate balance that requires a lot more context and training, maybe on the AI's part. But who knows, maybe with Super Intelligence or AGI we'll get there.

[01:14:54] But I've always like, I've struggled with that piece of it personally.

[01:14:57] Paul Roetzer: Yeah, agreed.

[01:14:59] Inside Anthropic’s IPO: Explosive Growth, Massive Commitments, and Catastrophic Risks

[01:14:59] Mike Kaput: Alright, so [01:15:00] this past week, the Wall Street Journal also obtained Anthropics Draft IPO Prospectus. So these are the financial disclosures that are prepared for selling shares to the public. This offers a look at how quickly the company is growing, what that growth costs and the risks investors will be taking by investing in Anthropic.

[01:15:21] Now there's some interesting details in here, Paul. So according to the prospectus, Andros revenue reached $16.3 billion in the first half of 2026. That's up a huge amount from 1.1 billion a year earlier in the same period. That's almost 15 fold growth. Interestingly, this revenue relies heavily on a handful of customers.

[01:15:42] Just two customers accounted for about one fifth of its revenue in this period. At the same time, computing and infrastructure spending roughly quadrupled from 2.3 billion to nearly 9 billion. Yet for this period. There's also this big loss that Anthropic reported. This is an [01:16:00] $80.7 billion net loss. Now, most of this apparently was not cash spent running the business.

[01:16:06] About 72 billion was an accounting charge tied to how the company raised money. Um. Basically, some investors provide funding through agreements that can turn into Anthropic shares. The estimated value of those investors financial claims increased and Anthropic recorded that increase as a loss. So basically the value of what it promised those investors went up.

[01:16:27] It does not seem like it paid out $72 billion in cash to operate the company. However, Anthropic is also staring down some very real, very big future. Bills, the journal reports. The company has roughly $450 billion in cloud commitments over seven to 10 years. These are mostly under contracts that Anthropic cannot cancel, so these signed agreements create obligations, stretching gears into the future.

[01:16:53] Anthropic will definitely need. More revenue or financing to meet them. Not to mention these major cloud [01:17:00] contracts require payment regardless of how much capacity Anthropic actually uses. Anthropic says its major cloud partners have not supplied all the capacity at once and increasingly view them as a competitor.

[01:17:12] And finally, the prospectus also spends considerable time warning investors about financial risks connected to AI safety. Andro explicitly warns it could owe damages for harm caused by its models such as stolen data or security incidents. The company even says that a catastrophic AI incident could cause damage so large that they may not be able to obtain enough insurance to cover it.

[01:17:35] So Paul, these are some leak details that if they end up being true once Anthropic IPOs are certainly some food for thought.

[01:17:41] Paul Roetzer: Alright, breaking news. They just filed it. So the S one, oh,

[01:17:44] Mike Kaput: there you go.

[01:17:45] Paul Roetzer: The S one registration is out. Um. Like 20 minutes ago, they just released this or an hour ago. So Anthropic put it up.

[01:17:53] The SEC site has it. so I'm just scanning my feed right now. So they've officially filed [01:18:00] a potential mid-November IPO. Okay. Looks like could raise up to 100 billion at a valuation approaching 2 trillion.

[01:18:08] Mike Kaput: Wow.

[01:18:09] Paul Roetzer: Yeah. So, they've got information in this thing about, what are they calling 'em?

[01:18:16] Self-reinforcing loops. So they're talking about recursive self-improvement to build the models. they identify. Google and openAI's is their primary competitors. Okay. No mention of, meta or Grok, which makes sense. They're not, those two aren't major players in the enterprise space. But yes, this is happening.

[01:18:36] if you're, if you want to invest in Anthropics IPO start setting some money aside, it sounds like in the next maybe three to four weeks, you might get your opportunity to do that. Maybe two to three weeks even. and they did actually address the size of the thing. They said, well, no customer, lemme see.

[01:18:54] I find it accounts for more than 10% of their revenue. I think it was like four [01:19:00] customers total of 25% of their revenue or something like that. Yeah. So they are, they're definitely like tight in terms of the revenue mix. Um. Yeah, I don't see that one right now, but yeah, so, oh, here it is. While no single customer represented more than 10% of our revenue for the nine months ended September 30th, our top five customers represented approximately 25% of our revenue for those same nine month periods.

[01:19:26] Mike Kaput: Interesting.

[01:19:27] Paul Roetzer: Okay. There you go. Yeah, it's happening.

[01:19:29] AI Use Case Spotlight

[01:19:29] Mike Kaput: It's happening. All right, so now let's move on to our AI use case spotlight. Every week we give you a quick look under the hood at some real AI use cases we are exploring at SmarterX. So Paul, I'm gonna share something I've been doing with some skills and then.

[01:19:44] Here what you've been working on. So this week I have been using AI to reorganize and improve my AI skills. So I just wanted to briefly share how I'm doing that in case it's helpful. But first, what is a skill? Why should you consider building them? So a [01:20:00] skill is a set of reusable instructions that tell an AI tool how to perform a particular task.

[01:20:06] This lives in a folder. It starts with a file called skill.md, MD being a plain text document written in markdown. That folder can also include examples, reference materials, templates, and codes. So you can use skills with tools like Claude Code Codex, ChatGPT work, et cetera, that can read those instructions and follow.

[01:20:27] Process. So if you repeatedly use AI to like prepare a podcast, produce a newsletter, or develop a presentation, you can capture how you want that work done in a skill instead of just explaining it in project instructions or explaining it from scratch each time. So why should you? Bother building these versus just a project or built-in memory or context.

[01:20:49] a skill gives you a dedicated reusable process for a specific job that's much more robust than what you might fit into just a project that references a couple files. So again, with the [01:21:00] ability to run code, reference more extensive files, it can be really useful to have this all documented in one place.

[01:21:07] so I've got hundreds of skills at this point, many of which are critical to how I do my job. I'm creating new ones all the time or trying to for new use cases. But as Chris Penn at Trust Insights has pointed out, skills are basically becoming software. They direct how an agent does work. They can include executable code.

[01:21:26] They need to be design tested and maintained. So I'm lucky at this point. I've spent enough time and, you know, effort on the skills that I have that they work. Pretty well most of the time. and I've got a lot of effort and, trial and error in developing them and refining the results. But it doesn't mean just because I have a good skill that the underlying skills organized well or running efficiently.

[01:21:50] So given how critical these are, I wanted to bring some more engineering, quote unquote discipline to the side of it. And thankfully, Chris was also a huge help here because he created something [01:22:00] called the I Love Chris's naming, highly opinionated Agent, skill Fixer, which basically is just like his opinions on how you should fix skills.

[01:22:08] It's a packaged up skill you can buy, from Trust Insights, his company. And I've been using it to repair my existing skills. I'll link to it in the show notes. It's like 20 bucks. I found it very well worth it. Basically you download the skill, you point it at your other skills, it backs up, the original audits it and works through the repairs.

[01:22:28] So basically a big part of what it does is reorganizing what goes where. It'll keep the main instructions focused on directing the work. It'll move detailed guidance about the skill and the workflow into reference files that the agent reads when needed. So that speeds up the skill, uses less tokens.

[01:22:46] It actually, this is key for me because I don't know how to do this on my own. It puts repeatable operations into code because a lot of times there's just things that you're having AI do that realistically can just be done by deterministic code and that saves you a [01:23:00] lot of time and effort, and tokens.

[01:23:02] So it'll do all that for you and help with you, help. Work through that with your skills and then actually validate the changes and run before and after tests as well on the skill itself. so overall, just to kind of wrap up here, whether you use Chris's tool, I'd highly recommend going, checking out no, you know, affiliation or anything with it.

[01:23:22] It's just been super, super helpful. Whether you go use that or work through this with AI yourself, I think it's worth thinking through his advice here on how to make better skills. So first, ask your AI to audit the skill against the agent skill specification. This is an open standard. You can go find a link to it.

[01:23:41] We'll drop the link in the show notes develop by Anthropic. And basic says, here's how skills should be structured. and it seems to be a standard that others are adopting. Chris also recommends keeping your main instruction files under 200 lines, though that's his preference. He states not the full re, not like [01:24:00] a hard and fast rule.

[01:24:01] He also has it like keep their initial skills short and direct it to the different types of steps and point to supporting files instead of kind of. Having everything just in one file or set of instructions. And then basically things like put detailed guidance in references, consistent output formats and templates and things like logos and assets.

[01:24:21] So basically you're just pointing to a bunch of different files this skill might want to use. so yeah, Paul, that's what I've been kind of working on. It's dramatically improved the efficiency and token usage of some skills. It's overall improved the performance of them, though they were working pretty well to begin with.

[01:24:36] This is more optimizing them to, work in a streamlined way.

[01:24:43] Paul Roetzer: All right, so I used agents to find a new garage door, not super sexy. So we're gonna come back to yours. I'd rather talk about yours for a minute and, okay. So as you're explaining this, I can't help but. Feel the scope creep of all of this.

[01:24:57] And so I, you and I have had zero conversations about [01:25:00] this. Oh yeah. But like, this is my reaction to what you just went through one amazing two. How in the hell do, do you manage all of this?

[01:25:08] Mike Kaput: A hundred percent.

[01:25:08] Paul Roetzer: So I started actually thinking like the bitter lesson is going to be true. And this is more of a hypothesis that I have had two minutes to think about, while you were talking and I was listening to you.

[01:25:17] So the bitter lesson is basically like anytime humans think they can, like just find a better solution, like code, a better way to do things, eventually the smarter model just obsolete. Anything the human tries to do. This is starting to feel, to me, like trying to manage dozens or hundreds of workflows where the human told the machine how to do something and then the smarter AI model comes along and be like, you dumb human.

[01:25:44] Why did you write all these skills? Like I'm better at this than you are, than you will ever be at creating these skills. And so I start to see like, and I've actually seen versions of this with. Prompting frameworks. So I've seen a number of AI researchers recently say, you can pretty much forget any [01:26:00] prompting framework you ever learned because the model is basically just going to re-prompt without you seeing it and like structure how it needs to.

[01:26:09] And every time a new frontier model comes out, the way you talk to it changes. Yeah. The prompt framework changes. Memory affects this because memory's gonna remember how to work with you. You're not gonna need to give it all this knowledge base because it's gonna know all of that from your previous communications.

[01:26:25] And so like. Building these massive skill libraries that you have to then maintain and constantly adjust, is starting to feel bitter, less an esque to me, where you're almost saying, here's the 10 steps to follow in a workflow versus ai. Here's the outcome. I want you figure out the steps to follow. And as we define all these skills, like, and I don't know if this is like super practical, but if I just, if I build a project that says, I want you to be my co CEO and I give you like pretty basic instructions, like what I want that to be, you know who I am, you know my company because we've been working together [01:27:00] for two years now, Claude.

[01:27:01] Yeah. Like I don't need to give you a whole bunch of other stuff now. If I start telling you how to write my proposals, how to draft my emails, how to summarize blog posts for me, how to, you know, take podcasts to do this, it starts to fixate on doing it the way I told it to do it in October, 2026, and then four months from now.

[01:27:20] We're on G PT seven, and all the ways I told it to work six months earlier are basically obsolete now because the model just figured out a better way. But I'm forcing it to do it through the skills I built six months ago. Am I off here, Mike? Like, is that,

[01:27:33] Mike Kaput: no, I think

[01:27:34] Paul Roetzer: it's, is that a

[01:27:34] Mike Kaput: possibility

[01:27:35] Paul Roetzer: that we just obsolete everything you just said with a better

[01:27:37] Mike Kaput: model?

[01:27:37] I think it's, I think it's yes and no because I think it's, yes, what you said. For a lot of things I would not be creating skills to like work through strategic problems or solve things you don't know how to solve. Right? Yeah. You're giving an agent a goal, you wanna give the agent a goal and work with it in a natural, organic way to get to that goal.

[01:27:57] skills, I would say yeah, are much [01:28:00] more repeatable workflows. Like, I don't need AGI to reinvent how the newsletter gets done, or something like that. Right, right. So it's like, if I didn't know the best way to do the newsletter within the confines of what we have. Then, yeah, I think I would say here's the goal.

[01:28:16] The goal is we're starting the newsletter from scratch, which is totally valid and important approach to do moving forward for the time being. There's a lot of ways that that skill could actually dramatically shorten the amount of time that the model, the agent figures out how to do the thing, by knowing where certain things are, like templates, logos, assets, formatting, et cetera.

[01:28:39] So, yeah, I think for the time being for certain things, your issue is yes, the agent can go figure it out. Um. It creates a lot of, it takes a lot of time and tokens to do that. Mm-hmm. It may not. It may not pull certain things. It does.

[01:28:54] Paul Roetzer: So memory's really good and then it

[01:28:55] Mike Kaput: doesn't Yeah. Compress

[01:28:57] Paul Roetzer: everything.

[01:28:58] Mike Kaput: I just assume everything I'm [01:29:00] doing eventually, if I complain about it being slower or costly, that will not be the case. Yeah. You have to plan around that. So for me personally, I find immense value in some of the things I do that need to just happen and happen consistently really well. I have tried in the past to your point, to experiment with like, Hey, let's build a skill about like my priorities or achieving my personal goals, and like the moment that changes, this is wildly out of date, or frankly, the models changed and it's like you should just give them the goal instead of the steps to your point, that's absolutely an important consideration here.

[01:29:35] I think this is more for taking some complex but relatively consistent work. Completely off my plate and kind of learning a bit more about skills in the process. But for innovative stuff, you're not gonna, you don't know how to do it. Like AI is going to be a partner or do it on its own and figure out the best way to do it.

[01:29:54] Not you telling its steps to your point is like some dumb human over here, you know? [01:30:00] So it's a very interesting consideration. Absolutely.

[01:30:02] Paul Roetzer: Yeah. I hadn't thought about it until I started, my brain was like overloading, thinking about, it's a good way to think about that. Hundreds of these skills and having to maintain, it's

[01:30:08] Mike Kaput: gonna be like gpt, like this is my replacement for GPT for now.

[01:30:12] Yeah. And I'm good at AI creates these very fast. So keep in mind there's not like a huge amount of like work of me like handwriting these, but this is gonna be like GPTs at some point something's gonna change and it's gonna be like, yeah, skills don't really matter anymore. No, I would imagine.

[01:30:28] Paul Roetzer: Interesting.

[01:30:28] Mike Kaput: Yeah.

[01:30:30] Paul Roetzer: Yeah.

[01:30:30] Mike Kaput: But for the time being, if you are creating skills, this is a helpful way to reduce some tech and budget and things. Definitely do I Do you wanna talk quickly about your garage door? I'm curious though.

[01:30:42] Paul Roetzer: We're doing a renovation and I needed to find a garage door that worked within some pretty specific specifications that couldn't be like a standard garage door with a standard track.

[01:30:50] And I just gave the parameters and I was like, go, go find options for us. So my wife and I just kind of sat there and we were talking through it, looking at designs while it was off doing its thing. [01:31:00] And it That's awesome. Came back like seven minutes later and it's like, here's the things that would work for the, you know, approved blueprints that you've got and da, da, da da.

[01:31:06] And we took those and sent 'em to the builder and like, here you go. Like the, so we found, so yeah, it was just one of those like could be a frustrating time where like, I don't even know where to find these. Like, we've Google searched a bunch of times and so that was a pretty practical personal use case.

[01:31:21] I had

[01:31:21] Mike Kaput: people sleep on the fact that AI has revolutionized home ownership. Yeah. I'll be honest. Yeah. Like there is so many things I would be so screwed otherwise

[01:31:30] Paul Roetzer: not Yeah. Just run in your personal life like a hundred percent. I just, so many times I look at something like, yeah, let's just build an agent.

[01:31:36] We keep doing this thing. Like let's just. Create an agent for it. Yeah. Basic stuff, like not super sophisticated, like literally just go and say, build me the instructions for an agent to do this thing. That's, that's about the extent to which I will prompt it. Yeah. I don't get into like these crazy prompt frameworks or anything like that.

[01:31:51] I just like, you're smart. Go figure this out. Here's what I want it to do. Do it. Okay, great. Let's build it. Great. Automate it. Go.

[01:31:59] Mike Kaput: Yes. I think [01:32:00] that's generally the right approach.

[01:32:01] Paul Roetzer: Yeah.

[01:32:02] AI Product and Funding Updates

[01:32:02] Mike Kaput: All right, let's wrap up here with our AI product and funding news Paul. Just a few items on the docket this week. So first up, openAI's is bringing its AI models and Codex coding tools to Google Cloud, starting with a private preview for select enterprise customers with broader availability planned in coming weeks and there are purchases counting towards existing Google Cloud spending commitments.

[01:32:24] So it seems like you'll be able to use those within your current agreements and commitments. the software company 1Password announced 1Password for agents which lets users approve an AI agents login while keeping passwords hidden from the agent. There are muse integrations rolling out and Claude Duet and Chat GPT work Integrations coming soon.

[01:32:44] A startup called Arga co-founded by former openAI's Chief Research Officer, Bob McGrew emerged from stealth with $230 million in total funding and a $1.5 billion valuation to build AI software that helps manufacturing engineers create and update assembly [01:33:00] lines, and Real is apparently already using this technology.

[01:33:04] Disney required chatbot company character.ai to enter bankruptcy as part of a proposed technology licensing deal. Designed to leave legal liabilities with character.ai according to Wired with the license, still subject to an auction and court approval. The Department of Homeland Security finally published notice of an order requiring federal civilian agencies to remove Deepseek technology, including third party products built using it and barring future purchases of Deepseek

[01:33:35] Mike Kaput: Products and service there.

[01:33:36] Paul Roetzer: Right? I never, ever saw it coming that they would not allow Chinese open weight models. Who would've ever thought that that would happen?

[01:33:42] Mike Kaput: No kidding. And just one final reminder, like we talked about top of the episode, go to SmarterX dot ai slash pulse to fill out this week's survey. And Paul, thanks so much for breaking down very heavy topics this week, but no doubt important to [01:34:00] educate people on.

[01:34:01] Paul Roetzer: Everybody. Stay safe out there. Stay safe out there. Get a good IT partner. Make sure you and your CIO have good open lines of communications and have somebody check your sites for PHP  and like everything else that's gone haywire in the last week. Yeah, hopefully next week we are not talking cybersecurity, but we shall see.

[01:34:20] I feel like it's gonna become like politics. It's unavoidable to not at least address what's happening at times. So No kidding. We'll do our best to make it make sense and give you some actions that the average, practitioner or leader can follow. Alright, Mike, thanks so much. See you soon Heading into the office.

[01:34:38] I'll see you in a little bit. Alright. Alright. Thanks for listening to the Artificial Intelligence Show. Visit SmarterX.AI to continue on your AI learning journey and join more than 100,000 professionals and business leaders who have subscribed to our weekly newsletters. Downloaded AI blueprints, attended virtual and in-person events, taken online AI courses [01:35:00] and earned professional certificates from our AI Academy and engaged in the SmarterX slack community.

[01:35:05] Until next time, stay curious and explore ai.
Claire Prudhomme

Claire Prudhomme is the Marketing Manager of Media and Content at the Marketing AI Institute. With a background in content marketing, video production and a deep interest in AI public policy, Claire brings a broad skill set to her role. Claire combines her skills, passion for storytelling, and dedication to lifelong learning to drive the Marketing AI Institute's mission forward.

Recent Posts

[The AI Show Episode 248]: The AI Cybersecurity Crisis, Dario Fights Back on Safety, OpenAI’s Monitoring Blind Spots & Claude Duet

Claire Prudhomme | October 27, 2026

Paul Roetzer and Mike Kaput break down the AI cybersecurity crisis, Anthropic's safety defense, and its $2T IPO filing.

Podcasts

[The AI Show Episode 247]: OpenAI’s Mounting Safety Concerns, Anthropic’s AI Expectations Gap, MAICON 2026 Recap, AI Score & Gemini 4 Argon

Claire Prudhomme | October 20, 2026

David Robinson's OpenAI safety warning, Anthropic's AI workforce data, and the MAICON 2026 recap on Ep.247 of The AI Show.

Podcasts

[The AI Show Episode 246]: AI Answers - Live from MAICON 2026: AGI, AI Costs, and the End of GPTs

Claire Prudhomme | October 15, 2026

OpenAI discloses six rogue-agent incidents as the fight over pacing AI development becomes a political issue in Ep.241 of The Artificial Intelligence Show.

Podcasts