If you stop hiring entry-level staff and you quietly decimate the leadership pipeline, where do future managers come from once AI does the junior work?
That's one of 15 listener questions Paul Roetzer and Cathy McPhillips take on in this AI Answers edition, drawn from recent Intro to AI and Scaling AI classes.
The conversation runs from the practical (how to carve out time for AI, generic vs. custom agents, human-in-the-loop checkpoints) to the strategic (funding AI, token budgets, intelligence redundancy, and whether we're building businesses on "rented land").
Listen or watch below—and see below for show notes and the transcript.
Over the last few years, our free Intro to AI and Scaling AI classes have welcomed more than 60,000 professionals, sparking hundreds of real-world, tough, and practical questions from marketers, leaders, and learners alike.
AI Answers is a biweekly bonus series that curates and answers real questions from attendees of our live events. Each episode focuses on the key concerns, challenges, and curiosities facing professionals and teams trying to understand and apply AI in their organizations.
In this episode, we address 15 of the top questions from our July Intro to AI and Scaling AI classes covering everything from tooling decisions to team training to long-term strategy. Paul answers each question in real time—unscripted and unfiltered—just like we do live.
Whether you're just getting started or scaling fast, these are answers that can benefit you and your team.
00:00:00 — Intro
00:07:15 — What are the best ways to use AI in marketing?
00:10:54 — How do you carve out time for AI when your team is already slammed?
00:14:46 — Should a small company hire outside consultants to build agents?
00:19:19 — Who should own AI transformation in the enterprise?
00:23:49 — Are there generic agents everyone can use, or do you build your own?
00:25:58 — Can you build human approval checkpoints into an agent?
00:28:01 — How do you enable agents on sensitive data without adding risk?
00:30:31 — How should a company fund its AI investments?
00:34:11 — How do you budget and forecast total AI spend, including tokens?
00:37:03 — What does a right-sized AI vendor approval process look like?
00:40:33 — What happens to your data if an AI vendor is acquired or goes under?
00:42:48 — When should you disclose that AI was used to create the work?
00:46:03 — If AI does the entry-level work, where do future managers come from?
00:47:57 — Is an LLM really a black box, and is that ominous?
00:50:38 — Are we building businesses on rented land with LLMs?
This episode is brought to you by Marketing AI Month. All month, the AI for Marketing Course Series inside AI Academy is free (a $499 value): five expert-led sessions from Mike Kaput, the frameworks and tools the SmarterX team actually uses, and a professional certificate on completion. Enroll by September 30 (you don't have to finish by then - just enroll).
The month closes with a live AMA on October 1, where Cathy puts your questions to Paul and Mike; anyone enrolled can attend.
Enroll at SmarterX.ai/marketing.
Join us in Cleveland, October 13–15. Use code POD100 at registration for a VIP lunch with Paul and Mike on the final day. Register at MAICON.ai.
Disclaimer: This transcription was written by AI, thanks to Descript, and has not been edited for content.
[00:00:00] Paul Roetzer: Where does management come from? What do managers do in the future if we don't train entry-level employees? So if we stop hiring as many entry-level staff, then do we decimate the future managers and leaders of the company. Welcome to AI Answers, a special Q&A series from the Artificial Intelligence Show.
[00:00:19] I'm Paul Roetzer, founder and CEO of SmarterX and Marketing AI Institute. Every time we host our live virtual events and online classes, we get dozens of great questions from business leaders and practitioners. Who are navigating this fast moving world of ai, but we never have enough time to get to all of them.
[00:00:37] So we created the AI Answers Series to address more of these questions and share real time insights into the topics and challenges professionals like you are facing. Whether you're just starting your AI journey or already putting it to work in your organization. These are the practical insights, use cases, and strategies you need to grow smarter.
[00:00:57] Let's explore AI together.[00:01:00]
[00:01:03] Welcome to episode 236 of the Artificial Intelligence Show. I'm your host, Paul Roetzer, along with my co-host Cathy McPhillips. Welcome, Cathy.
[00:01:12] Cathy McPhillips: Thank you.
[00:01:14] Paul Roetzer: Cathy is our Chief Marketing Officer at SmarterX. If you are a regular listener to the podcast, you have heard Cathy many times before. So, Cathy is not taking Mike's job.
[00:01:24] Mike and I are still doing the weekly every Tuesday, and Mike is the host of the AI transformation series that we've started. We've done. Three of those now. And then Cathy and I do these special editions of, what we call the AI answers episodes. So these episodes are based on questions that we receive during our monthly intro to AI and scaling AI classes, as well as some questions that we get from virtual events.
[00:01:49] So these are all community driven questions and we just kind of go through. There's usually, I don't know, Cathy, what do we have, 15, 12 to 15 of these things?
[00:01:57] Cathy McPhillips: Yes, we do.
[00:01:58] Paul Roetzer: Cathy prepares a brief that [00:02:00] I look at as we start recording. it's a wonderful brief, but like I just kind of dive in and we go. So, yeah.
[00:02:07] So today's 15 questions. They are based on the, last two intro to AI and scaling AI classes that we did. The intro to AI was July 29th and, scaling AI was July 9th. So these are both from late July. And yeah, so the interesting thing, and we'll kind of get into this as we go through these questions, is the intro class I have been teaching since fall of 2021.
[00:02:34] So we've had north of 60,000 people register for that series. And back in 2021, pre-ChatGPT, you can imagine we were getting some very foundational questions, just very basic. Trying to understand what the technology was and where it was going and how to figure out use cases. And the intro questions have evolved.
[00:02:57] They have become much more advanced. So even the people that are [00:03:00] attending the intro class generally have really, really good questions. And then the scaling AI we have now done for probably going on two years. Cathy, I would say we've been teaching that one. Yep.
[00:03:10] Cathy McPhillips: Just about,
[00:03:11] Paul Roetzer: yeah, we've done 18 of those and so that's probably had 10 to 12,000 people I would guess go through that series.
[00:03:17] And that one teaches our five step framework for scaling AI in an organization. So those questions tend to be more. Organizationally driven team, department driven. They're thinking bigger picture about how to move beyond just pilot projects and actually scale it in the organization. So if you've never had a chance to attend either of those, we do continue to teach them.
[00:03:38] You can, good look in the show notes and there's links to sign up for the next ones. I think we have one coming up at the end of September maybe. Does that sound right, Cathy?
[00:03:45] Cathy McPhillips: We have September 10th is our next scaling ai. Okay. And September 30th is our next intro to ai.
[00:03:52] Paul Roetzer: Okay, so apparently I'm teaching a class next week.
[00:03:54] all right, so, so September, you can join us for both of these if you'd like. [00:04:00] All right. So
[00:04:00] Cathy McPhillips: actually no, it's, it's, it's on Friday, actually, it's on the 11th because you're not gonna be in town on the time. I'm, so we had to shift that date. Yes.
[00:04:06] Paul Roetzer: Okay.
[00:04:06] Cathy McPhillips: We did
[00:04:07] Paul Roetzer: all, so, okay. So today's episode is brought to us by marketing AI Month.
[00:04:12] This is a brand new thing, and I'm gonna let Cathy, kind of tee this up for us and then we will. Get into the q and A.
[00:04:20] Cathy McPhillips: Do you also, do you think you ever need to say that I'm gonna, you people are worried I'm gonna take Mike's job.
[00:04:24] Paul Roetzer: I do. Because I think that these episodes with you, Cathy, are so popular and people might dip in and just want to hear you and I talk and they don't realize there's this broader show.
[00:04:34] So I don't know. I think like I never know when someone enters into the artificial intelligence show world, and it might be their first experience with it is
[00:04:43] Cathy McPhillips: fair
[00:04:44] Paul Roetzer: and AI answers on like a YouTube clip. I don't know. So yeah. I always try and set it up that way.
[00:04:49] Cathy McPhillips: I'm not taking Mike's job. I don't want Mike's job.
[00:04:52] We are all, we are all good.
[00:04:53] Paul Roetzer: I don't want Mike's job either.
[00:04:56] Cathy McPhillips: So thank you for being here. this episode is brought to [00:05:00] you, brought to us by Marketing AI Month. We launched that Tuesday of this week. So this month we are doing something really fun. We've got this amazing AI for Marketing Core series as part of AI Academy all month long.
[00:05:14] You can get it free. It's valued at $499. So you'll get the series. It's five expert LED sessions by Mike, practical Tools, frameworks that we actually use as well. And a professional certificate when you are done. So that's free through September 30th. You do not need to complete it by September 30th. You just need to enroll.
[00:05:31] By that date and to close out the month, which we're actually doing on October 1st, we are holding an a MA where, similar to this format. I'll be peppering Mike and Paul with all of your questions. Anyone that has registered for the, or enrolled in the course is eligible to attend. So if you only want the a MA re enroll anyways.
[00:05:50] But the course itself is amazing and I highly recommend it. I took it myself over the holidays and, kind of kicked off my year really strongly. So I was really excited about that. So if you [00:06:00] visit smarterx.ai slash marketing, you can enroll right on that page. Anything to add on that, Paul?
[00:06:07] Paul Roetzer: No, I take advantage of it.
[00:06:08] It's our most popular course series. AI Academy has, I think, 22 professional certificate course series right now. So the idea here was like, let's just accelerate marketing understanding and adoption because our experience with a lot of our AI Academy. business account customers is that marketing is often the tip of the spear.
[00:06:28] It's the department that's leading on AI adoption, but then also an essential part of communication and strategy to bring other departments in and drive AI adoption and, transformation through other departments, being communicated through the marketing team. So marketing to us is a compounding value for the.
[00:06:50] Industry, the more marketers we can get to understand this stuff and be adept at using it in a responsible way, the faster we can, you know, spread that throughout organization. So if [00:07:00] you're a listener and you're not a marketer, make sure to send that link to, to your marketing peers within your organization because it again, is open to everyone and, it's a great value for everybody
[00:07:11] Cathy McPhillips: for sure.
[00:07:12] Which takes us to our first questions. Let's jump right in.
[00:07:15] Cathy McPhillips: number one, what are the best ways to use AI in marketing?
[00:07:19] Paul Roetzer: I might turn this one back to you and say, how are you using it, Cathy? as our chief marketing officer. I'll, I'll keep it simple. And then if you have something to add, Cathy, I'd love to hear your perspective on this one.
[00:07:29] I just start with, what, what do you do? Like, I would look at the tasks and the workflows that you are expected to perform. Every day I would look at the goals that you're expected to achieve, and then I would go in and just ask your favorite AI assistant. You could use our JobsGPT tool if you want, but, or you could just give this prompt to any of your favorite assistants and say, I am a director of marketing.
[00:07:49] My job is this: here's the KPIs I'm responsible for, help me figure out like the top three to five ways that I could be using AI today. These are the tools I have access to. Here's the [00:08:00] data I can connect to. Just give it the information and let it help you figure that out. But the answer is, it's personal to you and your role in the company is how I kind of simplify that.
[00:08:11] Cathy McPhillips: Agreed. I think I would add, yeah, I'm using it a lot for strategy. We are doing, we are rolling things out on a regular basis, so me getting a headstart on getting all of the strategy out of my head onto paper. It's repurposing strategies that have worked in the past. It's building this knowledge base of all the things that we're doing.
[00:08:30] it and frequently helps me write 'cause I do enjoy writing, but it does give me a gut check on some things. I'm using some of our paid advertising to help me figure out new ways to message, looking at data, looking at trends that I might not be able to surface with my non-ST eyes. so one thing we've done on the marketing team is we have these.
[00:08:52] Documented processes for virtually everything that we're doing. So we're trying to, on a quarterly basis or whatever cadence we decide, running it [00:09:00] through saying, can we do something more? Can we do something more? So having the processes, having the use cases documented, and just because things are changing so rapidly, updating that regularly has been really helpful for our team.
[00:09:10] Paul Roetzer: Yeah. One example, and we don't have to get into great detail on this, but the marketing a AI month that we just talked about, we decided like last Wednesday that we were doing this, like it was a very spur of the moment, Hey, let's just give AI for marketing away. Like let's just try and accelerate awareness and understanding in the marketing industry leading up to our MAICON event, to our big conference in October, which is for marketers and business leaders.
[00:09:34] And so Cathy, I don't even know how she did it all, but. We launched that campaign three business days later, like it was, it was an entire thing, and I'm sure there was ways Cathy leveraged AI to help in the planning and the production of assets, things like that. But to launch a major campaign. In a matter of three business days is is wild.
[00:09:57] Like if you think, what could we have done [00:10:00] historically? Right now, I'm sure Cathy worked really long hours and it wasn't like she just hit a button and boom, here's your campaign. But even that, there's just no way you do that. We have a small team, like we, we don't have 20 marketers where Cathy's just like, all right, everybody go do your things.
[00:10:15] It's like, no, we get our hands dirty. We're actually in this, doing this stuff. So that's an example of like a very real. relevant thing that I'm sure we couldn't have pulled off without the support of ai.
[00:10:26] Cathy McPhillips: Right, and people, I mean, yeah, Jeremy Zimmer, our director of marketing for Academy, he was obviously integral in all of this may say, in social media, everyone.
[00:10:35] And then I built this brief and like, here's all of the things we could be doing with my knowledge, with AI's knowledge. And I told the team, I'm like, I know I'm missing something, so please, as humans go in and help me figure this out. What am I, what are all the pieces, parts that I'm missing? But yes, AI definitely gave me a very good roadmap to get it all done.
[00:10:54] Okay.
[00:10:54] Cathy McPhillips: Number two, I have a team that is mostly made up of AI beginners. They are curious and optimistic, but also time starved. The most common pushback I hear is I believe that if I invest time in ai, it will eventually generate efficiencies in my day-to-day work. But how do I carve out the time for now sales targets and client deadlines won't pause while I do this.
[00:11:13] Are there any good tips to respond?
[00:11:16] Paul Roetzer: I would do a forcing function of a workshop. it could be a one hour workshop, it could be two hours, it could be half day, however you wanna do it. But I would do an AI workshop where you teach them a framework. So we have a use case model we use. It's free. You can go, I mean, it's in our book.
[00:11:31] We teach it online. Like there's plenty of ways to go learn it, and you can apply that framework to your own company. But basically, let's say it's a, let's just pick 90 minutes. We are all gonna leave here in 90 minutes. With one to two AI use cases that we can implement starting tomorrow morning, that's gonna save us at least three hours a week.
[00:11:50] Like, just, just set some parameters. Because what it does is it forces you to look through your tasks and workflows that you do over, you know, any given week and say, [00:12:00] which are the ones that take more than three hours? Okay. Now you've narrowed that down to like, let's say there's five things on that list.
[00:12:05] Which ones have a repetitive process that we could automate elements of which ones require creation of strategy, documents or analysis of data. Like you go look for the use cases that are a good fit and you just pick one, and you build a project. You build an agent, like you just build something. So make the first like.
[00:12:23] You know, 20 minutes, setting the stage, make the next 25, 30 minutes brainstorming where people are prioritizing lists, make the last 30 minutes or so sharing what you've done or actually building the thing. so maybe you extend it to two hours and now you can actually have a build session with it. So I would, you have to force it.
[00:12:42] Like you have to have something that says, we aren't leaving here today until everyone has built a thing that's going to save them time. And then if that works, do it every other week or do it once a month and like. Then you just stack those things over time and all of a sudden everybody's got five agents or apps that are, you know, saving them 10, 15 [00:13:00] hours a week.
[00:13:01] Cathy McPhillips: Yeah. And I think showing them how, you know, I think,
[00:13:03] Paul Roetzer: yeah,
[00:13:03] Cathy McPhillips: like the jam sessions we're doing, where people on our team are opening up their computer, sharing it on the tv, we're all in the same room. We're online. We're watching them click the buttons and doing the things because that is daunting. Yeah.
[00:13:15] People like, okay, I think I can do that. I probably have the rain power and the capacity to do it. I just don't even know how and I don't have time to really figure it out. So having someone show you how to do it is, I think so valuable.
[00:13:25] Paul Roetzer: Yeah, and I don't know if I, again, I haven't really looked at these questions, so I don't know if I'm gonna get into this, and I'm not even actually a hundred percent sure I want to share this yet, but like I started a project this morning where I'm basically.
[00:13:37] Going through and trying to look big picture at where agents can be infused into my role. And as I was doing it, I was like documenting, journaling it like I normally would like, okay, I'm going in here, I'm creating this, I'm gonna test this agent. And, and so I would go through and like this whole dialogue with myself, but then I realized, wait a second, maybe I'll just turn on screen recording and I'll just like.
[00:13:58] Record this in Zoom or Google or [00:14:00] wherever, and then eventually take that and turn it into like demos for the team. But then I'm like documenting all the steps I'm going through, all the visualizations I'm seeing as well. So all of that is helpful. But I think you're right, Cathy, like just, you need to infuse whether they're workshops, build sessions, jam sessions, whatever you want to call 'em, those have to just become part of the routine and then it gives people the time and the permission to, to do these kinds of things.
[00:14:27] Cathy McPhillips: Right. The build sessions at MAICON this year. Not to plug MAICON. Yeah. But I will. I'm really excited about that because you bring your computer, you sit down and 45 minutes you leave with something tangible and it's just so helpful. Yeah. Even if it needs to be, you know, sussed out after you get home, but just being there and watching someone else do it is pretty cool.
[00:14:45] Okay.
[00:14:46] Cathy McPhillips: Number three, we are a small company. How should we think about using external consultants to help? The team is so busy doing day-to-day work that we are not making time to create agents and take us to the next level. We have lots of ideas, but need the time, skills. what thoughts do you have on that?
[00:15:02] Paul Roetzer: Yeah, I mean, there's a decent chance you can find consultants who come in and help you analyze your workflows. look for ways to drive innovation, look for ways to drive, you know, automate things, build some agents. I, I. I think that that's a short term fix. Honestly. I just feel like this is gonna be so critical to every organization's operational structure and the way they manage the company that you have to own it.
[00:15:32] Like, so I explained this a little bit on the podcast, but we're building SmarterX Labs within our organization. We're a small company. We have 19 employees. I think of labs as like the r and d unit, and their job is going to be to go to each team or department or business unit and work with them to say, where are the barriers right now to achieving your goals?
[00:15:57] What are the workflows that, you know, take the [00:16:00] longest? Or what are the major things that we should be doing as a company that we just don't have someone in that seat right now? Like, we can't do it. and so they're gonna basically function as. F forward deployed engineers. So that's the concept.
[00:16:13] We're talking about the podcast where you basically have people who go in, look for things to build, problems to solve, and then they build them for them. But what I believe the reason we have to internalize this is because we have to maintain that, that ip, like we have to maintain that knowhow within the organization because.
[00:16:31] If we go in and we solve something with Cathy and her team on the marketing side, it may translate over to the sales side or the ops side. Like an agent we build here might just need a couple of tweaks and all of a sudden it's a sales agent too. And so because AI being infused into all these areas is so critical to what we're trying to build, I just see it as like absolutely fundamental skill we have to build internally.
[00:16:55] So if you're gonna rely on consultants in the early going, I think that's fine, but I [00:17:00] would, I would be. Training or developing people who can function in that role moving forward. And it could be a, you know, a marketer who's just, you know, also pretty technical, like very comfortable in the data and the systems that's trained to become like an AI ops person.
[00:17:16] And it doesn't, not, you don't have to go hire software development, people. You don't have to go hire AI engineers. They can be business people, you know, knowledge workers with domain expertise, who are the, just the most advanced users of the AI platforms you use, so,
[00:17:29] Cathy McPhillips: right.
[00:17:29] Paul Roetzer: That's how I would think about it.
[00:17:30] Cathy McPhillips: You know, you think about subject matter expertise, you know, being, like you said, domain expertise, institutional knowledge, like a consultant can help you with most of that stuff. So, or so do you start building notebooks and knowledge bases. Now, if you're going to start using someone on the outside.
[00:17:48] Paul Roetzer: I mean, if you're, if you're gonna bring in a consultant, they're just gonna come in initially and like document processes, audit workflows, they're gonna look at what the eyes going.
[00:17:55] Again, all, all could really be valuable, right? Like if you truly don't have the time to do this, and like the [00:18:00] alternative is no one does it then. Yeah, like hire somebody, right? To at least get this moving, but. I don't know. I just, it's one of those things I think is so critical to every company that I would be very aggressively looking at.
[00:18:13] What is that like defining what that role is and seeing do we have someone on our team who could, this is a career opportunity for them, like they would work really well. We've seen this and companies, Cathy, you and I both know people who are in AI ops roles today, who. 12 months ago, 18 months ago, were were marketers or they were project managers or like whatever.
[00:18:32] So this to me is that whole creating, seeing an opportunity for a career path for yourself, defining it and then going and building it.
[00:18:39] Cathy McPhillips: And when we start this internally, I would like to be tribute and I would like you to tear apart my department and tell me all the things that I will swallow, my pride, and I can't wait to see what comes out of it.
[00:18:50] Paul Roetzer: I think it is just like a totally collaborative thing. I think it's more of like you have a wishlist, like you have roles you'd love to fill and it. I think this division is like, okay, Cathy, what are the five roles you would put on [00:19:00] your team today if you had the approval to hire 'em? And they might look at it and say, I think we can get you like 50% of the way there on three of these in like two weeks.
[00:19:08] It's not like, it's not gonna be your FTE, but like we could probably build something that'll at least give you that functionality. That's how I'm thinking about it. And then you kinda like go down from there.
[00:19:17] Cathy McPhillips: I'm ready. Okay.
[00:19:19] Cathy McPhillips: Number four, who should be the point person for AI transformation in the enterprise And some organizations it's the CFO.
[00:19:26] Project manager who understands the systems and knows where the data sets are hidden. Has this been your experience?
[00:19:32] Paul Roetzer: I don't know that there's a right answer for this one yet. I do think it depends so much on the skillset of this, this person. So I've de, I've seen one where it was like a chief legal officer who became
[00:19:46] Like the head of transformation because that person was just also extremely qualified on the AI side and understood the business deeply. I've seen it being driven. We had our first AI transformation spotlight that [00:20:00] Mike did with Good Karma Sports. That is a CEO driven transformation. I mean, Ty, who was, you know, did the interview was telling he's a, he's a key, key driver of the whole thing, but that is being very much led from the top by Craig at that organization.
[00:20:14] And so that's a, this is a CEO who got into it, said This is the future of the company I'm all in. And he is like right there every week in every meeting about AI strategy. So in that case, I don't know that they have a Chief AI officer, but it's probably Craig, like he's the driver. I've definitely seen where the CIO does it if that C-E-C-I-O has deep business knowledge, like if they're not just thinking about it from a technical perspective.
[00:20:39] we've definitely seen instances where like, Dan Slagen, who's speaking at MAICON this year, marketer, CMO and Dan just sort of emerged into being an AI transformation leader at a major company. Because he just became the most adept at the technology. So I don't think that there is a right answer. I really do think it is.
[00:20:58] Where is your organization at? [00:21:00] Where is the CEO in the organization at? Because I'm a believer that if the CEO is extremely AI forward. That is probably the person who should be at the top of the chain when it comes to spearheading the transformation. obviously doesn't have the ability to be in every detail, but the CEO should be the driver.
[00:21:20] that's probably gonna be the most successful organizations is when it is a top three priority for the CEO and they're actively involved in everything.
[00:21:28] Cathy McPhillips: Yeah, it just opens up that permission and enablement when CEO's in that role.
[00:21:33] Paul Roetzer: Yeah, like I actually, I, this is totally behind the curtain stuff. I actually messaged our team today 'cause I was working on something, pretty significant for our organization.
[00:21:44] And so I just messaged, I was like, Hey, can you guys send me the excerpt from our AI policy? Regarding connectors because, Tracy and Elizabeth are gonna laugh when they hear this. Now, because I am, I may be instituting an r and d [00:22:00] exemption, that's a term I just made up. I literally put in parentheses to them, and like approving things myself.
[00:22:07] so I'm not bypassing our approval channels and our AI policy, but I am,
[00:22:12] Cathy McPhillips: I mean, it kinda like you are,
[00:22:13] Paul Roetzer: I'm accelerating them. And so within like. 10 minutes I had, I was copied on emails to our IT team and our lawyers saying that, you know, basically our CEO is planning to accelerate approvals, we'll call it on our AI policy.
[00:22:29] Cathy McPhillips: So now let's think about companies. Think about companies we know where the CEO has gone. Rogue.
[00:22:34] Paul Roetzer: Paul, let's not. Yes. Oh no. Tracy did her job. She was like, here's the 15 things that I'm basically like want to check before I like. Go back to Paul, but like, and they're all really smart things. It's like, yeah, I know.
[00:22:47] Like I, I'm not gonna just like hit the buttons, but, yes, when the CEO is, has an urgency, like, Hey, I'm doing a presentation on this in seven days and I need these connections in place right now. [00:23:00] Then things happen and you do it within the safest way possible, but you have to also like an exam. Again, I don't wanna like spend our whole time on this, but like Tracy's thing is, okay, cool, you turn that on.
[00:23:12] Does everyone have access? Right. Do we have to like communicate with everybody not to use the access you just granted. So while I want to move fast, I, I, that is why I reached out to her first and said, I'm going to do this Within our guardrails, we have to set that example. But we also have to find a balance here because I have no choice.
[00:23:30] Like I have to do this in the next seven days. So how do we do this in, in the most, responsible way possible related to our guidelines and policies?
[00:23:41] Cathy McPhillips: Yeah.
[00:23:42] Paul Roetzer: So you may have, you may have some access in your ChatGPT PT account later today that
[00:23:45] Cathy McPhillips: I'm
[00:23:45] Paul Roetzer: gonna go. Doesn't look familiar.
[00:23:49] Cathy McPhillips: Okay.
[00:23:49] Cathy McPhillips: Number five, are there generic agents everyone can use?
[00:23:53] Or for agents to be beneficial for us, we need to build them on our own.
[00:23:57] Paul Roetzer: It depends on what platform you're in. So this is actually [00:24:00] the initiative I'm, I, I'm referring to, I'm, I'm working deeply on integration of agents into SmarterX. And I had this, I haven't talked about this internally. So again, I always, I, it's not even a joke like people think I'm like making this up.
[00:24:11] Maybe when the podcast where I'm like, yeah, no one on our team knows. Literally no one on our team knows about some of the things I'm working on, and they hear about it on the podcast. So a couple weeks ago, I started, a personal initiative. To deeply infuse agents by the end of this calendar year, and in essence, figure out what that looks like moving forward.
[00:24:32] All the connections. How we're gonna structure 'em through the SmarterX Labs division, things like that. And so, it, I have a forcing function that I have a presentation next week related to a, in Agentic Enterprises. And then I'm teaching an AI executive workshop at MAICON on October 13th, where I need to actually present a model for this.
[00:24:53] So I'm deep in the agent world at the moment, and what I'll say is like, an example here would be if you're in chat [00:25:00] pt, they have. Standard agent templates that you could go in and just start with those, like one that pops my, is like a marketing strategy agent. They've prebuilt the skills for it. They've built the system instructions for it.
[00:25:14] All you do is go in and give it a knowledge base or connect it to. Like HubSpot or whatever it is you want to connect it to if you're allowed to. so I guess the short answer is yes. Like there are generic agents if you're in copilot or you're, if, if you're in Google, ChatGPT, Claude, they've prebuilt some agents that are more universal.
[00:25:35] But most likely you're gonna be building things that are specific to your role or to your organization because for them to be really valuable. They're going to need, a knowledge base that is personalized to you or your company, and they're going to need connections to data sources that your administrators are going to have to approve.
[00:25:56] Cathy McPhillips: Got it. Okay.
[00:25:58] Cathy McPhillips: Number six. I love this question. Can you set up checks within an agent that require it to pause for human approval before taking specific actions?
[00:26:07] Paul Roetzer: I wouldn't build an agent that you couldn't do that with. So, yeah, it just depends on how technical you are. like where you're building these agents, but again, if you're working, I, I'm gonna assume most people that listen to our podcast are probably not like Advanced Codex users, cursor users.
[00:26:30] Like they're probably not. The Now I may be wrong. I, again, we don't really, it's hard to know your podcast audience. It's just all these people, it's quite opaque. so I'm assuming that most of our listeners are more, marketers, sales professionals, business leaders, that are not deeply technical coders, software developers.
[00:26:51] So we're we, I'm just gonna use the collective. We here are likely. We're Salesforce users and we're gonna be trying to figure out how to use the Salesforce agents that are [00:27:00] available to us. We're gonna be building copilots in Microsoft. We're gonna be building, ChatGPT agents. There's instructions within there that where you're going to say, this is the checkpoint where you're gonna do, or you're gonna give it read, read only access, where it can't actually go and send something or do something.
[00:27:17] So. The vast majority of us are likely only working within building agents where humans are in the loop or in the lead. And yes, you should know when an agent would be taking an action and you should be the one that giving it permission to do that. In low risks of scenarios, you may give it permission to do the actions without approval.
[00:27:38] Like example there, send Cathy a, a summary, report every Sunday night of marketing performance data the week prior. And it goes to HubSpot. It gets the data, it pulls it summarize it sends it to her. So it's gonna take the action because it's just between me and Cathy. There's no risk of it sending to our customer base, that, that kind of thing.
[00:27:59] Cathy McPhillips: [00:28:00] Okay, cool.
[00:28:01] Cathy McPhillips: Number seven. For organizations with many data sets containing personal and sensitive information, a complex enterprise architecture, legacy systems, shared, shared services across very different departments and pockets of AI forward teams, how can agents be enabled for those teams without creating risk for the rest of the organization?
[00:28:20] Paul Roetzer: With great care that, that is, like, when you're talking about these agents that are accessing sensitive information, let's say like a healthcare system, you know, a law firm, a bank wealth management company, like whatever, like there's just, we all have sensitive information. We could even say SmarterX.
[00:28:36] We have customer data, that we always obviously want to be extremely sensitive to. so anonymizing data before you put it into these systems, things like that, you, you have to have it and legal involved, like it's just. You. You can design what you want to have the outcome be. You can build prototypes of agents, you can do all of these things, but when it's gonna do things that present risk to an organization, especially if you're in a heavily [00:29:00] regulated industry.
[00:29:01] You, you have to know in advance the approval systems. You have to go through. You have to deeply understand your AI policies. And that's the example I was giving earlier with Tracy. Like, Tracy just taught an entire course and we'll get to some AI policy stuff. I see it highlighted below. she just taught a whole, AI Academy live course on our AI policy, which is a very, comprehensive document and.
[00:29:23] If you're gonna be building agents, you have to be familiar with that. You have to know what you're allowed to do and not allowed to do, what data you're allowed to put into it, what connections you're allowed to make. and so what we're trying to do as an organization is these sort of like. I, I, I'm just gonna make up a term here, like horizontal approvals, like, okay.
[00:29:40] If we want to connect HubSpot to ChatGPT, let's approve it once so that if Cathy or Tamara or Dia or anybody wants to do something between ChatGPT and HubSpot, they don't have to come ask for the connection permission every time it's already been connected. And then it's approved for read only access [00:30:00] for like.
[00:30:00] All these different uses. So you do something once and then it permeates across the whole organization with permissions. And so I think you'd be in a situation like that, like if you're in this kind of enterprise where you're gonna be accessing personal sense of information, doing these complex architectures.
[00:30:16] Try and try and solve for as many of those with a one-time permission and training process as you can so that like everybody's not coming and trying to get approvals are basically the same thing,
[00:30:29] Cathy McPhillips: right.
[00:30:31] Cathy McPhillips: Number eight, how should a company fund its AI investments over the next few years? And where should I expect the new spending to come from?
[00:30:37] Paul Roetzer: Oh boy. it's gotta be dynamic. I don't know, like, this is a really hard question to answer. it like, I'm gonna just like, I'm thinking out loud here. People is my first thing. So like when I'm thinking about SmarterX, if I, I'll just take it to make it personal. So for our company over the next three years, where am I gonna put AI investment dollars?[00:31:00]
[00:31:00] My first investment is the people to help me figure out the answer to that question. So the people who think 24/7 about our AI investments. That could be human capital, it could be token capital. It's like, where's our spend gonna go? What platforms are we gonna invest in? But first you have to have the right people to make those decisions.
[00:31:19] second is gonna be. My platforms. So what are our token budgets? Where, how are we going to optimize those token budgets? so
[00:31:30] Cathy McPhillips: how do you even budget for token budgets?
[00:31:32] Paul Roetzer: Yeah. Like do, how do you like teach people what that even means and how do you, right. You manage 'em. and then third would be probably more of an infrastructure thing.
[00:31:41] Like do I need to be buying Mac Minis? Do we need to be buying GPUs? Like are there, are there business cases from a hardware perspective? Where we actually need to be, like thinking differently about the company, to where we have access and do we need to be building. And I guess the fourth would [00:32:00] be, is there gonna be an increasing argument that we should be training our own model at some point?
[00:32:04] You know, I think that a lot of companies are gonna head in that direction where they're gonna get these open weight, more efficient models and they're just gonna. You know, do reinforcement learning, you know, training on them to be adept at their specific industry and business case. So, yeah, I guess that's four.
[00:32:21] And again, I've never actually thought deeply about this question at all. so that's, I reserve the right to change my answer at some point,
[00:32:32] Cathy McPhillips: but it is hard for, I met with the group this morning and they're on a three year cycle where they're trying to plan out the next three years. Like, how do you even do that?
[00:32:39] And as leadership, are you able to give your team permission to just as intelligently as you can make those decisions knowing that they're going to change?
[00:32:49] Paul Roetzer: Yeah. I think three years is hard, but every company's doing, every enterprise obviously thinks in three to five year roadmaps, like they're all looking in that direction.
[00:32:58] I think you need to [00:33:00] make decisions that are as, flexible as possible. As like evergreen as possible. So you're not like locking. I don't know, like one of the ways I think about this one is, you know, if we go really, really deep with ChatGPT as an example, and everything starts running through that and all the connections are for ChatGPT and then something happens to openAI's or like their models or whatever, and it's down for two hours, two days, two weeks.
[00:33:29] What happens to the company? There's no redundancies. There's no backup plan. And so that, that's a, you know, betting on a single provider of intelligence isn't a super scalable thing probably. And so when you're looking at three years, a strategy would be like, we need intelligence redundancy. And, and that, that scales, that like becomes a strategy that lives over three years and actually starts to affect the decisions you make because we've set a [00:34:00] priority around redundancies.
[00:34:02] so that, that
[00:34:03] Cathy McPhillips: would be, hold that thought because I do have another question later. I don't want you to answer right now.
[00:34:07] Paul Roetzer: Okay. All right. Yeah. So
[00:34:08] Cathy McPhillips: let's end it. Right. Let's end that right there. All
[00:34:10] Paul Roetzer: right. That's the end.
[00:34:11] Cathy McPhillips: Okay.
[00:34:11] Cathy McPhillips: Number nine. Kind of going into this a little bit beyond subscription costs, how are companies actually budgeting and forecasting total AI spend, including training and education time set up and integration, internal token usage and token usage driven by external user or customers?
[00:34:27] Is anyone doing this well enough to have a real forecasting model, or is everyone still finding out when the invoice arrives? Are there any resources you would suggest?
[00:34:36] Paul Roetzer: The best I've heard. I don't even know if best is the right term here. A model I have heard is literally just sending or setting a cap each month.
[00:34:44] Like here's our, here's our budget for the month. There are definitely more and more organizations trying to figure out the token optimization and management. There was just something from Google. I think it was on Monday I saw this, where they're starting to [00:35:00] offer that kind of capability within their systems to help you manage it.
[00:35:03] There's startups emerging that do token optimization and basically route you to the, like the most efficient models. I saw, I. Something, I think it was this week where openAI's is now actually experimenting with outcome-based pricing. So part of this is we're developing systems for the way that AI is priced right now, but most of these labs and providers of the intelligence know that this is not a viable, scalable system.
[00:35:32] Like per token, utility based pricing works sort of for developers, but once you start getting it into. Departments like marketing, sales, operations, hr. That's a really, really bizarre way to try and price stuff. Like if it's not seat based, with unlimited usage or it's not outcome-based, if like the outcome is very predictable and able to be attributed specifically to the [00:36:00] model's work, then it gets, it gets kind of tricky.
[00:36:03] So, yeah, I don't, I don't know, like an awareness of how the tokens are being used, what the different models charge per token. It's very early. I don't have a great answer for like, go look at these three companies because they're doing it perfect. I don't know anybody who's actually solved this yet, including, some of the big tech companies that are responsible for building the models.
[00:36:27] I don't know that, other than providing unlimited uses of tokens to their employees, but even meta just changed, like they've made changes to the models that they're using. For this reason because they, they couldn't manage the token budgets and they're a builder of their own models, like
[00:36:43] Cathy McPhillips: Right.
[00:36:43] Paul Roetzer: So
[00:36:45] Cathy McPhillips: and if you do know somebody, listener,
[00:36:47] Paul Roetzer: yeah.
[00:36:47] Cathy McPhillips: Reach out. We'll have 'em on the transformations.
[00:36:49] Paul Roetzer: Yeah, totally.
[00:36:52] Cathy McPhillips: The next three questions are from the Academy Live. We had on Tuesday this week with Tracy, our COO, and then Samantha, our legal counsel. So [00:37:00] these were, I thought these just fit in well with some of these discussions.
[00:37:03] Cathy McPhillips: So the first one is, what does a right sized AI vendor approval process look like?
[00:37:09] How long should a review take? What happens to employees while they're waiting and how do you catch it? When a vendor update introduces new integrations or changes how an approved tool handles data, and I know that you are not doing, you don't treat that OPS is doing this, but As the leader, what thoughts do you have on this?
[00:37:27] Paul Roetzer: I would start early. I mean, we've, we've been in situations with some customers of ours where they got excited about instituting, you know, different AI technologies into the organization. They'd made the business case for it. Then they go to procurement and they would be told there was like a multi-month wait time before procurement would even look at anything.
[00:37:47] So it starts with really just understanding your organization's procurement process and what kind of risk and security considerations need to be made as we start, you know, getting into the AI technology that might require access [00:38:00] to certain data sets. And, yeah, I think it really is about an understanding of the process and then from there.
[00:38:07] It's going through, and building that in. So if, if you know that it's gonna be three months before you can get the system you want, or the approval for the use case you want, or the connection you're looking for, you need a plan B about what that, workflow looks like without those tools. So, yeah, I think, again, this one is very, subjective to the organization.
[00:38:30] And what their approval processes look like and how they're defined within the AI policy. So I know in our case, we have to submit now through form requests to the operations team. When you want a new vendor, if you want to make a connection to a data source. And then I think the team says like within 48 hours or something, they'll at least give a preliminary, Hey.
[00:38:52] We're gonna have to look deeper into this. This is gonna reque kind of the example I gave with Tracy earlier. Like, I put in a request IG we'll call it. [00:39:00] and then there's a, you know, within a few minutes there's an email to the legal team. There's an e email to the IT team because for the ops team to make their decision.
[00:39:09] They need the expert perspective from those two bodies to make the informed decision around the vendor or the connection that's being requested. So yeah, it's gonna vary by company to company, but the best thing you can do is deeply understand. That process before you get started, so you don't run into that obstacle when you're ready to go.
[00:39:28] Cathy McPhillips: For sure. And in one instance that we had internally, the request was put in, it took longer than normal because it was brand new, brand new tech, and we were trying to figure out, well, ops was trying to figure all of it out with legal and. There were two, there were some redundancies because things were being done twice, one the old way, and one trying to do it with a new tool.
[00:39:47] So there was a minute of double work, but in the long run, it's going to be so much better now that everything got approved the right way.
[00:39:53] Paul Roetzer: Yep. And I'll just say, being on the other end of this, you know, we have our AI Academy, which is a technology [00:40:00] platform, and so we have to go through procurement, especially for these bigger enterprises, and you have to meet some very stringent standards that are only getting.
[00:40:09] Stricter as we go. And so Tracy can attest like, I mean, she spends a fair amount of her time getting through procurement systems and she could probably sit here all day telling you about all the new requirements over the last like 18 months as a result of ai.
[00:40:24] Cathy McPhillips: Yeah. There was one she did last week or two weeks ago that she was just like, this is the wildest form if ever filled out.
[00:40:30] Paul Roetzer: Yeah. They get crazy but necessary.
[00:40:31] Cathy McPhillips: Yeah. Yeah.
[00:40:33] Cathy McPhillips: Number 11. When reviewing an AI vendor's terms and data policies, what should companies look for regarding what happens to their data? If the vendor is acquired, goes bankrupt or sells its assets? How can organizations manage or mitigate that risk or sells the data for training purposes?
[00:40:50] Paul Roetzer: I, I, I'm like this one, I just honestly feel like my best answer is you, you have to rely on your legal team. Like [00:41:00] everybody's legal team's gonna have different appetites for risk. They're gonna look at it through different lenses. It is just simply not a decision that a, like a traditional business leader, department leader, you, you don't have the expertise and likely you don't have the permit in your company.
[00:41:18] To make those decisions, you, you have to involve legal and whatever other channels are required. And hopefully, you know, based on our previous question, that's all clearly document documented in the I policy. And if it's not, it needs to get documented so that you can follow this, these procedures. And then, you know when it's a non-starter, so you know you're getting really excited about this new AI tool, you go in and pull up their terms of use, you throw it into Chad, GPT and say.
[00:41:45] Oh, tell me how they handle our data and immediately it pops up with like three red flags for you. Another way you could do it is, take your AI policy related to vendor data usage. Drop that into ChatGPT thread or [00:42:00] Gemini or wherever and say, Hey, here's our AI policy. Or build a project that has that AI policy baked into it.
[00:42:06] I'm going to give you a contract for a new vendor. Can you tell me how this relates to our. Terms of use requirements and, flag for me what I should highlight for our attorneys.
[00:42:16] Cathy McPhillips: Right?
[00:42:16] Paul Roetzer: That's, I do that kind of stuff all the time where I'm just like, here's this doc, here's this doc. Compare them and tell me what I should ask the attorney
[00:42:23] Cathy McPhillips: for Sure.
[00:42:25] And then, you know, just thinking about when they change the terms and conditions or when something, I mean, there's so many, it's not like approve it once and it's done. It's like
[00:42:32] Paul Roetzer: Correct.
[00:42:32] Cathy McPhillips: Set up some normal cadence of just reviewing everything to make sure it's still.
[00:42:37] Paul Roetzer: Yeah, like when Anthropic came out with Fable 5, they changed their data retention policy where they kept everything for 30 days and I, they, a lot of people were up in arms about that.
[00:42:47] Cathy McPhillips: Right.
[00:42:48] Cathy McPhillips: Number 12. When should a company disclose externally that AI was used to create content or complete work? Is there a meaningful threshold or does it depend more on the nature of the work and its potential consequences?
[00:43:00] Paul Roetzer: This gets in again into AI policies, it everybody needs to make these decision decisions for themselves.
[00:43:06] I feel like it's a bit of a sliding scale where more and more it's just gonna be kind of assumed and expected that AI played a role. And I think like tagging every single thing with like AI health here or AI did this, it, I don't think most people are gonna care most of the time. So I think that there's some basic rules that at least I try to live by, and I hope our company tries to live by, is like, when authenticity matters, you should be doing the work.
[00:43:34] It should be human in the lead. Like it's okay if you're using AI to edit or, you know, verify ideas and things like that. But, You know, it should be you and you should stand behind that work. this came up on episode 2, 34 or 35 of the podcast we were talking about that one, author who used it and got called out on it, and he's like, of course I use it.
[00:43:56] I use it all the time. Like, whatcha talking about? I, [00:44:00] I generally like to default to, if you're using it can be something as simple as like. Co-authored with Claude, or Claude supported it in the editing and, you know, revisions of this document, or Claude helped me with brainstorming, whatever, like just disclose it because, it doesn't do any harm.
[00:44:17] But I don't think we're at the point where we need to make like a huge deal of it anymore. right. Yeah, again, it's you gotta make these decisions in your ad policies and then provide that guidance to your team because everybody's wondering. It's like, I don't know if I'm supposed to tell somebody I didn't actually look at their email like my agent is now answering my emails for me.
[00:44:35] In that case, I would just say, yes, you should disclose it if your agent is answering your emails for you. but yeah, I think it's a very personal decision and it needs to be decided in the policies.
[00:44:46] Cathy McPhillips: I was working on something last week and I can't even recall what it was. But I kind of went into it blind, not really knowing what I was gonna be working, like, what I was gonna be doing.
[00:44:56] And I used AI a lot.
[00:44:59] Cathy McPhillips: And I put that in the, [00:45:00] when I distributed it to the team, I was like, this is Claude written with Cathy supporting it. Yeah. Like, I honestly, I don't know. So this is my first pass I'm learning and just so people looking at this like, this doesn't sound like her. it's so, it's so obvious when it's not, you know?
[00:45:15] Yeah. But I just want people to know.
[00:45:17] Paul Roetzer: Yep. That
[00:45:19] Cathy McPhillips: where I was on that, in that process.
[00:45:20] Paul Roetzer: I like one example, and you could attest to this, Cathy, like sometimes we'll be working on a problem and I'll go into ChatGPT deep. Like, here's the exact problem, here's the context you need. Here's what I'm trying to think through.
[00:45:30] Build a brief for me, like I just need to like think about this and it'll build a brief and I will take 10 minutes. I'll read the brief, I'll give it some thought and like, this is really good. I will send it to the team internally and say this is completely unedited. I have reviewed this and this is a really good starting point, and then I turn it over to the team so they know I didn't do it, but they know I've signed off on like, this is a good direction.
[00:45:51] Come back to me when you, you all have made progress on this. so yeah, I just, I don't know, I don't think there's like rigid rules around this. I feel like it's [00:46:00] just evolving as our use of the models changes.
[00:46:03] Cathy McPhillips: Okay.
[00:46:03] Cathy McPhillips: Number 13, entry level staff have historically learned their trade by implementing task-based work assigned by their managers.
[00:46:11] If that work is now going to be assigned to ai, how do entry-level staff learn their trade? And where do future managers come from?
[00:46:18] Paul Roetzer: My theory is apprentice programs. I talked a little bit about this on the podcast. this week, it was 2 35. I think it came up because there was that research that came out about.
[00:46:29] apprenticeships were actually like the number one thing, and it just, that happens to be what I'm talking about for MAICON. My opening keynote is the architect, the Orchestrator and the Apprentice. It's a theory and that it's very just a theory on what the future of work looks like and what the different roles we're gonna need within organizations are.
[00:46:47] my belief is that we can actually accelerate learning and expertise through apprenticeships, with the support of like guided learning from ai. [00:47:00] Personalized learning. but I'm not sure yet. I have six weeks to figure out what that exactly that is. I, I've been thinking about this model since March of this year.
[00:47:10] I don't know. This has been my great debate where, like, where does management come from? What do managers do in the future if we don't train entry level employees? So if we stop hiring as many entry level staff, then we decimate the future managers and leaders of the company. And I don't know that, again, kind of like the earlier question, I don't know of an organization that has this figured out.
[00:47:34] Like it's, I've talked to lots of companies and there are lots of AI transformation leaders and HR leaders at big companies that are just starting to ask these questions themselves. And, and nobody seems to really know where to go with it yet.
[00:47:48] Cathy McPhillips: Yep. 41 days in case you're wondering till my can. No pressure.
[00:47:53] Paul Roetzer: 41. Oh. I only have three talks to create in that time period.
[00:47:57] Cathy McPhillips: number 14, you [00:48:00] described the actual mechanics of an LLM practice predicting the next word as a bit of a black box. Is it really? And if so, isn't that a little ominous?
[00:48:09] Paul Roetzer: It is really a black box. there's a field of study called mechanistic interpretability where they try and understand why the models do what they do.
[00:48:18] The LLMs itself is making predictions based on proximity of, of like words to each other within a graph. So this is like an, a very oversimplified way to try and help people visualize this. Imagine all of human language as like. This three dimensional cube and within that cube is like every word in the human language.
[00:48:40] And how close those words are to each other sort of shows a probability of them occurring next to each other. And so when ChatGPT starts writing something, it's in essence going through this dimensional graph and it's like. Knowing that, you know, Cathy sat on the chair and it knows [00:49:00] that chair is a likely word connected to where Cathy would sit.
[00:49:03] 'cause people sit on chairs. Could be a bench, could be the table, could be all these things. But in essence, like imagine in that cube, those words all appear near each other. Sat chair, desk, whatever. That's kind of like how it works. The way to think about the language model, the closest proximity to it is the human brain, and that would be like saying, well, do we really not know why?
[00:49:26] Paul picked the words he just picked when he was explaining this? Like, no, we actually don't, like you could put me under an MRI machine and it might see. What neurons in my brain are firing when I'm talking, but it can't drill in and say, why did he choose the word he chose? it's likely just on my training and in my experience as a human of the words that appear near each other in the English language.
[00:49:50] So that's kind of it. Like it, yeah. It's ominous. Yeah. It's weird. Like we don't really understand this intelligence that we've created. [00:50:00] All we know is that we give it more and better data and we give it more chips to train on that. It seems to just get smarter and more capable of simulating human behavior and language.
[00:50:11] And that's kind of it. Like that's the basis for the first scaling law of AI that has now stayed true for, you know, the last decade, which is more data. More computing power. The bigger the model, the better it seems to get. there was a, this sort of ominous thing that Ilya Sutskever said back in the day at openAI's, which was, they, they just wanna learn, like these models just want more information and they just get smarter.
[00:50:37] Cathy McPhillips: Okay.
[00:50:38] Cathy McPhillips: Number 15. The nice thing about these questions is that I get to decide the question, so I actually dropped in my own question.
[00:50:43] Paul Roetzer: Oh, okay.
[00:50:43] Cathy McPhillips: my husband's been traveling for a week and I've had a lot of free time. So this is, these are the things I think about. okay. When I worked with Joe Pulizzi at the Content Marketing Institute for years, he always emphasized the importance of building on land.
[00:50:57] You own your blog. Your [00:51:00] email list, your audience. We've seen, we've seen, what can happen when people go all in on rented land like LinkedIn, Twitter, or another platform, and the algorithm algorithm changes products, audiences. audience becomes harder to reach. The platform disappears. Now.
[00:51:15] People are building products, audiences, and sometimes entire businesses on custom GPT and other AI platforms. Are we watching the same story repeat itself with LLMs and how should we think about the risk of building on rented land?
[00:51:29] Paul Roetzer: That's a deep thought to have.
[00:51:31] Cathy McPhillips: I know A lot of free time. Not really free.
[00:51:34] Not really free time, actually. Alone time.
[00:51:36] Paul Roetzer: Yeah. I think it kind of relates back to the thing I mentioned earlier about. If you build on a single platform, you're sort of a prisoner to what happens to that platform. I look at this a lot now with like how transferable. skills are between agents and platforms.
[00:51:55] The system instructions that are used to build AGI, PT can very easily be [00:52:00] copied and pasted into Claude, and you can build the same capability over in Claude. So I think that there's an inherent risk here. It's probably part of the reason why some organizations are now choosing to take open weight models and train their own models.
[00:52:13] It's just a more reliable system. The model lives on internal servers. You're not. You know, captive to a cloud service going down and you no longer have access to it. So I do think that more organizations will start to build redundancies and backup plans in, at a very surface level. You can start by making sure everything is documented.
[00:52:35] We, we've done this again internally at SmarterX. We're now, at least from a governance perspective, documenting when apps and agents are created by different people. Ideally you include what the system instructions are, what the knowledge base they have access to is so we can go back and audit it. And so let's say like worst case scenario, you build everything around ChatGPT, you've got 25 agents that are basically running the company.
[00:52:58] You've got 75 [00:53:00] skills that those agents use to do what they do. And then chat, PT goes down for. Five days. You could worst case, go grab the skills, recreate 'em in Claude, or you just mirror everything in Claude, like someone's job literally might just be maintaining a mirrored image of the ChatGPT instance in a second platform so that if something goes down, you just flip a switch, everybody moves over to Claude and continues their work.
[00:53:27] I would imagine there are organizations that. Are actively doing what I just described. I haven't talked to them. It's just one of those things you can kind of make an assumption that someone is ahead of the game doing this because we are talking about entire companies running on this and now you have companies laying off thousands of people under the assumption that agents can do a lot of this stuff.
[00:53:47] Well, if those agents aren't available, work doesn't happen. So there needs to be redundancies built.
[00:53:52] Cathy McPhillips: Yeah, and I was talking to Claire, who, is our producer of the podcast and works on our, content. We were talking today about [00:54:00] how Mike had actually done, talked about this in one of his courses for Academy, about how to extract some of that from one tool to be able to upload it and put it in another tool or how to document it and what that process is.
[00:54:13] Yep. So it really is a critical step of all of this. All right. That is number, that is 15 questions.
[00:54:19] Paul Roetzer: That's are good questions. I don't, I feel like those are all different questions too. Like, I don't know how I've answered any of those. Like maybe like the marketing use cases has probably been answered at different times, but it changes.
[00:54:27] But the rest of these are, relatively original. Really good.
[00:54:32] Cathy McPhillips: You don't, you don't look at this beforehand, but Claire and I do.
[00:54:34] Paul Roetzer: Yeah.
[00:54:34] Cathy McPhillips: So you did a good job. She does a lot of, she does a lot of, has this been asked 20 times already? Yeah. Let's not ask it again. So, yeah.
[00:54:42] Paul Roetzer: Yeah. Good stuff.
[00:54:44] Cathy McPhillips: So thank you, Paul.
[00:54:45] We'll, you'll see a Mike next Tuesday back on the podcast if you're interested in learning more about. marketing AI Month or MAICON, our big event coming up in October. Please let us know. I'd love to help you get you to both of those
[00:54:57] Paul Roetzer: things. Get registered. Come to Cleveland. We've got Oh, do the pod [00:55:00] because we have the POD 100, which gets you a VIP lunch with me.
[00:55:03] And Mike's we're doing on that the final day, the October 15th of VIP lunch for anybody who's uses the POD100 when they register. So yeah, MAICON.ai get registered. We'd love to see there.
[00:55:13] Cathy McPhillips: POD100. Thanks, Paul.
[00:55:15] Paul Roetzer: Thank you. Thanks for listening to AI Answers to Keep Learning. Visit smarterx.ai where you'll find on-demand courses, upcoming classes, and practical resources to guide your AI journey.
[00:55:29] And if you've got a question for a future episode, we'd love to hear it. That's it for now. Continue exploring and keep asking great questions about ai.