The Future of AI in Production: Our Best Advice for Entrepreneurs and Developers

[00:00:00] Welcome to the Convergence Podcast. I'm your host, Ashok Sivanand.

[00:00:07] No one is like not using Gen.AI now, right? With the launch of ChatGPT.

[00:00:12] Now we have like these virtual employees and these virtual agents.

[00:00:14] I'm actually super optimistic in the area of technology and AI.

[00:00:19] On this show, we'll deconstruct the best practices, principles, and the underlying philosophies behind the most engaged product teams who ship the most successful products.

[00:00:36] This is what teams are made of.

[00:00:38] Hey folks! Thank you so much for following along and listening to the Convergence Podcast this year.

[00:00:46] We're closing out 2024 with some of the most relevant and valuable advice, insights, and stories from our guests.

[00:00:54] 2024 was another remarkable year for the progress of generative AI, and that's what we're focusing on today.

[00:01:00] We saw updates in the legal and regulatory space.

[00:01:05] Things like the European Union proposing new regulations addressing safety, privacy, transparency, and bias in artificial intelligence.

[00:01:15] We saw numerous lawsuits emerge around intellectual property and copyright laws.

[00:01:20] And it's yet to be seen whether companies like OpenAI have violated laws when training their models.

[00:01:28] The massive use of energy is also a growing concern, not just in the private sector, but also amongst policymakers now, as more companies and more investment is going into generative AI.

[00:01:41] We also saw some major technological advancements.

[00:01:46] At OpenAI, they continued to release models to the market, including reasoning models like 01 and 03, and also made video generation a lot more accessible through their Sora product.

[00:02:00] Apple got into the game with Apple intelligence in their latest release of iOS and their latest iPhone.

[00:02:07] And Meta disrupted a lot of the business dynamics, I think, as well as the accessibility of AI when they launched open source models of Lama that they claim to rival GPT for.

[00:02:20] On the investment front, we saw a lot of capital flow toward AI in 2024.

[00:02:26] OpenAI is now valued over $150 billion.

[00:02:30] And that's similar to companies like Pfizer, Comcast and AT&T.

[00:02:34] And Elon Musk's XAI raised $6 billion in total this year, putting them over 50 bill in valuation.

[00:02:41] We also saw emerging companies like Grok, Grok with a Q, raise impressive capital, especially as the need for faster inference and lower cost becomes more critical to the viability and usability of AI applications.

[00:02:57] NVIDIA sales continued to grow and they're now valued as the third most valuable company in the world.

[00:03:05] On the application front, we saw use cases beyond content generation, task automation and decision making.

[00:03:13] It's become a lot more typical now to see video and AI generated content even in journalism.

[00:03:19] This is perhaps supercharged by the US presidential elections that happened this year.

[00:03:24] We heard companies making some major personnel changes like Klarna, for example, implemented an AI customer service agent to do over 75% of the tasks that were carried out by a team of over 700 people before.

[00:03:39] AI generated art also exceeded a million dollars in the sale.

[00:03:44] A lot of our guests this year have experimented with generative AI to enhance both their products and their product development.

[00:03:51] And we've compiled some of our favorite clips about generative AI from 2024.

[00:03:59] We'll have links in the show notes to all these segments and their full episodes in case you want to hear the rest of the episode with that guest.

[00:04:10] First up, let's create a scenario where maybe you have a niece or nephew who's like getting out of high school, going to college and, you know, the college curriculums are going to take a little bit of a while to get updated.

[00:04:26] Maybe you tried talking them out of college, but they want the experience and they want to learn computer science.

[00:04:30] What's the advice to set them up best for that AI enabled world being a computer scientist or software engineer?

[00:04:40] I think maybe if maybe I should start by like a few observations that I'm kind of seeing that is happening in the industry.

[00:04:47] I definitely think there's an element of like you writing perfect syntax.

[00:04:52] It's kind of a skill that is going to zero in the sense that it's like you don't have to be great at writing perfect Python or .NET or C sharp.

[00:05:01] And an element is probably going to write that for you in the future that you're going to declare your intent.

[00:05:06] And then you're going to have an element drafting the code for you.

[00:05:09] And there's also an element of like, what does that mean?

[00:05:12] It means that things like domain expertise, you really knowing about a specific vertical, you know, and how about system architecture, how you actually translate requirements from humans and understanding the intent.

[00:05:25] That's probably going to be much more valuable than you just being a human compiler that can compile whatever this is to a syntax because it is probably going to do that.

[00:05:34] Right.

[00:05:35] And I also think like that's the element of like, if you are in school today, you do your core computer science, like having that strong foundation, understanding systems and how they work is going to be critically important.

[00:05:48] You can reason about these new systems.

[00:05:50] But there's also an element of just embracing these tools of like, if you're starting to program today, like use Copilot, like that can really help you to become like the famous 10x or 100x developer because suddenly you can express your intent and have

[00:06:05] had the system kind of help you.

[00:06:06] So that's definitely an element of just embracing this new reality of like the task, rewriting syntax and understanding different systems is that the elements can probably do that for you.

[00:06:19] So I really think about like you enabling your domain knowledge and really, really you learn to understand like the requirements are probably going to be much more important than the act of writing code.

[00:06:33] Now we have like these virtual employees and these virtual agents that actually like listen to all the onboarding material and can actually provide like the right answers.

[00:06:42] I think that's going to be a lot of very interesting kind of like verticals like you know, like this in legal accounting, etc.

[00:06:48] Is that I'm definitely seeing a lot of like tools being used, starting to be adopted by law firms where they kind of have like a paralegal as a service.

[00:06:57] So I think that can look up cases, do case work, find references, etc.

[00:07:01] Which I think is incredibly exciting and same for accounting.

[00:07:05] It's like accounting in the simplest sense is looking at numbers that turns out systems are really good at that.

[00:07:10] So I think we think that there's a lot of these kind of new kind of like levels of productivity that are being enabled that are super, super exciting.

[00:07:20] But that's also just like tying it back to like the handcrafted code.

[00:07:25] I definitely think that there's a lot of software developers in the world that probably didn't anticipate them being first in line to kind of get some of the work automated.

[00:07:35] And so I definitely think this is going to be a very interesting transition because there's probably a lot of software engineers in the world that are very content of being, if I say like a human compiler,

[00:07:46] is that I will take whatever and I'll write converted to code where like the job going forward is probably going to be much more human centric.

[00:07:53] It's probably going to be you working with your customers if because the act and actually writing the code is going to be much, much faster.

[00:08:01] So it's going to free up a lot of creativity and also a lot of your time to kind of do other things.

[00:08:05] And I think this is actually having a lot of it's probably going to having a lot of interesting kind of like second order effects of how we think about modern software engineering teams

[00:08:14] that do we need as many PMs if the EM or the IC engineer can actually talk to customers directly.

[00:08:22] And that's a lot of these things, these skills, I think that it's going to like be blurred a little where maybe a PM can actually use a Devon of the world to update the marketing side and scaffold the new feature.

[00:08:33] So I think it's going to be a very interesting kind of rewiring of product teams in the next five years.

[00:08:47] Next, we have Derek Ferguson, the chief software engineer at Fitch Group, who also shares a similar optimism to Kenneth around the need for software engineering talent growing,

[00:08:59] albeit with a different scope than it likely currently looks.

[00:09:04] I'm actually super optimistic in the area of technology and AI.

[00:09:10] I hear from time to time from software engineers, have I chosen the wrong profession because our computer is going to be doing all the coding going forward, you know, if you can get this.

[00:09:20] And I am constantly reminded of a lunch that I had with the man who was Microsoft's head of developer tooling for many years.

[00:09:34] Soma was his name.

[00:09:36] I mean, there is so much demand for technologists, for good technologists who are able to understand a requirement that might not be perfect,

[00:09:50] have the clarifying conversations that you and I have talked about today, get it to a point where it can actually be built,

[00:09:59] and then built in a good way that's secure and easily expandable, etc., etc.

[00:10:07] And what I think will happen with this perfusion of tools is maybe initially you do see a little bit of a dip in demand

[00:10:20] because now you've enabled non-developers to come into the fray and put together these version 0.1s of their applications.

[00:10:29] But if history is any guide, and I think it is, five years from now, six years from now, however long, I don't think it's long-term, it's more medium-term,

[00:10:39] you're going to see an explosion of need for software engineers unlike anything we've ever seen before

[00:10:46] because what winds up happening is supply drives demand.

[00:10:51] You will get a lot of folks who come in and get their ideas 90% done, but that last 10% really needs somebody who has a software engineer's knowledge and discipline.

[00:11:06] And then the thing is, I think that these ideas that are 90% done will sadly probably be 50% in the wrong direction,

[00:11:15] so there's going to be that rework that needs to be done to say, boy, you know, if you brought us in earlier, we could have, you know, we would have done this, that, and the other thing.

[00:11:24] So there's going to be, and then besides that as a feature, we're going to have people thinking about putting software places software has never existed before

[00:11:36] because there will be such a profusion, there will be this profusion of ability.

[00:11:42] Therefore, it might initially bring the cost down of software development to say, hey, let's put software in this kind of a device, let's put software in that kind of device.

[00:11:50] What if we had software that did this, this, this, and this?

[00:11:54] Then because you have all this, all these places that are running software that never ran it before,

[00:11:59] that contributes into the demand for having experienced software engineers also.

[00:12:04] So, boy, I'm, I, I tend to be an optimist in life in general, but in this case, I am insanely optimistic.

[00:12:13] Um, yes, it is going to drive the supply of software, but that supply is going to drive the demand like I think nothing we've seen before.

[00:12:20] So if you've chosen to go into software as a career, you've made the right choice. This is, this is the place to be.

[00:12:29] Fostering an engaged product organization and aligning them with the principles around lean, human centered design, and agile will more than likely lead to successful business outcomes for your organization.

[00:12:42] But getting started or getting unblocked can be hard.

[00:12:45] This podcast is brought to you by the player coaches over at integral.

[00:12:49] They help ambitious companies like you build amazing product teams and ship products in artificial intelligence, cloud, web, and mobile.

[00:13:00] Listeners to the podcast can head on over to integral.io slash convergence and get a free product success lab.

[00:13:09] During this session, the integral team will facilitate a problem solving exercise that gives you clarity and confidence to solve a product design or engineering problem.

[00:13:20] That's integral.io slash convergence.

[00:13:24] Now back to the show.

[00:13:35] Kenneth comes back to talk about how he thinks the field of developer tools and creating delightful developer experiences will evolve as folks adopt generative AI.

[00:13:47] How is this latest wave of artificial intelligence going to change developer experience for you?

[00:13:53] I think like at a very high level, like my, my thesis and my mental model is that I do think we are moving into a world where 80% of all code in one way or another will be written by an LLM or an agent.

[00:14:06] That's really like the, I think my, my, my, my foundational thesis.

[00:14:09] And I think we'll be looking back at the dark ages of handwritten code.

[00:14:15] It's like these programmers, were they like hand in a crafting code by hands without any assistance?

[00:14:22] What were they thinking?

[00:14:23] Right.

[00:14:24] Similar way that you, you go to like the hipster coffee shop and they're making the coffee.

[00:14:30] That's going to be a space for artisanal kind of crafted experiences.

[00:14:33] But the vast majority of software is probably going to be more industrialized and produced by, by machines and LLM similar to how we think about like the de facto worker.

[00:14:44] So I think that that's a lot of very interesting, like second and third order effects.

[00:14:46] So what does it mean in a world to provide great developer experience when the main customer is actually no longer the developer, but the tools that the, that the developer is using?

[00:14:57] And so what does it mean to be providing like almost like a AI first or AI native developer experience where you're actually optimizing the interfaces and the abstractions towards the LLM or the agent that is going to interface with the system and not necessarily the human.

[00:15:13] Because the developer might be transitioning to more of like the, the classic hovering art director meme role where they are like orchestrating different LLMs and tools kind of people producing stuff.

[00:15:25] And so they're no longer the direct consumer of your API or whatever future abstractions that we have.

[00:15:31] I think that's going to be like a lot of change in how we think about developer experience.

[00:15:35] But I should also say that I think this experience is going to be very gradual.

[00:15:40] I don't think code is going away.

[00:15:43] I rather think that we're going to have much more code in the world that is now going to be drafted or written by LLMs and agents.

[00:15:50] And there's still going to be a heck of a lot of systems out there, but we are moving to a world where in a few years, there's probably an inherent like code bases and stacks that has mainly been written by an LLM.

[00:16:02] And we need to start to reason about that.

[00:16:10] Farhan Thaur, the head of engineering at Shopify, in one of our most popular episodes this year, talks to us about their team's early access to GitHub's co-pilot and some of the parallels that he sees to the benefits of pair programming.

[00:16:26] Fair enough. And I mean, you can kind of look at the plus plus on that is using co-pilot, right?

[00:16:33] Which you could argue is like a pair, even if your pair is sick or whatever.

[00:16:38] 100%.

[00:16:38] Using co-pilot.

[00:16:40] And then is this pairing mindset?

[00:16:41] I know you're huge adopters of co-pilot.

[00:16:44] Did that pairing mindset, if you had to guess, make it easier to adopt co-pilot?

[00:16:51] Yeah, for sure.

[00:16:52] I mean, I think if you watch all the videos from like GitHub universe, you'll see that they use the pairing analogy quite readily because it's like a pair programmer.

[00:17:00] I would say for me, you know, what it does is for leaders who used to code a lot, it helps you get unrusty quick because you might be in a language you're not as familiar with as your primary language and co-pilot can really help you not get stuck and not feel like you have to start from zero.

[00:17:18] So I think it's good for that.

[00:17:19] It does like what you mentioned.

[00:17:21] It does give you this feeling of not being alone.

[00:17:25] Like I'm not alone making this, writing this program.

[00:17:28] I actually have somebody who's helping me.

[00:17:30] And that I think has a little bit of the pairing nuance.

[00:17:34] One thing we tried, which I couldn't really get working.

[00:17:37] And I think they just deprecated it, but maybe it'll come back was for accessibility.

[00:17:42] They had like a voice co-pilot.

[00:17:44] So you could almost like hear it and talk to it.

[00:17:48] And then you're like really like impairing because you're not like, you're not even typing to the chat.

[00:17:52] You're actually like talking out loud.

[00:17:53] So I hope that they, I hope that there's more innovation in that area.

[00:17:57] Cause I think it's all then you can really like have a pair program where it's talking to you.

[00:18:06] Derek from Fitch is back and he shares how the Fitch group rolled out the adoption of generative AI augmented development to their team of over 600 developers.

[00:18:17] On the software development side.

[00:18:21] I started the year with a mandate that by the end of the year, all of our software developers.

[00:18:27] Got a team of about 600 would be empowered and using generative AI tools.

[00:18:33] We weren't exactly sure what to expect from it in terms of productivity boosts, but our early experimentation indicated it was going to be big.

[00:18:46] So what we did as a kickoff, we took four or five of our recent university graduates.

[00:18:57] And at our big annual senior leadership meeting at the end of last year, we brought these fresh out of university employees up on stage and we set them at a table.

[00:19:13] And we got from the audience a mandate of an application that they'd like to see built and myself and my peers and our boss Fitch's CIO did our, you know, 15 minute a piece presentations up, you know, leading up to an hour.

[00:19:33] And by the end of the hour, we were able to say, developers, have you finished the application?

[00:19:41] Then they built it.

[00:19:44] It's a little bit of, you know, a little bit of circus.

[00:19:47] Never, never hurts.

[00:19:48] Right.

[00:19:48] Showmanship is an important thing in business as with anywhere.

[00:19:52] It was a real danger too, by the way, we had, we had no backup plan.

[00:19:56] The backup plan, if it hadn't worked, would have been probably me running off this running off the stage crying with.

[00:20:03] Would have been the backup plan, but.

[00:20:05] Burn the ships, I think they call that.

[00:20:06] Burn the ships.

[00:20:08] Exactly.

[00:20:08] Yeah.

[00:20:08] Yeah.

[00:20:09] If you've got no alternative, you're, you're, you're going to make it.

[00:20:11] Um, I guess we would have said that we failed fast, right?

[00:20:14] You know, this is a fail fast culture and we, we just failed.

[00:20:17] So, um, but yeah.

[00:20:19] And so they, they built this application, which just in that one hour did stuff that would have taken a squad probably a week prior to that.

[00:20:27] Um, so that set our expectations fairly high.

[00:20:32] And we have been following the process of putting our AI generation, uh, coding tools of choice in front of all of our developers.

[00:20:42] This year I sort of perceive as leading the horse to water.

[00:20:47] Uh, it's been a very prescriptive, supportive approach to say, here are the tools.

[00:20:53] Next year I want to switch footing.

[00:20:55] And this is perhaps a piece of advice I would give at some point in the inflection cycle.

[00:21:01] You have to switch from being prescriptive about the approach to, and this is true of all management, being prescriptive about the desired results.

[00:21:13] Um, from what we have seen and from what the industry also reports, the productivity boosts associated with these tools can be anywhere from 20 to 50%, depending upon the nature of the developer.

[00:21:27] Um, we see more productivity gains for the newer, less experienced developers.

[00:21:33] And it sort of tails off a little bit more as you get more towards experienced developers, but it never, never, never failed, never shrinks to zero, even with the most experienced of developers.

[00:21:46] And, um, as a result, we're going to set our productivity targets, double digits, uh, for, for next year, just based on the emergence of these tools.

[00:21:58] But it is, um, it's a change in approach from saying, here are the tools, please use them to saying, here's the expectation around productivity boost.

[00:22:08] And, um, you can achieve that through whatever means you think is best, but we're saying we think these, these coded AI tools are, are the way to go.

[00:22:22] Next up, we have the renowned product author Akash Gupta, and he shares advice on some of the mistakes that AI companies make as they go to market.

[00:22:36] What was interesting to me, perhaps like the most obvious advice is still the most profound, which is that the common mistakes that lead to AI tools.

[00:22:45] Shopify startups to dying, we're like universal, you know, getting hyped about what's in the lab verses actual product.

[00:22:52] So if anybody's looked at rabbit or humane pin, you know, the demos and the ads looked pretty cool.

[00:22:58] Like with the rabbit device, you could order door dash from bed, but anybody who's used the rabbit device.

[00:23:04] Now today is like the door dash rabbit is currently down.

[00:23:07] It's like, it's a pretty low quality experience.

[00:23:10] It's not like Apple showing a demo and then the product, they actually deliver on it.

[00:23:13] And so that's one of the most common things you see with AI startups.

[00:23:18] Like that, we kind of cataloged based on the failures that came to.

[00:23:22] I have had a chance to use the rabbit.

[00:23:25] I didn't try the DoorDash feature, so I didn't get that disappointment there.

[00:23:29] What are some of the other pitfalls that you're noticing amongst the AI startups that you've

[00:23:35] had a poke into?

[00:23:36] Yeah, so the most common thing is falling for shiny object syndrome.

[00:23:40] So after it works in the lab, falling for shiny object syndrome.

[00:23:45] So basically, for example, when Dali came out in 2022, you saw so many startups, basically

[00:23:52] Dali for X, Dali for Y, or Dali wrapper.

[00:23:55] And the idea was text image.

[00:23:58] But it all just was clearly using Dali, had that exact same distinctive style.

[00:24:03] And so everybody could realize that it was built on top of Dali.

[00:24:06] And the company that actually benefited the most was OpenAI.

[00:24:08] And as they released iterations on Dali, that's what people used.

[00:24:13] And so it's a common example where something new is released.

[00:24:16] And this isn't just with Dali or ChatGPT.

[00:24:19] So we profiled this company called Olive, which maybe people don't know about.

[00:24:23] They actually raised $902 million.

[00:24:25] And they were started in 2012.

[00:24:27] And they were an AI company.

[00:24:29] Is this the company that does warranties or is it different Olive?

[00:24:32] I don't believe they did warranties at all.

[00:24:34] So they had like 24 products, though, interestingly.

[00:24:36] And their company, their CEO was like really happy that they pivoted so often.

[00:24:42] And the VP of product at Olive actually reached out to me after we published this piece to say that all five of these pitfalls were spot on for Olive.

[00:24:50] And what she said was that with respect to shiny object syndrome, like those 24 products I mentioned, like basically every time something new is released, they wanted to take advantage of it and apply it to some industry.

[00:25:01] And that really wasn't like committing to like one problem.

[00:25:06] Like you want to fall in love with one problem.

[00:25:08] And then it doesn't matter whether it's AI or not, like you want to solve it.

[00:25:12] So that's like the critical thing.

[00:25:14] And then if AI is the best way to solve it, then you're an AI startup.

[00:25:17] So pitfall three is irresponsible deployment.

[00:25:21] So there was this company called Clearview AI that people probably heard of.

[00:25:25] It had like almost $100 million raised.

[00:25:27] And the idea was it's a facial recognition startup.

[00:25:30] And the problem was that they scraped hundreds of millions of profiles off Facebook.

[00:25:36] And people didn't like that when the NYT released its front page.

[00:25:41] It might have not been front page, but released its expose on the company.

[00:25:45] And it went to the point where they actually lost in the court of law and they had to cease operating as a company.

[00:25:53] So irresponsible deployment is definitely a problem with AI.

[00:25:56] Okay.

[00:25:57] Pitfall four is prioritizing flash over function.

[00:26:00] So essentially, this is the example of Quixi, which a lot of people probably heard of because they famously took out a Super Bowl ad.

[00:26:08] So their entire idea was, you know, we're going to have deep learning search engine for apps.

[00:26:14] And their search engine for apps was going to be on Android fighting against the world's best search company, Google.

[00:26:21] Of course, they lost that battle after burning over through $160 million.

[00:26:27] And it was basically flash.

[00:26:28] Like they were just trying to advertise like these deep learning models that they were using, but they weren't focused on the functionality of people are clicking our first result 17% more than Google search engine.

[00:26:40] People weren't doing that.

[00:26:41] And then the fifth pitfall is raising too much too fast.

[00:26:44] So this is probably the most controversial or the hottest take that we had, but, you know, everybody wants to like raise $900 million for their foundational AI model startup and take $20 million in secondaries and become rich in two years.

[00:26:57] But the reality is, you know, the average company that's succeeding in AI has been around for a long time, whether it's OpenAI or NVIDIA or anybody else, all these big tech companies.

[00:27:09] And so what you actually probably want to do is the hotter thing nowadays, which is, you know, build a profitable company from day one.

[00:27:17] Grow as your profits grow.

[00:27:18] Grow as your profits grow.

[00:27:53] As opposed to hinder people using it.

[00:27:56] So there's a machine learning policy.

[00:27:57] What's that for?

[00:27:59] So there were two big things that I think was important about us actually having a policy.

[00:28:03] One is just from as an organization, this is a tool people are going to be using.

[00:28:07] You need to assume they're using it.

[00:28:09] So you want to put some guardrails out there to make sure that you have policy in place about how they should and shouldn't be using this new tool.

[00:28:16] But the other was actually to encourage people to use it, not just to prevent them from doing dumb things or risky things with it.

[00:28:24] But we realized that people weren't using the tool and getting the benefits as much as they could because we hadn't clarified.

[00:28:29] Hey, here we want you to go experiment with this.

[00:28:32] We are looking to learn where this helps us win.

[00:28:35] And we want you to share the learnings, maybe even spend an hour and make some mistakes and realize it's not worth it.

[00:28:39] And then come tell everyone else about that.

[00:28:41] So you're the last person that has to waste that hour.

[00:28:44] And so the policy allowed us to clarify that that was what we were expecting from people and gave them a lot more safe space to fail and learn and then find the wins with the tools.

[00:28:59] Next, we've got Doug Cramon, the head of customer care from ESPN.

[00:29:05] Doug shares how their team at ESPN uses AI to replace some of their carry agents tasks.

[00:29:12] But more importantly, how they have this awesome real time analysis tool powered by AI that augments their care agents and helps convert difficult complaints to loyal customers, especially as they transition their team from a cost center to a profit center.

[00:29:28] I think technology and automation even more broadly is finding its way more and more into customer service.

[00:29:34] And I have noticed maybe an anecdotal trend that I'd love for you to kind of give, you know, point me a thumbs up or a thumbs down on where I find that companies that tend to think of their customer service teams as a profit center in the long term, as opposed to think of them as a short term cost center, tend to find more ways to invest in enabling their teams with stuff.

[00:29:58] And the folks who are maybe thinking of more of a short term cost center are looking to replace their human agents with AI agents.

[00:30:07] And in your case, what I'm hearing is that you're using AI to augment your human team and give them superpowers as opposed to replacing them.

[00:30:17] So tell me if that's right or wrong.

[00:30:20] And also, what was the thought process behind that?

[00:30:23] You know, we're at a crossroads right now where AI is moving so quickly and evolving so fast that we need to determine what is it that our customers want.

[00:30:37] I can't answer that question, but my customers can.

[00:30:41] So by surveying them, we understand the dynamic that they wish to have.

[00:30:45] You know, the fact is that my bots right now are dealing with, you know, haven't left the dynamic of Uncanny Valley.

[00:30:55] They're still kind of weird and kind of creepy and they're not humanistic.

[00:30:59] And we make sure that when you speak with our chat bots, for example, they represent themselves as, I'm just an automated assistant.

[00:31:07] They're not human because if they tried to pass themselves as such, that would look weird and would be creepy.

[00:31:16] So we're at also a point where we know that in many cases, AI can still make mistakes.

[00:31:23] And so we need to make sure that those that we've hired and spent a lot of money training can, in fact, become even better specialists in what is required to support the fan.

[00:31:39] Now, does that mean in a year or so I might need fewer of them?

[00:31:43] Yes, because certain things that are very rote can be supported by generative AI and things like RPA, where if it's what we call MACD's moves, adds changes and deletes to an account.

[00:31:56] I wish to cancel.

[00:31:58] You know, a bot with RPA can do that.

[00:32:01] I wish to upgrade to a different plan.

[00:32:03] We'll take care of that for you and a bot can do that.

[00:32:05] But if it's something that requires conversation and deeper dive discussion, a description of an issue that must be troubleshooting, it's going to require their agent.

[00:32:20] Now, I'm a practitioner of customer service.

[00:32:23] What can I use to improve upon and augment my agents, make them better?

[00:32:30] It's an easy ask.

[00:32:32] I then interview, I'm again, I am the bedside ethnologist and I ask my agents, what's making you less productive?

[00:32:41] What's making you less efficient?

[00:32:44] What's negatively impacting your image as a care agent, right?

[00:32:48] And what's concerning you with regards to safety, security and stability?

[00:32:52] We call that model the RIPES model.

[00:32:54] It's also known as the SPIRE model.

[00:32:57] I memorize it as RIPES.

[00:32:58] Revenue, image, productivity, efficiency, safety, security and stability.

[00:33:03] So I ask my agents, what is hindering in regards to any of those elements?

[00:33:09] Well, what would that be?

[00:33:10] In my knowledge base or I take it back in my situation where if I were a care agent, if you ask me a question and I'm new to fantasy football, I've just been certified.

[00:33:22] In a shop, you ask me, I have a question about, you know, when brackets lock or something like that.

[00:33:30] I probably have to go and type that question in to the knowledge base and wait for a macro, an article that might describe it.

[00:33:39] Wouldn't it be easier if generative AI was just listening and swooped in the answer, not just as a macro, but elegantly wrote it out as if I, the care agent, were saying the answer to you?

[00:33:52] That would turn up time.

[00:33:54] Efficiency makes me more efficient.

[00:33:56] Also, at the end of the conversation, I have to summarize the conversation at the end, right?

[00:34:03] That should be something that the AI does.

[00:34:05] And immediately when I say, I appreciate your time, have a great day, boop, it immediately puts in the summarization.

[00:34:13] And then it dispositions it too because I have to build back to other departments.

[00:34:17] So those things make our agents much more efficient.

[00:34:22] Thus, they can take more contacts and they result in better CSAT scores and they're happier.

[00:34:28] That means I need fewer agents.

[00:34:30] That's a positive.

[00:34:32] So am I still a center?

[00:34:34] Yes.

[00:34:35] So can AI be used to make me profit center?

[00:34:40] Most definitely.

[00:34:41] Some companies have the ability to cross-sell and upsell.

[00:34:45] They have product sets that care can constantly do that for.

[00:34:48] And that's great.

[00:34:49] Some companies don't.

[00:34:51] And if they don't, maybe there's other ways to add value.

[00:34:55] And that also helps move you to a profit center because when it comes to the life cycle of the customer and how much they spend,

[00:35:01] if you're able to quantify how long they are using the service or using it more, that quantifies us as a profit.

[00:35:08] So in what we do, for example, our care agents, of course, they can upsell an opportunity.

[00:35:14] So we're using generative AI to analyze the conversation and determine other moments that are a plus one.

[00:35:24] What does that mean?

[00:35:25] If we're talking about fantasy and you've asked me that question about when do rosters lock and, you know,

[00:35:33] what is my waiver order because I need to pick up a running back because my current running back or quarterback, let's say, is injured, like Dak Prescott said, injured, you're going to have to pick up a replacement quarterback.

[00:35:47] Well, wouldn't it be great if I explained to you the answer?

[00:35:51] But then the generative AI in my CRM system swoops in one, two, three articles about who to sit and who to start this week in fantasy to make you a better fantasy manager.

[00:36:06] And then I can share those articles with you.

[00:36:09] One might be a free article, but then I can share the other two and say, hey, Shope, if you like that article, that's great.

[00:36:15] Here's some more insight that will make you a better manager.

[00:36:18] And these two sit behind the paywall because they're on ESPN Plus.

[00:36:22] If you're interested, I'd love it if you subscribe to ESPN Plus if you found this valuable.

[00:36:26] What we do is we're able to floodlight those articles and track all of them.

[00:36:30] So we can see, did the customer click on those articles to read them and did they buy?

[00:36:36] And that's pretty impressive to see click to conversions that air for you.

[00:36:41] So now we're using generative AI to offer extras to make you a better player in the game and a better fan of ESPN because we're always looking out for you, the fantasy manager.

[00:36:51] So you win your league, you win your week because we're offering you exciting, interesting content.

[00:36:57] And I just didn't solve your issue.

[00:37:00] I actually solved for something else.

[00:37:03] I kept ESPN sticky.

[00:37:05] I made sure you understood how to manage the waiver wire and how to work in waiver order, a selection of a quarterback.

[00:37:14] But then I said, you know what?

[00:37:15] I'm not a prognosticator.

[00:37:17] I don't write articles about who are the best quarterbacks to pick up for this week.

[00:37:22] If I said that to you, I'd be fired.

[00:37:24] That's not what Gary just did.

[00:37:25] But I can say, hey, here's some great articles by our award-winning writers that you might benefit from and allow you to win your week.

[00:37:34] How amazing is that?

[00:37:35] I personalized it for you and made the experience better.

[00:37:39] So we're doing a lot of that.

[00:37:41] And suddenly we're seeing those conversions, which quantifies us as trying to make sales and acting as profit-centered.

[00:37:48] So we're doing a great deal of work to make sure that we enhance the fan experience by giving them tools to be a better thing.

[00:38:07] Farhan from Shopify is back.

[00:38:09] And he talks about how Shopify is using generative AI in their product to improve their merchant experience.

[00:38:16] Merchants, of course, are Shopify's customers.

[00:38:19] And help those merchants with focusing on growing their online stores instead of some of the toil required with operating an online store.

[00:38:29] No one is not using Gen.AI now, right, with the launch of ChatGPT and these tools.

[00:38:35] I think there's a few different ways to think about it.

[00:38:37] One is what are all the things that we're building for our merchants, right?

[00:38:41] We want to make sure that we can save them time and they can spend all their time building great products and building their community.

[00:38:47] And so right away we had something called product descriptions, which is making it easier for them to generate these descriptions that are SEO-friendly and vibrant for buyers.

[00:38:56] We have a tool, Inbox, which is a way for our merchants to offer this to their buyers.

[00:39:04] So a buyer comes to the storefront and asks questions like, where's my order?

[00:39:08] Can I exchange this for red?

[00:39:10] Is there a discount?

[00:39:11] And an LLM could reply quickly.

[00:39:14] And depending on your workflow, you might want to read it first or you might want to have an auto-reply as a merchant.

[00:39:19] But again, it makes that workflow much easier.

[00:39:22] Toby put out a cool video about Sidekick.

[00:39:24] It's kind of like your co-founder.

[00:39:26] And you can have it help you run your business.

[00:39:30] So again, you're not alone, right?

[00:39:32] And Sidekick is really your person to help you manage your business and do admin operations in Shopify via chat.

[00:39:41] So there's lots of these examples.

[00:39:43] And then for employees, right?

[00:39:44] GitHub Copilot.

[00:39:45] We're testing other tools from other companies as well to see can we remove mail from people's work, right?

[00:39:54] We use AI summaries a lot in the company where you might have a lot of text.

[00:39:58] You might want to just summarize it or you might want to generate ideas.

[00:40:01] So there's all sorts of places that we're trying to take advantage of this.

[00:40:06] And I think it's still very, very early days.

[00:40:08] Like there's a lot of toil that's being reduced, but there's a lot more to come.

[00:40:13] Thank you so much for tuning in to this best of episode about generative AI.

[00:40:24] Stay tuned as we roll out more of the best of 2024 episodes.

[00:40:29] We compile insights and stories about things like delightful developer experiences,

[00:40:34] OKRs, product roadmapping and prioritization,

[00:40:38] leadership and culture in product organizations, and a lot more.

[00:40:43] Make sure to hit that subscribe button so you get notified when those episodes launch soon.

[00:40:55] Thank you for joining me on the Convergence podcast today.

[00:40:59] Subscribe to the Convergence podcast on Apple Podcasts, Spotify, YouTube, or wherever you get your content.

[00:41:07] If you're listening and found this helpful, please give us a five-star review.

[00:41:10] And if you're watching on YouTube, hit that like button

[00:41:13] and tell me what you think about what you heard today.