Cold Open And Big Changes

SPEAKER_01 0:04

I don't know. I'm not sure the world's ready for it.

SPEAKER_00 0:06

Like, I have these tools, but I have no sort of foundational understanding of how they knit together.

SPEAKER_01 0:11

What I'll say is there's things that have rewired how I think.

Welcome Back And New Roles

SPEAKER_00 0:18

Hello world. Welcome to the Tech Overflow podcast. I'm Hannah Clayton Langton.

SPEAKER_01 0:23

And I'm Hugh Williams, and we are the podcast that explains technology to it's either Curious Minds or Curious People, Hannah. I think we've used both this season.

SPEAKER_00 0:31

Yeah, curious people have curious minds, so I think we can sort of swap between the two.

SPEAKER_01 0:36

Yeah, but I like the Curious Minds. I think you dropped that maybe episode seven or something, and I've kind of stuck with it. I really like it.

SPEAKER_00 0:41

Okay, we'll we'll officially make the change for season three, Curious Minds.

SPEAKER_01 0:45

Sounds good. Sounds good. It's nice to see you. It's been a while.

SPEAKER_00 0:47

It's been a while. It's great to be back in the studio together. Unfortunately, it is the virtual studio, but um, I'll take what I can get. And yeah, it's do you want to tell the listeners what you've been up to?

SPEAKER_01 0:56

Yeah, I think uh sometime around episode seven or so, I actually took a job in the United States. So I've moved back to LA where I'm spending most of my time and uh working for Paramount. So it's uh it's pretty exciting. It's good to be back in the game, Hannah.

SPEAKER_00 1:08

Back in the game. It's so tech guy that you just like had to go on very short notice out to the west coast of the US to take up some big, it's like an exec role as well. Hugh is so humble, guys. Hugh, what is your job title?

SPEAKER_01 1:20

Uh I think it's executive vice president, executive in residence.

SPEAKER_00 1:24

Yeah, that's so freaking cool. Anyway. How are you? How are you? How have you been? Well well, one more point on LA because it really makes me laugh. Hugh guys is like always in a Waymo. I have a friend and colleague who had a call with Hugh the other day, and I was catching up with her, and she was like, she was like, Hugh was in a Waymo in the call, and I could like see LA like through the palm trees through the sunroof. And I was just like, Yeah, he's just loving his best Waymo life.

SPEAKER_01 1:51

I am. It is such a great product. Uh I've uh I think I've taken 45 rides now in four weeks, Hannah. I'm basically using it all time.

Waymo Robotaxi Realities In LA

SPEAKER_00 2:00

Yeah, guys, that was 28. Last time was it, it wasn't even that long ago. Okay, this is strong for me to do it. And you did a quick video for socials about the unexpected challenges of the Waymo's. I mean, you obviously are raving about the customer experience and are very loyal now. But what were the for people who haven't seen socials, can you just run us through that feedback? Because it's quite interesting.

SPEAKER_01 2:18

Yeah, I noticed two things, Hannah. Um, the first is because there's no social pressure inside the car, people behave badly. So I've hopped in quite a few Waymo's and there's been rubbish in the Waymo. Like I jumped in one and there was like half-eaten chicken wings wrapped in a napkin just dumped on the ground. There was another one that's just stunk of weed, which is uh maybe a bit of an LA thing. But um, you know, like people, because there's no social pressure, because there's nobody else in the car, you can kind of do whatever you want. And unfortunately, some people do. So I think that's a hard one for the Waymo folks to solve. I'm not sure, but needs needs work for sure. And the second thing's really interesting is that there's a lot of Waymo's in LA. I particularly see them on Beverly Boulevard in the mornings, like they just stream down there. So I'm assuming their parking lot for the Waymo's is somewhere there. So you can sort of see them streaming out to go to work, as it were. And I think LA people, because there's so many, have learnt that you can basically cut off a Waymo, you can pull out on a Waymo, you can cross the road in front of a Waymo, and the Waymo will be so polite that it'll let you do it. So, you know, if you don't get honked at by somebody who, you know, is just driving their regular car when you're pulling out into the traffic, you just pull out in front of a Waymo and the Waymo just gently breaks and lets you in. And LA people have totally figured that out.

SPEAKER_00 3:25

That sounds frustrating for the Waymo passenger.

SPEAKER_01 3:28

Yeah, yeah, though, such a polite car though. It, you know, it's it's a confident driver, but it's also such a polite driver. And I also think you because you're not engaged with the you know, the Uber driver or whatever it is, you you can you just sort of find yourself looking out the side window, or maybe you open up your phone or you sort of read a book or something and you pay very little attention because there's nobody to interact with. So it's a little bit like just you know traveling in a capsule or something. It's it's a really different experience once you get used to it. So I'm off in LA, I've got this brand new job. What about you? What's news in your world?

Product Work And Last Mile Logistics

SPEAKER_00 3:57

Well, I also have a brand new job, which is very exciting. So I have just accepted a product role in my current company. So I'm gonna be a product director at Ocado. I don't even think I've ever mentioned that I work for Accado. If you're a UK listener, you you'll know us from the retail business by actually work for the Global Technology Company where we do end-to-end um solutions for online grocery, including really cool robotic warehouses and front-end and super smart last mile. And I will be a product director in the last mile logistics space. So that's a real moment for the podcast, I'd say.

SPEAKER_01 4:30

I'd say it's a great moment for the podcast. And the last mile is super hard too. So, you know, I think that's where a lot of the margin gets eroded in the business, right? Is getting it out of the van and getting it up the stairs and waiting for the person to open the door and you know, unpacking and all those kinds of things. So, you know, good, hard problem, I think, for you, Hannah.

SPEAKER_00 4:48

Well, and I'll be working mainly in what we call the delivery experience space, which is like how new customer missions, so super short lead times or more flexible window slots rather than like a one-hour slot, how that all sort of plays through the system so that you can still optimize for your last mile, but you have a cohesive customer experience and you might fulfill from different locations. Anyway, it's I'm really excited about it. And when I started the podcast, I didn't really know what product was, I have to say, even though I work a lot with our product management leadership team. So yeah, it's a real coming together moment of all the things I've learned from you, Hugh, and the listeners will have learned a long way.

SPEAKER_01 5:25

Makes the podcast worthwhile because a reason for starting it when I think back to you talking me into it was hey, I want to learn some stuff about tech. Can you explain it to me? And somehow that turned into a podcast and and now it's had a real tangible outcome. This is super cool.

Season Two Highlights And Favourite Moments

SPEAKER_00 5:37

Exactly. And it's maybe that's a good segue to sort of a a few reflections from us because I keep thinking, like, we're nearly done with season two. It's been, what, it's been about 18 months. And I think back to that original like problem I had to solve quite a lot because I have learned so much. And it's almost like once you know something, you see it everywhere, and you realize that you sort of go over your head the rest of the time. But like, I not only am able to apply the learnings that you've taught me and the listeners like on this journey, but also I'm able to engage better with the next question or the next topic, which is super fulfilling and like basically achieving the mission. Yeah. What what about you? How end of season two? What are your thoughts?

SPEAKER_01 6:15

I really enjoyed season two. I feel like um, and I'd love to hear what our listeners think and also what you think, but I feel like we're much more confident podcasters. I feel like we're putting together episodes that are really, really great. I just love the interviews that you did the last couple. I I feel like you're such a natural. Um, I really, really enjoyed you speaking to Derek. And and then last week the interview with John was was fantastic. And I reflect back to the Mez episode where we Mez and I had a great conversation and we learned a ton, but one thing I figured out is you're a better interviewer. I think we're I think we're really we're really onto something, Hannah. I'm looking forward to a season three.

SPEAKER_00 6:47

Yeah, I've had amazing feedback on this season. Season one was a I think a really important sort of cutting our teeth on all things podcasting and all things working together and what sort of the right way to pitch technical topics to curious people or curious minds was. But um, yeah, it's felt like a great build. I was gonna ask you your favorite episodes, you maybe have already given that away.

SPEAKER_01 7:09

Well, they're the ones I guess I enjoyed the most because I wasn't terribly involved in the production, you know, and so it was really great for me to just be a listener and listen to you speaking to some some smart experts. So I thought that was fantastic. But I'd say looking back, I think there was a point after our first episode came out where we just you and I scrambled and we said, let's do an episode on vibe coding. And we took the brave step of coding live and running something live and recording that. And it was a it was a very organic episode that just sort of happened, and I thought it was great. I loved that episode.

SPEAKER_00 7:39

It's our best performing. I mean, it came out early, so we'll have to see the stats, but um, that's definitely top of the leaderboard at the moment, right? For season two.

SPEAKER_01 7:47

Yeah. And I just think the spontaneous nature of it, the fact they had a demo, it was really organic and just happened, I think um made it a little bit special in my mind. Because we obviously do a lot of prep for most of the other episodes, right?

SPEAKER_00 7:58

Yeah, that one was it's quite interesting, actually. Speaks to maybe a broader piece that the advent of AI and LLMs really in the public consciousness has timed really well with the sort of genesis of the podcast. And that was a sort of like coming together of, as you say, like us having a really good time recording, but also something that's super relevant to pretty much everyone right now, whether or not they would identify as a curious listener wanting to learn about tech, like it suddenly hit the mainstream in a really big way. So that was a fun way of sort of making the most of that and and learning. And I have some thoughts on vibe coding, which I'll get to later because I've done a little bit more experimenting since we last spoke, but that was for sure fun.

SPEAKER_01 8:35

And what about your favorite episode? Do you have a favorite episode?

SPEAKER_00 8:38

Uh uh, Derek, Derek Connell was just such a dream to record with. Um, I know he's a listener, so hi Derek. But um, honestly, that that's the Tay story is it's a pretty crazy story. It's and it's so relevant. Like I'd I'd heard some of it before because I've worked with Derek for a little bit, but um the relevance to some of the AI learnings that the industry took. So interesting. And Derek just he's a great storyteller. He was we had a lot of fun recording in person, which I I obviously love an in-person record. So that was one of my favorites. Um, yeah, I think I'll put that as my favourite. John, I also, John Lilly, last week's episode. John also tells a great story about what it means to be an investor. And and he's got some crazy, I mean, you guys will have heard last week, and if you haven't go back and take a look, but like the way Derek was the same, the way they reel off, like, oh, Mark and Bill, and like, you know, obviously Mark Zuckerberg and Bill Gates. It's it's really kind of like, I can't believe I'm speaking to these people, pinch me type moment.

SPEAKER_01 9:30

Yeah, yeah, yeah. And how about owning Instagram for four days and doubling your money and Mark Zuckerberg, you know, talked them into selling the company to him. And I mean, just crazy stuff, right? Great stories, true Silicon Valley stuff.

SPEAKER_00 9:41

And what about favorite moments more generally? Like, I definitely have mine, but I'm interested to see if you say the same one.

SPEAKER_01 9:47

Um, I don't know about a moment, but what I'll say is there's things that have rewired how I think. So I remember deep in the Mez episode, it it may not have really landed with many of our listeners, but Mez said that education's fundamentally changed. And and you know, and that meant a lot to me. You know, I'm involved in lots of charity work in the education space, and I know that schools are struggling with how to use AI in a school and these kinds of things, reasonable things that you'd struggle with. And Mez made the point that most of us now, when we want to get educated, just turn to our LLM and ask a question. And so it's actually a profound education tool. And that really just landed with me that that's one of the things that's really changed in the last little while is that LLMs are a primary source of education. And I think that's very, very challenging for universities, for schools, for how we think about learning, you know, lifelong learning, all those kinds of things. And so that's just been buzzing in the back of my brain since he since he uttered that sentence. It's changed how I think.

SPEAKER_00 10:41

That one stuck with me too, because we when we think how LLMs are changing the face of many different aspects of our lives. I'm not in education and I don't have any children education. My mom's a teacher, but she's um primary school. So I don't know how LLM forward primary schools will be. Hopefully, it's an exciting thing and it means more access and democratizes, you know.

SPEAKER_01 11:01

Access to education, yeah, absolutely. I mean, I think it's it's so profound. I mean, we're all talking about coding and productivity at work and these kinds of things, but I think the change in how we get educated is really, really profound. So I I was I was just it really got my brain buzzing when Mes said that. What so what about you? Favorite moment?

SPEAKER_00 11:17

Mine is so much less philosophical, but it still blows my mind. So, guys, he was in New York and he was recognized on the streets of New York by a tech overflow listener.

SPEAKER_01 11:28

How cool is that? I feel like our podcast's getting out and about, and nice to bump into a listener. We took a selfie, I know uh it's a friend of yours, and I think she sent you the picture and all that stuff.

SPEAKER_00 11:37

So Well, yeah. So here's the flip, here's the flip side. Um, so she, Monica, hi, if you're listening to me, she is she's actually my oldest friend in the whole world. And I woke up at 6 a.m., bit groggy, and I opened my phone and there's a photo of you and her that she sent me. And I was like, is this AI? Like, what is going on? But yeah, she's she's New York based. She had it was funny. I asked her, like, did you wonder why Hugh was in New York? Like famously, he lives in Australia. And she was like, No, I just it just really looked like him. So yeah, and then you guys got a selfie, so that was fun.

SPEAKER_01 12:06

Yeah, yeah, it was awesome. It was awesome, made my day. So thanks, Monica. That's awesome.

Trying To Build An AI Agent

SPEAKER_00 12:10

Yeah, amazing. Okay, so it wouldn't be season two wrap if we didn't talk about AI, which has ended up being a sort of repeat feature topic for us for obvious reasons.

SPEAKER_01 12:20

Had to be.

SPEAKER_00 12:21

Yeah.

SPEAKER_01 12:21

Had to be, right? Yeah.

SPEAKER_00 12:23

And I have been on an AI journey, and I kind of want to tell you a little bit more about that because um we haven't really chatted since I tried to build my agent. Spoiler.

SPEAKER_01 12:33

Oh, no, we haven't. Tell me more.

SPEAKER_00 12:36

Okay, so I was inspired by Mark Cargis, actually, another New York resident and a friend of the podcast. So I was catching up with Mark. Hey Mark. Yeah, and he was telling me that he uses his agent to book his pickleball courts because they book up really quick and everyone knows when it opens, and they just jump on it and you can never go to court. And so he, Mark being Mark, was like, Well, I can just solve this problem with technology. And he built an agent and he was like, I now have too many pickleball courts, like I have to cancel them now. Like, what a flex. Anyway, so I was like, this is a great idea for my reformer Pilates. Like at my gym, the Reformer Pilates books up. I don't even know when those classes open, but it's very rare to get a slot. And so I was like, I'll build an agent to help with my reform Pilates booking. So super excited. I sat down one Sunday, I had like everything ready. And I have to say, it wasn't quite as smooth a process as so a lot of the people that have encouraged me to vibe code are like yourself, Hugh, or my boss James. Like they're like engineers, they're like engineers with executive careers. So you guys are like very well qualified. You all say things like, yeah, anyone could do it. Like, you know, you could figure it out, like you're smart. And here's the analogy that I have landed on. Like, I feel like someone has told me, yeah, these new skis, like you don't even need to learn how to ski. You can just like they'll just take you down the mountain and you'll like love it and it'll be really fun. And then I've arrived in a ski resort with a suitcase full of equipment, but I don't know how to put the boots on. I don't know how to put the skis on. And like I have these tools, but I have no sort of foundational understanding of how they knit together. So like I have I had a visual code viewer on my computer and I was like dropping all this code in, but it was like, put this text in the terminal. I'm like, what's the terminal? Like I was sort of just going back and forth between Claude and this code terminal. Anyway, so I did build an agent, but I didn't manage to build an agent advanced enough to put my Pilates. So but I have built an agent that's integrated with my my phone calendar. But yeah, what's your reflection on that? Like, I I think there's a sort of context gap for people who like me are enthusiastic, but like don't really know where to start.

SPEAKER_01 14:38

Yeah. I was talking to my friend Emma the other day about this too, actually. And I think I think the interesting thing is, you know, when you've when you've done a lot of software engineering, you kind of know what's easy and what's hard. And when something's hard, you kind of know where to start. Like you think, oh, here's the thing I should build first. This will be the simplest thing to get working, and then you kind of get that working, and then you build the adjacent thing, and you end up stitching it all together and you end up with a solution. And I think if you haven't done that, like if you're not a software engineer and you don't think that way, you kind of can't tell easy from hard. And you also perhaps can't think through sort of where to start and where to end. And so you end up just expressing what it is you want without expressing any of the how of how to actually do it. And then it gets you can get pretty stuck, I think. So it's good when you want to make a web page to you know figure out the cost of a meeting, like we did back in episode two. But I think when you try and do something complex that you want to run all the time and keep checking for the Pilates and then you know, carry out a booking and log in and send you an email and whatever else, you know, it's a little bit harder than it seems, I think.

SPEAKER_00 15:39

Well, Mark made it sound easy because he was like, Oh, you can just save your login details for your gym app in this, like, I won't even, I'll technically botch this explanation, but I understood it was like the agent doesn't even need to know your login details, you can like store it in the separate type of file and then it can just access it and like plug it in. And that does sound very sensible, but I think I needed to maybe walk before I tried to run. Well, in fact, this is even more embarrassing. I think maybe you'll find it funny. I tried to like build a virtual machine, like I did was I like created an AWS, like I set up this whole account. I the one thing I was able to do intuitively was I was able to link my bank account to the AWS platform that I was using. That bit is very if something discharged me directly, but I was never able to get anything up and running. So yeah, I think I need to maybe we could do an episode on it or even like a sort of, I don't know, like a how-to video.

SPEAKER_01 16:29

That'd be a good episode. Why don't we try and build it and I could I could kind of coach you through the building and then we edit it up to be an episode? That'd be a good episode.

SPEAKER_00 16:36

Yeah, because I know that people are curious, they want to use it. Like I would love to have some agents doing things more proactively for me because that feels like the difference, right? But I just need to situate myself a little bit better with the tools that I've got. And and it's an interesting thought in a world where companies are really promoting AI in-house. So, like, we have a big effort at the minute for the what we're calling like business operations folks to better leverage AI because there's some really interesting use cases in some of the support spaces in sort of typical back offices. But people like me, like with the best will in the world, we still need some better coaching. And I think there's been a little bit of a gap where like it's lowered the barrier to entry so much, the engineers are super excited, but it's not lowered it all the way.

SPEAKER_01 17:20

No, no, I think you're totally right. You know, people are assuming that it's some kind of autopilot that you can just deploy and magic will happen. But the reality is it's still a co-pilot, right? It still sits alongside you and you have to kind of work with it, not you know, not abdicate the task to it. And and so I think people haven't quite figured that out yet.

SPEAKER_00 17:36

Okay, well, my goal is still to build that Pilates booking agent. So we can definitely have another go at that.

SPEAKER_01 17:43

Yeah. My hot tip would be don't do the booking part yet. Just keep checking the page to see if the bookings are open and then get it to send you a text or an email or something. I'd start with that and then you do the booking yourself and then work up to the booking.

SPEAKER_00 17:54

Okay. Yeah, yeah, yeah.

SPEAKER_01 17:55

Break it down. Break it down.

Token Usage And AI At Work

SPEAKER_00 17:56

Yeah, yeah, yeah. Okay, awesome. So more on that to come, guys. Maybe I'll license my um Pilates booking app when I actually eventually have it. But I think the topic of AI and workplace is interesting because my husband's workplace is going really hot on like you must use AI, and if you don't use it enough, we will tell you you're not using it enough. But the way that they measure enough, quote unquote, is like token usage. Maybe we should just you could just break down what that actually means in practice. But my view is that that is a pretty blunt instrument. But maybe you could explain what token usage means in the context of AI, and then your thoughts on that as like the metric for War Places to check people are using it.

SPEAKER_01 18:32

It's an interesting metric. Look, I think there's probably two things you could measure. One is the number of prompts that somebody had issued, which may be a proxy for the amount of time they've used it. The other thing you could measure is the complexity of the tasks that you're getting it to do. Counting tokens is a good way to measure the complexity. So I guess it separates people doing shallow things from people doing complex things. So, you know, if you if you ask it to build an app for you, you will use a lot of tokens. If you ask it to just quickly edit an email or something, then you won't use terribly many tokens. And that task will finish quickly. So it's really a measure of of task complexity. So I guess what they're trying to do is figure out who's doing uh who's doing complex and important things versus who's doing shallow things.

SPEAKER_00 19:09

And a token is like a measure of compute, right? That you're using up. Is that a fair way to describe it?

SPEAKER_01 19:16

Yeah, I think that's a reasonable way to say it. And certainly these companies sell usage by the token. So it's a reasonable proxy for the amount of work that they're having to do.

SPEAKER_00 19:24

Okay, but then I have so many questions that come out of that. So the the tokens is an interesting point because when I use Claude, I use up my free tokens pretty quickly. And then when you we actually had a few questions come through about this, when you sign up to a subscription, you're basically like paying for a number of tokens per month or per day. I don't know how it works exactly, but people must be, I'm sure this must be happening. If people want to do cool, exciting things with AI and they know that they have tokens at work and they don't want to pay for the tokens on their own time, there must be a ton of people playing with their cool side hustle ideas on their work clawed account, surely.

SPEAKER_01 19:59

I'm sure. I'm sure. I think you know, people will uh people will find the most efficient path. I'm not sure it's the greatest idea given an IP questions and all that kind of stuff. If you've been in on work time on the work computer, work definitely owns it in almost every jurisdiction. I know enough about IP law to know that one.

SPEAKER_00 20:14

That's an interesting point. I'm desperate to do an IP episode. It is much more interesting than it sounds, guys, if you don't know anything about IP and technology. Um, so there's a little teaser there. But um, AI at work, again, my view is that well, my view is twofold, and I'd love your thoughts. So within this sort of engineering and coding space, there's a really clear path to efficiency, and there's still a lot to figure out, but like I think it's pretty lined out for folks that there's some big change there coming. And then in the sort of broader use case for LLMs at work, I'm not sure in the business operations space, and that's quite wide. Like I run a strategy team currently, like that's a very different use case to maybe like the accounts payable, which might be more straightforward way to leverage AI. But I'm not sure anyone's like really stamped out exactly the best way of using it. And I guess there's a learning journey and getting your hands dirty is an important way to learn. But I'm not sure I'm convinced when you hear some of these companies saying, like, oh, we've cut this team because it's now gonna all gonna be done by AI, I'm not sure they've worked out the how yet. I think it's just more like a oh, we we want to do things cheaper. And maybe that's right. Maybe necessity is the sort of the master of problem solving. And if you cut people off, then they'll figure out a way to do it with the models. But uh what's your thoughts? Have you do you think anyone's cracked it?

SPEAKER_01 21:25

I don't think anyone's really cracked it, Hannah. Uh look, perhaps the the large AI companies themselves have cracked it. I mean, you would hope they're kind of using their own product uh in in interesting ways, but I would say most companies definitely haven't cracked it. I think what most companies have done is they've gone into this thinking that AI can replace people and then they've figured out the hard way that really it's it's something that sits alongside people and helps people. And so I just don't think, you know, the dream, if you like, has has quite become a reality for most companies yet.

SPEAKER_00 21:54

But I do think getting your workforce on it and experimenting is the way to get there. So it's not to say anyone's Doing anything wrong. I just think we all need a little bit more time to iron out what the new world of work looks like. I guess in a way that we all started working from home and everyone was like, oh, is remote working the new thing? And now it's turned out that, well, it's kind of a hybrid. And in some industries, it's just not a thing at all. And in some industries, it's not even hybrid, it's like fully remote. So there's sort of a spectrum that we need to figure out along the way.

Listener Q&A On LLM Tiers

SPEAKER_01 22:20

Yeah. Yeah, that's it. So Hannah, it's become a little bit of a tradition, maybe. When we certainly did it in our season one rap, that we do a little bit of listener QA in our rap episode. What do you think? Do a few questions?

SPEAKER_00 22:32

Yeah, let's make it a tradition. I've got some of them in front of me. I'm going to pick and choose the ones that I am most interested in hearing the answer to. So first one is a little bit of a continuation of the conversation that we were just having. And we had it in a few different forms, actually. But the question was just like, can you explain the difference between the different tiers of LLMs? So, like I think we mentioned the different paid subscriptions and the security levels if they change with the different tiers. And that one was from Sam. Thank you, Sam.

SPEAKER_01 22:58

Hey Sam. How are you doing? Uh it's a good question. It's a good question. So maybe I'll give a sort of a Claude answer to this, but the answer generally applies to Gemini and OpenAI and ChatGPT. So right now, Claude has three tiers of model. I think our listeners will have heard of them. So Haiku, Sonnet, and Opus are the three tiers of models. Opus is the big model, if you like. So small, medium, and large. The big model is Opus, and that's the model that you use for, you know, reasoning, writing code, uh, building an agent. You know, it's the expensive model that you use for the hard-hitting tasks. The mid-tier, which is Sonnet, uh, is used for sort of the bulk of kind of real work. So it's a bit cheaper, it's a little bit faster. And then the small model is for the high volume, really low stakes stuff. So if you're looking to do lots of work, low stakes, you want to do a lot of processing, you want to do it very quickly, then Haiku is for you. So different kinds of model tiers of capability. All of the companies have those tiers. The other thing I should say is you can actually download a model and install it on your own computer and not use them at all. And those models that are local on your computer, so you might install that on your laptop, maybe you've got a Mac Mini, those kinds of things, those models have typically less capability than the low-end models, but they can be very, very fast. That's a way you can just avoid paying at all, is actually just get a model running on your own computer when you've got low-stakes tasks where you don't want to pay.

SPEAKER_00 24:27

Okay, I have a bunch of questions. So, more sophisticated models. So Opus, I think you said, is the most sophisticated one, that uses more compute.

SPEAKER_01 24:35

Yep, uses more compute. It's slower, it's more expensive for the company to run. They they spend a lot more effort training that model. It's their frontier model, if you like. So it's the best model that they've got, uh, the most powerful model that they have. And clearly, because of all of those things, it's the one that costs the most for you to use.

SPEAKER_00 24:50

Okay. And so if you have a subscription that gives you a certain number of tokens and you have lower stakes tasks, you want to toggle to use the less costly, quote unquote, model to get the answers that you need.

SPEAKER_01 25:02

Exactly. So you might switch to haiku to do some high volume tasks where the accuracy you need is not that high.

SPEAKER_00 25:08

Okay. And then the version where you have a model locally on your laptop, for example, you need to hook it up to something to power the compute, right?

SPEAKER_01 25:16

No, you actually run it literally on your computer, Hannah. So, you know, you literally install it on your laptop and actually run it locally on your laptop. So this is things like um, some of our listeners will have heard of Lama, they'll have heard of Gemma, they'll have heard of Mistral. Um, these kinds of models are actually models you can download and actually run on your computer. And and because they're just running on your computer at home, the model isn't as sophisticated and the answers aren't as good.

Privacy Rules And Data Retention

SPEAKER_00 25:39

Interesting. And um, what's the implication from a security perspective, if there is any?

SPEAKER_01 25:44

So obviously, if you're just running a model on your own computer at home, then the data isn't leaving your computer, and obviously, you know, it's as secure as your computer is at home. So that's that's one answer. I think the answer as to what happens to your data when you're using one of these commercial providers is a little bit complex. I think we talked about it a little bit back in one of our earlier episodes. But basically, if you're using the consumer plan, so these are things called, you know, like free, pro and max in in the case of Claude, then you can opt in and out of training. We've talked about that. So you can choose whether your data is used for training or or not. So that's one thing. If you do allow it for training, then your data will be retained for up to five years. If you turn it off, then it'll only be retained for 30 days. And if you delete a chat, then it will be excluded from future training. So that's sort of the consumer answer, if you like. If you work in a company and your company has a commercial license, whether it's for the your team or whether it's for the whole of the company, the enterprise, if you like, then you've negotiated commercial terms with that provider, right? So you've had a conversation with the anthropic guys or the open AI guys or with Google and you've negotiated some terms. And usually those terms would be things like look, you're not allowed to use any of our data as input to training. And you might have agreed some other some other kinds of terms around data retention, you know, where the data can be, can be stored, you know, which countries, whereabouts, those kinds of things. But you negotiate that commercially. And then the third way you can get into these tools is by using an API. So you can actually, you know, behind the scenes be chatting to these things. So, you know, we talked about open claw, I think, back in an earlier episode. If you're using open claw, then you're chatting to, and you decide you want to use Claude as a model, then you're chatting to it through an API. In that case, the data's never used for training by Anthropic and their data retention is very, very short. So typically using it through an API, you're getting less data retention, less use of your data than if you're using it through the consumer interface, you know, whether you've installed the app or you've gone to the website.

SPEAKER_00 27:48

We had another question from Zaniela, which is sort of of that same theme. It's a bit of a build. So if you don't have your privacy settings off, so if you're just using normal default, train the model, could someone eventually ask Claude something about me? Because Claude will know who I am and it will probably know that I'm the Hannah Clayton Knight in the works at a cardo just because I'm giving it those signals. If you asked Claude something about me personally, like would it ever spit out insights it's gleaned about me?

SPEAKER_01 28:17

I mean, look, nothing, you know, I liked how Mez sort of framed things that were impossible. He'd say it's incredibly improbable or whatever it is, but I I think I could confidently say that's impossible. I mean, it's not a database, right? So it's not like your chat is being stored somewhere and then other people can search and find your chat. I mean, that's just not fundamentally how it works. So when it comes to training, you know, your data is added to a vast pool of data. Remember, there's trillions and trillions of words to use in training, and you're added to this vast pool, and the output is a model, and the model has certain weights, and then people go and use that model to do things, right? So I think the most significant thing that could happen is you're slightly influencing style and tone and what words follow what words. So you're, you know, you're part of a giant pool. And the consumer data that's used in training is a relatively small part compared to the whole of the internet as well. Um, and so you're really very literally a drop in an ocean. Maybe that's even being generous when it comes to your chats being used in training, and and it's not like the data's retained, all you're all you're doing is becoming part of the creation of a of a model, which is a many month process, cost you know,$100 million or whatever it is. So you're you're becoming part of a model's behavior. Your your data's not really searchable by anybody else.

SPEAKER_00 29:33

Okay, and then someone else asked me, would they be selling that data? But I think the way that I would frame it in my head, correct me if I'm wrong, is they well, they already are selling the output. So they've taken all the data they can find ever, or at least a you know, huge swather data, they've used it to train the model, and then they are driving commercial outcomes from the model itself. So they wouldn't need to go and sell a load of consumer data because they've already leveraged data for commercial means and that's their sort of main product.

SPEAKER_01 30:03

Yeah, exactly. I mean, these companies are never gonna sell your data. And it's like, you know, Microsoft, Google, Amazon, Meta, these folks are never gonna sell your data. This is the moat, right? Is the fact that they they have all of this data and they can use this in smart ways to build products. So the last thing these companies are gonna do is sell data.

How Contactless And Apple Pay Work

SPEAKER_00 30:20

Okay, well, that's probably reassuring to some folks. Um, I'm gonna move us away from AI, easy as it is to be sucked into lots of discussion about it. Um, Becky asked, how does contactless payments work? It has always amazed me. And I would second Becky's question. So can you do like a 30-second version of how it works?

SPEAKER_01 30:36

Yeah, nice one, Becky. Um, let me let me do my best to do the 30-second version, though. I think I've used six or seven other seconds now, so I need to speed up. Um, your contactless card has a chip in it. When you take that card and you move it close to a payment terminal, your chip is energized by the payment terminal. So that's where it gets its power from. Um, it only works when you're about four centimeters, let's call it an inch and a half away from the terminal. And as soon as it becomes energized, what happens is the chip on the card will send through an encrypted version of your card number. And also it'll mix in the fact that this is a very specific transaction at a very specific time. So this little key that comes off your card, that comes off the chip in your card, is a one-off thing that represents the card number and this particular transaction, and that goes into the terminal. And then what the terminal does is it makes a very quick decision about should I just approve this because it's a small transaction, you know, I've seen this card before, whatever it is, or do I need to send it all the way through to your bank to get a yes or a no? And Visa and MasterCard are the kind of rails, if you like, where this communication happens. So really it's Visa and MasterCard that do all the communication between this little terminal and your bank. And when eventually this arrives at your bank, it probably takes, I don't know, 100 milliseconds, a tenth of a second. When it eventually arrives at your bank, your bank's job is to say yes or no. And obviously, no if you've got no money, it's a good reason. Um if your card's blocked, you know, these kinds of things. Um, but they can also make a fraud decision. They could say, Hannah just bought something in London and now she's in Nigeria. That seems a pretty unlikely thing. And so your bank will say yes or no. If your bank says yes, comes back to the terminal, transaction approved.

SPEAKER_00 32:17

Okay, but I have a follow-up question. Because when I think of contactless now, I think of Apple Pay, which is obviously like my favorite thing. I think I've mentioned before. I love it. So when I'm using Apple Pay to get on the tube, is the difference there just that there's like a virtual version of that chip in my phone?

SPEAKER_01 32:32

Yeah, that's right. That's exactly right. And it's a little bit more secure than a card because uh, you know, your phone has face ID or you had to enter your code in order to make that work. So you've got this added layer of security beyond a physical card, which obviously somebody could steal out of your wallet and go and use for a payment, where it's a lot more difficult to do that with a phone.

SPEAKER_00 32:50

Well, and I think that's why at least in the UK, there are limits on contactless payment. There used to be like a 30 pound limit. I don't know what it is today, but like after a certain amount of spend, it would need you to enter your PIN number. But with Apple Pay, that doesn't apply because they've verified using your face and that's pretty secure. So you you end up sort of never needing your pin ever, and then sometimes I need it because I'm using my card for some reason and I feel like a criminal because I don't know the PIN number.

SPEAKER_01 33:16

No your PIN number. Yeah, yeah. And again, like if you go back to the original explanation, right? When that chip or your phone gets very, very close to the terminal, there's a routing process that goes all the way through to your bank, and your bank can respond any way it wants. And one way it could respond is say, Hey, you need to answer the pin. I want to see the pin before I'm going to approve this. It could just approve the transaction, it could block the transaction, it could send you a text, it could cause somebody to call you, whatever it is, right? So your bank can make a decision and that decision ends up getting all the way back to you at the terminal.

Overhyped Tech Trends VR And AR

SPEAKER_00 33:44

It's crazy. Like if you think about how quickly things are changing. Okay, um, Sarah has asked us what tech trend is currently getting way more attention than it deserves.

SPEAKER_01 33:56

You know what I'm gonna say, Sarah, and thank you for the question. Is uh VR and AR. I was walking down, it's name-dropping a bit, Hannah, I was walking down Melrose Place in West Hollywood the other day, and I walked past the Meta store. So Meta's got a giant store there, and it's just full of Ray-bans with, you know, cameras in them, this whole AR thing, and there was nobody in there. So it tells you all you need to know.

SPEAKER_00 34:16

Yeah, it hasn't really landed and it's been around for quite a long time.

SPEAKER_01 34:20

Yeah. And I think, you know, VR putting on goggles and waving your arms in the air, you look like an idiot. Like, I don't know how that ever works and is socially acceptable. So I think, you know, VR maybe for some very specialist applications, you know, commercial applications, maybe sure. AR, you know, this idea of you're basically wearing glasses and there's a heads-up display and you've got a camera on the front and things. I don't know. I'm not sure the world's ready for it. I can see some great applications, like you're in a bar and you want to watch the sports scores or whatever, and they're projected up onto your lenses, maybe great plays from the game or whatever it is while you're at the bar and things. But I I think it's a little bit weird having a camera in your glasses and being distracted by this heads-up display.

SPEAKER_00 34:59

Yeah, it's maybe like a step too far. It's interesting. I've heard that speaking about some of the applications where it it maybe has stuck, I I have heard that it can be used for like trauma therapies and stuff, where if someone's really afraid of like spiders, you can hold a spider that isn't really a spider, and maybe that helps people like through their trauma. I should probably do that if you can remember when I we saw that Huntsman at your house. I probably need that. But um generally speaking, it feels a little bit to me like the Nintendo Wii. Do you remember the Nintendo Wii? And it was like, oh, no one's gonna do sports anymore because we're all gonna be playing tennis in our front rooms using this week. And like it was fun for a couple of years, but I don't think that really stuck either.

SPEAKER_01 35:36

No, no. So I don't know. I just think it's overhyped. It's been overhyped for over 10 years now and hasn't really gone anywhere.

SPEAKER_00 35:43

And what about um I I think this would probably be my answer. I don't know if it's still hyped, but I remember all the noise around NFTs. I don't even remember what that stood for, but it was like virtual art and people were non-fungible tokens.

SPEAKER_01 35:55

Okay, tokens. There we go again. It's basically like sticking a, you know, a little hologram ID thing on something. You know, it's it's just saying this is authentic.

SPEAKER_00 36:03

Okay, so kind of like blockchain.

SPEAKER_01 36:04

Blockchain is the underlying technology that sits under NFTs. Yeah.

SPEAKER_00 36:08

Okay, again, I mean, we could probably do an episode on blockchain.

SPEAKER_01 36:10

Please let's not do that.

unknown 36:13

Fair enough. Okay.

SPEAKER_01 36:14

Certain topics are banned. That's one of them.

Is The SaaS Pocalypse Real

SPEAKER_00 36:16

Do we have time for one more? Yeah, sure. Okay, uh, Marwan asked us, is the SaaS pocalypse real? So that was like when the markets panicked that the advent will be back on AI again, but like the advent of LLMs would massively impact the sort of value creation of all the SaaS companies, right? Because suddenly everyone could use LLMs to like make their own software. Is that a fair explanation?

SPEAKER_01 36:38

I think it's a yes and a no. Like I think the devil's in the details, right? I think you have to go company by company. I think it's very, very true today that if you've got a small SaaS product in your company that does something fairly small, that it's quite possible that somebody could vibe up an alternative to that and solve this problem some other way. So I think in a company, you should absolutely be asking, do we really need to sign this contract now or could we rebuild this ourselves? But clearly for really complex software, you know, if you think about things like workday or you think about things like Salesforce or the cloud providers and things, you can't you can't vibe code that solution unless you're only using a small part of it, in which case maybe you maybe you ask that question. But the large SaaS providers that are deeply embedded into companies, deeply wired into companies, become part of the ecosystem of that company. I I don't think they're under any significant threat, at least today.

SPEAKER_00 37:30

Okay, so that's maybe one to sort of watch.

SPEAKER_01 37:33

Yeah. Look, you know, if we wind back to season one, I kind of said that, you know, vibe coding was pretty cool, but it wasn't going to replace software engineers anytime soon. And I was still writing code, and that's changed a lot. So I think, you know, it's hard to predict the future. So maybe just come back to this one in season three, Hannah.

SPEAKER_00 37:48

Well, yeah, and actually maybe that's a nice place to end because one of the things that Mez said that has really stuck with me is that his, you know, he's obviously a futurist and his time horizon against which he feels comfortable making predictions or asserting anything, like really with force. He said that that's shortened quite a lot. And that is one of the things that stuck with me is like, well, no one really knows, and we probably know less than we did before. So maybe we all just need to live in the moment, put the V R A R away and I don't know, read a book, go read Derek's book. Go read the paper copy of Derek's book. Hugh, by the way, guys, told me earlier that he's in the book and there's like a chapter about him.

SPEAKER_01 38:27

So I don't know if there's a chapter, Hannah. I haven't read the book yet. I'm still waiting for my autograph copy to arrive from Derek. He's very kindly sent me one. So I'm gonna look interested to find out what part I play in the book. But um, I feel really uh humbled and honored to be interviewed for the book.

SPEAKER_00 38:40

So um well, we'll see how it turned out. Well, I've got the book and I started it a couple of days ago. So I will let you guys know where we land on more insight from Hugh Williams.

Season Three Plans And Farewell

SPEAKER_01 38:49

Speaking of putting things down, walking away, you know, taking a break from tech.

SPEAKER_00 38:55

He's leaving the pocket.

SPEAKER_01 38:56

No, we're just getting it.

SPEAKER_00 38:57

I'm done with season two, Hannah. So it's gonna be a break till season three. Season three. So the original listeners will know that I pitched hard for a trip to Melbourne and I did indeed go there in February of this year. So season three is gonna be all about Hannah in LA. What do you think?

SPEAKER_01 39:11

Sounds good. Sounds good. Any excuse, but sounds great.

SPEAKER_00 39:15

That's what my husband said. I think that the moment that you and I are in a Waymo together in LA will be a real coming together moment of all of this hard work.

SPEAKER_01 39:24

Yeah, we can do a bit of karaoke in the Waymo. This is gonna be great. Carpool karaoke in a Waymo.

SPEAKER_00 39:29

We didn't have you as a karaoke. This is going right. Yes, and um, we can go and visit all of the guests that we've interviewed who are all based on the west coast of the US.

SPEAKER_01 39:40

Yeah, maybe find some new guests. Um definitely interviews are really exciting. We had more interviews this season. I think they were a ton of fun. So it'll be interesting to see what the mix looks like next season, Hannah.

SPEAKER_00 39:48

Well, it will look something like California, that is what I can assure you.

SPEAKER_01 39:52

So we've been the Tech Overflow Podcast, and I hope you've enjoyed season two. We'll be still on socials. So you can find us on Instagram, LinkedIn, X, YouTube Shorts, and TikTok.

SPEAKER_00 40:04

And you can always find us at TechOverflowpodcast.com. Remember, do rate, subscribe, share us with your friends, colleagues, family. You know we're a lot of fun. Share the love.

SPEAKER_01 40:14

Yeah, absolutely. And that'll inspire us to come back for uh season three.

SPEAKER_00 40:18

Yeah, we'll see you in season three, guys. Bye. Bye, Hannah.