Hugh Williams: 0:04

A lot of companies are resistant to experimentation because involves humans letting go of control.

Hannah Clayton-Langton: 0:09

I definitely do that where I pick up the phone and then minutes later I'll be like, what was I even doing here?

Hugh Williams: 0:14

We shouldn't be afraid of gathering data and learning. What we should be afraid of is thinking we're right.

Hannah Clayton-Langton: 0:21

Hello world and welcome to the Tech Overflow podcast. I'm Hannah Clayton Langton:.

Hugh Williams: 0:25

And I'm Hugh Williams, and we are the podcast that explains to curious people.

Hannah Clayton-Langton: 0:30

We are indeed. And today we are coming to you from Hughes Place and So I'm still down under and still loving life. Today I tried my first magic, which is a type of coffee. I don't know if I'm gonna get in trouble for saying this. It was just like a really, really good flat white. Is that fair?

Hugh Williams: 0:48

I think that's fair. The way it's done is you uh you grind the coffee, put it in the machine, and just run it into one cup when you'd normally it between two, and you kind of stop it halfway and then so get the best bits.

Hannah Clayton-Langton: 0:57

Oh, yeah.

Hugh Williams: 0:58

And then you know, put in the milk, the coffee art. Life's good.

Hannah Clayton-Langton: 1:01

It was great. So, what are we talking about today?

Hugh Williams: 1:03

We're talking about Hannah, why you can't put your phone down.

Hannah Clayton-Langton: 1:07

It is true, guys. I can't put my phone down, which my husband and my sure, can attest to. But I'm hoping we're gonna learn that it's not just me.

Hugh Williams: 1:15

It's not just you. I pick up my phone way too often. In fact, I think I've got to a stage in my life where I pick it up to do something and I find myself just drawn to an app. I open up the app, I go deep into this app, and then a of minutes later, I can't remember exactly why I picked my up in the first place. But phones are super, super addictive, and we're gonna really dig into that topic.

Hannah Clayton-Langton: 1:33

Yeah, I'm not gonna reveal my screen time numbers not proud of them, but I definitely do that where I pick up the phone and then five minutes later I'll be like, what was I even doing here? And sometimes I'll close an app and I will literally navigate to it, like Instagram or maybe Slack if it's if work-related. And I've just closed it and I'm my brain is like looking for more stimulation or something.

Hugh Williams: 1:58

Yeah, absolutely. And look, the companies that build these apps are trying to you to engage with their app, stay engaged where they're at. And today we're gonna chat a little bit about the the methods they use to do that, some things around sort of and how they try and improve their apps and and their web as well, and really explain that it isn't all about you, Hannah. It's not that it's not some character failing, it's uh you big tech, and big tech is winning.

Hannah Clayton-Langton: 2:23

Wow. Okay, and is are we gonna learn something like it's just like tip-top product management that's drawing me back?

Hugh Williams: 2:29

Yeah, there are lots and lots of bright people working in large companies, and their mission is to drive engagement and figure out ways to keep you engaged, Hannah.

Hannah Clayton-Langton: 2:38

I am very engaged. Okay, so Hugh, as a product management professional, if after my quote unquote engagement, like what behavior are user?

Hugh Williams: 2:49

So I want you to use the app or the web experience in a in a way, and that's gonna depend a little bit on whether it's like a Facebook or an Instagram that's more of a social or something like Google Maps, you know, guiding you from A B or Google Search. So what engagement means is is going to vary between the apps. But inside of these companies, they are definitely in some way and they're doing some aggregate counting. So, for example, a lot of these big companies track what's DAO, which stands for daily active users. And so they're very, very interested in the number of people use the product every day, and they've probably got a goal driving up DAO over the year. And so all the product managers, all these super smart who are working in these teams, they've probably got DAO and they're probably really trying to make sure that whatever it is that they build drives a daily habit.

Hannah Clayton-Langton: 3:38

Okay, so just thinking back to one we've talked about So for a like a social network type app like LinkedIn or engagement is basically me scrolling, maybe hitting a like. Whereas for an app like Uber, which is or any ride-sharing which is much more transactional, the engagement's about me like converting, requesting a ride. Yeah. Buying something. Makes sense, makes sense. Okay, so engagement is one metric, or maybe it's a suite of metrics that might look slightly different for different But can you give us some other metrics that these companies are gonna be tracking?

Hugh Williams: 4:13

Yeah, absolutely. Great question, Hannah. So Mao is another one. You can guess what that is.

Hannah Clayton-Langton: 4:18

Monthly active users. That's it. Ding ding ding. Ding ding ding.

Hugh Williams: 4:21

And so, you know, it's gonna be a much bigger number than right? So DAO is the number of users that are using you on a day. Mao is the number of users who've used you in a particular So you're typically gonna get a much, much larger number. What often companies are interested in is the ratio between two things, right? So if you think about what percentage of the daily users are you monthly, that's pretty interesting, right? So what they call the Dow Mao ratio. So take the number of daily users, divide it by the number of monthly users, and you get kind of the percentage, if you like, of folks who are using you as a daily habit, who are in your user base that are using you monthly. And so what you'll often find in these companies, and the social companies, is that they're trying to drive up the mal ratio. So they're trying to take that latent user base who are using the app on a monthly basis and trying to activate them to be daily. So the down mount ratio is one of the other things that they

Hannah Clayton-Langton: 5:15

And presumably the mal user base is attractive because they you know you've used them before, you might have lapsed as a sort of more loyal user, and that's a sort of lower hanging fruit than having to bring someone brand new in, get them register, get them to create the habit.

Hugh Williams: 5:30

Exactly, Hannah. And you know, when I was working on Google Maps, we had about a billion Mal. So about a billion users used us monthly, but not that many used it daily. So what was what was happening there was when a user know how to navigate from A to B. So you're going to somewhere unfamiliar, you'd you'd Google Maps, you'd love it, it'd help you get to a place, but it wasn't a daily habit. So it wasn't something that you were always using to guide you through the world. It was something you were using when you didn't know how to through the world. And so we're pretty interested in the down mount ratio. How do you drive that up? How do you turn Google Maps more into a daily product?

Hannah Clayton-Langton: 6:05

That makes sense. Because I don't need Google Maps for my commute. Maybe if I'm looking at if the trains are on time, if I'm a like more exploratory travel on the weekend, then that's first thing I fire up.

Hugh Williams: 6:16

Yeah. And look, they did a lot of work on Google Local, probably after I left, to be honest, where they really did a lot on opening reviews, photos, the latest photos, getting people to take more photos. So trying to give you a real feeling of the vibe of places. And so that helped it become much more of a daily habit. It wasn't just a driving app.

Hannah Clayton-Langton: 6:36

Okay. So for all apps, you have your user base and your frequency of use. And then I assume for Amazon, it's going to be about ads to basket, that sort of thing. For Instagram, it's when I like or maybe send a post to or I hit up a story or something. You know, I add something to my story. And then Uber, it's going to be about actually ordering the Uber WhatsApp. I mean, do WhatsApp even have to track? Like, isn't everyone just using it all the time?

Hugh Williams: 7:05

Yeah, well, I guess they've got, you know, they've got some competitors. You know, we see Apple has the iMessage ecosystem, Facebook True. There's lots of people, other things that people use. So certainly if I was at WhatsApp, I'd be, I'd be tracking Mao, I'd be tracking Dow, I'd be tracking the ratio between and making sure that I was continuing to drive that up.

Hannah Clayton-Langton: 7:23

That's true. There's this really weird phenomenon where like I can be to the same person on Instagram, WhatsApp, and then like Facebook Messenger, probably less so because I've aged that a bit. But it's like a WhatsApp conversation is different to an Messenger conversation, even though it's both via my phone. That's like my boss.

Hugh Williams: 7:54

Yeah. But certainly, you know, these are the kinds of things that all these teams are tracking. And of course, there's a competitor release as a feature. So, you know, if we're working in one of these companies release a feature that's really compelling for our users, you know, the user base will shift slightly.

Hannah Clayton-Langton: 8:06

Yeah, yeah, for sure.

Hugh Williams: 8:07

And uh, you know, our job is to respond to that and make sure that we continue to drive up our metrics. I'll throw one more metric in the mix too, Hannah. Uh, there's a thing called 3d7, 4d7. Have you heard of that one?

Hannah Clayton-Langton: 8:18

Okay, no, but I'm guessing it's three days out of four days out of seven.

Hugh Williams: 8:22

Absolutely. And so you're interested in users who are fairly engaged.

Hannah Clayton-Langton: 8:26

Oh my god, I'm I'm 7d7 Instagram. I'm 8d7 Instagram.

Hugh Williams: 8:31

Well, they will love you because I bet they're tracking 3d7 4d7 to see if people are, you know, pretty engaged, using it of the time throughout the week. Um, so that's another popular metric.

Hannah Clayton-Langton: 8:42

I once dated a guy who worked for a dating app, which is a meta, and he was an analyst there and he was telling me the metrics that they were tracking. And I know that Hinge says stuff like our main objective that you delete the app, but that's obviously just marketing. And he was talking about like it was like messages exchange, and then he was like, Oh, we can tell when you've you've a phone number or something like that. And it was it was pretty interesting stuff, actually.

Hugh Williams: 9:06

Yeah, absolutely. Actually, dating's a hard one. I, you know, I worked at Tinder for a little while, and right. I mean, people are super, super engaged, and that's fantastic, and you want them to be engaged and using the product and know, carrying out the tasks, meeting people, that kind of But then when they get into a happy relationship, they vanish. You hope. Yeah, you hope they do. Yeah, happy relationship. You hope they uh they vanish. I guess we should be a bit more uh open-minded about it, but vanish, and then of course, you know, if that relationship work, they appear again and uh become super, super engaged. So often, you know, users are active for a while, inactive for a while, and then they come back.

Hannah Clayton-Langton: 9:41

I was gonna say that must be difficult to predict, but then I bet you have people who disappear off for like three weeks and then they're back again.

Hugh Williams: 9:48

Yeah, absolutely. There's a lot of that going on. And uh we used to have a sociologist that Tinder and her job to really think about people and dating and definitely had people in the team who really thought about dating from a that wasn't just tech.

Hannah Clayton-Langton: 10:00

This ultimately sounds like it's this is an example of product management, right? Like if the goal of Instagram or WhatsApp is to have me their app three or four D7, then they are absolutely that goal.

Hugh Williams: 10:12

Yeah, they must love you, Hannah. You're even uh an advocate for their ads.

Hannah Clayton-Langton: 10:17

Yeah, it's pretty sad.

Hugh Williams: 10:19

Uh so you might ask the question though, like how are they are they driving up 3D7, 4D7, the down mouth ratio, whatever is they're driving up? And the answer is they're running a lot of experiments. So a lot of a lot of experimentation is really the the if you like.

Hannah Clayton-Langton: 10:33

Okay, and so experimentation, as I've understood trying out different features of a product with different user groups, and then basically looking at how users in their behavior and basically finding out which size of button or color of a button people are responding best

Hugh Williams: 10:51

Yeah, absolutely. And you know, folks like Google and Meta who own Facebook and Instagram are running tens of thousands, if not hundreds of of experiments every year and trying to highly optimize things that, you know, are as simple as you say, as you know, fonts, sizes of buttons, whether the buttons have rounded all these kinds of things.

Hannah Clayton-Langton: 11:10

And I'm sure that if I picked up a version of Instagram was like two or three years old, it would just feel really But it's all these tiny little changes that over time evolve the product.

Hugh Williams: 11:23

Yeah, absolutely. And this is a relentless machine, right? Like this is a machine that's just running the whole time.

Hannah Clayton-Langton: 11:28

And if they've got millions or billions of users and have a super rich data set across which to run all these

Hugh Williams: 11:35

Absolutely. And you know, if you worked on Google search, you can probably get a signal within, you know, seconds, if not minutes, uh, you?

Hannah Clayton-Langton: 11:44

Here's a question, because I've got a brother that China, and sometimes I use their e-commerce websites if I'm like sending them a gift and it's very different UX. So does something like Instagram have different UX for its or Asian users versus its European users, or is it trying to optimize across both?

Hugh Williams: 12:05

Uh I can't speak to Instagram specifically, but what I you is that particularly large alphabet languages, so we're about things like Chinese, Japanese, and Korean in particular, the experiences are very different for users who you know and type these languages. And one of the reasons is it's quite hard to type. Yeah. So, you know, when you've got hundreds or thousands of characters, it's much, much harder to type. And so you'll find often that their experiences are much if you like. That's how it feels. Yeah. There's a lot more things to click on because it's a lot to click than it is to type.

Hannah Clayton-Langton: 12:39

Ah, that's interesting. Okay. Yeah. So my husband is a native Arabic speaker, and when he types to his family, he uses the English alphabet, it replicates sounds of the Arabic script. But he says it's much easier than typing in the Arabic on WhatsApp.

Hugh Williams: 12:54

And you know, think of, I mean, Arabic and and Hebrew are but Chinese, Japanese, and Korea are in a whole different ball game because the alphabet is so, so much larger, which isn't case with with Hebrew and Arabic. So when I've worked in these large companies, um, I when I was at eBay, we used to just let the Korean team do Korea because certainly our Western intuition around what the should be, the way we experimented with it, how we thought it, just did not sell properly.

Hannah Clayton-Langton: 13:21

That makes I I always had noticed that the Asian websites are, as you say, like they felt noisier to me. But when you break it down to the fact that it's easier to than to type in the large alphabet languages, that's just like the most obvious thing I never would have thought of myself. So there's different versions sensibly as you broke it of an app, depending on where you're from and what UX will best suited to you. But how does that work? Like my brother lives in China and he would download in Mandarin because he speaks Mandarin, and then he comes holiday to London. Does his app change?

Hugh Williams: 13:55

No, no, not usually. So typically what an app is built for is what we call a country language pair. For example, if um we've got listeners in the US, they get a US experience, but you choose whether you get a US experience or a US Spanish experience. So probably when your uh brother downloads the app in China, probably got a Chinese Mandarin experience. Yeah. And when he comes to the US or he's over in the UK, probably that way. So unless he changes his country because he's permanently to be there. So Amazon's probably a better example where you know, if come to Australia and you open up your Amazon app, you're going to be shopping in the UK unless you actually go and change your country to Australia.

Hannah Clayton-Langton: 14:41

Okay, so this has happened to me, and it will surprise for me to tell you that I clicked on an Instagram ad yesterday because they they promoted a dress that looked great. And I clicked on it and it said, Oh, are you in Australia So it must have been twigged because I have the location on that I'd moved.

Hugh Williams: 14:56

Yeah, absolutely.

Hannah Clayton-Langton: 14:57

So we've done a little bit of a a deep dive on UX there, I know that UX is heavily guided by experimentation, and we're gonna maybe call it A B testing. And just to state possibly the very obvious, that's you have one group A and you have one group B. Yep. And you're pushing out a different feature to group A and B, and you're looking at your metric success in group A group B, and that's how you decide which is the better

Hugh Williams: 15:23

Exactly that. Exactly that. And there's a lot of art and a lot of science in doing that, a little bit more science than art, but these large companies have huge experimentation test bed environments where you can actually go and do this in a pretty scientific way and actually figure out what's the right experience for your users. And when you're letting mini flowers bloom, whether it's A, or A, B, C, D, E, and F, you're gonna pick the winner and gonna make the winner the default for everybody who's out

Hannah Clayton-Langton: 15:49

And doesn't this? I have a few questions. So I'll throw them all out and then you can answer them in order that makes most sense. But like, would you notice if you're part of an A B Because sometimes you sign up to be in a beta version of app, but A B testing would you notice? And then, like, are they selecting user groups based on attributes? So we want to test this feature on users who are 2D7 because we want to get them 4d7. And Hannah doesn't need any testing because she's already

Hugh Williams: 16:18

A couple of great questions there. I'd say you can figure out that you're in a test group by down and looking incredibly carefully at your experience versus say a friend's experience.

Hannah Clayton-Langton: 16:29

As long as they're on the same operating system, right? Because we we talked last season about Android versus iOS and how those experiences might be different purposefully or one's ahead of the other.

Hugh Williams: 16:40

Yeah, yeah, exactly. And so, you know, I'd say if you open up your iPhone and a iPhone, you're both sitting at the kitchen bench and you sit and you really study this carefully, you will be able to spot some differences. And so things to look out for are things like colors, fonts, of fonts, those kinds of things, sizes of images, typefaces, kinds of things. And you you'll be able to spot some differences for sure.

Hannah Clayton-Langton: 17:01

Interesting. And and they are choosing user groups based on what they're looking to drive.

Hugh Williams: 17:07

I'd say that's probably the edge case. I'd say in general, what they're doing is they're trying to a representative part of the population.

Hannah Clayton-Langton: 17:14

Okay.

Hugh Williams: 17:14

So they'll typically randomly assign some part of the to be A and then randomly assign a different part of the to be B. And they'll know that if there's a difference in engagement those two, that that's a statistically significant we call it, that they could, you know, then ship to everybody.

Hannah Clayton-Langton: 17:32

And I know from working for a B2B to C tech company. So we sell a B2B product, but it's for the businesses we work with to interact with their consumers. So B2B to C. That can be tricky because you're asking your clients you their customers for experimentation. And we don't always get the level of excitement or as we'd want to improve our product because our clients are super protective of their customer base.

Hugh Williams: 17:57

And I'd say probably also they don't realize the magic of right? I think you have to see it up close inside a tech company to just how powerful this is. I mean, I'll tell you a quick story. The the most important, significant A-B test I've ever been in was slightly increasing the size of the images in the search results page at eBay. Instead of just showing users, you know, small thumbnails, showed them medium-sized thumbnails. That experiment alone put more than 400 million US dollars of revenue into the city.

Hannah Clayton-Langton: 18:30

That's crazy.

Hugh Williams: 18:31

Right. And it's such a small adjustment. It wasn't even technically very hard. Yeah. Right? We're just showing slightly larger images. And I think when you've seen that up close, when you've just how powerful it is to be able to measure changes on the base and understand a change in engagement, then you want more of it. And so I'd say a lot of companies are resistant to because it involves humans letting go of control and letting have control and also being more about testing any idea not right. Companies get quite resistant to that. But once you've seen it up close, uh, you just want to do more of it.

Hannah Clayton-Langton: 19:06

Okay. And I imagine it's very rare that you actually like lose 10% of your users overnight because you've changed the size of a font and people have just deleted the app, right? Because if you're a company that's worried about I presume that's the worry, right? That you're gonna lapse a bunch of users because you made the wrong decision.

Hugh Williams: 19:24

Yeah, absolutely. And I think you have to have a budget. So you have to have a think about kind of if we're gonna put our users in different treatments, how much are we willing here to do that? And you also want to be monitoring these metrics all of the right? And making sure that if treatment D is not a great and we're statistically sure that that is a poor treatment, we just turn off D as soon as we possibly can, right?

Hannah Clayton-Langton: 19:43

Okay, and when you say budget, that's not budget for like to make changes to the platform. That's budget because if your conversion drops by not point two percent because of the way you formatted the button, you want to have planned for that eventuality rather than lose money.

Hugh Williams: 20:01

Yeah, absolutely.

Hannah Clayton-Langton: 20:02

And I assume this is kind of like we talked about in previous episodes about how reported behavior is very different be very different to actual behavior. Like we think or maybe want to behave in a certain way, and then our habits shine through with the reality. And I guess that's why emotionally led decisions around UX be pretty flawed because you think you want a button that's this color in this corner of the page, but you've not actually tested that that drives the results that you're after.

Hugh Williams: 20:29

Yeah, and I've had lots of arguments with UX people where, you know, they've said, hey, it doesn't fit our sort of, you know, corporate ethos, the style guide, whatever else it is, and I want to do that. And I always say, like, let's just run the test. And then when we've got some data, we can have the about who's right, who's wrong, whether the style guide needs changing, whether you're right, that we shouldn't be afraid gathering data and and learning. What we should be afraid of is thinking we're right.

Hannah Clayton-Langton: 20:53

Okay. And is there sometimes like a change fatigue? Like people famously don't like change, and then Apple their iOS operating system and like I can't find the button on the phone book. But is am I really gonna come round to loving it, or do they can you sometimes get it wrong?

Hugh Williams: 21:11

You can sometimes get it wrong. Where experimentation is powerful is when you're optimizing existing experience, right? So we're talking about things like changing colors and font button sizes, colors, these kinds of things, sizes of boxes, how large an image is. Experimentation's wonderful for that. And that'll help us find the the local maxima, best of the existing experience. If you want to go do something completely different, useless.

Hannah Clayton-Langton: 21:34

I feel like Apple do this a lot. Like they're always changing my operating system, and can never find anything.

Hugh Williams: 21:38

And they're not a data company, right? They are a hardware company, right? So if you look at the largest companies by market cap, you you've got folks like we'll skip over NVIDIA and we'll skip over Apple for a second. You've got Meta, Google, Microsoft, Amazon. Those folks are consumer companies that are all about data software. Apple is not one of those. Apple is a hardware company. That makes wonderful, wonderful hardware. I love their ecosystem, but they are absolutely not a data

Hannah Clayton-Langton: 22:05

Okay. And that's why I sometimes find their updates

Hugh Williams: 22:08

Yeah. And so the designers run Apple, not the experimentation

Hannah Clayton-Langton: 22:13

Okay. That makes sense. So assume in your time at eBay and Google Maps, et cetera, done your fair share of experimentation. So what would you say is like the biggest surprise or biggest failure or most counterintuitive experiment outcome came across?

Hugh Williams: 22:30

Yeah, so obviously we talked about the experiment of the size of uh images at eBay and that being certainly the successful uh experiment that I've ever been involved in. I should say to finish out that story, um, you can guess what idea the product manager had who was working on that, the idea they had.

Hannah Clayton-Langton: 22:47

Make it even bigger.

Hugh Williams: 22:48

That's it. And that uh delivered even more gains to the company so it did. Okay. So you can guess what they did after that.

Hannah Clayton-Langton: 22:55

No, because I'm gonna say then you're gonna have like one hit per search.

Hugh Williams: 22:59

Yeah, and eventually the images got so large that as a team we had to say, look, you know, we need to we need to stop increasing the size of the images. So then the PM went and worked on uh increasing the quality the images, which also mattered. Okay. Um, so this this was a gift that just kept on giving. Turned out larger the images, the more people shop, higher quality of the images, the more people shop. And it was one of the most important things in the period of uh eBay when when I worked there.

Hannah Clayton-Langton: 23:23

Okay, any other cool experimentation stories?

Hugh Williams: 23:25

I can tell you a ton. I'll I'll tell you one where I uh I kind of came last, as So another eBay story. So we were thinking about there's a button on eBay that says So we obviously, you know, bid on an auction, right? And we were sitting there one day and we thought, I wonder if bid's the right word. Like, should we say bid now? Bid soon. Um, should it say bid and maybe it has a price that you could bid at? So you could say bid at$4.20. It's like, oh, this is an interesting theme. How do we get people bidding more on auctions and and do that? So we came up with, I think it was five different ideas of what the bid button could look like. And I was sure I was gonna win this. I know so much about eBay, I've been involved in this part of eBay for ages. My treatment's gonna win for sure. I came stone cold last.

Hannah Clayton-Langton: 24:11

Oh no.

Hugh Williams: 24:12

Stone cold.

Hannah Clayton-Langton: 24:13

I thought you were gonna tell me that you you nailed it and you guessed the winner now.

Hugh Williams: 24:16

No. And look, the takeaway for me is what we should do is have many flowers blooming, lots of different ideas, make easy. As long as it's legal, it's not deliberately destroying the you know, whatever it is, we should get these ideas out as as we can, as frequently as possible, and measure what happens, and then you know, let the data speak and then let the winner win.

Hannah Clayton-Langton: 24:38

I'm loving these stories. Have you got one more for us? Because there's they're they're really uh they're giving me pause for thought.

Hugh Williams: 24:44

Yeah. My good friend Michael Cameron, uh, who I've worked with at you know, he went off and founded a company called Rome to

Hannah Clayton-Langton: 24:50

Oh, yeah, I do know that one.

Hugh Williams: 24:51

I love Rome to Rio. Melbourne Company. Um is it? Yeah, it is indeed. And uh, I was working with Michael a few years ago when he was still at Rome to Rio, and he told me a little bit about some that they ran. And look, maybe just we just dive in. I'm sure many of our listeners have used Rome to Rio. It's a fantastic product. Great for getting from A to B when A and B are a long way apart. It's basically a short summary. So if I want to go from Melbourne to Manchester, I want to out what flights I can take. Like, you know, do I take a train from Heathrow? You know, whatever else it is. So fantastic product for that. The way they make money is uh they get a referral fee if you some accommodation or you book a car after you've done that through something like booking.com. So Michael and his team highly, highly incented to get to click a little part of the screen that said book or book car rental. And they played around with a bunch of different treatments of how do you say that? Like, how do I say book accommodation? Should I say book a hotel, book accommodation, book a room, a room? Um, what about best priced accommodation? Best priced car rental. So we can we can sit here and brainstorm lots of different That's exactly what they did. And then they tried some different treatments, ran those as a big A B test, and they actually increased this is a small startup, right? They increased their revenue by over $250,000 a year by the text on the part of the screen that said book accommodation.

Hannah Clayton-Langton: 26:15

Because it just was more uh engaging unconsciously to users.

Hugh Williams: 26:19

Yeah, absolutely. Turns out colours of buttons matter too. If you go and look at a lot of the travel sites, you'll find all the buttons are green. Not a coincidence. Green signals go. Green signals go, and so you get these sites that are blue or purple or whatever else it isn't all the buttons are

Hannah Clayton-Langton: 26:34

Huh. Do you know what? When I was a kid, I remember being told that McDonald's red and yellow because it made you eat fast and got you like out of the restaurant so more people could come in. And I guess this is the virtual equivalent of that sort of behavioral nudging.

Hugh Williams: 26:49

Yeah, absolutely.

Hannah Clayton-Langton: 26:50

Oh, wow. Okay, so colors, sizes, placement. How you say things. How you say things.

Hugh Williams: 26:56

Absolutely. It all matters and it all drives engagement. There's an army of incredibly smart, well-qualified, young product managers in all of these companies who are incredibly hard to drive up whatever metric it is that they want to drive up. And so, Hannah, it's not really your fault that you can't put your phone down. It's kind of you versus an experimentation engine that's by lots and lots of smart mods.

Hannah Clayton-Langton: 27:20

And is there also an element here of like the of stuff? So Google Maps is maybe not going to be as personalized to but like my Instagram feed or TikTok feed, if I had I keep going back to it because the content, it makes me It's something that I'm interested in. And I assume that's good like processing of all the data that I'm feeding in with my use and then applying that to the and images that are in the app and then pushing me the ones to keep me scrolling, to keep me reopening the kind of thing.

Hugh Williams: 27:54

Absolutely. And look, one of the biggest areas of experimentation in these large companies is search and recommendations. And the news feed or the feed in any social app is a is a of that. And so you tend to get the brightest set of folks working on search and recommendation style problems and feed style And you know, their job is to make sure that you keep on

Hannah Clayton-Langton: 28:15

And back in the early days of Instagram, I remember that hashtag was really big or or and maybe Twitter as well, it was like you hashtag your content, and that's how the app or the algorithm knows what you're pushing, and then it link you to relevant like recipients of your content. But that they're not really used anymore. So how are they doing that now? They just know that it's a picture of like, I don't know, on toast.

Hugh Williams: 28:37

Yeah, absolutely. So, you know, today things are a lot more sophisticated than used to be. And so things like processing the images as they're trying to identify objects that are in them, and also other of them, you know, are they a pleasing image? Is it well focused? Those kinds of things. Is um there a single object in the image? All of these kinds of things are the sort of metadata that can extract from it. And again, you can put this into the algorithms, into the AI and the AI system, you know, will be able to serve the that are, you know, probably great for all customers, great for you.

Hannah Clayton-Langton: 29:11

Does videos make that easier or harder? Because like what was it a few years back, Instagram moved being very much a still image-based app with still content to videos and reels and everything's moving all time. Is that going to be easier or harder to identify that

Hugh Williams: 29:27

Uh it's definitely a little bit harder, but there's lots and of work that's gone into video over the years. So, first of all, what happens with video is you divide it into shots. And so if you're watching a video, you'll find, you know, that cuts between scenes in different sorts of ways. So there's, you know, hard cuts, there's fades, there's wipes, there's all kinds of things that happen. So you basically cut the cut the video up into all of its scenes, and then you can basically process the scenes to kind of understand what's in those scenes. So it's definitely more work than a still image, but you can of a video as a series of still images, and particularly when you break it up into scenes or shots, um, you know, it's a lot easier to process and try and understand.

Hannah Clayton-Langton: 30:03

Okay, so it's a combination of relevant content and user experience.

Hugh Williams: 30:10

Yeah, exactly that. And lots and lots of experimentation to mine my That's it.

Hannah Clayton-Langton: 30:15

Okay, so I have revealed my cards as an avid Instagram and user. And I was trying to think about like if I could only keep apps in my phone, and I'm not including here like my banking, because I obviously need my banking and my like multi-factor authentication, because as we know from our privacy episode, that's just good sense. But I'm thinking about the like more optional social apps I'm using. And I've decided that my top three that I couldn't live are Instagram, WhatsApp, and Google Maps. Like, I I really don't think I could get anywhere without Maps. But what would you say yours are?

Hugh Williams: 30:48

That's a great question. I know if I had my my screen in front of me, the answer's too hard. I mean, first of all, I'm gonna agree with you on Google Maps. I'm a little bit biased, I guess, since I worked on it, but uh, but I love that product. Yeah, it's such an amazing product. The team's done so much amazing work. Look, sadly, I've got Gmail pegged to the bottom of my screen. I think that's a fantastic email app. I'm still primarily an email kind of person. I have five different email accounts. So, you know, that's something I just can't do without to of function. And then maybe my third one's a little bit more fun, is I I my Whoop. Um, and Whoop takes up, I don't know, like a quarter of my my home screen.

Hannah Clayton-Langton: 31:24

Okay, so Whoop is like a wearable fitness tracker, right?

Hugh Williams: 31:28

Yeah, exactly. So, you know, you can think of it as a fancy heart rate monitor. So it's always tracking my heart rate and what's called heart rate variability, which is your ability for your heart to pace.

Hannah Clayton-Langton: 31:38

Okay, so you're a big wearables fitness data guy, which we probably do an episode on at some point because people are into it. I love my garment. It's I didn't make my top three, but I do love the the fitness tracking.

Hugh Williams: 31:49

Let's get somebody on from that part of the world because I our listeners would love to know how these wearable devices

Hannah Clayton-Langton: 31:55

And definitely. I think it's safe to say that your Whoop app improves your more than Instagram improves my life.

Hugh Williams: 32:02

I don't know. Whoops got me being very, very religious about when I go to and when I get up in the morning, which probably make me a lot less fun.

Hannah Clayton-Langton: 32:07

I can attest to this. Hugh is all over his uh his fitness optimization.

Hugh Williams: 32:12

Yeah, yeah, yeah. It's so important.

Hannah Clayton-Langton: 32:14

Okay, so there's a few more insights into us as co-hosts. That's been a super awesome episode. I don't know that it's gonna help me stay off my phone. Well, do you know what's gonna help me stay off my phone is the sunshine and your pool. That's what's gonna help me stay off my phone. But once I'm back into the cold depths of the English I think I'll be uh straight back on my addictions.

Hugh Williams: 32:36

I'll give you one life hack though, and it works for me is put your phone away every Sunday. Just don't take it out of the cradle and uh pick it up again Start with uh 1D7, Hannah.

Hannah Clayton-Langton: 32:45

But what about my Google Maps? I guess I gotta keep it local on a Sunday. I know where the farmer's market is and I know where the gym is.

Hugh Williams: 32:51

Yeah, get somebody else to navigate for you.

Hannah Clayton-Langton: 32:53

Yeah. I'm also, do you know what though? Apple Pay. I should have put I don't know if I would put that in my top three or if that's just a sort of table stakes, but I I never hold a physical card anymore.

Hugh Williams: 33:03

Maybe on Sundays.

Hannah Clayton-Langton: 33:04

Maybe on Sunday. Yeah, fair enough. Okay, well, I will take my New Year's resolution with me, we will see you next time, guys. If you have enjoyed this episode, please like, subscribe, and rate the podcast.

Hugh Williams: 33:17

Yeah, and you can find us at techoverflowpodcast.com where got all the episodes from season two and season one, and we're also on LinkedIn, Instagram, and X.

Hannah Clayton-Langton: 33:28

Awesome. And I think we should go and take a look at the UX of here, because maybe we can drive some addictive behaviors getting uh some of those buttons in a really cool place.

Hugh Williams: 33:38

Yeah, I think I'll just make the images bigger.

Hannah Clayton-Langton: 33:40

Yeah. All right, guys, we'll see you next time. See you next time. Bye.