When you’re shopping for a new gadget online, there’s a good chance you consulted the reviews on YouTube. 

So many of them are well produced and very thoughtful takes on the latest computer or camera equipment, services, even food and toys. But are they unbiased? Our guest today, Michael Swart, says a lot of them are practically paid commercials without even telling you. And he has a way to tell the difference.  

He says content creators are incentivised to do reviews that cause consumers to buy the product, which results in a commission for the content creator. He said that when this occurs, creators are required to post a notice, but in practice this rarely happens. But he’s found a way around that, for consumers willing to take the time to download a browser extension he created.

Mike is a 2019 graduate of Princeton, with a degree in computer science. He’s now working as a software engineer at the transportation app company Lyft.

Links:

Study: AdIntuition tool study

Study: Endorsements on Social Media

Video: Can you trust video-based reviews?

 

Transcript:

Aaron:

From the Princeton University School of Engineering and Applied Science, this is Cookies, a podcast about technology security and privacy. On this podcast we’ll discuss how technology has transformed our lives, from the way we connect with each other, to the way we shop, work, and consume entertainment. And we’ll discuss some of the hidden tradeoffs we make as we take advantage of these new tools. Cookies, as you know, can be a tasty snack — but they can also be something that takes your data.

When you’re shopping for a new gadget online, there’s a good chance you consulted the reviews on YouTube. So many of them are well-produced and thoughtful takes on the latest computer or camera equipment, services, even food and toys. If you’ve never seen an unboxing video, I’d recommend giving one a try. Pretty much anything can be featured in a review video, but are they unbiased?

Aaron:

Our guest today, Michael Swart, says a lot of them are practically paid commercials without even telling you. And he has a way to tell the difference. Mike is a 2019 graduate of Princeton with a degree in computer science. He’s now working as a software engineer at the transportation app company, Lyft. This podcast is brought to you by the Princeton School of Engineering and Applied Science. Let’s get started. Mike, welcome to the podcast.

Mike Swart:

Hey Aaron, thanks for having me.

Aaron:

Where are you joining us from today?

Mike Swart:

I’m calling in from San Francisco, California.

Aaron:

So before we get started, we should note that you also have an illustrious career in water polo.

Mike Swart:

Yeah, well, I was at Princeton University. I was lucky enough to play on the varsity men’s water polo team for four years. And I have nothing but positive things to say about that experience. I met some of my best friends on the team and we played in some big games and had some tough practices. And I look back really fondly on that time.

Aaron:

And you balanced that with some pretty rigorous work in computer science.

Mike Swart:

Princeton University does a great job of supporting athletes and students and allowing student athletes to do both things. From 4:30 to 6:30 at night, there aren’t any classes, and so there’s enough time for practice. And then everyone can take any class that they want. You’re not restricted in what classes you can take and there’s great resources to help students. So, the school does a great job of helping everyone manage.

Aaron:

Let’s talk about, we call this affiliate marketing. Why would a reviewer accept money from a company whose product they’re reviewing?

Mike Swart:

Yeah, this is a really great question. So, social media has really blown up recently and some content creators, they can get millions of views on some of their posts. And so brands see this as a really great way to target marketing. A lot of content creators have a specific type of content that they make. So, someone could be a food blogger or a makeup artist or something like this. And so, a brand that then is in the marketing space for makeup, they can then target their ads at people that are watching videos about makeup. And so it’s a really great way for brands to target their advertising.

And then these content creators, YouTube is a free platform. And so it’s a great way for the content creators to make money based off of the work that they are doing to create these videos. So for the brands, it’s great because they get targeted advertising and for the content creators it’s great because they can actually make money off of their content. And so it’s a really great system for those two parties. The one party that gets left out in that equation is the viewers of the video who, most of the time, don’t actually know that that content is sponsored because the content creator doesn’t disclose that they’re making money based off of those purchases.

Aaron:

Let’s talk about bias in these reviews. Let’s say that there is a financial relationship between these two parties. Do these video makers state negative things about these products or would they hesitate to do that if they actually believe that? And are these reviews biased?

Mike Swart:

Yeah. So, that’s a really great question. There’s a couple of types of social media marketing and the one that we focused on is called affiliate marketing specifically, which the way that that works is the content creator only makes money when someone buys the product that they are supposed to be promoting. And so in that system, the content creator is incentivized to actually give a positive review because then they’ll generate more money for themselves. In other types of social media marketing, it’s possible that they could give a more unbiased review, but because they’re incentivized by the percentage of the proceeds that they’re going to generate, they’re much more incentivized to give a positive review.

Aaron:

So, they literally get a commission.

Mike Swart:

Yeah. So, you could look at it as someone views the video, then they click a link or use a coupon code on the brand’s website. And then they pay with their credit card and a typical structure could be like 85% of that money that the brand makes goes to the brand and 15% goes to the content creator. And so each time that someone presses buy on that website, the content creator is making money. And again, they’re really incentivized to try and get people to press that buy button so that they can make more money.

Aaron:

Okay. So how did you get involved in this area? There was a team involved here. Can you tell us about some of the other members of the team?

Mike Swart:

Yeah, definitely. So, while I was looking for a thesis topic, I reached out to professor Marshini Chetty, and she was talking about this really cool project that her and Professor Arvind Narayanan, and then Arunesh Mathur, who’s a PhD candidate, they were all working on this project in the affiliate marketing space where they were just looking into what is affiliate marketing, why is it an issue, and these type of things. And so then when I was talking to Professor Chetty about my thesis topics, she said, “Hey, we have this really interesting project that we’ve been working on, but we haven’t actually used any of it yet.”

So, they had done all of this really amazing research. The paper that they wrote and had won an award at the conference that they presented at. And she said, “We’ve done all this really great work, but we haven’t actually used it anywhere. Would you want to build a browser extension?” So basically she just placed this really cool project right in my lap for my thesis. And so I worked during my senior year on creating the browser extension that we called AdIntuition that actually on YouTube flags this sponsored content, because the content creators don’t actually disclose it.

Aaron:

How pervasive is the practice of affiliate marketing?

Mike Swart:

Yeah, so we did some research into this. We had users who downloaded the extension, we tracked some of the videos that they were watching and things like that. And over 60,000 videos, we found that 7.5% of the videos contained affiliate marketing, which is a pretty large percentage of the videos, in my opinion. And so if you watched 15 videos on YouTube, chances are you’re going to see one that has affiliate marketing in it. And so it’s definitely very prevalent across the platform.

Aaron:

When they do have affiliate marketing, is that usually disclosed? Are they required to disclose?

Mike Swart:

Yeah, that’s a great question. So, they are required to disclose. The FTC has rules for online marketing and social media marketing. But the problem is that the FTC doesn’t actually enforce those rules. And so only about 10% of videos that have affiliate marketing disclose. And then another issue is that there are different types of disclosures. So one would be just saying, “Hey, this video contains an affiliate link.” And that’s actually not that descriptive because YouTube descriptions can be very long. And so it can be confusing if right at the top of the video, it says, “This contains an affiliate link.” But then it’s kind of hard to tell what actually was affiliate, was everything affiliate? And so that can be confusing.

Another one says that, “Hey, if you click a link in the description, you’ll support my channel.” And that’s also a little bit confusing because they make the jump for you that there’ll be making money, but they don’t say it explicitly. And so then the final one is an explanation disclosure, which says, “This video contains affiliate marketing and I will make money if you click on the link.” And so that one occurs the least often, but it’s the only one that users are consistently able to tell that a video has affiliate marketing if that one’s present. And so that one occurs a little bit less than 2% of the time. And because it doesn’t happen very often. Most of the time that users are watching videos, they actually don’t know that the video contains affiliate marketing.

Aaron:

So how can you tell if the video has affiliate marketing?

Mike Swart:

Yeah. So, as part of the research that we did, we looked into different ways that we could tell. And the nice thing about affiliate marketing is that the brand needs to know where the content was sourced from in order to pay the content creator based off of the percentage of the proceeds that they generate. And so because the brand needs some sort of tracking mechanism, we can then reverse engineer that to find those tracking mechanisms from the beginning.

And so one of those tracking mechanisms is coupon codes. A content creator could say, “Hey, use my code, Mike10 at checkout.” And so the brand is able to track where the buyer came from. And the buyer needs to use that coupon code in order to get some sort of deal or something like that. So, the buyers incentivized to actually use the coupon code and then the brand benefits a lot from that, because they can see where the buyer actually got that information from that they could get this deal.

And then there are two different types of links that also brands use. One of them is they’ll go through an affiliate marketing service. So, some companies, their whole business is, “We’ll do all of the tracking for you, so you don’t have to.” And so as part of that original research that I was piggybacking off of, they built this list of about 60 link patterns to these brands that help track affiliate marketing. And then finally some brands that are bigger, they might just do the tracking themselves based off of query patterns and the URL string.

And so we looked for some common query patterns in the URL stream. And so those were the three methods that we used. Affiliate marketing services, coupon codes, and then query string patterns. They’re called UTM parameters mostly. Urgent tracking module parameters. And we found that those are actually often used also to show that affiliate marketing happens.

Aaron:

You’re listening to Cookies, a podcast about technology, security, and privacy. Brought to you by the school of Engineering and Applied Science at Princeton University. We’re speaking with Michael Swart, a 2019 graduate of Princeton. In our next episode, we’ll speak with Prateek Mittal, an associate professor of electrical engineering. We’ll speak about adversarial machine learning. But for now, back to our conversation with Michael Swart, in which he discusses what a thoughtful consumer should do after they know that content is being sponsored in a video they’re watching on YouTube.

So, you’ve built a browser extension. How does this work and how effective is it at identifying monetized links?

Mike Swart:

Yeah, so we built a browser extension. We called it AdIntuition. It’s available for Chrome and Firefox. And basically what it does is those three different sources or signals of affiliate marketing, what we do is whenever the user loads a YouTube page, we search the description of the YouTube video for any of those three signals. So, we look at all of the links and look at their redirects and see if any of them go to either an affiliate marketing service, or if any of them have tracking parameters on them. And then we also built a coupon code classifier that checks to see if any coupon codes exist in the video description.

And then if it does see any of those signals, what it does is across the top of the page, right under the search bar, it gives an explanation disclosure, which I explained earlier is the only one that users are actually able to tell that affiliate marketing was present. And then in the description, it highlights the link or coupon code that we found as a signal of affiliate marketing. And so we, we thought that this was also a helpful addition because the user not only knows that affiliate marketing is present, but also knows exactly where in the description it’s mentioned. And so, for example, if a link is highlighted, they know when they click on that link, if they purchase something, then the content creator will make money.

Aaron:

So, you’ve actually tested this extension on users, right? I mean, what kind of experience have they had?

Mike Swart:

Yeah, so we ran two different user studies. One of them, we did that tracking that I was talking about to tell how users interact with the tool, what type of videos contain affiliate marketing, and things like that. And we also were able to tell what the prevalence of affiliate marketing is. And so that was a pretty useful user study. And then we did another one to test does the tool actually work? So, does the explanation disclosure across the top work? And does the link highlighting work? And we found that compared to someone who isn’t using the extension, users were able to more easily tell when affiliate marketing is present in a video. So by proxy, the extension does work. And if a user is using the extension, they will be more informed about affiliate marketing content in the videos that they’re watching.

Aaron:

So, how can people find this extension? It’s AdIntuition, one word?

Mike Swart:

Yeah. So it’s AdIntuition, it’s all one word. And on the Chrome and Firefox web stores, you can download it as a browser extension and use it for yourself. And so if you do any YouTube watching, it could be a cool thing to go and download the extension and see if the content that you watch actually contains affiliate marketing. One interesting thing about affiliate marketing is it does depend on the content that you’re watching. So, some content does not contain much affiliate marketing and other content contains way more affiliate marketing than the average channel. And so a lot of it depends on browsing habits, but again, we found that averaged out over the 60,000 videos and almost 300 users that use AdIntuition in our original study, we found that 7.5% of those videos contained affiliate marketing. So, it’s still a pretty large percentage. but it does depend on the videos that each individual user watches.

Aaron:

What sort of videos tend to have them and what sort of videos don’t?

Mike Swart:

Yeah, that’s a great question. So like I was saying earlier, brands, they try and partner with affiliates that have a similar content to what the brand is actually selling. And so makeup tutorials could often have affiliate marketing because any form of cosmetic company could want to have more pointed advertising. So, that would be one area. Whereas there are some bigger YouTube channels that are sponsored by network television stations. Those ones often don’t have affiliate marketing because they have other like ways to make money outside of affiliate marketing. And so it’s less prevalent in like those big network things.

Mike Swart:

But again, downloading ad intuition, it’s really hard to tell. Those network television stations tomorrow could start using affiliate marketing and the cool thing about the tool is that we don’t need any user input to tell if affiliate marketing happens, because we have these patterns that we are automatically generating. So, new videos are included in it. And so again, if a network television station started participating in affiliate marketing tomorrow, the tool would be able to detect that.

Aaron:

Do you view these reviews just in your off time? Do you ever go looking? Have you ever actually stumbled upon an affiliate link just casually?

Mike Swart:

Definitely. You know, building the tool, I did a lot of research into these types of things and I watch a decent amount of YouTube on my own time and I have AdIntuition installed on my computer and sometimes I’m very surprised about what videos contain affiliate marketing. And sometimes it’s not even explicit to the content that’s in the video, it’s just in the description.

Mike Swart:

So Amazon has an affiliate partner program. And what you can do is you can partner with Amazon and you go in actually, as the content creator, and say, “Hey, I’d like to make money based off of proceeds that I generate.” And so any vlogger or someone like that could do this, and then you’ll see links in the description that say, “This is the camera that I use. This is the microphone that I use.” And they go down the list of all their tech and those types of things. And every one of those in some videos is an affiliate link to an Amazon affiliate’s website. And so, they never mention that they’re making money off of those proceeds. And so it’s a little bit confusing because the video wasn’t about the camera that they’re using, but they will make money if you buy that camera.

Aaron:

If you want to be a thoughtful consumer, once you’ve downloaded the extension and you’re aware of which videos are monetized and which are not, what else should a good consumer be doing?

Mike Swart:

Yeah. So after downloading the extension, that’s pretty important so that you know the sponsored content exists, then thinking about, “Does that change my opinion about the content that is being sponsored and the brand that is sponsoring the content?” So, if it was some makeup tutorial, again, and the review was sponsored or not sponsored, those might change your opinion about whether the brand is actually a brand that you want to buy. If an unbiased review is actually unbiased, then maybe the content is slightly different than if the brand is sponsoring the content.

Aaron:

So, what needs to be done at a policy level, as well as just best practices for companies to address this issue?

Mike Swart:

So, I think that the FTC, they have these rules in place, but they aren’t really enforcing them. And I think it’s also pretty difficult to find these cases, like as the FTC, it can be difficult for them to find cases of this just because without some automatic tool like AdIntuition, it can be a lot of hours to go through and try and find these things. But I think that there needs to be some level of accountability from the top level, at the FTC, saying that this is not allowed and you will get in trouble if you don’t do it, that being not disclosing sponsored content. And then from the platform perspective, YouTube or Google Chrome, both of those have some sort of responsibility, I think as well. YouTube, for example, they have the functionality for content creators to say this video contains sponsored content, but they don’t enforce content creators needing to actually use that disclosure.

And so it could be very useful for YouTube to actually go in and say, “Hey, if you don’t include disclosure of sponsored content, then we will take your video down.” Because without any sort of that enforcement, content curators will continue to not disclose because in the current system, there’s no negative effects of them not doing so. And so from the FTC level, I think that may need to start cracking down on it. And from the YouTube level, I think that they need to also start enforcing that disclosure must be present for sponsored content.

Aaron:

Is it just YouTube, or are there other platforms where people typically consume this content?

Mike Swart:

Yeah, I think every social media platform has this. Basically anywhere where someone can gain a significant following, these types of practices will be present. So Instagram, I think a lot of times people see #ad or #sponsored. But again, not everyone does it. And then Pinterest, in that original study that my thesis advisor had done, they looked at Pinterest as well as a source of affiliate marketing and a place where affiliate marketing was happening. But it really can happen anywhere, Facebook, and so across all social media platforms this can definitely happen.

For our study, we just focused on YouTube so that we could get a really good knowledge of the whole ecosystem in YouTube. And YouTube has a couple of issues, too. Like videos can be a little bit longer. Descriptions can be very lengthy. And so having this tool, we thought helped users to actually tell that there was affiliate marketing and we got a better sense of affiliate marketing in general, rather than going really wide across a lot of social media platforms.

Aaron:

So finally, why don’t you tell us a little bit about what you’ve been doing since graduation?

Mike Swart:

Yeah. So after I graduated, I took a job at Lyft where I’m currently a software engineer and I’m in San Francisco and I’ve really been enjoying it so far and am excited to see what else I can do there.

Aaron:

Mike Swart is a class of 2019 alumnus in computer science at Princeton. I want to thank our recording engineer, Dan Kearns. Cookies is a production of the Princeton University School of Engineering and Applied Science. This podcast is available on iTunes and other platforms. Show notes are available on our website, engineering.princeton.edu. Mike is joining us today from San Francisco. Thank you for taking the time to speak with us, Mike.

Mike Swart:

Yeah, Aaron, thanks so much for having me on. I really appreciate it and really enjoyed talking to you.

Aaron:

I enjoyed it a lot.

Aaron:

The views expressed on this podcast do not necessarily reflect those of Princeton university. I’m Aaron Nathans, digital media editor at Princeton Engineering. Watch your feed for another episode of Cookies soon, when we’ll discuss another aspect of tech security and privacy. If you have a moment, please give us a good review. It helps. Thanks for listening. Peace.