Online Ad Experiment: Comparing Ad Channels

I recently ran a small online advertisement experiment with the goal of  increasing the number of subscribers to this Substack publication, and also to write this post about the experience. I focused my experiments on two advertisement channels: LinkedIn and Google search.

LinkedIn was an obvious choice as this is the only social network where I (and also this publication) have presence and followers. It is also a social network focused on professional networking and professional topics which fully aligns with the essence of this publication.

I picked Google as well because it’s not targeting specific people based on their profile details and historical usage of a social network, but rather targeting anyone using a search engine who is trying to find very specific information on the Internet at that specific point in time. Google’s strategy is very different to targeting specific people in social media so I thought it would be interesting to try both channels and compare results.

Both of these advertisement channels have some key differences and similarities which I will try to explain below:

Billing

LinkedIn charges you for the total number of ad impressions served.

Google will charge you only for the clicks made on your ads regardless of how many impressions they end up serving.

Campaign Optimization Goal

LinkedIn will charge more per impression depending on your campaign goal (e.g. brand awareness, website traffic, etc.) They add different features to the ad depending on your goal (for example some campaign goals will add a “follow” button to the ad and some goals would not). However, in my experiments, picking any goal other than the most basic one had no impact on clicks or conversions but made the cost per impression (and the campaign in general) substantially more expensive without any benefits.

Google lets you optimize your campaign on clicks or conversions without any extra cost. To make my campaign optimize for conversions (my real advertisement goal) I needed to set up a Google Analytics account in Substack, link my Google Analytics account to my Google Ads account, and create a “goal” in Google Analytics to track every time it detected a “Signup” event in the website. 

Audience Targeting

LinkedIn provides an automatic way to generate “Lookalike” audiences on their network using their proprietary data and models. I leveraged this feature using all the users who have interacted with my publication’s LinkedIn page as an input. Their service then created a big audience of users in their network who are “similar” to my input users. They don’t disclose much about how these models work but I’m sure they use some sort of user similarity scores based on their profile information and their behavior on the platform.

LinkedIn also has an “Insights Tag” that you can add to your website/publication directly and build the “lookalike” audiences based on the visitors to your website. However, I wasn’t able to try this feature as the only way to set this up in Substack is with Google Tag Manager and, unfortunately, Substack’s implementation of Google Tag Manager code is broken (I have reported this to Substack but they said that this is currently not a priority.)

To set the targeting in Google I provided some relevant search keywords related to my ad (I started with “learn data science”) but it also has an automatic feature that lets Google experiment and suggest other keywords that will work well with your ad. It does this as it’s running the campaign so I let it run for a couple of days to gather all their keyword recommendations and use those too for my campaign. They ended up recommending more than 300 more keywords that I added to the campaign (e.g. “ml and ai”, “be a data scientist”, “path to become data scientist”, etc.)

Partner Ad Networks

LinkedIn lets you expand the reach of your campaigns by targeting specific LinkedIn users outside of LinkedIn (by using display ads on other websites). They achieve this by partnering with ad networks and publishers (other websites) in the industry. The cost per impression is reduced substantially when your ads run there so I also experimented with this feature.

Google also lets you expand your campaigns with two types of partners. Search partners, who also operate websites that are search based (users inputting a query to find information), and ad network partners which lets them run your (display) ads in random websites around the world. The cost per click also decreases when your ads run with partners so I also experimented with these. The search partners made a lot of sense to me as the targeting mechanism is similar to Google search, but the ad network partners didn’t as we aren’t really targeting a specific user in those random websites like we are with LinkedIn. That said, Google still charges per click and not per impression in those random websites so I was still interested in trying it out.

Ads

In LinkedIn I set the campaign to use all the postings I’ve made on my publication's LinkedIn page as ads.

In Google I gave them a few headers and descriptions about my websites and they ran experiments to find the best combinations of these for different queries.

Experiment results

Experiments

• LA: LinkedIn campaign leveraging ad network partners.

• L: LinkedIn campaign only (no partners)

• GSA: Google campaign leveraging search and ad network partners.

• GS: Google Campaign leveraging search partners.

• G: Google campaign only (no partners)

Metrics

• CPM: Cost per mille in dollars (cost per thousand impressions - industry standard)

• CTR: Click through rate or percentage of impressions that lead to a click.

• CPC: Cost per click in dollars

• SUR: Signup rate or percentage of users that sign up to the publication after clicking on the ad.

• CPS: Cost per signup in dollars.

Except for Google search, it is common for all publishers (websites), social networks, etc, to charge based on impressions. As a data scientist with experience in ad tech I’m not thrilled about having to pay for impressions. Impressions will be rarely correlated to my campaign goals (publication signups in this case) and there is no guarantee all these impressions are really being optimized for my benefit or, even worse, that some nefarious entities or users in the complex advertisement network are committing ad fraud.

Lets compare the experiment results using the CPM lens:

GSA experiment gave us the lowest CPM compared to all others. It is also interesting to see that although the Google experiments seem to have lower CPM in general, there is not a clear winner platform-wise as LA has a lower CPM than G.

As discussed earlier, Google charges based on clicks which in my opinion is a more valuable metric for advertisers than impressions. Clicks can sometimes be somewhat correlated to the campaign goals but unfortunately clicks are still far from perfect. Clicks can still not be correlated at all with the goal in some campaigns and fraudsters can also abuse these.

Lets compare the experiment results using the CPC lens:

GSA is still the clear winner also when using this metric. Note that based on cost per clicks the Google platform is also a clear winner over the LinkedIn platform both in terms of cost (which is not surprising to me given LinkedIn charges per impression and Google per click) but also in percentage of users clicking on impressions. Another interesting thing to note is that G has the higher CPC for the Google experiments but G also has the higher click rate, which shows that impressions on the Google network are a lot more relevant to the users than impressions that show up on the partners. However, those impressions come at a higher click price whenever the user clicks on them.

Let’s now focus on signups, which is what I really care about for these experiments. I don’t care about impressions or clicks if these are not leading to signups at the end of the funnel.

Lets compare the experiment results using the CPS lens:

Note how GSA, our previous CPM and CPC winner, is now at the bottom of the Google experiments with GS now being the winner on cost per signup as well as signup rates. GS is closely followed by G. Also note how G and GS had a huge difference in CPM but in terms of signups they both perform very well (which is what I as the advertiser really care about).

GS is the clear winner here in terms of how much I need to pay per signup on this campaign. This makes sense to me as this experiment leverages search keyword  targeting while also displaying some of the ads on cheaper (per click) search partners. The google advertiser network in GSA is not adding value and losing us money as I originally speculated. 

Unfortunately, in my experiments, LinkedIn was way more expensive and much less effective than Google. I was also disappointed that my following on that social network didn’t grow much with this campaign. This is the social network where I have a presence and I assumed that some extra advertisement could easily expand that following. However, this was not the case. 

On the other hand Google was a very effective channel. They had the lowest cost per click by far, the highest click through rate by far, and more importantly, substantially higher signup rate and substantially lower signup cost, which is all I really cared about.

I hope this article is helpful for when you are evaluating your own ad campaigns.