I created this data-led process for testing ads because I couldn’t predict if an ad would succeed or fail. I was frustrated that the ads should technically work according to the analyst or the designer, but half the time they didn’t.
This method uses facts to inform the creative design process.
- It minimizes the risk of publishing an ad that doesn’t work.
- It maximizes the chances of creating ads that hit the objectives.
Some advertisers get wrapped up in pixels, tags, and targeting, and forget they’re still trying to talk to a human being.
Many other advertisers are so invested in their ideas that they forget to test them and often miss what’s important to the customer.
This ad testing methodology will prompt your creative design process so that you create ads to target people at the various stages of the consumer journey. You will use data to make incremental improvements and reduce the risks to your business. No more generic ads.
As a bonus, I’ve included templates for you to get started with and some tools I designed to help visualize it all
Contents
Ad Testing Process
The challenge that a lot of advertisers have is knowing what to say and when best to say it.
Some companies focus on their products, for example, Cactus Club Cafe showcases its menu and drinks.
Some focus on their community.
Others on their culture. Here you can see that Ford focuses heavily on the culture of the brand and the communities that it fosters and enables.
Some concentrate on brand building.
Yet others, go straight for the sale.
Use the Facebook Ad Library to find some examples of your favourite brands’ ads and see what they do.
Getting ‘what to say’ and ‘when to say it’ right from the start is difficult. It takes time and systematic process to polish a rough diamond. It’s no different for developing and optimizing an ad. You need a series of step-by-step actions that will lead you to your desired outcome.
If you want an ad that sparks interest with the best customers, you need a process.
PLANNING & RESEARCH
Define your business goals
The best campaign strategy for you will always need to be crafted specifically to match the overall goals of your business.
You wouldn’t start a journey without knowing where you’re going, so make sure you know where you’re headed (your goal) and how you’re going to get there (your strategy).
If you’re unsure what your strategy is, or how to define it, I recommend reading my posts on data-led marketing strategies and finding your target market.
Understand your campaign objectives
If you have multiple campaigns each with different conversion objectives, for example, brand awareness, lead generation, or sales, you’re going to need a separate testing program for each. You can use the same step-by-step process, but you cannot compare them alongside each other.
The ad copy for a lead generation campaign will likely be to-the-point and driving the user to convert. The advertiser will want to highlight the key features that separate them from their competitors.
Someone looking for a time-sensitive product or service is in a different part of the purchase funnel and has different needs. The advertiser may add time-sensitive offers and include location-relevant information.
If you’re running a branding or traffic campaign, you are more likely focused on the product, the solution it provides, or the problems it solves.
If you’re generating e-commerce sales with video and display ads, well, that’s very different again. Consumers may not be aware of the products, so the job of the advertisement is to educate and inform.
Know your marketing channels
If you’re running your ads across many channels, you’re going to need to develop specific ad copy and creatives for the requirements of the platform, and that can vary a lot.
Each platform has a particular ad requirement. Facebook, for example, has many different formats, and they are always changing.
It’s best to Google the latest specs to see for yourself.
Likewise, there are various requirements for video, text, and display ad formats on Google, Microsoft, Pinterest, etc.
Knowing the channels also means you need to consider the way the platforms’ algorithms work and the way consumers use the platforms.
Google & Microsoft Search Ads
I use search platforms like Google and Microsoft Ads, to typically target consumers using three types of search query.
- Navigation queries from customers looking for a particular website or webpage. A user might search for “RBC” to find their bank account rather than entering the URL into a browser’s navigation bar.
2. For Information queries like “how”, “where”, and “who”, the user is looking for information and answers to a question.
3. Transaction search queries from users that intend to convert. They often include brand and product names like “Dyson V7” or more generic terms like “chargeable hand vacuum” and sometimes include phrases like “buy” or “purchase.” From the query, we can assume the searcher is considering a purchase soon.
Recently, I’ve seen people complaining about how expensive search advertising has become. However, the value or search depends on your strategy, how hard you are pushing for your objectives, and if you’ve accurately costed your target acquisition costs.
Google ads use a ranking metric called Ad Rank to determine where a sponsored ad is shown on a page relative to other ads. This is determined by several factors that you have control over:
- Your bid amount
- Auction time
- Expected clickthrough rate
- Ad relevance
- Landing page experience
Microsoft ads use a similar score.
Facebook & Instagram Social Ads
Unlike Google’s Ad Rank, Facebook uses a Total Value metric to determine the winner of the ad placement.
Why? Well, a social network exists to feed the engagement of its users. It’s crucial that a social network ensures the content users see continues to satisfy their needs, or they risk losing them.
For that same reason, the metrics used to determine the Total Value of your ad are:
- Your bid
- Ad quality
- Estimated action rates
You may also find with social ads you have to update them significantly more because of the way users engage with it. If they see the same content over and over, it goes against what the platform is trying to achieve.
Other Ad Platforms
I’ve only scraped the surface here, but if I don’t stop, I’ll never publish this article. I will add more in the near future and I encourage you to investigate the following platforms to see if they would be suitable for your needs.
Heuristic research
Research for new ads usually begins with an experience-based assessment. The assessments help you uncover areas of interest that you can validate with qualitative and quantitative research.
You know your business and your customers better than anyone. Start with a structured question and review assessment of your ads and what you think your customers would need to see to achieve your objective.
- What do you think is the biggest problem your product/service solves?
- How significant would the change be for your best customer?
- Are the changes lasting?
- How does your customer measure success?
- What’s the risk to your customer if they don’t take action?
- Who is this for?
Use the structured analysis data sheet to guide your heuristic analysis.
Quantitative research
Your analytics data should be the next step in your research. Look at data from the last three to four months for ideas and inspiration.
Starting with a web analytics tool, like Google Analytics, try to uncover the content that interests your customers.
Where do they spend their time and where do they go next? How do they behave on the site? Are their purchases planned or impulsive? Do they make recurring purchases?
Use the search query reports in Google and Microsoft ads to get a better understanding of the underlying issues that drive users to convert.
What are they in the market for? Are there common search queries or keywords they use? Is their purchase a luxury or a necessity?
Open up the Audience Insight tool on Facebook and try to put build a picture of your best customer’s interests and demographics.
What’s their age and gender? Do they have common interests?
In the Google Search console, you can get a very top level view of the search queries people are using to uncover your site organically.
What are they in the market for and what language do they use?
SEO tools like SEMRush and Ahrefs are my go-to competitor research tools.
Now find out, what channels and formats worked in the past?
What placements worked well?
Open up your old ad reporting.
- Were there specific keywords that were included in the ad?
- What were the landing pages?
- Review the creative used, was there a specific theme?
- Were the customers targeted with an offer?
- Did the ad describe a pain, risk, problem or solution?
The data from your quantitative analysis will reveal what your audience does and how it behaves but it won’t tell you why your audience behaves in those ways. You can use quantitative analysis to focus your attention on what to test.
And last on my list is customer surveys.
Qualitative research
Customer observations, interviews, and focus groups are all great sources of qualitative data but they can’t replace a conversation with your best customers. Pick up the phone, send an email, and utilize messenger to start conversions.
Become your customer
- In relation to your product or service, what’s their biggest problem?
- What would the best outcome for your customer be?
- Would the results last a long time?
- In their eyes, what does success look like and how do they measure it?
- What’s the risk to your customer if they do nothing?
- Who is the person that your offer solves a problem for?
Talk to your current customers
- Do they work or don’t they work?
- Where do they work and what do they do?
- Before they decided to purchase from you, what was the biggest problem they were hoping to fix?
- Did they feel like there was something that you offered that others couldn’t or wouldn’t provide?
- What was their biggest hesitation when deciding and how did they overcome that hesitation?
- Which social media platforms do they use?
- What are their hobbies and interests outside of work?
- What websites or blogs did they use for research?
Scrape group and forums for ideas
Quora
To uncover why certain ad might work well, research online groups and customer forums to find out more about a particular need.
Open up Quora and search for your product or service.
Saveonit, a client of mine, sells enterprise level software. Many of their ideal customers are using Microsoft Access and tend to have the same problems.
With a little research on Quora, I found exact issues many customers had about Microsoft Access.
- They didn’t believe it was a scalable platform
- It’s inefficient and difficult to maintain
- They wanted to know what the most popular desktop database was on the market
- Corruption of the data was a common problem
- Many felt that access wasn’t an enterprise solution
- Poor security was often highlighted
- A lot of people felt Access was hard to deploy
There was so much material here for us to start testing ads with. So, I created an ad to address each of these issues.
Find a subreddit where you think your target customer might hang out. Then take a look at some of the most popular recent threads. Look at the language people use to describe what they like and don’t like about the current options on the market.
In the same way that we use Quora, we can harvest ideas and themes from Reddit.
Amazon Reviews
People on Amazon don’t hold back with the reviews. For that reason, the product and book reviews are a great place to mine brutally honest feedback.
What language do they use? Why did the product help, or not help? What was it like before they bought it, and what was it like after?
In most cases, the more you know about your customer, the more you can create an ad that speaks to them.
Create an ideal customer avatar
If you haven’t already, using the data you have gathered from your research, create a customer avatar. You’re are not limited to one. It’s better to have multiple accurate avatars than one general avatar.
- What are their interests?
- Where do they spend their time online?
- What are their concerns?
- Why would they work with you?
- What is it that drives them to buy your products and remain loyal over a long time?
You can use the customer persona worksheet to help.
The founders of Lululemon believe so strongly in the power of customer avatars that they created a second brand named after the avatars, that brand is Kit & Ace.
AD CREATION & PUBLISHING
Ensure consistency
Do you already have a look and feel to your ads? Have you ever noticed that some advertisers all have the same tone, voice, or theme to their ads? That’s because that have spent the time to create a style guide.
Arguably, the perceived brand quality will also impact your cost and performance. Using a consistent style should help you maintain that brand quality, and thus, lower costs and improve performance.
The style guide you design usually aligns with your brand, a visual cue that it’s one and the same. If your brand is targeting Generation Z then you recognize they move seamlessly between brands, they’re bound to their devices and they aspire to have security and stability. Your ads should also follow those same themes.
For my client, Pangea Maps, we created a style guide to define the rules and aesthetic of the brand.
Ours was simple. It established acceptable composition, colors, fonts, filters, and hashtags.
Your style guide doesn’t have to be complicated. It’s just important to have a guide to keep things consistent and professional.
Experiment with styles and formats
People use different digital channels for various purposes, and your ads have to reflect that.
For search ads, think about the company tone. Is it jokey or to the point?
YouTube is a search engine and so you may choose to inform or entertain.
Snapchat ads may need a very different style to your Instagram and LinkedIn ads.
People use Pinterest for curating ideas, bookmarking, inspiration, and promotion. Depending on the type of user you’re trying to engage with, the ad would be different.
Create employee stories
Experiment with the types of content you want to share with your audience.
For Pangea Maps, we tested feature stories about the founder, Tom on Instagram, and what is going on in his life.
Culture content
We used Pangea Map’s Instagram page to tell a story and share their company culture.
Product demos
Of course, we also shows off the amazing maps in carousels, collection ads, and videos of the laser cutter.
Dynamic ads
Most of the major platforms are now introducing responsive or dynamic ads to their platforms, and in my experience, they work well.
They let you create an ad that adapts to show more relevant messages to your customers based on what device they are using and what the platform knows about them.
There are a few different types; for example, there are the fully dynamic versions where the platform builds your ad based on a landing page. Or Ad customizers which adapt your text ads to what someone is searching for.
With the newer formats, you can enter multiple headlines, descriptions, and images. Over time, the ad platform will automatically test combinations and learn which perform best.
By adapting your ad’s content to match potential customers’ interests or search terms, you can improve the relevance of your ad, which will impact your campaign’s performance.
Personalized content
When we create ads, we try to personalize an ad as closely as possible to a user. But what does this mean?
Realistically, person to person personalization isn’t possible. Instead, we create the illusion of personalization by using micro-segmentation.
For Pangea Maps, we showed the audience maps of their local coastline or a coastline they had recently visited. It was an emotional hook. GARRR!
For search ads on Google and Microsoft, we would try to include the keywords that we are targeting in the ad copy.
Alternatively, you can use dynamic ad formats to insert the keywords your customers use into the ad copy.
Copywriting formulas
I rarely start from scratch. Instead, my ads come from proven templates that I have used with success over and over.
The Classic Digital Ad
An important goal of your ad is to keep moving the prospect along to the next step of the customer journey.
With each step, they learn a little more and start to understand better the problem you solve. From Awareness to Consideration, to Purchase.
Creative and headline – Get their attention
Opening line – Spark a match inside them. Relate to an issue they have.
The body – The fire burns brighter. They find what you have described valuable, or at the very least it describes the problem they are facing.
Final line – When they’re burning hot make an offer.
Call to action – Get them to take action
E.M.E.R.I. Method
The E.M.E.R.I scoring system is a method I learned and repurposed from The Uncommoners Club. Created by Ami Sanyal for writing engaging social media posts, I use it for writing high-engagement social media ads.
I start by drafting a post as usual, then, ask, would my ideal customer find this post–
- Educating: Are you teaching them something?
- Motivating: Are you pushing them to succeed?
- Engaging: Are you using direct call-to-action?
- Revealing: Are you letting prospects peek behind the curtain or glimpse the human side to your business?
- Interesting: Does your ideal customer care?
You get a point for each one you can put a tick next too. If you score below four, work on improving it.
Checklist Method
There’s a lot squeezed into this checklist, and it won’t come together without some dedication.
For the Checklist method, we force ourselves to stop and think about each of these things. We cannot fit them all into one ad, but we try to put as many in as possible.
Pain, Empathy, Challenge, Risk Method
The ad copy for each ad should focus on one of four emotions:
Pain – Describe the frustration and difficulties that the reader feels, and your happy customers felt. Talk about the solution that made your customers lives more accessible or better. Tell the reader how life was before, how it was after, and the steps from A to B.
Empathy – Explain your experience, your understanding, and how it impacted you. Share why you’re offering this solution.
Challenge – Throw a spanner in the works by challenging everything they believe about the problem. What’s the common idea? Now, tell the reader the opposite is true. If you only have the same thing as everyone else to say, why would they listen?
Risk – There are some obvious risks, like not buying insurance. But what are the risks involved in not purchasing a holiday, a new car or a new SaaS? Think about the missed opportunities to travel, the dangers of an old car, and the lost revenue from out of date software. If the reader doesn’t take action soon, what could it cost them in the long term?
AD TESTING PROCESS
So now you’ve got your bits and bobs together, it’s time to get some ads live! But, of course, like the calculated marketer you are, there are several processes that you’ll want to have in place before hitting publish.
For every round of testing, you create several ad variations. There’s no limit to the number of times you launch a series of testing. Each round is simply an opportunity for you to learn more and improve performance.
Evaluate which ad performs best, and then create a hypothesis. Based on that hypothesis, you can create a new round of testing.
Create your testing hypothesis
When I talk about testing, I’m talking about testing a hypothesis or idea. You’re going to take your assumptions that you formed from your research and turn them into insights.
Then, you’re going to calculate the financial risks involved in choosing to implement the results of your tests. The reason you do this is that you are trying to reduce the likelihood of making a poor decision.
The hypothesis is generally formed around an idea you have and something you want to learn about your customers. This is what you were gathering in the research phase.
In each round of testing, you could be looking for answers to different questions.
To use split test tools or not
When launching your new ads, you have two options. You can choose to use the ad platforms split testing tool (Google ads, and Facebook ads both have AB testing tools you can use), or you can put your ads in one ad set/ad group without using the native split testing tools.
If you choose not to use the internal testing tool, you risk corrupting the data from your test. It doesn’t mean you can’t do it, it just means that there is a higher risk that the results will be inaccurate.
The problem with all the major ad platforms is that you cannot accurately test variables against each other without using their internal split or A/B testing tools.
The problem lies in the way the platforms deliver their ads. For example, if I create two audiences with standard campaign setup, Audience A and Audience B, there is no way for me to only show Ad One to only Audience A, and Ad Two to only Audience B.
The Ad Platform may inadvertently show Ad One to Audience B or vice versa. This means that my sample audience is corrupt. They may have been influenced by the message from the competing ad, and my test is now invalid.
By using the platforms’ split testing tools advertisers can create multiple ad sets and test them against each other to see what produces the best results.
The benefits to using the split testing tools:
- Divide your audience into random, non-overlapping groups
- Only allows you to test one variable at a time
- The cost per results are accurately calculated and compared
There are downsides to using the split testing tools:
- You are usually limited by the number of variations you can include
- You cannot call them early. You must go at their pace
If you want to learn more about running an accurate split test, I suggest reading my how-to guide on A/B testing.
Testing template
To help you, I’m going to share some example testing rounds so you can visualize the ads.
Test One – Themes
The goal is to find out what themes our potential customers respond to.
- If the ad copy is related to the customer’s pain
- If we are empathetic
- If we challenge their beliefs
- The risk of not taking action
How do they respond?
The goal is to find the themes that move target customers closer to the objectives outlined at the planning phase.
Now stop. Review. Hypothesize.
Why did one theme perform better than another? The next step is to form a test based on your hypothesis. Wish to prove or disprove your belief.
If, for example, you found that the “risk” ads worked the best, you would start a new round of testing. Create four new ads that follow the same theme (the new set of ads should include a “risk” variation.) Can you repeat the previous results?
Using the methods described in the A/B testing post you can work out if your hypothesis is wrong. You can also work out the potential cost to your business.
Test Two – Ad Copy
Once we know what themes work, we can create ad copy specific around those themes and test them further — refining the pain points, expanding the audiences also.
In this test, we would change, descriptions, headlines, titles, and call-to-action buttons.
Try adding a credibility factor to the ad, or a time-sensitive offer.
Test Three – Engagement
Engagement’s important because it can unlock the scale accelerator. When designing and optimizing ads, I’m trying to hit a scale accelerator.
If you are optimizing your ads for leads or sales, many argue that your clicks, click through rate and cost per click metrics are irrelevant. I believe that’s wrong because there exists a point that we call a scale accelerator. If your metrics reach a certain point, it can unlock dramatically reduced costs and raise the number of conversions an ad can earn.
These accelerators exist because only a few ads can achieve and maintain a high level of performance. These creatives shoot straight to the top of the ad auction and win all the high-quality impressions very cheaply allowing them to in turn buy more, leaving advertisers with poor performance to be punished with higher costs and poor placements.
The exact algorithm that each platform uses to rank an ad isn’t public knowledge, but Google does give us some information on Ad Rank and Facebook tell us how the ad auction works.
Our objective is to raise the engagement metrics because the ad platforms will reward our ads for better engagement metrics with cheaper costs and scale in the auctions.
Test Four – Creatives
Testing creative formats are one thing, for example, image vs. video.
The exciting findings are usually what’s in the creative. You could test;
Using pattern disruption to break up the monotony of what a user sees.
Something very relatable, for example, recognizable local landscapes or points of interests.
Happy customers enjoying the product.
Video testimonials.
One audience or more
Decide if you’re going to test your ad variations against one audience or more than one.
If you only have small data accounts, you may be forced to run your ads across many audiences. If you have a large account, adding ads to multiple audiences can also be a great way to test hypothesis or market research.
The downside to this is that if you are reviewing the performance at scale, not the performance of the individual ad groups, some ad groups may perform better but the weighted average will show you the overall winner.
If you’re testing high volume or your best-performing audiences, for example, your brand terms, your best keywords or your lookalike audiences you would be best to use single audience testing as you will find the best ad for each targeting type.
Success metrics
Once you have determined your hypothesis for testing, you need to determine how you will pick a winning ad, what metrics will tell you that your ad tests are winners and losers.
There’s a lot you can choose from. Conversion rate, conversions, click through rate, cost per reach, cost per landing page view, cost per impression. Ultimately I review these metrics in aggregate.
You will almost definitely find that some ads have rock bottom cost per view or cost per landing page view but often very high cost per conversion. For that reason, this is something that you will need to assess yourself based on your business goals and campaign objectives.
How to Use labels for ad testing
Labels are one of the most useful features on the ad platforms. For every ad you create add a label.
The current best performer is labeled with “Control” and your “Variation” ads are labeled accordingly.
Usually you’ll have at least four ads running at a time, and having the labels in use and visible will help you quickly determine how the test is going.
Optimization Process
Gather enough data
When determining minimum data, there are two considerations:
- What is your testing metric
- How much data do you generate each month
To declare a winner, you must have collected a minimum amount of data. That minimum will be based on the size of your sample audience, the number of impressions, clicks, and or conversions you have achieved.
If you are keen to learn about the methods described here, I suggest reading my blog post on A/B testing.
If your ads don’t have enough data, you shouldn’t pause ads or make adjustments as it will skew the experiment results.
You need to avoid situations where just one or two people can significantly affect your data.
In this hypothetical example, our control ad has delivered 234 impressions and ten clicks, so the click-through rate is 4.27%. If each ad delivers one more impression, but one ad delivers a single click the click-through rate for that ad will change dramatically and have a significant impact on which ad is seen to be the better performer.
I used abtestguides.com Bayesian Calculator for my examples here.
Calculate your sample size
This is important to do, and if you’re using a built-in split testing tool, it will probably do this for you. If not, use one of the many online audience size calculators to work out how big your audience sample size needs to be.
Use the audience size calculator to estimate how long the test will need to run.
Factor in timeframes
Consideration the variance of time. Unfortunately, the impact of time on your testing cannot be calculated purely with math.
As the timeframes change, so does your target users behavior – this is why we need to take into account not just the data, but the timeframe of the data. I recommend you use at the very least a minimum of one week of data.
Identifying a winner
Deciding on a winner is an easy process if you have completed the steps described above:
- Testing hypothesis & test type
- Testing metrics
- Minimum data
- Maximum data
- Confidence factors
If you’re ready to take action:
- Pause the losing ad variations
- Examine the ads for themes that worked
- Create a new round of testing
Templates Used in This Post
Structured analysis data sheet
Ad QA checklist
Search ad templates
This post is part of a series to help marketers better understand their data. If you would like to see a change in the way you start A/B testing, it begins with a conversation. Reach out to me at mike@knowndigital.com.
