There is no doubt the rapid changes to the ads ecosystem make it much harder to measure the true value of marketing. Fortunately, geo experiments offer a great way to quantify ad effectiveness via lift. Facebook purposely created GeoLift, a leading open-source tool that makes it easy to design and run robust geo experiments. This article provides everything you need to know about GeoLift and how it helps measure incrementality.
What Is Incrementality in Marketing?
Incrementality is the lift or increase in the desired outcome provided by marketing activity. The outcome could be profitability, revenues, conversions, web visits, or awareness. Incrementality testing and measurement provide a true incremental contribution of your paid media at the ad, campaign, tactic, or channel level. Incrementality in marketing is specifically necessary for channels where ad impressions are challenging to map and measure, like social channels such as Facebook, Snap, Pinterest, and more. With incrementality measurement, you can determine:
- Waste to eliminate and opportunities to scale, expand, and divert media spend for optimal growth
- The media channels, publisher, campaign contributions to the marketing goals
- Whether to launch new campaigns or ads to increase contribution to conversions at the portfolio level
Why Is Incrementality the Gold Standard of Measurement?
Incrementality-based measurement allows you to determine the actual value of your marketing efforts. Essentially, a campaign’s incremental effect is the difference between what you observed as the results of a specific campaign and what could have happened if it didn’t take place.
FREE GUIDE
Digital Marketing Attribution and Measurement Roadmap
We layout a roadmap of all the measurement and attribution tools direct response advertisers should use to better understand their media’s impact.
What Is GeoLift?
GeoLift is Facebook’s open-source solution designed to measure lift at a geographical level. The tool utilizes advanced synthetic control methods to deliver insights that help you make informed decisions established in incrementality and measure the true value of your marketing campaigns. GeoLift provides unmatched solutions for geo experimentation, spanning data ingestion, power analysis, and market selection to inference. Besides, the tool’s open-source nature makes it transparent and fully reproducible. The tool is highly adaptable to the needs of various businesses, making it ideal for users who want to employ other assessment strategies due to data limitations.
Importance of Lift Analysis
Lift analysis is a method for determining how a marketing campaign affects specified indicators. It is a technique for tracking variations in metrics such as clicks, conversions, engagement, impressions, and more. It measures the percentage rise or decrease in each metric for users who were the recipient of the campaign against a control group. Ideally, the control group that was not presented with the campaign is called the lift.
Credit: https://mackgrenfell.com/blog/conversion-lift-tests-are-dead-transitioning-to-geo-experiments
In a nutshell, lift analysis is a technique that compares users who receive a campaign vs. those who do not receive the campaign to gauge the better group off. This provides excellent insights into the effectiveness and impact of the campaign. Determining the lift through the use of a control group is the most effective way to determine the true impact of your ad campaign. The following are some of the ways lift analysis can help your business:
- Determine whether your ad campaign is triggering short-term spikes in engagement or long-term increases in loyalty and app usage.
- Determine the ROI of each marketing strategy,
- Save valuable time and avoid targeting your campaigns in areas with high clicks but low sales.
- Track repeat conversions instead of one-time conversions
What Does GeoLift Do?
GeoLift comes with a pretest dataset included in the package. The package, which consists of the simulated data of 40 cities in the USA across a period of 90 days, is comprised of three variables, namely:
- Location (city)
- Y (total number of conversions each day per location)
- Date (in “yyyy-mm-dd” format).
GeoLift can effectively perform the following tasks:
Handle Data Processing
After you feed the dataset into GeoDataRead, it will assess the dataset and perform other related tasks. These include handling locations with missing data by configuring the ideal method of imputation. It will also transform the dates in the date column to timestamps. Since it is a data-backed process, the time series would indicate a similar pattern being shared across multiple locations.
Carry Out Power Analysis
Power analysis is the vigorous and comprehensive statistical analysis to extract crucial information from data. Power analysis is an essential step to a successful test. It helps you find vital parameters such as:
An optimum number of test locations: To find out the optimum number of locations where running your test would deliver the best results, you have to leverage the NumberLocations() functions in the GeoLift package. The function is designed to simulate many tests and evaluate their outcomes.
Insights: The plot of Average Power vs. Number of Locations helps you observe how to get a well-powered test even from a few locations. It delivers a perfect start to your analysis so that you run the test in a few preferred locations and still get optimum inferences from it.
Probable action: You can choose two to three locations to run the test in and obtain the best results with minimal effort.
Determine the ideal test and control location: Once you determine the number of locations, you can move ahead to work out the best combination of test locations to run the test. To achieve this, leverage the GeoLiftPower.Search() function. The function helps provide you with a ranking of various location combinations. This helps you achieve the following:
- Insight: You can see the ideal test locations combinations to focus your campaigns
- Probable action: You can implement your test in one of the top-ranking markets while putting all the other cities in the dataset in the holdout group.
Determine the minimum detectable effect to get substantial results: Although GeoLiftPower.Search() reveals the test and control markets that can reliably produce well-powered results in synthetic control tests, it can’t calculate the lift you need to obtain a powerful test. Consequently, you must now employ the GeoLiftPowerFinder() method designed to investigate the Minimum Detectable Effect that a combination of test and control groups can produce.
Find out a suitable test duration and ballpark for the budget required to do a test: Once you have determined all your obtained parameters, you can now find out the ideal duration for the test and the reasonable budget you need to run it. To do this, find out the Cost Per Incremental Conversion (CPIC) to use the GeoLiftPower() function. The CPIC enables you to obtain the approximate value of the budget you need for a test.
Determine Actual Lift
GeoLift can also help you compute the actual lift that your marketing campaign has achieved. You will leverage the GeoLift_Test dataset included in the GeoLift package for this purpose. The database comes with resultant sales after the 15-day test period. Ideally, the GeoLift() function takes the GeoLift data frame as an input along with information about the test. For example, the cities in the treatment group, the time the test began, and the time it concluded.
FREE GUIDE
Digital Marketing Attribution and Measurement Roadmap
We layout a roadmap of all the measurement and attribution tools direct response advertisers should use to better understand their media’s impact.
Benefits of Using GeoLift to Measure Incrementality
Help Brands Invest in Data-Based Decision-Making
Advertisers often face myriads of issues with their measurement frameworks. One of these challenges is a tendency for brands to spend a bulk of their budgets on one or two digital channels, making it more challenging to determine the ROI of investing in new approaches. GeoLift can be a valuable tool to overcome these challenges. A data-based approach of ad performance can be achieved because ads are served to a subset of people and not a similar subset. By controlling other factors that could influence performance, lift tests enable advertisers to measure instrumentality and determine the direct impact caused by marketing activity.
Evaluate Media Mix More Effectively
When it comes to lift studies, brands often have a tendency to focus their spending on one or two approaches only. This means that advertisers get only the picture of specific channel performance rather than overall performance. GeoLift tools help advertisers maximize the benefits of lift testing by delivering a comprehensive view. Because geo lifts allow you to apply geographical boundaries to test and control groups, you gain the flexibility needed to understand better the performance of different media sizes rather than just individual channels.
Help Understand the Incremental Impact
GeoLift can help marketers understand the incremental impact of their campaigns vs. the budgets allocated. Determining the ROI of investing in new marketing approaches can be challenging for most advertisers using traditional techniques. Lift tests help brands understand relative channel performance and effectively compare diverse media mixes.
How to Implement a Geo Experiment
Follow the steps outlined below to set up a geo experiment:
Step 1: Geo Identification
Your first step is to determine the coverage area you would like to run your test at. For example, if you are targeting the entire US market, it may make sense to have states as your geos. If you are targeting a specific state market, consider leveraging DMAs or zip codes further to scale down your region of interest to smaller units. To achieve robust statistical measurement, avoid the temptations to select geos that are too small such as the zip codes. Ideally, people tend to move across geo-boundaries meaning the volumes of conversions will be too low with small geos.
Step 2: Geo Randomization and Assignment
After defining the market and its partitioning, the next step is randomly assigning each geo to the treatment or control group. Robust randomization ensures minimal differences between two groups aside from the ad serving. This function prevents possible biases. This step ensures the treatment and control groups are as similar as possible before you launch your campaign. Unfortunately, differences in geos may be prevalent, and if not controlled, it can hurt the design and accuracy of the test. It is advisable to do a preliminary analysis to determine the best geos to incorporate in the treatment and control group to avoid this.
Step 3: Measuring a Geo Experiment
After you set up the experiment, use econometrics methodologies such as Difference in Differences or Synthetic Controls to quantify the ad incrementality. This is basically a critical step in determining the number of conversions that would not have happened if the ad was not serviced.
GeoLift: FAQs
Do I Require User-Level Tracking to Run Activity in Specific Locations?
GeoLift works by dividing the country geographically to make a control/test split. This means that it won’t require user-level tracking. All you need to run GeoLift successfully is the ability to only include activity in specific locations. After running the test, you will determine whether your spending resulted in a significant uplift in the areas you spend compared to your control geos.
How Does GeoLift Testing Differ from the Standard Lift Tests?
A crucial question that most marketers ask is how GeoLift testing is different from standard lift tests that ran in the past with Facebook. Ideally, with the old way of making conversation, lift tests on Facebook compared control vs. exposed at that moment in time. With GeoLift, we evaluate what we anticipate would happen had ads not occurred using pre-test period data relative to the control. We then calculate the difference between what would not have happened and what actually happened during the test. Another key difference is that lift tests are more comprehensive and complicated than the standard lifts tests you may have run in the past.
What Are Facebook’s Brand Lift Tests?
Brand lift is a type of lift test where you utilize brand polling and other brand awareness measurements to gain an understanding of the true value of your Facebook advertising and how well it performs independently of your other marketing approaches. Brand lift works by choosing a representative sample of people who qualify to see your adverts. The sample group is further subdivided into the test and holding groups to allow for causal inference techniques to be used to measure the impact of advertising activity.
Key Takeaway
GeoLift is a powerful lift measurement solution and an ideal tool to analyze your advertising strategies. It can help businesses gain greater insights and data-backed decisions while course-correcting their marketing methods. GeoLift is designed to be relevant in the ever-changing advertising ecosystem, thereby helping brands correctly quantify the effect of their marketing. In a nutshell, GeoLift delivers an effective way for brands to move ahead into the future and overcome typical marketing challenges common with traditional techniques
If you have further questions or need help to get started with this, too, don’t hesitate to contact us.
