How to Use Lookalike Modeling to Better Segment and Target Ad Audiences

  1. Can this entire setup run smoothly and with a low runtime to maximize revenue?

Hyperopt

In a nutshell, Hyperopt allows us to quickly train and fit multiple sklearn-models across multiple executors for hyperparameter tuning and can search for the optimal configuration based on previous evaluations. As we try and fit multiple models per campaign, for multiple campaigns, this allows us to quickly get the best hyperparameter configuration, resulting in the best loss, in a very short time (e.g.: around 14 minutes for preprocessing and optimizing a random forest with 24 evaluations and a parallelism-parameter of 16). Important here is that our label is the propensity to click (i.e., a probability), rather than being a clicker (a class). Afterward, the model with the lowest loss (defined as — AUC of the Precision-Recall), is written to MLflow. This process is done once a week or if the campaign has just started, and we get more data for that specific campaign compared to the previous day.

PandasUDF

After we have our model, we want to draw inferences on all visitors of our sites for the last 30 days. To do this, we query the latest, best model from MLflow and broadcast this over all executors. Because the data set we want to score is quite large, we distribute it in n-partitions and let each executor score a different partition; all of this is done by leveraging the PandasUDF-logic. The probabilities then get collected back to the driver, and users get ranked from lowest propensity to click, to highest propensity to click:

Conclusion

In short, we can summarize the entire process as follows

  • Max: Highest Precision-Recall AUC of an evaluation within the daily hyperopt-run
  • Min: Lowest Precision-Recall AUC of an evaluation within the daily hyperopt-run
  • St Dev: Standard deviation Precision-Recall AUC of all evaluations within the daily hyperopt-run

--

--

We are the tech team behind the digital products of all DPG Media’s brands and internal apps!

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store