Mass-model campaign optimization platform with a fully automated retraining pipeline

The client is the largest online marketplace in Central and Eastern Europe, operating across 10+ countries with an integrated ecosystem serving millions of users. Their platform connects 128k merchants with 21 million customers, facilitating 70 million item sales monthly through comprehensive marketplace, advertising, and data analytics services.

Before implementing the AI-powered optimization platform, the client faced several critical advertising challenges:

• Manual campaign optimization – Media buyers had to manually track and adjust campaign variables, creating operational bottlenecks and suboptimal performance
• Volatile traffic patterns – Fluctuating user behavior and market conditions caused performance degradation without automated adjustment mechanisms
• Fixed pricing inefficiency – Using the same bid price for all impression opportunities led to overpaying for low-value ads and missing high-value opportunities
• Scale management complexity – Managing advertising campaigns for 128k merchants required automated solutions beyond manual capabilities
• Performance inconsistency – Lack of real-time optimization resulted in higher costs per click and lower conversion rates across merchant campaigns

The company needed an intelligent advertising platform that could automatically optimize campaigns in real-time, minimize costs, and scale efficiently across their massive merchant network.

Click events were rare compared to non-clicks, creating training challenges for accurate conversion probability prediction models across different merchant campaigns.

Solution: We implemented resampling techniques including SMOTE (Synthetic Minority Oversampling Technique), class weighting in algorithms, and specialized loss functions to balance datasets through over-sampling and under-sampling strategies.

Predictions must be made in milliseconds to participate in RTB auctions, requiring optimization between model accuracy and inference speed for thousands of concurrent campaigns.

Solution: We prioritized latency over accuracy in model selection and tuning, implementing lightweight models that meet strict timing requirements while maintaining sufficient predictive performance for CPC optimization goals.

User consumption behavior varies vastly across different product categories – people buy dog food, consumer electronics, or luxury perfume differently, making a single model ineffective for all campaigns.

Solution: We developed separate models for every user behavior class, classifying all product categories into logical classes by consumption behavior patterns. Classification was based on marketplace taxonomy (Electronics, Baby Items, Mobile devices) and brand taxonomy, later implementing more complex statistically-defined similarity classification.

Categorical variables like user demographics and ad IDs led to high-dimensional feature spaces, creating computational and storage challenges for real-time bidding systems.

Solution: We employed dimensionality reduction techniques like PCA and special encoding methods including target encoding and binary encoding that don’t increase training data dimensionality while preserving predictive power.

User behavior patterns, ad creatives, and competition landscape change over time, causing model performance degradation as training data distributions shift from current market conditions.

Solution: We built automated monitoring for data drift with active learning and periodic retraining processes, including repeatable drift identification, threshold-based alerting, and proactive model replacement mechanisms.

Most campaigns start on merchant requests without manual optimization, requiring fully automated pipeline covering model selection to continuous improvement across hundreds of diverse campaigns with varying durations.

Solution: We created a comprehensive automated pipeline with pre-trained model repository, permanent performance monitoring, gradual model switching (5%, 10%, 50% up to 95-98%), and unlimited lightweight model deployment capability handling thousands of models without infrastructure impact.

We permanently and automatically evaluate the model’s performance within each campaign. Once the degradation signs are detected, the re-training or replacement process is automatically triggered. Also, model replacement can be initiated according to the schedule.

We switch to the new model gradually (e.g., 5%, 10%, 50%, and up to 95-98%). We always leave some campaign traffic without going through the ML CPC optimization model.

It is worth mentioning that model performance is a complicated criterion. It was initially set into empirical values but tuned throughout the system lifecycle per actual performance.

We use lightweight models, so training and deployment are never a bottleneck.
Furthermore, our proposed infrastructure can handle as many models as needed,
allowing us to create thousands of models in fully automated pipelines without any
infrastructure performance impact.