🤖 machine learning

Click prediction boosting via Bayesian hyperparameter optimization based ensemble learning pipelines

This paper proposes a Bayesian hyperparameter optimization-based ensemble learning pipeline that combines feature elimination with stacked models to improve hotel click prediction accuracy for online travel agencies by approximately 10%.

Original authors: Çağatay Demirel, A. Aylin Tokuç, Ahmet Tezcan Tekin

Published 2026-07-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Çağatay Demirel, A. Aylin Tokuç, Ahmet Tezcan Tekin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a giant, bustling digital marketplace where millions of people wander around looking for the perfect place to sleep. Before they book a room, they often hop onto "meta-search" sites—think of these as massive comparison shops like Kayak or TripAdvisor—where they can see prices and reviews from dozens of different travel agencies at once. To get noticed in this crowded marketplace, travel agencies pay for ads. But here's the tricky part: they usually pay only when someone actually clicks on their ad. If they guess the price wrong, they might spend a fortune on clicks that never happen, or worse, they might bid too low and miss out on thousands of potential customers. So, the big question for these agencies is: "If we set our price at this specific number, how many people will actually click?"

To answer this, scientists use a branch of computer science called machine learning, which is basically teaching computers to find patterns in data rather than just following rigid rules. One popular method is "ensemble learning," which is like forming a super-team of detectives. Instead of relying on just one detective who might miss a clue, you gather a whole squad, let them each investigate the case, and then combine their theories to get a much more accurate answer. This paper dives into that world, trying to build the ultimate team of algorithms to predict hotel clicks for online travel agencies.

The authors of this study set out to solve the click-prediction puzzle using a massive dataset from a major online travel agency in Turkey. They didn't just throw one algorithm at the problem; instead, they built a sophisticated pipeline to create a "dream team" of predictors. First, they cleaned up the messy data, filling in missing gaps and adding helpful context like weather reports and how close a day was to a holiday. Then, they used a smart filtering process (based on a tool called XGBoost) to throw away the noisy, unhelpful information, leaving only the most important clues.

Next, they trained ten different types of mathematical models, ranging from complex tree-based systems to simpler linear equations. But the real magic happened when they combined them. They tested four different ways to mix these models: simply averaging their answers, giving more weight to the smarter models, and two advanced techniques called "stacking" and "blending." Stacking is like having the first team of detectives write a report, and then a second, senior team reads those reports to make the final verdict. Blending is similar, but the senior team also gets to look at the original clues again while reading the reports.

The results were quite clear. While individual models were decent, the team effort was significantly better. The paper suggests that the best approach was the "stack ensemble" model, specifically when the senior team used simple tools like linear regression to interpret the reports from the first team. This top-performing model achieved a score (called an R² of 0.639) that was about 10% better than the best single model on its own. The authors found that simpler models worked best as the "final judges" because the heavy lifting was already done by the first layer of models. They also noted that more complex models didn't necessarily make the final decision better; in fact, sometimes they made things worse.

In the end, the study suggests that by carefully selecting features, tuning the settings of the algorithms, and combining multiple models in a smart way, online travel agencies can get a much clearer picture of how many clicks their ads will generate. The authors propose that this method is a promising direction for the industry, though they admit there's still room to grow. They hint that in the future, they might try adding even more types of models, like artificial neural networks or specialized tools for handling categories, to see if they can squeeze out even more accuracy. For now, however, the evidence points to a simple truth: in the world of predicting clicks, a well-coordinated team of diverse thinkers beats a lone genius every time.

Technical Summary: Click Prediction Boosting via Bayesian Hyperparameter Optimization Based Ensemble Learning Pipelines

Problem Statement
The paper addresses the challenge of predicting the number of clicks a hotel will receive for a specific bid amount on meta-search bidding engines. This prediction is critical for Online Travel Agencies (OTAs) managing Cost-Per-Click (CPC) advertising campaigns, as the product of the bid amount and the predicted number of clicks defines the total campaign cost. The problem is characterized by high variance in daily data, the presence of external factors (seasonality, events, location), and the need to model non-linear relationships between features and click volume.

Methodology
The proposed system follows a five-component pipeline involving data retrieval, preprocessing, feature selection, hyperparameter optimization, and ensemble modeling.

Data Preprocessing and Feature Engineering:
- Data Sources: The dataset combines daily reports from a meta-search platform with data from a major Turkish OTA. External features, such as daily weather, exchange rates, and proximity to public holidays, are integrated to capture economic and seasonal contexts.
- Cleaning: Missing values in categorical and numerical columns are imputed with the mode and mean, respectively.
- Feature Generation: A sliding-window (time-delay) approach calculates statistical aggregates (mean, standard deviation) over 3, 7, and 30-day periods. Time-series features undergo seasonal decomposition (Trend, Seasonality, Residual) to create additive components.
- Encoding and Scaling: String-based features are one-hot encoded or binarized, and all features are normalized using min-max scaling.
Feature Selection:
- The authors employ an XGBoost-based recursive feature elimination algorithm. This process ranks features by importance and iteratively removes the least significant ones. The elimination stops when the $R^2$ score of a reduced feature subspace no longer exceeds the previous iteration's score, effectively reducing dimensionality while retaining predictive power.
Hyperparameter Optimization:
- Bayesian Hyperparameter Optimization is applied to tune XGBoost, LightGBM, and Stochastic Gradient Descent (SGD) models. This probabilistic approach is chosen over Grid Search or Random Search to efficiently locate global optima for non-convex functions within fewer epochs. The optimization metric is the $R^2$ score.
Ensemble Learning Strategies:
- The study evaluates ten base regressors: XGBoost, LightGBM, SGD, Lasso, Lasso Lars, Ridge, Bayesian Ridge, Huber, Passive Aggressive Regressors, and Elastic Net.
- Simple Averaging: Two methods are tested: a simple mean of predictions and a weighted average, where weights are derived from the individual $R^2$ scores of the base models.
- Stacking: Predictions from the ten base models form an intermediate dataset. This dataset is fed into four meta-regressors (XGBoost, Lasso, Bayesian Ridge, and Linear Regression) to generate final predictions.
- Blending: Similar to stacking, but the intermediate predictions are concatenated with the original, dimensionally reduced feature set before being fed into the same four meta-regressors. This approach effectively creates artificial features to expand the feature dimension.

Key Contributions

Pipeline Integration: The paper presents a comprehensive pipeline that integrates time-delayed feature engineering, XGBoost-based dimensionality reduction, and Bayesian optimization specifically for hotel click prediction.
Comparative Ensemble Analysis: It systematically compares four distinct ensemble strategies (simple average, weighted average, stacking, and blending) across 46 model combinations.
Meta-Learner Selection: The study investigates the efficacy of different meta-learners, finding that simpler linear models often outperform complex tree-based models when used as the final layer in a stacking architecture.

Results

Individual Models: Ten base regressors were tested individually. The best-performing single model was Ridge Regression with an $R^2$ score of 0.579. The worst performer was Elastic Net with a negative $R^2$ of -0.012.
Ensemble Performance: All ensemble methods outperformed the individual models.
- The top-performing models were three Stack Ensemble variants (using Linear Regression, Lasso, and Bayesian Ridge as meta-learners), all achieving an $R^2$ score of 0.639.
- Blend Ensemble models with Lasso and Bayesian Ridge followed closely with an $R^2$ of 0.638.
- The Weighted Average approach ranked third with an $R^2$ of 0.597.
- Other combinations, such as stacking with XGBoost or blending with LightGBM, performed at the level of isolated models.
Performance Gain: The authors conclude that the best ensemble model improves prediction performance by approximately 10% compared to the best single machine learning model.

Significance and Claims
The paper claims that ensemble learning pipelines, specifically those utilizing stacking with simple linear meta-learners, significantly enhance the accuracy of hotel click predictions compared to standalone regressors. The authors attribute this success to the ability of ensemble methods to reduce variance and leverage the complementary strengths of diverse algorithms. They note that the "level-1" regressor (meta-learner) acts primarily as a mediator, suggesting that simple algorithms are often sufficient for this role when the "level-0" learners have already captured the complex patterns. The study emphasizes that while domain knowledge and feature engineering are essential, the structural combination of models via stacking offers a robust direction for improving CTR and click volume prediction in meta-search environments.

More like this