Competition-Aware CPC Forecasting with Near-Market Coverage

This paper proposes a competition-aware CPC forecasting framework that leverages semantic, behavioral, and geographic signals derived from Google Ads auction logs to approximate latent market competition, thereby significantly improving forecast stability and accuracy for medium-to-long horizons compared to traditional baselines.

The Gist

Imagine you are running a lemonade stand in a very busy, crowded city square. You want to know how much you'll have to pay to get a customer to stop and buy your lemonade tomorrow, next week, or next month.

In the world of online advertising, this "price" is called CPC (Cost-Per-Click). But here's the catch: unlike a lemonade stand where you can see the price tag on the sign, the price of an online ad is decided by a secret, lightning-fast auction. You can see how much you paid yesterday, but you have no idea what your competitors are bidding, what their "quality" is, or how many other people are trying to sell lemonade in the same spot today.

This paper is like a detective story about how to guess that secret price when you only have half the clues.

The Problem: The "Blind" Auctioneer

The authors studied the car rental industry (think Hertz, Enterprise, etc.) using billions of data points from Google Ads. They found that trying to predict the future price of an ad just by looking at your own past prices is like trying to predict the weather by only looking at your own backyard. It works for a few minutes, but once the wind changes direction (a competitor changes their strategy, or a holiday hits), your prediction fails.

The market is partially observable. You see the result (the price), but you don't see the cause (the competition).

The Solution: The "Sherlock Holmes" Approach

Since the advertisers can't see the competitors directly, the researchers built a system to approximate the competition using three clever "clues" (proxies):

1. The "Semantic Neighborhood" (Reading the Mind):

   • The Analogy: Imagine you see someone searching for "cheap car rental." You know that someone else searching for "affordable car hire" is probably looking for the exact same thing, even though the words are different.
   • The Tech: They used AI (specifically "transformers") to read the meaning of keywords. If two keywords mean the same thing, they are neighbors in a "semantic graph." If your neighbor's price goes up, yours probably will too.

2. The "Behavioral Dance" (Watching the Moves):

   • The Analogy: Sometimes, two lemonade stands are on opposite sides of the square and use different words, but they both get crushed when it rains. Their sales patterns move in sync, even if they aren't next to each other.
   • The Tech: They used a method called Dynamic Time Warping (DTW). This is like watching two dancers and seeing if their steps match, even if one starts dancing a second later than the other. If two keywords have "danced" to the same rhythm in the past, they are likely competing for the same customers.

3. The "Geographic Map" (Knowing the Territory):

   • The Analogy: A rental car ad in "Paris" faces a totally different crowd than one in "Rural Nebraska." The competition is local.
   • The Tech: They mapped keywords to specific cities and countries. This acts as a stabilizer. Knowing where the demand is helps predict how wild the price swings will be.

The Experiment: Testing the Tools

The researchers didn't just invent these clues; they tested them against the best tools in the business. They compared:

• Old School: Simple math formulas (like ARIMA).
• Modern AI: Huge "Foundation Models" (like TimeGPT or Chronos) that are trained on massive amounts of data.
• The New Approach: Graph Neural Networks (GNNs) that use the "neighborhood" maps they built.

The Results: What Worked?

Here is the "take-home message" in simple terms:

1. Short-term vs. Long-term:

   • If you need to know the price for tomorrow, the "Graph" method (looking at neighbors) works best because it catches immediate ripples in the market.
   • If you need to plan for 6 weeks or 6 months, the "Foundation Models" (the big AI brains) combined with Geographic clues worked best. The big AI is great at seeing the long trends, and the geography keeps it from getting confused.

2. The "Competitive Frontier":

   • The biggest wins weren't on the easy, cheap keywords. The biggest wins were on the expensive, volatile keywords (the "high-stakes" auctions).
   • Why it matters: If you get the price wrong on a cheap keyword, you lose a few dollars. If you get it wrong on a high-stakes keyword during a busy season, you could lose your entire marketing budget. This method saved the most money exactly where it was needed most.

3. Less is More:

   • Throwing every possible clue into the mix actually made the predictions worse (like trying to solve a puzzle with 1,000 extra pieces that don't belong). The best results came from selecting the right clues (like just geography and semantic neighbors) rather than dumping everything in.

The Big Picture

This paper proves that even when you can't see your competitors, you can still guess their moves if you look at the context.

By combining what the words mean (semantics), how the prices moved in the past (behavior), and where the customers are (geography), advertisers can build a "crystal ball" that is much more accurate than just looking at their own history. It turns a blind guess into an educated strategy, saving money and preventing budget disasters in the chaotic world of online auctions.

Technical Summary

1. Problem Definition

The paper addresses the challenge of forecasting Cost-Per-Click (CPC) in paid search advertising (specifically Google Ads).

• The Core Difficulty: CPC is an endogenous outcome of a real-time auction determined by competitor bids, quality scores, and platform dynamics. However, advertisers only observe their own realized CPC, clicks, and spend. They cannot observe competitor bids, full auction states, or the complete competitive landscape.
• Partial Observability: This creates a "partial observability" problem. Traditional autoregressive models (relying only on past CPC) fail at medium-to-long horizons because they cannot account for regime shifts, competitor reallocations, or localized demand changes that are invisible in a single advertiser's history.
• The Goal: To approximate latent competition using observable proxies (semantic, behavioral, and geographic) to improve forecasting stability and accuracy, particularly for high-value, high-volatility keywords.

2. Data and Context

• Dataset: A large-scale dataset of 1.66 billion advertising logs from the car-rental sector (2021–2023).
• Sample: After filtering for relevance and data quality, the study focuses on 1,811 keyword series observed over 127 weeks.
• Characteristics: The data exhibits heavy-tailed distributions (Pareto-like), where a small number of high-intent keywords generate disproportionate volume and cost. The market is geographically localized (airports, cities) and highly competitive among a concentrated set of major players.

3. Methodology: Competition-Aware Forecasting

The authors propose a framework that operationalizes latent competition through three distinct proxy families and evaluates them via two representation routes.

A. Competition Proxy Extraction

1. Semantic Neighborhoods:
   • Source: Keyword text.
   • Method: Uses pretrained transformer embeddings (all-MiniLM-L6-v2) to generate 384-dimensional vectors.
   • Logic: Keywords with similar user intent compete for the same inventory, even if lexically different.
   • Output: Pairwise cosine similarity defines semantic neighbors.
2. Behavioral Neighborhoods:
   • Source: CPC time-series trajectories.
   • Method: Dynamic Time Warping (DTW) with a Sakoe–Chiba band constraint.
   • Logic: Captures keywords that experience synchronized CPC shocks due to shared demand cycles or auction reallocations, even if their text differs.
3. Geographic Intent:
   • Source: Keyword text and gazetteers.
   • Method: Extraction of hierarchical location indicators (Continent, Country, City).
   • Logic: Auction pressure is spatially segmented; demand in specific cities or countries creates localized competitive regimes.

B. Representation Routes

The proxies are integrated into forecasting models in two ways:

1. Route 1: Exogenous Covariates: Proxies (e.g., lagged neighbor CPC, geographic indicators) are fed into models as additional input features alongside historical CPC.
2. Route 2: Relational Priors (Graph Structure): A fixed semantic keyword graph is constructed where nodes are keywords and edges connect the $k=10$ nearest semantic neighbors. This graph is used as an adjacency matrix in Spatiotemporal Graph Neural Networks (STGNNs) to propagate information across the network.

C. Forecasting Architectures Evaluated

The study benchmarks a broad model universe across three categories:

1. Classical/ML Baselines: SARIMAX, XGBoost, Random Forest, MLP, LSTM, GRU, and TabPFN.
2. Time-Series Foundation Models (TSFMs): Chronos-2, TimeGPT, and Moirai, augmented with the competition proxies as covariates.
3. Spatiotemporal Graph Neural Networks (STGNNs): DCRNN, GConvLSTM, and GraphWaveNet, utilizing the semantic graph for message passing.

4. Key Results

The evaluation uses Symmetric Mean Absolute Percentage Error (sMAPE) and RMSE across three horizons: 1-week, 6-week, and 12-week.

• Performance by Horizon:
  • 1-Week: STGNNs (specifically DCRNN) perform best, capturing immediate dynamics and cross-keyword spillovers.
  • 6-Week & 12-Week: Covariate-augmented TSFMs (specifically Chronos-2 with geographic intent) dominate. They provide the lowest error rates (e.g., 27.14% sMAPE at 6 weeks vs. 35.04% for the best baseline).
• Impact of Competition Awareness:
  • Models incorporating competition proxies consistently outperform univariate or weakly contextual baselines.
  • Geographic Priors: Coarse geographic encoding (Continent level) acts as a powerful stabilizing prior, outperforming finer-grained (City/Country) encodings which tend to fragment the signal.
  • Semantic Priors: Semantic-neighbor CPC signals provide complementary gains, especially for graph-based models at the 6-week horizon.
• The "Competitive Frontier" (High-Risk Segments):
  • The value of competition-aware modeling is concentrated in the high-CPC, high-volatility quadrant (the "competitive frontier").
  • In this segment, proxy-based augmentation significantly reduces error compared to standard models, directly addressing the highest budget risks.
• Feature Selection vs. Stacking:
  • Indiscriminate stacking of all available proxies degrades performance (increasing error by ~3–5 points).
  • Selective, theoretically grounded augmentation (e.g., Core + Geo + Sem CPC) is strictly superior.

5. Key Contributions

1. Reframing the Problem: Positions CPC forecasting as a partial observability problem, arguing that latent competition must be approximated via observable proxy structures rather than purely autoregressive extrapolation.
2. Operationalization at Scale: Successfully constructs and validates three types of competition proxies (semantic, behavioral, geographic) across a massive industrial dataset (1.66B logs) and a diverse model universe (including foundation models and GNNs).
3. Horizon-Dependent Insights: Demonstrates that the optimal modeling strategy depends on the planning horizon:
   • Short-term: Relational graph structures (STGNNs).
   • Medium/Long-term: Covariate-augmented foundation models with geographic priors.
4. Practical Robustness: Shows that these methods do not just improve average accuracy but specifically stabilize forecasts in the most economically critical (high-cost, high-volatility) market segments.

6. Significance and Limitations

• Significance: The paper provides a scalable, data-driven framework for advertisers to "see" the invisible competitive landscape. It proves that combining domain-informed proxy construction with modern foundation models and graph neural networks yields tangible business value in auction-driven markets.
• Limitations:
  • Vertical Specificity: Results are based on the car-rental sector; generalization to less concentrated or different verticals requires further study.
  • Static Graph: The semantic graph is fixed and does not capture evolving keyword relationships, market entry/exit, or shifting competitive regimes over time.
  • Data Scope: While "near-market coverage" is broad, it is not a full market census, relying on a specific dataset provider.

Conclusion: The paper concludes that under the informational asymmetry of advertising platforms, advertisers can significantly improve CPC forecasting by investing in principled competition proxy construction (semantic, behavioral, geographic) rather than relying solely on historical time-series data.