Gravity Falls: A Comparative Analysis of Domain-Generation Algorithm (DGA) Detection Methods for Mobile Device Spearphishing

This paper evaluates the effectiveness of traditional and machine-learning DGA detection methods against a new semi-synthetic smishing dataset called Gravity Falls, revealing that current tools struggle to generalize across evolving threat actor tactics like dictionary concatenation and themed combo-squatting, thus highlighting the need for more context-aware detection approaches.

The Gist

The Big Picture: The "Gravity Falls" Mystery

Imagine you are trying to stop a group of thieves who are sending text messages to millions of people. These messages look like they are from your bank, the post office, or the DMV, but they are actually traps designed to steal your passwords or money.

To do this, the thieves need a lot of "houses" (websites) to send people to. But if they build one house, the police (security software) will find it and shut it down. So, the thieves use a Domain Generation Algorithm (DGA). Think of this as a magical machine that can print out millions of unique house addresses every day. If the police shut down one, the thieves just print a new one and keep going.

This paper is about a specific group of thieves (called the "Smishing Triad") who have been active from 2022 to 2025. The researchers collected a list of all the "houses" these thieves used and called it "Gravity Falls."

The goal of the paper was to test our current "security guards" (detection tools) to see if they could catch these thieves. The researchers wanted to know: Do our security tools work on mobile phone scams, or are they only good at catching computer viruses?

The Four Seasons of the Thief's Evolution

The researchers noticed that the thieves didn't just use the same trick every year. They evolved, like a character in a video game leveling up. The "Gravity Falls" dataset is divided into four "seasons" (clusters):

1. Cats Cradle (2022): The "Random Noise" Phase

   • The Trick: The thieves used addresses that looked like gibberish, like xk9j2m.com.
   • The Analogy: Imagine a thief wearing a mask made of static noise. It's obvious they aren't a normal person.
   • The Result: Our security tools were very good at catching these. They looked at the "noise" and said, "That's not a real word, it's a trap!"

2. Double Helix (2023): The "Word Salad" Phase

   • The Trick: The thieves started combining real dictionary words, like apple-banana-fruit.com.
   • The Analogy: The thief stopped wearing a static mask and started wearing a suit made of random grocery items. It looks like a real word, but it doesn't make sense.
   • The Result: The security tools got confused. Because the words were real, the tools thought, "Hey, that looks like a normal website!" and let them pass.

3. Pandoras Box (2024): The "Package Scam" Phase

   • The Trick: The addresses started looking like real delivery services, like usps-delivery-track-99.com.
   • The Analogy: The thief is now wearing a fake FedEx uniform. They are mixing real brand names with a tiny bit of random numbers at the end.
   • The Result: The tools struggled. They saw the "FedEx" part and thought it was safe, missing the tiny random numbers that gave it away.

4. Easy Rider (2025): The "Ticket Fine" Phase

   • The Trick: The addresses looked like government fines, like dmv-speeding-fine-pay.com.
   • The Analogy: The thief is now wearing a fake police uniform. They are using fear (you might get a ticket!) to trick you.
   • The Result: The tools were mostly blind to these. They looked too much like legitimate government sites.

The Test: Can the Security Guards Catch Them?

The researchers took four different types of "security guards" (detection tools) and asked them to look at the list of 40,000 websites from Gravity Falls.

• The Old School Guards (Shannon Entropy & Exp0se): These tools look at the math of the letters. They ask, "Is this string of letters too random?"
• The AI Guards (LSTM & DGAD): These are smart computers trained to recognize patterns, like a dog that has been trained to sniff out bad guys.

The Results:

• On the "Random Noise" (2022): The guards were heroes! They caught almost all of them.
• On the "Word Salad" and "Fake Uniforms" (2023–2025): The guards failed miserably.
  • The Old School Guards couldn't tell the difference between a real word and a fake one.
  • The AI Guards, which are usually very smart, got tricked. They had been trained on old data (mostly computer viruses) and didn't know how to handle these new, clever mobile phone tricks.

The Main Lesson: The "One-Size-Fits-All" Trap

The biggest takeaway from this paper is that security tools are too specialized.

Imagine you have a security guard who is an expert at spotting people wearing clown masks. He is great at his job. But then, the criminals stop wearing clown masks and start wearing realistic business suits. The guard, who only knows how to spot clowns, lets the criminals walk right past him.

The paper argues that:

1. Current tools are too focused on old threats. They are great at catching the "gibberish" domains used by computer viruses, but they are terrible at catching the "clever word tricks" used in text message scams.
2. We need smarter guards. We need security systems that understand context. They shouldn't just look at the letters; they should look at the message, the sender, and the brand being impersonated.
3. AI might need a human touch. The researchers found that even a basic AI chatbot (like Claude) could spot the theme of the scams better than the specialized security tools. This suggests that in the future, we might need to combine human-like reasoning with computer speed to catch these evolving thieves.

In Summary

The "Gravity Falls" study is a warning to the security world: The bad guys are changing their costumes, and our security guards are still looking for the old ones. If we don't update our tools to recognize these new, clever text-message scams, we will keep losing our data and money to these digital pickpockets.

Technical Summary

1. Problem Statement

While Domain Generation Algorithms (DGAs) are well-studied in the context of malware Command-and-Control (C2) and email phishing, there is a significant research gap regarding SMS spearphishing (smishing).

• The Gap: Existing DGA research focuses heavily on enterprise-internal threats or known malware families. It lacks evaluation against mobile-specific threats where users operate outside organizational perimeters with fewer security controls.
• The Challenge: Adversaries in smishing campaigns rapidly rotate hostile infrastructure using DGAs to evade detection. Current detectors, often trained on static malware datasets, may not generalize well to the evolving tactics, techniques, and procedures (TTPs) used in mobile phishing, which blend dictionary words, brand tokens, and randomization.

2. The "Gravity Falls" Dataset

To address this gap, the authors introduced Gravity Falls, a new semi-synthetic dataset derived from real smishing links collected between 2022 and 2025.

• Source: Hyperlinked URLs from SMS, iMessage, and email-to-SMS.
• Attribution: Linked to a single threat actor (associated with the "Smishing Triad" ecosystem) based on behavioral patterns (e.g., spoofed senders, specific TLDs, infrastructure reuse).
• Structure: The dataset is organized into four chronological clusters representing the evolution of the threat actor's DGA tactics:
  1. Cats Cradle (2022): Short, randomized strings (5–8 characters) with generic TLDs. Used for target validation via fake CAPTCHAs.
  2. Double Helix (2023): Dictionary wordlist concatenation (dual words) in the domain name and TLD.
  3. Pandoras Box (2024): Themed combo-squatting focused on package delivery (Amazon, USPS). Features keyword splitting between subdomains and domains, plus random suffixes.
  4. Easy Rider (2025): Themed combo-squatting focused on government/toll fines (DMV, EZ-Pass). Uses brand associations with randomized suffixes.

3. Methodology

The study employed a many-to-many comparative evaluation, testing multiple detection tools against the four experimental clusters and a control group.

• Control Group: 10,000 benign domains randomly selected from four Top-1M lists (Alexa, Cisco, Cloudflare, Majestic) to serve as a baseline.
• Experimental Groups: 10,000 domains per cluster (approx. 50% malicious, 50% benign padding).
• Detectors Evaluated:
  • Traditional/Heuristic:
    • Shannon Entropy: Calculates information content based on character probability.
    • Exp0se: A signature-based detector using entropy, consonant count, and string length thresholds.
  • Machine Learning (ML):
    • LSTM Classifier (MiaWallace0618): Uses one-hot encoding of TLDs.
    • COSSAS DGAD: A Temporal Convolutional Network (TCN) trained on Shadowserver data, providing substring and overall domain assessments.
• Metrics: Precision, Accuracy, and Recall were calculated, treating Top-1M domains as benign.

4. Key Results

The performance of all detectors was highly tactic-dependent, showing a clear degradation as the DGA evolved from pure randomness to semantic/brand-based manipulation.

Cluster	Tactic	Best Performer	Key Finding
Cats Cradle	Random Strings	Exp0se & DGAD	High precision/recall (>90%). Traditional heuristics excel at detecting pure randomness.
Double Helix	Dictionary Concatenation	DGAD (Low)	Significant Failure. All tools struggled. Dictionary words lowered entropy, causing high False Negatives (low recall).
Pandoras Box	Themed Combo-Squatting	Exp0se (Moderate)	Mixed results. Exp0se detected some, but ML tools failed to generalize.
Easy Rider	Themed Combo-Squatting	Exp0se (Moderate)	Similar to Pandoras Box; high precision but low recall for ML models.

• Critical Observation: While tools performed well on the 2022 "random string" domains, their recall dropped drastically (often to <5%) for the 2023–2025 clusters.
• ML Limitations: The LSTM and DGAD models, despite being trained on large datasets, failed to generalize to the specific "dictionary + brand + minor randomization" patterns found in the later clusters.
• Anecdotal Insight: Large Language Models (LLMs) like Claude were observed to be surprisingly effective at identifying the thematic consistency across clusters, suggesting a potential role for LLMs in future detection.

5. Key Contributions

1. New Dataset: Introduction of Gravity Falls, the first semi-synthetic dataset specifically focused on the evolution of DGA tactics in SMS spearphishing over a multi-year period.
2. Empirical Evidence of Generalization Failure: Demonstrated that state-of-the-art DGA detectors (both traditional and ML) trained on malware C2 data do not generalize to smishing-driven domain tactics that utilize dictionary words and brand squatting.
3. Benchmarking: Provided a reproducible benchmark for evaluating future DGA detection tools against evolving mobile phishing TTPs.

6. Significance and Implications

• Defensive Strategy Shift: The paper argues that relying solely on lexical heuristics (like entropy) or standard ML models is insufficient for mobile phishing. Defenders must adopt a layered approach:
  • Use fast heuristics for obvious random strings.
  • Rely on contextual analysis (message content, hosting infrastructure, brand abuse policies) for dictionary-based and combo-squatting domains.
• Future Research Direction: The results suggest a need for context-aware detection that incorporates semantic understanding (potentially via LLMs) rather than just statistical string analysis.
• Operational Reality: The study highlights that adversaries are actively evolving their infrastructure to bypass static signatures and standard ML models, rendering "one-size-fits-all" DGA detection obsolete for mobile threats.

7. Limitations

• Semi-Synthetic Nature: The dataset mixes observed domains with predicted patterns for sinkholing, which may introduce bias.
• Sampling Issues: Accidental duplication in the dataset occurred due to human error in collection, though this inadvertently helped verify tool consistency.
• Baseline Imperfection: The "benign" Top-1M lists are not perfectly static or purely benign, though they are standard proxies in the field.
• Tool Versions: The study used specific versions of open-source tools, which may not reflect the absolute latest proprietary capabilities.

Conclusion

"Gravity Falls" reveals a critical vulnerability in current cybersecurity defenses: DGA detectors are ill-suited for the evolving, semantic-heavy tactics of mobile spearphishing. As adversaries move from random strings to sophisticated brand-squatting and dictionary concatenation, traditional and standard ML approaches fail to maintain high recall. The paper calls for a paradigm shift toward context-aware, multi-signal detection strategies to effectively combat mobile eCrime.