Evaluating targeted anti-fraud communication in rural China: a cluster-randomised intervention simulation

Technical Summary: Evaluating Targeted Anti-Fraud Communication in Rural China

Problem Statement
Digital inclusion in rural communities often exposes users to fraud risks that are unevenly distributed, creating a gap between digital access and digital security. While existing literature addresses digital inequality regarding access and general skills, there is a lack of evidence on whether these inequalities translate into specific fraud susceptibility. Furthermore, current fraud-prevention studies often rely on self-reported awareness rather than performance across both fraudulent and legitimate messages. A critical gap exists in comparing risk-targeted communication against equally intensive, non-tailored education to isolate the incremental value of personalization without confounding it with intervention dose.

Methodology
This study is a methodological simulation based entirely on synthetic data, designed to evaluate the architecture of a prospective cluster-randomised field experiment in rural China. No human participants or real villages were involved.

• Data Generation: The study employed a fully item-level synthetic data-generating process (DGP). It generated discrete responses for five Digital Security Capability (DSC) items, fifteen Protection Motivation Theory (PMT) items, and Q1/Q2/Q3 responses for ten scenario forms (five fraudulent, five legitimate). Composite scores (DSCI, SBFSS, recognition, false alarms, domain-risk scores) were calculated from these raw responses.
• Experimental Design: The simulation mimicked a two-arm, dose-matched, cluster-randomised trial with 48 villages (960 adults).
  • Targeted Arm: Participants were assigned the two intervention modules (M1–M5) corresponding to their two highest baseline domain-risk scores.
  • Active Control Arm: Participants received two modules randomly permuted from the targeted arm's pool, ensuring identical arm-level module frequencies but no person–message alignment.
  • Intervention: Both arms received a 25-minute session covering a universal safety rule and two specific risk modules.
• Analysis Plan: The primary analysis used a linear mixed-effects model to estimate T0→T1 and T0→T2 contrasts in the Scenario-Based Fraud Susceptibility Score (SBFSS). Secondary outcomes included recognition, false alarm rates, and PMT constructs. The study utilized an ADEMP (Aims, Data-generating mechanisms, Estimands, Methods, Performance measures) framework to assess operating characteristics via 500 Monte Carlo replications.

Key Contributions
The paper makes four specific methodological and conceptual contributions:

1. Conceptual Framework: It conceptualizes digital fraud vulnerability as a downstream dimension of digital inequality (specifically digital-security capability) rather than a purely technical failure.
2. Integrated Measurement: It integrates practical digital-security capability, scenario-based discrimination, and PMT constructs into a single testable framework using item-level data.
3. Design Isolation: It specifies a design that separates personalization from intervention dose by comparing targeting against a dose-matched active control, addressing the confounding of "more time/content" with "tailoring."
4. False Alarm Metric: It treats false alarms towards legitimate messages as a substantive outcome, allowing improved detection to be distinguished from generalized distrust.

Results
The simulation evaluated the recoverability of the proposed design under specific synthetic assumptions:

• Primary Outcomes: The mixed model successfully recovered the T0→T1 contrast of −0.193 (95% CI −0.275 to −0.111; P = 3.97×10⁻⁶) and the T0→T2 contrast of −0.145 (P = 0.000753), demonstrating the design's ability to detect the programmed targeting effect.
• Secondary Outcomes: After Holm correction, only the T1 recognition contrast remained detectable. False alarm contrasts and most PMT-related contrasts (self-efficacy, response efficacy, protective intention) were not detectable, illustrating that the simulation does not force all theoretical mechanisms to succeed.
• Heterogeneity (H6): The interaction between baseline capability and treatment effect was only partly recovered; a significant interaction was found at T1 but not T2, and the pattern for baseline susceptibility did not consistently match the hypothesis.
• Monte Carlo Operating Characteristics: Under the null hypothesis, rejection probabilities were close to the nominal 0.05 level (0.058 and 0.054). Under non-null scenarios, the design demonstrated high power (0.928 to 0.988), confirming the statistical properties of the specified simulation.

Significance and Claims
The authors explicitly state that these findings represent properties of the specified simulation and demonstrate design behavior, not real-world intervention effectiveness. The study does not claim that targeted communication works in rural China; rather, it provides a "fully specified empirical blueprint."

The significance lies in the transparency of the simulation:

• It allows assumptions, measurement logic, and analysis plans to be scrutinized before a real field implementation.
• It demonstrates that a complex, item-level simulation can recover primary contrasts while allowing secondary hypotheses (such as specific mediation pathways or equity effects) to remain unconfirmed, thereby avoiding the tautological trap of synthetic data where every hypothesis is programmed to succeed.
• It offers a reproducible computational source for evaluating the feasibility of a future prospectively registered, ethically approved field trial.