💻 computer science

Building Resilience to Misinformation: A Cross-National Development of the Digital Media and Information Literacy Scale (DMILS)

This paper introduces the Digital Media and Information Literacy Scale (DMILS), a psychometrically validated, multidimensional instrument developed through cross-national studies in the US and Singapore to provide a concise yet comprehensive metric for assessing and improving resilience against misinformation.

Original authors: Sijia Qian, Cuihua Shen, Huiyi Wang, Hichang Cho

Published 2026-05-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sijia Qian, Cuihua Shen, Huiyi Wang, Hichang Cho

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a house to protect yourself from a storm of fake news, deepfakes, and online tricks. For a long time, experts have been arguing about what kind of tools you actually need to build that house. Some say you need a hammer (digital skills), others say you need a blueprint (information knowledge), and many have been using different, mismatched toolkits that don't fit together.

This paper introduces a new, all-in-one "Digital Media and Information Literacy Scale" (DMILS). Think of it as a master blueprint and a standardized toolkit that finally measures exactly what people know and what they can do when navigating the internet.

Here is how the authors built and tested this new tool, explained simply:

1. The Problem: A Messy Toolbox

Before this study, researchers had many different ways to measure "media literacy," but they were like trying to measure a car's speed with a ruler, a stopwatch, and a thermometer all at once.

Some tools only asked, "Do you feel confident?" (Subjective).
Some only asked, "Can you actually do it?" (Objective).
Some focused only on tech skills, while others focused only on news facts.
Many were old and didn't account for modern things like AI or algorithms.

The authors wanted one single, reliable yardstick that could measure everything: what people think they know, what they actually know, and how well they can use that knowledge in both the digital world and the news world.

2. The Solution: A Four-Part Compass

The authors designed a new scale based on a simple map with four directions (a 2x2 grid):

Digital Knowledge: Do you know what terms like "algorithm," "phishing," or "hashtag" mean?
Digital Skill: Can you actually use your phone or computer to solve problems or find things?
Information Knowledge: Do you understand how news is made, who writes it, and why?
Information Skill: Can you actually check if a news story is true or false?

Crucially, they measured each of these in two ways:

The "Confidence" Check (Subjective): Asking people, "How familiar are you with this?"
The "Reality" Check (Objective): Giving people a quiz with true/false questions and multiple-choice puzzles to see if they actually know the answer.

3. The Test Drive: Two Countries, One Tool

To make sure this new compass worked everywhere, the authors tested it in two very different places: the United States and Singapore.

The US is like a chaotic, open marketplace with many different voices and lots of political arguing.
Singapore is more like a regulated, organized city with a centralized flow of information.

They surveyed nearly 1,500 people in total. They didn't just ask questions; they gave them tasks, like:

The "Fake News" Test: Showing people 16 headlines and images (some real, some fake) and asking them to spot the liars.
The "Treasure Hunt": Asking people to find specific facts on the internet (like "Who is the Prime Minister of Croatia?") to see if they could actually retrieve accurate info.

4. What They Found

The results were like finding a key that fits every lock:

It Works Everywhere: The scale measured literacy the same way in both the US and Singapore. It didn't matter if you were in a chaotic marketplace or an organized city; the tool gave consistent results.
Confidence vs. Reality: There was a gap. People who spent a lot of time on social media often thought they were very good at spotting fake news (high confidence), but when they took the actual quiz, they often missed the mark. The "Reality Check" (objective scores) was a much better predictor of whether someone could actually spot a lie than their "Confidence Check."
The Best Predictors: The study found that Information Skills (the ability to critically evaluate news) and Objective Knowledge (actually knowing the facts about how tech and news work) were the strongest shields against believing fake stories.

5. The Result: A Short and Long Version

The authors created two versions of this scale:

The Full Version: A detailed 18-question survey plus 16 quiz questions. This is for researchers who need deep, precise data.
The Short Version: A quick 8-question survey plus 8 quiz questions. This is for teachers or organizations who need a fast check-up without taking up too much time.

The Bottom Line

This paper didn't just invent a new test; it built a common language for the world. Before, if a researcher in New York and a researcher in Singapore wanted to compare their results, they couldn't because they were using different rulers. Now, they have the same ruler.

The study proves that to truly understand if people can handle the modern information storm, we need to stop just asking them how they feel and start testing what they can actually do. This new scale gives educators and policymakers a clear way to see where people are struggling (e.g., "They know how to use the app, but they don't know how to check the source") so they can build better training programs to fix those specific gaps.

Technical Summary: Building Resilience to Misinformation via the Digital Media and Information Literacy Scale (DMILS)

Problem Statement
Despite the critical role of media literacy in mitigating misinformation and supporting democratic deliberation, the field suffers from conceptual fragmentation and empirical inconsistency. Existing measures are often fragmented, outdated, or narrowly focused, failing to capture the full spectrum of competencies required in rapidly evolving digital ecosystems. Common limitations include:

Conceptual Silos: Distinct literacies (digital, news, algorithm, AI) often operate in parallel rather than as an integrated framework.
Dimensional Gaps: Many scales measure only skills or only knowledge, neglecting the interplay between the two.
Measurement Modes: A heavy reliance on subjective, self-reported assessments fails to distinguish between perceived competence and actual ability, a distinction crucial for identifying overconfidence and literacy gaps.
Contextual Limitations: Prominent scales were developed over a decade ago, primarily in Western contexts, limiting their relevance to contemporary technologies (e.g., generative AI) and non-Western cultural environments.

There is a lack of a concise, comprehensive, and psychometrically sound instrument capable of tracking these competencies across diverse national contexts to evaluate interventions and build a resilient information ecosystem.

Methodology
The authors developed and validated the Digital Media and Information Literacy Scale (DMILS) through a two-study process involving three nationally matched samples ( $N = 1,498$ ) from the United States and Singapore. The development followed rigorous scale construction guidelines (e.g., Boateng et al., 2018; Carpenter, 2018).

Conceptual Framework: The authors proposed a multidimensional framework distinguishing between:
- Domains: Digital Media (technical/procedural) vs. Information/News (journalistic/verification).
- Competency Types: Knowledge (understanding concepts) vs. Skill (ability to execute tasks).
- Measurement Modes: Subjective (self-reported) vs. Objective (performance-based).
Study 1 (Item Development and Reduction):
- Process: A literature search across 26 journals yielded 74 empirical studies. Items were mapped to the framework, resulting in an initial pool of 56 subjective and 30 objective items.
- Sample: 499 US participants recruited via Prolific.
- Analysis: Exploratory Factor Analysis (EFA) and item reduction criteria (loadings, cross-loadings, communalities) refined the subjective scale to 18 items. Objective items were revised based on difficulty ( $P$ -values) and discriminability.
Study 2 (Validation and Generalizability):
- Sample: 501 US and 498 Singapore participants.
- Analysis:
  - Construct Validity: Confirmatory Factor Analysis (CFA) and Rasch modeling (Item Response Theory) were used to test the factor structure and item fit.
  - Measurement Invariance: Tested to ensure the scale functions equivalently across US and Singapore contexts (configural, metric, and scalar invariance).
  - Convergent Validity: Correlated with social media use and news consumption.
  - Predictive Validity: Regressed against two behavioral outcomes: Discernment (ability to distinguish true vs. fake news headlines and images) and Information Retrieval Accuracy (ability to find factual answers online).

Key Contributions

The DMILS Instrument: A validated, multidimensional scale comprising an 18-item subjective battery and a 16-item objective knowledge test. A short-form version (8 subjective, 8 objective items) is also provided.
Integrated Framework: The scale operationalizes a unified framework that simultaneously captures digital and information domains, knowledge and skills, and subjective and objective modes of measurement.
Cross-National Robustness: The scale demonstrates measurement invariance across the US and Singapore, addressing the need for culturally applicable tools in global misinformation research.
Diagnostic Utility: By integrating subjective and objective components, the scale allows researchers to quantify the "gap" between perceived and actual literacy.

Results

Structural Validity: Both EFA and CFA supported a four-factor structure (Digital Knowledge, Digital Skill, Information Knowledge, Information Skill). A higher-order model confirmed that these factors could also be treated as a unidimensional construct for composite scoring.
Reliability: The subjective scale showed excellent internal consistency ( $\alpha = 0.93$ ).
Measurement Invariance: The scale exhibited configural, metric, and scalar invariance across US and Singapore samples, confirming its applicability in both contexts.
Predictive Validity:
- Discernment: Both subjective and objective DMILS scores significantly predicted the ability to discern true from false news. Notably, objective measures were stronger predictors than subjective self-reports. Among sub-dimensions, Information Skill was the strongest subjective predictor, while both Objective Digital and Information Knowledge were strong predictors.
- Information Retrieval: Composite DMILS scores predicted retrieval accuracy. However, when decomposed, individual sub-dimensions were less predictive, suggesting the composite score captures a broader utility for retrieval tasks.
- Convergent Validity: Subjective DMILS was positively correlated with social media use, whereas objective DMILS was negatively correlated, highlighting a potential discrepancy where heavy users feel more competent but possess less actual knowledge.

Significance and Claims
The paper claims that the DMILS provides a "parsimonious yet multidimensional yardstick" for the field. Its primary significance lies in:

Standardization: Offering a common metric for cross-national research, enabling the comparison of media literacy interventions and outcomes across different cultural and political environments.
Intervention Design: Enabling educators and policymakers to diagnose specific competency gaps (e.g., high digital skills but low information knowledge) to tailor interventions rather than applying generic "critical thinking" approaches.
Addressing Overconfidence: By distinguishing perceived from actual competence, the tool helps identify populations (such as heavy social media users) who may be overconfident in their abilities, a factor linked to misinformation susceptibility.
Theoretical Advancement: The findings support the theoretical distinction between digital and information literacies, as well as between knowledge and skills, arguing against treating media literacy as a monolithic construct.

The authors maintain a modest stance, noting that while the scale is robust, future iterations may require more challenging items to better discriminate among highly literate populations, and that qualitative methods may still be necessary to capture aspects difficult to quantify.