Psychometric Validation of the Education and Assessment of Genetic Literacy (EAGL) Measure

This study psychometrically validates the Education and Assessment of Genetic Literacy (EAGL) measure in a large US sample, confirming its multi-domain structure and revealing significant interactions between education, personal connection to autism, and knowledge comprehension while finding no geographic disparities.

The Gist

Imagine that "genetic literacy" is like a map of how well people understand the complex instructions that make up life (our DNA). For a long time, the maps scientists have been using to measure this understanding have had some holes in them: they didn't test if people truly understood the concepts, and they hadn't been rigorously checked to see if they were accurate or reliable.

This paper is about building and testing a brand-new, high-quality map called the EAGL (Education and Assessment of Genetic Literacy).

Here is how the researchers built and tested this new map:

1. The Blueprint and the Test Drive
The team took a group of 2,708 people from across the US and asked them to take the EAGL test. Think of this test as a multi-lane highway with three different lanes:

• Lane 1 (Subjective Knowledge): Asking people, "How confident do you feel about your genetics knowledge?"
• Lane 2 (Objective Knowledge): Asking people, "Can you actually answer these specific facts correctly?"
• Lane 3 (Knowledge Comprehension): This is the new, special lane. Instead of just memorizing facts, it asks people to explain how complex conditions work. To test this, they used autism as a specific example, treating it like a complex puzzle rather than a simple fact.

2. What the Test Revealed
When the researchers looked at the results, they found some interesting patterns, like finding hidden landmarks on their map:

• The "Autism" Connection: People who had a personal connection to autism (like having a family member with it) were much more confident in their knowledge (Lane 1). However, this confidence didn't necessarily mean they knew more facts (Lane 2). This confirmed a known phenomenon: people often feel they know more about a topic when it touches their personal life, even if their actual facts aren't any better.
• The Education Interaction: When looking at how well people understood the complex "autism puzzle" (Lane 3), the results depended on a mix of their education level and their personal connection to autism. It wasn't just one or the other; the two factors worked together like gears in a machine to influence understanding.
• City vs. Country: The researchers wondered if living in a big city versus a small town changed how much people knew. They found that, surprisingly, it didn't matter. Whether you were in a metropolitan hub or a rural area, your overall genetic literacy levels were about the same.

3. The Final Product
After running these checks (using statistical tools called factor analysis to ensure the map was accurate), the researchers confirmed that the EAGL works. They now have two versions of this new map available for anyone to use:

• The EAGL-long: A detailed, multi-lane version that has already been tested on thousands of people.
• The EAGL-short: A streamlined, three-lane version that is just as accurate but quicker to take.

In short, this paper didn't just make a new test; it proved that the test is a reliable tool for measuring how well society understands the complex world of genetics, specifically highlighting that understanding complex conditions requires more than just memorizing facts.

Technical Summary

Technical Summary: Psychometric Validation of the Education and Assessment of Genetic Literacy (EAGL) Measure

Problem Statement
Genetic literacy is a critical metric for understanding society's engagement with genetic science. However, existing widely-used measures suffer from two primary limitations: they often lack specific assessments of knowledge comprehension, and they frequently lack rigorous psychometric validation. This gap hinders the accurate measurement of how individuals truly understand complex genetic concepts.

Methodology
To address these deficiencies, the authors conducted a psychometric validation of the Education and Assessment of Genetic Literacy (EAGL) measure. The study utilized a large sample of 2,708 participants from the United States. The validation process employed both Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) to test the structural integrity of the instrument.

The EAGL instrument was designed to include standard subjective and objective knowledge subscales, alongside a distinct "knowledge comprehension" subscale. This unique subscale specifically utilizes autism as a case study for a complex genetic condition to assess depth of understanding. The researchers further investigated demographic and contextual variables, including education level, personal connection to autism, and geographic location (metropolitan vs. non-metropolitan status), using regression analyses to examine their relationships with genetic literacy scores.

Key Results
The analysis yielded several statistically significant findings:

• Interaction Effects: A significant interaction was observed between education level and personal connection to autism regarding knowledge comprehension ($F=3.68, p=0.003$).
• Subjective-Objective Gap: Participants with a personal connection to autism scored significantly higher on the subjective knowledge section ($F=19.52, p<0.001$) compared to those without such a connection. However, this elevation was not observed in the objective knowledge section, thereby corroborating previous demonstrations of a subjective-objective knowledge gap in science literacy.
• Geographic Factors: The study found no significant main effects of metropolitan versus non-metropolitan status on overall levels of genetic literacy.

Contributions and Significance
The primary contribution of this work is the successful validation of the EAGL measure, which now offers two distinct, multi-domain instruments available for broad application:

1. EAGL-long: A comprehensive, multi-faceted measure previously tested in thousands of participants.
2. EAGL-short: A validated three-factor version derived from the current study.

The paper asserts that these tools accurately capture the construct of genetic literacy, specifically addressing the prior lack of knowledge comprehension measures and psychometric rigor. By providing validated instruments that distinguish between subjective confidence and objective understanding, the study enables more precise examination of society's interaction with genetics.