📄 medicine

Measuring What Future Physicians Know About Contraception: Development and Validation of the Clinical Contraceptive Knowledge Test (CCKT) in Vietnam Using Item Response Theory

This study developed and validated the 18-item Clinical Contraceptive Knowledge Test (CCKT-18) using Item Response Theory among Vietnamese medical students, demonstrating it to be a robust, unidimensional, and precise instrument for assessing contraceptive knowledge and guiding curriculum improvements.

Original authors: Nguyen Dinh Ky, Nguyen Quynh Anh, Nguyen Ngoc Minh, Pham The Lam, Do Hoa Cham, To Khanh Linh, Nguyen Dang Linh Chi, Pham Hong Anh, Nguyen Thi Hong Hanh, Nguyen Tien Thang, Nguyen Thi Huyen Anh, Nguyen

Published 2026-07-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nguyen Dinh Ky, Nguyen Quynh Anh, Nguyen Ngoc Minh, Pham The Lam, Do Hoa Cham, To Khanh Linh, Nguyen Dang Linh Chi, Pham Hong Anh, Nguyen Thi Hong Hanh, Nguyen Tien Thang, Nguyen Thi Huyen Anh, Nguyen Manh Thang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to measure how well a group of future doctors knows about birth control. In the past, researchers used a "ruler" that was a bit broken. It treated every question as if it were equally important, and it only gave you a single number (like a test score out of 100) without telling you what the student actually knew or where they were struggling. It was like trying to measure the height of a mountain range with a single, flat tape measure; you get a number, but you miss the peaks and valleys.

This paper describes the creation of a brand new, high-tech "laser scanner" called the Clinical Contraceptive Knowledge Test (CCKT-18). Here is how the researchers built it and what they found, using simple analogies.

1. The Problem: The Old Ruler Wasn't Enough

The researchers noticed that medical students in Vietnam (and globally) often get good grades in school but still lack the practical know-how to advise patients on birth control. The old tests they used were based on "Classical Test Theory."

The Analogy: Think of the old test like a bag of marbles where every marble is the same size. If you count them, you know how many you have, but you don't know if some are heavy and some are light. The old test assumed every question was equally hard and equally important.
The Goal: They wanted a tool that could measure the difficulty of each question and how well each question separated a "know-it-all" from someone who is guessing.

2. Building the New Tool: A Four-Step Construction

The team didn't just write questions and hope for the best. They built the test in four careful phases, like constructing a sturdy bridge:

Phase 1: Gathering the Bricks (Item Generation): They looked at global health guidelines (from the WHO) and educational theories to write 27 initial questions. These questions covered everything from basic facts to complex clinical decisions.
Phase 2: The Quality Control Check (Expert Review): Six experts in reproductive health and education looked at the questions. They acted like strict editors, removing one question that didn't fit and polishing two others until they were perfect. This left them with 26 questions.
Phase 3: The Stress Test (Pilot Testing): They tried the 26-question test on a small group of 100 students. It worked well, but they wanted to make it even sharper.
Phase 4: The High-Tech Scan (Large-Scale Validation): They gave the test to over 700 medical students across five universities in Vietnam. They used a sophisticated mathematical method called Item Response Theory (IRT).
- The Analogy: If the old method was a simple scale, IRT is like a 3D printer that analyzes the shape of every single piece of data. It looks at how hard each question is and how well it distinguishes between students who really know the material and those who are just guessing.

3. The Result: The CCKT-18

After analyzing the data, the researchers realized they didn't need all 26 questions. Some questions were redundant (asking the same thing in different ways) or didn't do a good job of telling students apart.

The Trim: They cut the test down to 18 questions.
The Shape: They expected the test to measure three different skills (remembering facts, understanding concepts, and applying rules). However, the math showed that for these students, all these skills were so tightly connected that they acted like one single super-skill. So, they kept the test as a single, unified measure.
The Performance: The new 18-question test (CCKT-18) is highly precise. It can accurately measure a student's knowledge whether they are a beginner or an advanced learner. It's like a camera with a perfect zoom lens that stays sharp at every distance.

4. What the Test Revealed

When they used this new "laser scanner" on the students, they found a few interesting things:

Fairness: The test worked exactly the same way for male and female students. No one was unfairly advantaged or disadvantaged by the questions.
The Learning Curve: The test showed that students were generally good at the "easy" stuff (basic facts) but struggled more with the "hard" stuff (applying guidelines to complex real-life situations).
The "Clinical" Gap: The test noticed that students who had already started their clinical rotations (working in hospitals) answered certain tricky questions better than those who hadn't. This suggests the test is good at spotting who has actually learned the practical skills versus just memorizing the textbook.

5. Why This Matters (According to the Paper)

The paper concludes that the CCKT-18 is a valid, reliable, and robust tool.

The Analogy: Before, educators were trying to guess how much water was in a bucket by looking at the color of the water. Now, they have a precise measuring cup.
The Benefit: Because the test is so precise, it can tell educators exactly where the gaps in knowledge are. Instead of saying "The class needs more study," they can say, "The class is great at remembering facts, but they need more practice on how to apply those facts to real patients."

Important Note: The paper strictly focuses on creating and validating this measuring tool. It does not claim to have fixed the problem yet, nor does it say exactly how schools should change their classes based on these results. It simply provides the high-quality "ruler" needed to measure the problem accurately so that future work can begin.

Technical Summary: Development and Validation of the Clinical Contraceptive Knowledge Test (CCKT)

Problem Statement
Accurate assessment of contraceptive knowledge among medical students is hindered by the limitations of traditional evaluation tools. Existing instruments, such as the Contraceptive Knowledge Assessment (CKA), rely on Classical Test Theory (CTT), which aggregates raw scores under the assumption of equal item weighting. This approach fails to provide a precise measure of latent clinical competence and cannot independently estimate item difficulty and discrimination. Furthermore, there is a lack of validated, Item Response Theory (IRT)-based instruments for assessing contraceptive knowledge in Vietnam or internationally, despite the critical need to quantify educational gaps to inform curriculum reform. Given that Vietnam faces high rates of unintended pregnancies and abortions, and that medical students often lack practical knowledge of modern contraceptives and WHO Medical Eligibility Criteria (MEC), a more robust psychometric tool is required.

Methodology
This study employed a cross-sectional, multicenter design involving undergraduate medical students across five universities in Vietnam. The development and validation of the Clinical Contraceptive Knowledge Test (CCKT) followed a rigorous four-phase sequential framework:

Item Generation: An initial 27-item scale was constructed based on WHO guidelines and mapped to Bloom's Revised Taxonomy (Recall, Understanding, Application). Items utilized a 'True/False/I don't know' format to minimize guessing.
Expert Review: A panel of six specialists assessed content validity using the Item-Level Content Validity Index (I-CVI) and Scale-Level Content Validity Index (S-CVI/Ave). This phase resulted in the removal of one item and the revision of two, yielding a 26-item instrument.
Pilot Testing: A pilot study (n=100) utilized CTT to screen for item difficulty, discrimination, and internal consistency (KR-20 = 0.854).
Large-Scale Validation: The main study recruited 723 students, with 636 retained for analysis after excluding invalid responses. The researchers applied a two-parameter logistic (2PL) IRT model.
- Dimensionality: Parallel analysis and scree plots were used to determine the number of latent factors.
- Model Selection: Seven competing models (ranging from unidimensional to multidimensional 2PL and MIRT specifications) were evaluated using fit indices (CFI, TLI, RMSEA, SRMR). The principle of parsimony guided the selection of the final model.
- Item Reduction: Items were iteratively removed based on low discrimination ( $a < 0.80$ ), local dependence ( $|Q3| > 0.20$ ), or misfit ( $S-X^2, p < 0.05$ ).
- Validation Checks: Differential Item Functioning (DIF) was tested across sex and training stages. Sensitivity analyses were conducted using a "strict-clean" dataset (n=545) to ensure model robustness.

Key Results

Instrument Refinement: The process reduced the initial 26 items to a final 18-item instrument (CCKT-18).
Structural Validity: Although exploratory analyses suggested potential multidimensionality, empirical evaluation supported a unidimensional structure. The unidimensional 2PL model (Model 7) demonstrated excellent fit indices (CFI = 0.983, TLI = 0.980, RMSEA = 0.036, SRMR = 0.039), comparable to more complex multidimensional models. Inter-factor correlations in multidimensional models were extremely high ( $r > 0.80$ ), indicating a single dominant latent trait.
Item Parameters: The final items exhibited discrimination parameters ranging from 0.687 to 2.374 and difficulty parameters from -3.86 to 1.08.
Reliability and Precision: The scale demonstrated high measurement precision with a marginal reliability of 0.844. The Test Information Function (TIF) peaked at $\theta = 0.462$ , providing high precision (reliability $\ge 0.80$ ) across the ability range of -0.784 to +1.51.
DIF Analysis: No significant DIF was found across sex, confirming measurement invariance. However, four items showed significant DIF across training stages (preclinical vs. clinical), specifically regarding clinical decision-making competencies (e.g., IUD selection, missed pill management), reflecting differences in curricular exposure rather than bias.
Robustness: Sensitivity analyses confirmed structural stability, with negligible changes in model fit ( $\Delta CFI = 0.003$ ) and item parameters between the main and strict-clean datasets.

Significance and Claims
The authors claim that the CCKT-18 is a valid, reliable, and robust instrument for assessing contraceptive knowledge among medical students. Its primary significance lies in its methodological advancement over legacy tools:

Precision: By leveraging IRT, the CCKT-18 moves beyond raw scores to provide a precise, competency-oriented evaluation of latent ability, allowing educators to identify specific curricular deficits (e.g., gaps in higher-order application skills) rather than just general knowledge levels.
Educational Utility: The instrument supports a shift from traditional score-based assessment to competency-based evaluation, aligning with frameworks that emphasize clinical reasoning.
Global Context: The study corroborates international findings that medical students often lack practical application skills despite strong theoretical knowledge, but provides a standardized, psychometrically rigorous template for cross-cultural assessment.
Future Application: Within the broader V-COC Study framework, the CCKT-18 is positioned as a foundational tool for nationwide surveys to characterize the contraceptive knowledge landscape in Vietnam, aiming to drive targeted curriculum reforms in reproductive health education.

The authors acknowledge limitations, including the use of convenience sampling, the cross-sectional design, and the concentration of item difficulty in the moderate range, which may limit discrimination at the highest ability levels. They note that future work should include longitudinal studies and criterion-related validity assessments.