📄 social_science

“It feels like an activity, not a test”: Developing and pilot-testing a sticker-based Likert scale for adolescent museum visitors at the Huangmei Opera Museum

This paper reports on the design and pilot testing of an engaging, sticker-based Likert scale instrument that successfully assesses five key experience dimensions among adolescent visitors at China's Huangmei Opera Museum, demonstrating its viability as a practical quantitative evaluation tool for this demographic.

Original authors: Li Yang, Liliana Soares, Rute Gomes

Published 2026-07-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Li Yang, Liliana Soares, Rute Gomes

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine walking out of a museum after a cool trip, and instead of being handed a boring, text-heavy survey that feels like a pop quiz, you're given a sheet of paper and a handful of colorful stickers. Your job? Stick a sticker on a cloud-shaped spot that matches how you felt. That's exactly what researchers tried at the Huangmei Opera Museum in China, and they found that for kids and teens aged 8 to 17, this "sticker game" actually works as a real science tool.

The Big Idea: A Game, Not a Test
The main finding here is that turning a survey into a sticker-placement activity is a feasible way to get honest answers from young museum-goers. The researchers suggest that this method feels more like an activity and less like a test, which helps keep teenagers engaged. In their pilot run with 30 visitors, the stickers did the job: every single questionnaire was completed with no missing answers, and the kids used the full range of options (from "strongly disagree" to "strongly agree") rather than just picking the same spot over and over.

What This Tool Is NOT
It's important to know what this study doesn't prove. The authors explicitly rule out the idea that this sticker method is a "solved problem" or a fully perfected tool. They argue against the notion that it is currently a stable, multi-dimensional psychometric instrument ready for immediate, wide-scale use. The paper makes it clear that while the format works, the specific questions inside need serious tuning. For instance, the study found that some questions about "interactivity" got almost everyone to stick their stickers in the "disagree" corner, while questions about "wanting more fun" got everyone to stick them in the "agree" corner. This suggests the tool is currently too "clumpy" in some areas to measure subtle differences perfectly.

How Sure Are We? (The "Pilot" Reality)
Think of this study as a "dress rehearsal" rather than the opening night. The authors are careful to state that their results are preliminary and based on a small group of 30 people. They don't claim to have proved that this method is better than a traditional paper-and-pencil survey; they simply showed it is possible to use it in a real museum without kids walking away.

Here is what the data actually showed, based on their measurements:

Completion Time: It took the group an average of 13 minutes and 14 seconds to finish, with the fastest taking 10 minutes and 2 seconds and the slowest taking 16 minutes and 2 seconds.
Independence: Most kids (23 out of 30, or 76.7%) could do it entirely on their own after a quick demo. However, 5 kids (16.7%) needed a little help now and then, and 2 kids (6.7%) needed help the whole time.
The "Direction" Glitch: A few kids got confused about which way the scale went. They might have said "I loved it!" out loud but stuck their sticker on the "I hated it" side. The researchers suggest this isn't because the kids were lying, but because the visual map of the stickers was a bit tricky to read in the moment.

The "Cultural Awareness" vs. "Interactivity" Surprise
When the researchers looked at the sticker patterns, they found some interesting splits.

Cultural Awareness: The kids generally felt they understood the opera and the history. The average score for this was 3.83 (on a scale of 1 to 5), with a "Cronbach's alpha" (a measure of how consistent the answers were) of 0.43. This is a modest score, suggesting the questions were okay but not perfectly tight.
Interactivity: This was the big surprise. The average score for how much kids liked the interactive parts was only 1.92. In fact, 90.0% of the answers for one question and 86.7% for another landed in the "disagree" range. The internal consistency for this part was so low (a correlation of -0.02) that the researchers suggest these two questions might actually be measuring two different things, or that the interactive exhibits just weren't hitting the mark for this group.
Wanting More Fun: On the flip side, 100% of the kids agreed they wanted more fun experiences, and 100% said they would come back if the museum used new display methods.

The Takeaway
The authors suggest that while this sticker-based approach is a promising, child-centered way to gather data, it needs more work before it can be considered a "gold standard." They propose that future versions should add more questions to the "fun" and "interactive" sections to get a clearer picture, and they need to fix the visual layout so kids don't mix up left and right.

In short, the sticker method is a viable, engaging tool that successfully collected data from 30 teenagers without any dropouts, but it's still a work in progress. It's a great start, but the researchers are clear: don't treat these results as the final word, but rather as a solid foundation for building a better, more reliable tool for the future.

Technical Summary: "It feels like an activity, not a test": Developing and pilot-testing a sticker-based Likert scale for adolescent museum visitors

Problem Statement
Museums increasingly seek systematic feedback from younger audiences to improve interpretation and educational programming. However, evaluating the experience of children and adolescents (ages 8–17) remains methodologically challenging. Conventional text-heavy questionnaires often fail in real-world museum settings due to tight time windows, heterogeneous literacy levels, variable attention spans, and the perception of surveys as "school-like" burdens. Existing age-appropriate quantitative tools are limited, and while pictorial or emoji-based scales exist, there is a need for instruments that are both engaging for adolescents and analytically compatible with standard psychometric techniques. The specific challenge is to create a measurement tool that captures complex experience constructs (e.g., enjoyment, interactivity, cultural awareness) without increasing response burden or compromising data quality.

Methodology
The study employed a field-based, instrument-oriented pilot design conducted at the Huangmei Opera Museum in Anqing, China.

Participants: A convenience sample of 30 visitors aged 8 to 17, recruited via intercept at the museum exit. All participants were accompanied by a parent or guardian.
Instrument: A one-page, sticker-based Likert scale was developed. Instead of ticking boxes, respondents placed Huangmei Opera-themed character face stickers onto a five-point agreement grid (ranging from "strongly disagree" to "strongly agree"). The instrument operationalized five experience dimensions: enjoyment, cultural awareness, interactivity, technological novelty, and revisit intention.
Procedure: Researchers administered the questionnaire immediately post-visit. Participants were given a brief demonstration and then completed the task independently. Researchers recorded completion times and observed response behaviors (e.g., hesitation, need for assistance, sticker placement accuracy).
Data Analysis: Data were coded numerically (1–5) based on sticker position. Analysis focused on two layers:
1. Measurement Performance: Feasibility (completion time, missing data), response process (independence, assistance levels), response distribution (differentiation vs. clustering), and preliminary internal structure (Cronbach's alpha, inter-item correlations).
2. Illustrative Experience Relations: Descriptive analysis of item-level patterns to explore associations among the five dimensions, serving as a secondary demonstration of the tool's utility.

Key Results

Feasibility and Acceptability: The instrument was highly feasible. All 30 questionnaires were completed with zero missing data. Completion times ranged from 10 to 16 minutes (mean: ~13 minutes).
Response Process: 76.7% of participants completed the task independently. 16.7% required occasional assistance, and 6.7% required continuous support. Observations noted that a minority of participants experienced momentary confusion regarding the left-to-right spatial orientation of the scale, occasionally placing stickers in columns opposite to their verbal judgments.
Response Distribution: The format successfully elicited differentiated responses. All five response categories were utilized across the dataset. However, specific items showed strong skew:
- Floor Effects: Items related to "Interactivity" (I9, I10) showed strong negative skew (means < 2.0), with 86–90% of responses in the "disagree" range.
- Ceiling Effects: Items regarding "desire for more fun experiences" (I12) and "revisit intention with new methods" (I13) showed strong positive skew (means > 4.7), with 100% of responses in the "agree" range.
Internal Structure: Preliminary reliability estimates were mixed, likely due to the small sample size and restricted variance.
- Cultural Awareness (4 items) showed modest internal consistency ( $\alpha = 0.43$ ).
- Enjoyment (2 items) and Interactivity (2 items) showed unstable or near-zero inter-item correlations ( $\alpha = 0.13$ and $-0.04$, respectively), suggesting these dimensions may not function as single reflective constructs in this specific pilot context or require more items.

Significance and Claims
The paper positions itself primarily as a methodological contribution rather than a comprehensive assessment of the Huangmei Opera Museum. Its core claims are modest and focused on the instrument's potential:

Feasibility of Sticker-Based Formats: The study provides initial evidence that a sticker-based Likert format is a practical, engaging option for quantitative evaluation with adolescent visitors in situ. It successfully transforms a rating task into a tangible activity, reducing the perception of a "test."
Analytic Compatibility: Despite the non-traditional response mechanism, the data produced are compatible with standard quantitative analysis (ordinal coding, descriptive statistics, and internal consistency checks).
Identification of Design Refinements: The pilot successfully identified specific usability issues, particularly regarding the spatial mapping of verbal judgments to the grid (directionality errors). The authors argue that future iterations must include stronger visual cues (e.g., arrows, direction checks) to mitigate these errors.
Limitations of Current Validation: The authors explicitly state that the instrument is not yet fully validated. The mixed internal structure results and the small, single-site sample mean the tool requires further item expansion (particularly for enjoyment and interactivity), cognitive interviewing, and larger-scale testing before it can be considered a robust psychometric instrument.

In summary, the paper demonstrates that a sticker-based approach can yield complete, analyzable data from adolescent museum visitors, offering a promising alternative to text-heavy surveys, while simultaneously highlighting the necessary steps for refining the tool's reliability and validity.

Technical Summary: "It feels like an activity, not a test": Developing and pilot-testing a sticker-based Likert scale for adolescent museum visitors

More like this