The impact of AI and peer feedback on research writing skills: a study using the CGScholar platform among Kazakhstani scholars

This study of 36 Kazakhstani scholars using the CGScholar platform reveals that while familiarity with AI tools correlates with openness to feedback, participants still highly value peer input for methodological guidance, suggesting that integrating AI with traditional peer feedback effectively enhances academic research writing skills.

The Gist

Imagine you are trying to write a very important, complex recipe for a new dish. You want it to be perfect, but you aren't sure if your instructions are clear, if the ingredients are right, or if the cooking steps make sense.

This paper is about a group of 36 scholars from Kazakhstan who were trying to write their own "recipes" (academic research papers). They were stuck in a kitchen where they had two new helpers:

1. The Robot Chef (AI): A smart computer program that can instantly check if you spelled "salt" correctly, if your sentences are too long, or if your recipe is formatted nicely.
2. The Fellow Cooks (Peer Feedback): Real human colleagues who taste the dish and say, "Hey, this spice is too strong," or "I think you need to explain why you chose this ingredient."

The researchers wanted to know: Do these scholars trust the Robot Chef? Do they prefer the Fellow Cooks? And does using the Robot make them more willing to listen to the Cooks?

The Setup: A Digital Cooking Class

The scholars were taking a special class at the University of Illinois, but they were doing it from Kazakhstan. They used a digital platform called CGScholar. Think of this platform as a giant, high-tech kitchen table where everyone can see each other's work.

The class had a specific plan:

• Week 7: The scholars let the Robot Chef look at their drafts first. The robot gave them a report card on grammar, structure, and clarity.
• Week 9: The scholars swapped papers with each other. The Fellow Cooks gave them advice on the ideas, the logic, and the "flavor" of the research.
• Week 12: They submitted their final, polished recipes.

Afterward, the researchers asked the scholars a bunch of questions to see how they felt about the whole process.

What They Found: The Surprising Results

1. The Robot is New, but the Cooks are Old Friends
The scholars admitted they didn't know much about the Robot Chef yet. Their "familiarity score" was quite low. It's like being handed a brand-new, complicated smartphone when you've only ever used a flip phone. They were a bit nervous about it.

However, they were very comfortable with the Fellow Cooks. Peer feedback is a traditional way of learning, so they felt safe and familiar with that process.

2. The "Openness" Sweet Spot
Here is the good news: Even though they didn't know the Robot well, the scholars were very open to taking advice. They were like eager students ready to learn. The study found that the more comfortable someone felt with technology (the Robot), the more willing they were to change their work based on feedback.

3. Experience Changes the Menu
The most experienced scholars (the "Master Chefs" in the group) had a specific request: they really wanted the Fellow Cooks to help them with the methodology (the "how-to" and the logic of the recipe). They knew the Robot could fix their grammar, but they needed a human to tell them if their research logic was sound.

There was a strong link found: The more writing experience you have, the more you value human feedback on the deep, tricky parts of your work.

4. The Power of the Hybrid Kitchen
The study concluded that the best kitchen isn't just the Robot or just the Humans. It's a hybrid kitchen.

• The Robot is great for the basics: fixing typos, checking formatting, and making sure the sentences flow smoothly.
• The Humans are essential for the soul of the dish: checking if the argument makes sense, if the ethics are right, and if the research is truly innovative.

The Takeaway (In Plain English)

This paper tells us that AI is a helpful tool, but it's not a replacement for human connection.

Imagine AI as a spell-checker on steroids. It can tell you if you used the wrong word, but it can't tell you if your story is boring or if your argument is weak. The Kazakhstani scholars in this study were happy to use the AI to polish their writing, but they still craved the wisdom of their human peers to make their research truly great.

The Bottom Line:
If you want to write a masterpiece, let the Robot fix your grammar, but make sure you invite your smartest friends over to taste-test your ideas. That combination is the secret sauce to success.

Technical Summary

Here is a detailed technical summary of the paper "The Impact of AI and Peer Feedback on Research Writing Skills: A Study Using the CGScholar Platform Among Kazakhstani Scholars."

1. Problem Statement

The study addresses the evolving landscape of academic writing in the digital era, specifically focusing on the limitations and potential of Artificial Intelligence (AI) versus traditional peer feedback.

• The Gap: While AI tools excel at correcting surface-level linguistic errors and structural coherence, they struggle with higher-order cognitive tasks such as evaluating critical thinking, argument depth, ethical considerations, and contextual reasoning. Conversely, traditional peer feedback is effective for these higher-order tasks but is often time-consuming and subject to variability.
• The Context: There is a lack of empirical data regarding how scholars from non-Western, non-English dominant backgrounds (specifically Kazakhstani scholars) perceive and utilize hybrid feedback models that combine AI and peer review.
• Objective: The research aims to determine how familiarity with AI tools and exposure to peer feedback processes influence scholars' openness to incorporating feedback into their research writing, specifically within the context of the CGScholar platform.

2. Methodology

The study employed a Design-Based Research (DBR) approach, utilizing an iterative cycle of design, implementation, and evaluation over one academic semester.

• Participants:
  • Sample: 36 Kazakhstani scholars (100% female) enrolled in a scientific internship at the University of Illinois Urbana-Champaign (UIUC).
  • Demographics: 58% were aged 46+, 25% aged 26–35, and 17% aged 36–45. All identified Kazakh as their first language.
  • Course: "Transformations in Higher Education: Teaching, Learning, and Research in a Time of Disruptive Change."
• Intervention Platform: CGScholar, a collaborative learning platform developed by UIUC and Common Ground Research Networks. It integrates AI-driven analytics with peer review mechanisms.
• Procedure:
  1. Preparation: Participants received a video tutorial in Kazakh to navigate the platform.
  2. Feedback Cycles:
     • Week 7: AI-generated feedback on drafts based on a specific rubric.
     • Week 9: Structured peer review process.
     • Week 12: Submission of final drafts revised based on both feedback types.
  3. Data Collection: A semi-structured survey (15 questions) was administered via Google Forms in both English and Russian.
     • Instrument: Included multiple-choice, Likert-scale (1–5), and open-ended questions.
     • Metrics: Measured familiarity with AI, familiarity with peer feedback, openness to feedback, and research writing experience.
  4. Rubric: The "Knowledge Processes Rubric" categorized feedback into six areas: Experience, Empirical Evidence, Conceptualization, Analysis, Application, and Presentation.
• Analysis: Quantitative data was analyzed using descriptive statistics and correlation analysis. Qualitative data underwent thematic analysis.

3. Key Contributions

• Hybrid Feedback Model Validation: The study provides empirical evidence supporting a hybrid model where AI handles linguistic/structural accuracy while peer feedback addresses methodological and argumentative depth.
• Cultural Contextualization: It offers specific insights into the adoption of AI tools among Central Asian scholars, a demographic often underrepresented in EdTech literature.
• Platform Specifics: It details the operationalization of the CGScholar platform's "Knowledge Processes Rubric" in a real-world academic setting, demonstrating how AI can be trained to evaluate specific academic dimensions (e.g., conceptualization, application).
• Correlation Insights: The study identifies distinct correlations between a scholar's experience level, their familiarity with technology, and their receptiveness to different feedback types.

4. Key Results

• Familiarity Levels:
  • AI Tools: Low familiarity (Mean Score: 1.92 on a 1–5 scale). Participants reported limited access or confidence in using AI for research.
  • Peer Feedback: Moderate familiarity (Mean Score: 3.67), indicating it is a more established practice in their educational environment.
• Openness to Feedback:
  • Participants showed a high level of openness to feedback generally (Mean Score: 3.83), suggesting a willingness to adapt regardless of the feedback source.
• Correlation Analysis:
  • AI Familiarity & Openness to Change: Moderate positive correlation ($r \approx 0.35$). Those more familiar with AI were slightly more open to making changes based on feedback.
  • Writing Experience & Peer Feedback Expectations: Strong positive correlation ($r \approx 0.68$). Experienced researchers placed higher value on peer feedback, particularly regarding research methodology.
  • AI Familiarity & Peer Feedback Expectations: Very strong positive correlation ($r \approx 0.82$). Scholars comfortable with AI tools also highly valued peer feedback for methodological guidance, suggesting they view the two as complementary rather than mutually exclusive.
• Skill Level: Participants reported moderate research writing experience (Mean Score: 3.5), indicating a transitional stage where targeted interventions could be highly effective.

5. Significance and Limitations

• Significance:
  • The study validates that while AI is currently underutilized by this demographic, scholars are open to its integration.
  • It confirms that peer feedback remains critical for high-level academic tasks (methodology, argumentation) even as AI adoption grows.
  • It suggests that integrating AI for efficiency (grammar/structure) and peers for depth (critical thinking) creates a balanced, ethically sound evaluation system.
• Limitations:
  • Sample Size & Diversity: The small sample (N=36) consists entirely of female Kazakhstani scholars, limiting generalizability to other genders, nationalities, or disciplines.
  • Longitudinal Gap: The study relied on pre-survey data and immediate feedback loops; it did not measure long-term skill retention or the sustained impact of these mechanisms over multiple writing cycles.
  • Language Bias: The bilingual survey (English/Russian) may have introduced interpretation variances.
• Future Directions: The authors recommend longitudinal studies to track skill retention over time and expanded research involving diverse academic audiences and disciplines to better understand cross-cultural feedback mechanisms.