FeedAIde: Guiding App Users to Submit Rich Feedback Reports by Asking Context-Aware Follow-Up Questions

This paper presents FeedAIde, a context-aware feedback system powered by Multimodal Large Language Models that guides app users through adaptive follow-up questions to generate more complete and valuable bug reports and feature requests, as validated by improved user ratings and expert assessments in a real-world iOS app evaluation.

The Gist

Imagine you are trying to tell a mechanic that your car is making a weird noise.

The Old Way (Traditional Feedback):
You walk up to the counter and say, "My car is broken."
The mechanic looks at you, confused. "Which part? When does it happen? Does it happen when you turn left? Is it a grinding sound or a squeak?"
You say, "I don't know, it just broke."
The mechanic sighs, writes down "Car broken," and sends you home. They can't fix it because they don't have enough info. You have to come back later, try to remember the details, and explain it again. It's frustrating for both of you.

The New Way (FeedAide):
Now, imagine the mechanic has a super-smart assistant (an AI) standing right next to you.
You say, "My car is making a noise."
The assistant immediately looks at your car's dashboard, sees you just drove over a pothole, and notices the engine light is flashing.
Instead of just writing "Car broken," the assistant asks you: "Did the noise start right after you hit that pothole?"
You say, "Yes!"
The assistant asks: "Was it a grinding sound?"
You say, "Yes, like metal on metal."
The assistant then hands the mechanic a perfect note: "User hit a pothole at 2 PM. Engine light flashed. Grinding noise heard immediately after. Suspect suspension damage."

The mechanic fixes it instantly. You leave happy.

---

What is this paper actually about?

This paper introduces FeedAide, a smart tool for mobile apps that acts like that super-smart assistant. The authors (from the University of Hamburg) noticed that when people use apps, they often leave feedback that is too vague (like "This app sucks" or "It crashed"). Developers get these vague notes and have to spend hours chasing users to ask, "Wait, what happened? What were you doing? What phone do you have?"

FeedAide changes the game by using AI (specifically a type called Multimodal Large Language Models) to talk to the user while they are reporting the problem.

Here is how it works, step-by-step:

1. The Trigger: You are using an app (like a gym app or a language learning app), and something goes wrong. Instead of just typing a complaint, you shake your phone or tap a "Report" button.
2. The AI Takes a Snapshot: The system instantly takes a "snapshot" of what you were doing. It sees your screen, knows what time it is, knows what version of the app you have, and even sees your recent clicks.
3. The Guess: The AI looks at that snapshot and guesses what you might be upset about. It might say, "It looks like your daily streak reset. Did you want to report that?"
4. The Conversation: If you say "Yes," the AI doesn't just stop there. It asks smart follow-up questions based on what it saw.
   • AI: "I see you were traveling to a different time zone. Did the streak reset when you crossed the border?"
   • You: "Yes!"
   • AI: "Okay, I'll tell the developers that the time zone change broke the streak."
5. The Result: The developer gets a rich, detailed report that includes the problem, the context, and the solution, all without you having to write a novel.

Why is this a big deal?

The researchers tested this on a real gym app with real employees. They compared the new AI system against the old, boring "type your complaint here" text box.

• For Users: They found the AI system much easier and more helpful. They felt understood because the AI already knew the context. They didn't have to struggle to explain things.
• For Developers: The reports they received were much better. The AI reports included details like "Steps to Reproduce" (how to make the bug happen again) and specific device info, which the old text-box reports almost never had.

The "Secret Sauce" (How it thinks)

The paper explains that the AI is trained to be a "feedback specialist." It's not just a chatbot; it's programmed to:

• Not be annoying: It asks short, simple questions.
• Be smart: It uses the screenshot and your recent actions to ask the right questions.
• Be private: It tries to only send the necessary info to the developers, though the authors admit they need to be very careful about user privacy in the future.

The Bottom Line

Think of FeedAide as a translator between the "User Language" (which is often messy and vague) and the "Developer Language" (which needs precise, technical details).

Instead of you struggling to describe a problem to a stranger who isn't there, the AI acts as a bridge, gathering all the missing pieces of the puzzle while you are still holding the picture. This saves everyone time, reduces frustration, and helps fix apps faster.

Technical Summary

Here is a detailed technical summary of the paper "FeedAIdе: Guiding App Users to Submit Rich Feedback Reports by Asking Context-Aware Follow-Up Questions."

1. Problem Statement

Mobile app developers rely heavily on user feedback to identify bugs and request features. However, a significant gap exists between what users report and what developers need:

• Vagueness: Users often submit unstructured, vague, or purely negative feedback lacking essential contextual details (e.g., device info, steps to reproduce).
• Inefficiency: Approximately 45% of developer responses to app store reviews require users to provide more details. Many users fail to reply, or the clarification process takes days, leading to frustration and delayed fixes.
• Limitations of Current Solutions: Existing in-app feedback tools are typically static text fields. While some research explores chatbots, few utilize Multimodal Large Language Models (MLLMs) to combine real-time context (screenshots, logs) with adaptive questioning to guide non-technical users during the reporting process.

2. Methodology: FeedAIdе Approach

The authors propose FeedAIdе, a context-aware, interactive feedback framework that leverages MLLMs to guide users in creating rich, developer-ready reports.

Core Workflow

1. Context Capture: When a user triggers the feedback system (e.g., via a "shake-to-report" gesture or button), the system automatically captures:
   • A screenshot of the current screen.
   • Device details (model, OS version).
   • App version and interaction logs (recent events, timestamps).
2. Prediction & Selection: The captured context is sent to an MLLM, which generates up to three feedback predictions (e.g., "Login failure," "Crash on call"). The user selects one or types their own.
3. Adaptive Follow-Up: Based on the user's selection and the captured context, the MLLM generates two targeted follow-up questions. These questions are designed to be simple for non-technical users but critical for developers (e.g., asking about recent password changes or time zone shifts).
4. Report Synthesis: The system compiles the user's answers, the original context, and a generated summary into a structured JSON report containing:
   • userIntentSummary: A natural language explanation of the issue.
   • developerSummary: A concise, technical summary of the root cause or feature request.
   • Full interaction history and metadata.

Technical Implementation

• Platform: Implemented as a reusable Swift Package for iOS applications using SwiftUI.
• AI Integration: Uses AIProxy as a secure proxy to communicate with major LLM providers (e.g., OpenAI), keeping API keys secure and enabling easy model switching.
• Prompt Engineering: The system uses a structured prompt divided into three sections:
  1. Settings/Role: Defines the LLM as a low-friction feedback system, instructing it to avoid generic "something else" options and maintain a non-AI persona to reduce user friction.
  2. App Description: Provides the LLM with a high-level overview of the app's functionality to improve prediction accuracy.
  3. Context Information: Dynamic JSON data including interaction logs, timestamps, and the screenshot.
• Constraints: The model is constrained to ask exactly two follow-up questions and must output structured data (JSON schema) to ensure parseability.

3. Key Contributions

1. Novel Framework: Introduction of FeedAIdе, the first approach to combine MLLMs with real-time app context (screenshots, logs) to dynamically guide users through a feedback reporting flow.
2. iOS Implementation: A fully functional, modular Swift framework that can be integrated into any iOS app with minimal effort (single view modifier).
3. Empirical Evaluation: A comprehensive study involving:
   • User Testing: 7 participants reporting 4 scenarios (2 bugs, 2 features) using both traditional text fields and FeedAIdе.
   • Expert Assessment: Two industry experts (an iOS developer and a UX specialist) evaluated 54 generated reports (28 bug reports, 26 feature requests) based on established quality models.

4. Results

User Perception (RQ1)

• Ease of Use: Participants rated FeedAIdе significantly higher (3.71/4) compared to the traditional text field (2.86/4).
• Helpfulness: FeedAIdе was rated as much more helpful (3.43/4) compared to the traditional system (1.14/4).
• Qualitative Feedback: Users appreciated that the system "understood" their context, reducing the cognitive load of explaining the issue.
• Critique: Users noted loading times (10–20s) were too slow and suggested a visible "Report" button was more intuitive than the shake gesture.

Report Quality (RQ2)

• Bug Reports: FeedAIdе reports scored significantly higher in Observed Behavior (1.50 vs. 0.93) and Steps to Reproduce (2.00 vs. 0.14). The traditional reports often lacked reproduction steps, whereas FeedAIdе automatically captured them via interaction logs.
• Feature Requests: FeedAIdе achieved 100% ratings for Summary and Description, whereas traditional reports were often just summaries with no description.
• Completeness: FeedAIdе reports consistently included metadata (e.g., Product Version) that was missing in 100% of traditional reports.
• Expert Insight: While quality improved, experts noted that some follow-up questions focused too much on refining a solution rather than uncovering the root problem.

5. Significance and Future Work

• Impact: The study demonstrates that GenAI-powered, context-aware feedback flows can drastically reduce the gap between user intent and developer understanding, minimizing the need for time-consuming asynchronous clarification rounds.
• Privacy Concerns: The authors highlight that sending screenshots and logs to vendor LLMs poses privacy risks. Future work should focus on on-device MLLMs (e.g., Apple's Neural Engine) to process data locally, ensuring privacy while maintaining high-quality feedback generation.
• Generalizability: The current study was limited to a single gym app and a small sample size. Future research should test the framework across diverse app categories and larger user populations to validate its scalability.

In conclusion, FeedAIdе represents a shift from static feedback forms to interactive, AI-driven reporting, proving that leveraging multimodal context and adaptive questioning significantly enhances both the user experience and the actionable value of feedback for developers.