Envisioning Sensemaking in Multi-Human, Multi-Agent Collaborative Knowledge Work

Technical Summary: Envisioning Sensemaking in Multi-Human, Multi-Agent Collaborative Knowledge Work

Problem Statement
Sensemaking—the process by which individuals and groups search, evaluate, interpret, and use information to construct durable understanding—is a cornerstone of knowledge work. The integration of Generative AI (GenAI) into these workflows introduces significant challenges, particularly in collaborative settings. While GenAI systems increasingly perform traditional interpretive functions (e.g., summarization, synthesis, thematic grouping), their involvement blurs the boundaries between human and machine sensemaking. This shift obscures the provenance of data, the weighting of evidence, and the handling of conflicting interpretations. In collaborative environments, this complicates the division of labor, undermines trust among team members, and risks reducing collective sensemaking to the mere assessment of AI output plausibility rather than the construction of defensible understanding. Current systems often treat AI as a peripheral assistant rather than an embedded participant, failing to support the accountability and traceability required for negotiated knowledge construction.

Methodology and Approach
This paper is a position paper that proposes a conceptual framework rather than reporting empirical experimental results. The authors' approach involves:

1. Theoretical Synthesis: Drawing on interdisciplinary perspectives from Human-Computer Interaction (HCI), information science, cognitive systems engineering, and organizational communication to define sensemaking as a constructive, dynamic, and often social process.
2. Principle Derivation: Identifying gaps in current AI-mediated workflows to formulate five specific design principles for multi-human, multi-agent collaboration.
3. Conceptual Framework Design: Proposing a "Dynamic Shared Representational Workspace" that operationalizes these principles. This framework is illustrated through a running scenario involving financial market analysis to demonstrate how the system functions in practice.

Key Contributions
The paper makes three primary contributions to the field of collaborative knowledge work and human-AI interaction:

1. Five Design Principles for Multi-Human, Multi-Agent Sensemaking:

   • D1 Dynamic Multi-layer Information Representations: Systems must externalize understanding into structured, editable representations spanning multiple levels of abstraction (from raw data to hypotheses).
   • D2 Active Identification and Bridging of Gaps: Systems should help users recognize and address what is not yet understood, making the construction of meaning visible rather than bypassing it.
   • D3 Critical Engagement with Information: Users must be supported in evaluating, reasoning, and reorganizing data sources rather than accepting ready-made conclusions.
   • D4 Verifiability: Systems must enable users to trace, justify, and explain the development of their interpretations.
   • D5 Accountability: Inputs must be visible and comparable across collaborators, with the evolution of shared understanding remaining traceable over time.

2. A Three-Agent Conceptual Framework:
   The authors propose a dynamic shared workspace populated by three specialized agents that manage the flow of sensemaking:

   • Partner Agent: Operates within an individual's workspace to support the "foraging loop." It assists in articulating information needs, retrieving content, reframing ambiguous ideas, and identifying gaps in reasoning (supporting D2 and D4). It ensures the human remains the primary reasoner.
   • Shared Space Agent: Operates within the collective workspace. It treats individual submissions as candidate additions, performing semantic scans to evaluate structure and fit against existing collective sensemaking. It manages the integration of new evidence and hypotheses without automatically resolving conflicts, preserving alternative interpretations for negotiation (supporting D1 and D5).
   • Orchestrator Agent: Manages the "submission and evaluation" process. It records the provenance and authorship of every input, ensuring that the history of contributions and the evolution of ideas remain visible and traceable (supporting D5).

3. A Unified Representational Structure:
   The framework posits that individual and shared views should draw from the same underlying multi-layered representational structure. This allows for synchronization between personalized individual workspaces and the collective space, ensuring that insights remain legible and coherent across different levels of abstraction and different collaborators.

Results and Claims
As a position paper, the work does not present quantitative experimental results or user studies. Instead, the "results" are the proposed framework and its theoretical alignment with established sensemaking models (e.g., Pirolli and Card's sensemaking loop). The authors demonstrate through a financial analysis scenario how their framework:

• Preserves the distinction between human reasoning and AI assistance.
• Maintains the visibility of the interpretive process (e.g., flagging gaps in reasoning or alternative data sources).
• Facilitates the negotiation of shared understanding by keeping competing hypotheses visible and linked to their evidence chains.
• Prevents the fragmentation typical of current multi-tool workflows by providing a coherent backbone for collaborative sensemaking.

Significance
The paper claims significance by addressing the critical need to redesign collaborative knowledge work in the age of GenAI. It argues that for AI to be a viable partner in sensemaking, systems must move beyond output quality metrics to support the process of sensemaking. By preserving provenance, authorship, and the evolution of interpretations, the proposed framework aims to sustain accountable, negotiated knowledge construction. This approach seeks to prevent the "black boxing" of interpretation that current generative AI systems tend to obscure, thereby enabling teams to maintain trust and coherence while leveraging the capabilities of specialized AI agents.