💻 computer science

BayesInsights: Modelling Software Delivery and Developer Experience with Bayesian Networks at Bloomberg

This paper presents BayesInsights, a novel Bayesian Network-based tool developed at Bloomberg that enables interactive visualization of causal dependencies in software engineering, successfully demonstrating its utility to senior practitioners in identifying delivery challenges and modeling developer experience.

Original authors: Serkan Kirbas, Federica Sarro, David Williams

Published 2026-04-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Serkan Kirbas, Federica Sarro, David Williams

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a massive, high-tech ship (Bloomberg's software engineering team). You have a dashboard full of gauges: speed, fuel consumption, engine temperature, and crew morale.

The Problem:
Right now, your dashboard tells you what is happening (e.g., "The ship is slowing down" or "The crew looks tired"). But it doesn't tell you why.

Is the ship slow because the engine is clogged?
Is the crew tired because the engine is making too much noise?
Or is the crew tired because the food is bad?

If you just guess and try to fix the engine, you might waste time and fuel, only to find out the real problem was the food. In the paper, the authors call this "ambiguous interpretation." You have all the data, but you can't see the invisible strings connecting the different parts of the ship.

The Solution: BayesInsights
The team built a tool called BayesInsights. Think of this not as a dashboard, but as a Crystal Ball or a Flight Simulator for your software team.

Instead of just showing you numbers, it maps out the "invisible strings" (causal links) between everything. It uses a mathematical model called a Bayesian Network, which is like a giant, interactive flowchart of "If this happens, then that will likely happen."

How It Works (The Analogy):
Imagine you are playing a game of "What If?" with a magic mirror.

The Map: The tool has a map of your team's world. It knows that "Focus without distraction" is connected to "Time lost to obstacles," which is connected to "Developer Happiness."
The Simulation: You can reach out and touch a part of the map. Let's say you click on "Time lost to obstacles" and say, "What if we cut this time in half?"
The Ripple Effect: The magic mirror instantly updates. It shows you that if you fix the obstacles, "Developer Happiness" goes up by 20%, and "Code Quality" improves by 15%. It shows you the ripple effect of your decision before you even make it.

How They Built the Map (The Recipe):
They didn't just guess the connections. They used a "Three-Layer Cake" approach to make sure the map was accurate:

The Foundation (Literature): They started with the "textbook rules" of software engineering (like the DORA metrics) to draw the basic lines.
The Flavor (Expert Opinion): They asked the ship's most experienced captains (senior engineers and managers at Bloomberg) to taste the map. "Does this connection make sense in our real life?" They tweaked the lines based on the experts' gut feelings and experience.
The Reality Check (Data Algorithms): Finally, they fed thousands of real survey answers into a computer algorithm. The algorithm checked: "Does the data actually support these lines?" If the data said two things weren't connected, the algorithm removed the line. If it found a hidden connection the experts missed, it added it.

The Results:
They tested this "Flight Simulator" with 24 senior leaders at Bloomberg.

96% of them said, "This is incredibly useful! It helps us see the real problems."
75% said the results were easy to understand.
Many said that in just 15 minutes of playing with the tool, they came up with concrete ideas to fix their teams' problems (like "We need to stop distracting our engineers so they can focus").

Why It Matters:
Before this tool, managers were like doctors guessing a diagnosis based on a fever chart. With BayesInsights, they have an MRI scan that shows exactly how one symptom affects the whole body. It moves software management from "guessing and hoping" to "knowing and planning."

In a Nutshell:
BayesInsights is a predictive simulator that helps software leaders see the hidden cause-and-effect chains in their work. It lets them run "What If?" scenarios to fix the root causes of unhappiness and slowness, rather than just treating the symptoms. It's currently being used by a few teams, and the plan is to roll it out to the whole company soon.

1. Problem Statement

As software engineering (SE) organizations grow, they generate vast amounts of data regarding delivery performance and developer experience (DevEx). However, current analytics solutions (e.g., dashboards tracking DORA metrics or SPACE framework surveys) suffer from significant limitations:

Lack of Causal Insight: Dashboards show what is happening (correlations) but not why (causation). They fail to explain how changes in one metric (e.g., deployment frequency) impact others (e.g., developer happiness or stability).
Ambiguity in Trade-offs: Teams struggle to estimate the consequences of engineering changes or predict root causes of deficiencies because metrics are often viewed in isolation.
Data Complexity: SE data involves complex interdependencies, hidden confounders (e.g., organizational culture), and biases inherent in survey data, making naive data-driven causal inference unreliable.

2. Methodology

The authors developed BayesInsights, an interactive tool utilizing Bayesian Networks (BNs) to model probabilistic and causal dependencies between software delivery metrics and developer experience factors.

A. Network Structure Definition (Hybrid Approach)

To construct credible Directed Acyclic Graphs (DAGs), the team employed a three-step iterative methodology:

Established Literature: The foundation was built using DORA metrics (Throughput and Stability) and causal links from existing literature. Nodes were mapped to specific survey questions (e.g., "Time Lost to Obstacles"). Causal idioms (Cause-Consequence and Definition/Synthesis) were used to hypothesize links.
Expert Insight: A structured survey was conducted with 8 senior DevX practitioners (avg. 10 years experience) at Bloomberg. They rated the strength of 24 potential causal relationships (Strong, Moderate, Weak, None). Relationships with a weighted score $\ge$ 0.70 were retained. This step refined the model to fit Bloomberg's specific context.
Structure Learning Algorithms: To validate and refine the expert-derived structure, the team applied Hill-Climbing (HC) and Peter-Clark (PC) algorithms to internal survey data (>2,000 responses, 20 questions).
- Technique: Bootstrap resampling was used to assign confidence scores to edges.
- Evaluation: Models were scored using the Bayesian Information Criterion (BIC).
- Result: The expert-refined model performed best (lowest BIC), followed by HC. PC added little value due to weak overlap. The final structure prioritized expert/DORA consensus, using algorithmic outputs only as candidates for refinement.

B. Parameterization

Data Source: Internal engineering survey data (Likert-scale questions treated as discrete categorical variables).
Conditional Probability Tables (CPTs): Computed for each node.
- Root Nodes: Marginal distributions ( $P(X=x)$ ).
- Child Nodes: Conditional probabilities based on parent configurations ($P(X|Parents)$).
- Smoothing: Bayesian-Dirichlet equivalent uniform (BDeu) smoothing was applied to handle sparse data and prevent zero-probability entries.

C. Implementation & Integration

Architecture: Client-server model.
- Backend: Python-based using Django and Django Ninja for RESTful APIs. Inference engine uses pgmpy.
- Frontend: TypeScript/React using React Flow for graph rendering and Chart.js for probability distribution visualization.
Functionality: Users interact with a dynamic graph. By selecting "evidence" (fixing a node's state, e.g., "Minimize time lost to obstacles"), the system performs real-time inference to update the probability distributions of all connected nodes, enabling "what-if" scenario analysis.

3. Key Contributions

Novel Tool (BayesInsights): The first interactive system to visualize causal dependencies between software delivery performance (DORA) and Developer Experience (DevEx) using Bayesian Networks in a large-scale industrial setting.
Hybrid Modeling Framework: A robust methodology for constructing BNs in SE that combines theoretical literature, domain expert knowledge, and data-driven structure learning, addressing the limitations of using any single source.
Industrial-Academic Collaboration: A successful case study of a partnership between Bloomberg and UCL, demonstrating how academic causal modeling techniques can be validated and deployed using real-world, large-scale engineering data.
Proof of Concept: Validation that causal modeling can effectively identify root causes and predict trade-offs in complex socio-technical systems where traditional dashboards fail.

4. Results & Evaluation

The tool underwent a mixed-methods evaluation involving quantitative performance benchmarking and qualitative user studies with 24 senior practitioners (Team Leads, Managers, Engineers).

Performance:
- Latency: Average response time of 24 ms for single inference requests.
- Scalability: Median response time under 40 ms with 50 concurrent users, confirming suitability for real-time exploration.
User Feedback (Qualitative):
- Usefulness: 95.8% of participants found the tool useful for identifying delivery challenges at team and organizational levels.
- Interpretability: 75% found the outputs easy to interpret; 83.3% understood how the model visualized metric changes.
- Adoption: 79.2% indicated they would use or recommend the tool.
- Immediate Impact: 25% of participants generated concrete improvement ideas (e.g., reducing distractions) within a 15-minute demo session.
- Trust: 70.9% expressed confidence in the tool's outputs for reasoning about delivery and DevEx.
Current Status: Deployed in early access to seven engineering teams at Bloomberg, with a roadmap for wider organizational rollout.

5. Significance

Bridging the Gap: BayesInsights bridges the gap between quantitative delivery metrics and qualitative developer experience, allowing leaders to see how human factors (e.g., focus, meaningful work) directly influence technical outcomes (e.g., throughput, stability).
Shift from Correlation to Causation: It moves software management from reactive dashboard monitoring to proactive causal reasoning, enabling teams to simulate the impact of interventions before implementation.
Scalable Causal Modeling: The paper demonstrates that with a hybrid approach (Expert + Data), complex causal models can be built and trusted even in noisy, survey-based environments, offering a blueprint for other large-scale software organizations.
Future Direction: The success of this project highlights the potential for "explainable AI" in DevOps, where causal models provide transparent reasoning for engineering decisions, moving beyond "black box" analytics.