Ethics in Online Software Experimentation: A Body of Knowledge and the Cactus Model

In the digital world, companies constantly tweak their software to make it better, faster, or more engaging. They might change the color of a button, rearrange a menu, or test a new way to show prices. To see if these changes work, they often run what are called online experiments. In these tests, real people use the software without always knowing they are part of a study. One common method is to show one group of users version A of a feature and another group version B, then compare how they behave. This approach helps businesses learn what people want, but it also places ordinary users into a situation where their choices and data are being manipulated to test a hypothesis. While this practice drives innovation, it raises a quiet but serious question: just because a company can run these tests, should they? And if they do, how can they ensure they are not hurting the people who make the software possible?

Two researchers from Brazil set out to answer these questions by looking at the ethical side of this digital testing. They noticed that while many companies run these experiments, the rules about treating people fairly are often vague, scattered, or ignored entirely. The researchers wanted to build a clear guide that software teams could actually use. They did not just list abstract ideas about right and wrong; instead, they gathered specific problems that happen during these tests and matched them with practical ways to fix them. Their goal was to create a model (the Cactus Model) that helps experts think through the moral consequences of their experiments before they ever launch a single test.

To build this guide, the researchers looked at a wide range of existing information. They started by searching for studies specifically about ethics in continuous and online experimentation, but they found very little written on the topic. Realizing that the problem was too big to solve with just those few papers, they broadened their search. They looked at ethical rules from related fields like artificial intelligence, general computer science, and even medical research. They read codes of conduct, technical guidelines, and academic papers to find common themes. From this massive collection of sources, they identified sixteen key ethical principles, such as honesty, fairness, and respect for a person's ability to choose for themselves. They then dug deeper to find forty-eight specific ways these principles get violated in the real world. For example, a violation might be hiding the true purpose of a test from a user, or designing a system that tricks people into making decisions that are bad for them.

Once they had listed the problems, the researchers found fifty-three ways to solve them. They did not just leave these as a random list; they connected them all together. They showed which specific solution fixes which specific problem, and how that solution supports the underlying ethical principle. This created a structured map of knowledge. It is a tool that says, "If you see this problem, here is the specific action you should take, and here is the value it protects." This map is the foundation for their main contribution, a new way of thinking about these tests called the Cactus Model.

The Cactus Model is designed to help the people in charge of these experiments make better decisions. The name comes from the idea of a cactus, which is a tough plant that survives in harsh conditions by having a protective outer layer. In the same way, the model provides a protective structure for ethical decision-making in the fast-paced, high-pressure world of software development. It does not tell a team exactly what to do, but it gives them a step-by-step way to think through the consequences of their choices. The model is built on four layers that work together.

The first layer is about looking closely at the experiment to find potential problems. The team asks, "What could go wrong?" They use the map of ethical problems to check their design. They might ask if the test is transparent, if it treats all users fairly, or if it respects the user's freedom to say no. They rate how likely a problem is to happen and how many people it might affect. This step turns vague worries into specific, identifiable issues.

The second layer is about weighing the consequences. Here, the team looks at the problems they found and asks, "Why might we want to ignore them?" Often, there is a business reason, like saving time or making more money, that tempts a team to skip ethical safeguards. The model forces them to write down that temptation and then compare it directly against the harm it could cause. They look at the impact on the users' trust, their privacy, and their well-being. This step ensures that the decision to proceed is not just a gut feeling, but a reasoned choice where the trade-offs are clear.

The third layer is about taking action. Once the team has weighed the risks and the benefits, they decide what to do. They can choose to run the experiment as planned, change the design to fix the problems, or cancel it entirely. If they decide to proceed, the model guides them to pick specific solutions from their map to protect the users. For instance, if they found a risk of unfair treatment, they might add a rule to ensure all user groups are treated equally. This step turns their ethical reasoning into concrete changes in the software or the testing process.

The final layer is about learning from the experience. After the experiment is over, the team looks back at what happened. They review the choices they made and see if their predictions were right. They write down what they learned so that the next time they run a test, they can do it better. This step is crucial because it turns a single experiment into a lesson for the whole organization, helping them avoid the same mistakes in the future.

The researchers tested this model with a real-world example involving a delivery app that wanted to test different pricing strategies. They showed how a team could use the model to spot that changing prices for different users might feel unfair or confusing. By following the layers, the team realized they needed to be more honest about how prices were set and ensure that vulnerable users were not being targeted unfairly. They then used the specific solutions from their map to adjust their plan, adding clear explanations for users and setting limits on who could be part of the test. This example showed that the model works not just for single tests, but for families of tests that happen at the same time, allowing teams to apply the same ethical standards across many different projects without starting from scratch every time.

The study concludes that while the tools to run these experiments are powerful, the tools to guide them ethically have been missing. The researchers have provided a body of knowledge and a thinking model to fill that gap. They emphasize that this is not a final solution that solves every ethical dilemma forever. Instead, it is a starting point that makes the invisible visible. It gives software teams a way to talk about ethics in the same language they use for code and data. By making the ethical reasoning process clear and structured, the model helps ensure that the drive for innovation does not come at the cost of the people who use the technology. The work suggests that with the right framework, companies can continue to learn from their users while still treating them with the dignity and respect they deserve.