💻 computer science

You may implement this later: Cofunctors as partial implementations

This paper proposes interpreting cofunctors (or retrofunctors) as partial implementations that defer specific backend choices, such as data representations and algorithms, until runtime based on system state.

Original authors: Vincent Wang-Maścianica

Published 2026-08-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Vincent Wang-Maścianica

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of software engineering, building a system often feels like assembling a complex machine where every gear must be chosen before the first bolt is tightened. Engineers frequently face a dilemma: they need to design the overall structure of a program, such as a database or a network service, but they cannot yet decide on the specific details, like which storage engine to use or how to handle data replication. Traditional methods for handling this uncertainty usually involve locking in a single set of choices for the entire system at the start, or waiting until the very end to fill in the blanks. This creates a rigid process where the path forward is fixed long before the destination is fully clear. The challenge lies in finding a way to build systems that can evolve, where decisions made early on can naturally shape the options available later, without forcing the programmer to commit to a final solution prematurely.

A researcher at the University of Oxford has proposed a new way to think about this problem, using a mathematical concept called a cofunctor to describe how software can be built in stages. The core idea is to treat a software system not as a finished product, but as a collection of obligations and choices that change as the system grows. Imagine a blueprint for a house that starts with a basic outline. As the architect adds a new room, the blueprint doesn't just get bigger; it also updates the list of materials needed. If the architect decides to add a second story, the blueprint might now require a stronger foundation, a choice that wasn't relevant when the house was just a single story. This new approach allows engineers to carry these evolving lists of requirements through the design process, ensuring that every new decision is compatible with the ones that came before it, while still leaving the final details open for later.

The paper argues that existing tools for managing software configurations are often too rigid. They typically require a global set of parameters to be defined at the very beginning, which means the system cannot easily adapt if a new requirement emerges mid-development. For instance, choosing to store data in a specific way might later force a decision about how to replicate that data across different servers, but standard methods struggle to link these two decisions dynamically. The author suggests that by viewing a software system as a "partial implementation," where the current state of the system dictates what choices are available next, we can create a more flexible engineering process. This is not just about delaying decisions; it is about structuring the system so that the act of making one decision naturally updates the menu of options for the next.

To demonstrate this, the author uses the example of a data storage system. Initially, the system might be defined simply as a place to keep data. At this stage, the engineer has not decided whether to use a local database, a remote service, or a specific file format. As the design progresses, the engineer might add a requirement for the data to be persistent, meaning it must survive power failures. This new requirement updates the system's state, introducing a new set of choices regarding durability. Later, if the engineer decides to replicate the data across multiple locations for safety, the system updates again. This second change might introduce a requirement for a transaction protocol, a detail that did not exist when the system was just a simple store. The beauty of this approach is that the system automatically checks for conflicts. If the engineer had chosen a simple file format that cannot handle transactions, the system would flag this as a conflict immediately when the replication requirement is added, rather than waiting until the code is written and failing later.

The researcher shows that this method allows for the creation of executable migration plans. Instead of just writing down a list of requirements, the system can generate a step-by-step plan for how to transform a basic store into a complex, replicated one. This plan can be built in stages, where each step is verified against the current state of the system. If a step is skipped or done out of order, the system can detect the error. For example, a plan that tries to replicate data before creating a durable store would be rejected because the necessary foundation does not exist yet. This ensures that the final system is built on a solid logical path, where every change is consistent with the history of previous changes.

One of the key findings is that this approach does not require the engineer to list every possible future state of the system in advance. In many traditional methods, one must define all possible configurations upfront, which can be overwhelming and often leads to a combinatorial explosion of options. Here, the system only tracks the obligations that are currently active. As new requirements are added, new choices appear, and as old requirements are satisfied, they disappear. This keeps the complexity manageable. The author notes that while the mathematical framework behind this idea is sophisticated, its practical application is straightforward: it simply provides a way to manage the flow of decisions in a way that respects the dependencies between them.

The paper also addresses why this idea has not been widely adopted in programming before. The term "cofunctor" has historically been confused with other concepts, leading to a lack of clarity about its specific utility. Furthermore, previous attempts to solve similar problems, such as those involving database updates or modular programming, often focused on different aspects, like keeping data consistent or merging code modules, rather than on the dynamic evolution of implementation choices. The author suggests that by reframing cofunctors as a tool for partial implementation, the concept becomes much more accessible and directly applicable to the daily work of software engineers.

Ultimately, the work offers a new perspective on how we build complex systems. It suggests that the best way to handle uncertainty is not to freeze the design in place or to leave it entirely open, but to create a structure where the design evolves naturally. By treating the software as a living document of choices and obligations, engineers can build systems that are robust, adaptable, and easier to reason about. The result is a method that allows for the assembly of intricate systems while leaving the final, concrete details for the moment when they are truly needed, ensuring that the path taken is always logical and consistent.

Technical Summary: Cofunctors as Partial Implementations

Problem Statement
In functional programming and software engineering, the standard model for implementation is the functor: a mapping from a specification category ( $S$ ) to a concrete implementation category ( $C$ ) where every specification object and morphism is immediately assigned a concrete instantiation. However, real-world software development often requires staged configuration, where implementation choices (e.g., data representations, algorithms, backend protocols) are deferred.

Current approaches, such as parameterized modules (ML functors), rely on a single global parameter type ( $P$ ) supplied upfront ( $S \times P \to C$ ). This model fails when implementation choices are state-dependent: a choice made at one stage (e.g., selecting a persistence backend) may introduce new obligations or constraints at a later stage (e.g., requiring a specific serialization format or transaction protocol). The challenge is to model systems where the set of available implementation choices evolves dynamically as the specification evolves, without requiring the enumeration of all possible final configurations upfront.

Methodology
The paper proposes modeling partial implementations using cofunctors (also known as retrofunctors). The methodology involves constructing a category of partial implementations ( $P$ ) based on a specification category ( $S$ ) and a functor of choices ( $Choice: S \to \mathbf{Set}$ ).

State-Dependent Choices: For every specification object $A$ , there is a set of choice states $Choice(A)$. These states represent partial configurations where some fields are fixed and others are open (obligations).
Transport of Choices: Specification morphisms $f: A \to B$ induce a "transport" function $f^*: Choice(A) \to Choice(B)$ . This function updates the implementation state by carrying forward existing choices and introducing new obligations or conflicts based on the specification change.
The Cofunctor Structure: A cofunctor is defined as a span $S \xleftarrow{p} \Lambda \xrightarrow{q} P$ $S p Λ q P$ where:
- $q$ is bijective on objects (identifying objects of $P$ with pairs $(A, c)$ where $A \in S$ and $c \in Choice(A)$ ).
- $p$ is a discrete opfibration, encoding the dependent menu of choices.
- The morphism map, impl, takes a specification change $f$ and a current choice $c$ to produce an implementation morphism. Crucially, the target of this morphism depends on the transported choice $f^*c$ .
Composition Laws: The implementation of a composite specification change $g \circ f$ is defined as implementing $f$ with choice $c$ , updating the state to $f^*c$ , and then implementing $g$ with the new state. This ensures that the order of operations respects the evolution of obligations.

The paper demonstrates this via an Idris implementation, utilizing dependent types to enforce that the implementation state (the choice) is correctly updated and tracked through every specification transition.

Key Contributions

Conceptual Reading: The primary contribution is the novel interpretation of cofunctors as partial implementations. While cofunctors (or retrofunctors) exist in category theory (e.g., in the context of delta lenses or internal categories), this paper explicitly frames them as a mechanism for managing deferred, state-dependent implementation choices.
Dynamic Obligation Management: Unlike static parameterization, this approach allows the "menu" of implementation choices to change dynamically. It supports scenarios where a later stage (e.g., replication) can introduce new requirements (e.g., transaction logs) that were not present in earlier stages (e.g., simple storage), and can detect conflicts (e.g., a non-transactional backend failing a replication requirement) without pre-defining all possible combinations.
Executable Migration Plans: The framework supports the construction of executable migration plans that compose implementation steps. It distinguishes between transports (changes driven by specification evolution) and refinements (filling in holes in the choice state without changing the specification).
Comparison to Alternatives: The paper argues that while tools like Backpack (for static mixin linking) or tagless-final encodings (for deferring interpreters) solve related problems, they do not natively support the state-indexed transformation of running systems where obligations are resolved incrementally and compositionally.

Results and Examples
The paper provides a concrete example of a storage system evolving from a generic Store to a PersistentStore and finally to a ReplicatedStore.

Scenario: Starting with a choice state containing a SQLite backend but an undefined serialiser and cache.
Process:
1. Persist: Transports the state, preserving the backend but recording a durability obligation.
2. Replicate: Transports the state again. If the backend is SQLite, it successfully introduces a protocol obligation. If the backend is a non-transactional FlatFile, the transport explicitly results in a Conflict state.
Outcome: The system generates a typed migration plan that composes these steps. The plan can be executed to produce a concrete system (e.g., SQLite/CBOR/LRU/Raft) or rejected if conflicts arise. The composition law ensures that a single-pass plan for "Persist and Replicate" is equivalent to the staged execution of "Persist" then "Replicate."

Significance and Claims
The paper modestly claims that the ingredients (cofunctors, staged configuration, dependent types) are not new. Its specific contribution is the conceptual reading that connects these categorical structures to the practical problem of partial implementation in functional programming.

The author asserts that the slogan "cofunctors are partial implementations" is a new, demystifying insight. This reading provides a formal basis for:

Managing composite partial implementations where choices depend on the current state of the specification.
Separating the environment of obligation management (the cofunctor) from the final semantics (the functor to the concrete category).
Handling dynamic migration and configuration in systems where the set of valid implementation choices is not static but evolves with the system's architecture.

The paper concludes that while cofunctors have been used in monoidal category theory and database update propagation (lenses), their application as a standalone tool for managing state-dependent implementation choices in functional programming has been previously overlooked, likely due to terminological confusion with contravariant functors and the dominance of database-theoretic perspectives in lens literature.

Technical Summary: Cofunctors as Partial Implementations

More like this