How Long-Lived Forks Break Open Source Collaboration
Open source is built around forks that briefly diverge and then merge back. Hyper-personalized software inverts that. Three problems get worse when forks stop being temporary.
I have been thinking a lot about forks in open source lately. Specifically, long-lived forks. Not short-lived forks used to implement a feature and upstream it, but forks that permanently copy a piece of software and maintain it forever.
What stands out to me is that I personally have never really forked something with the intention of maintaining the fork forever. Even though that is the promise of open source. I have read the MIT License more times than I can count, with its rights "to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software". And yet I have never seriously considered forking a project to make it my own.
It's just not worth it. Maintaining a fork is a serious endeavor. Pulling in updates to the upstream and resolving the merge conflicts from local changes is painful. It requires deeper knowledge of the original codebase and its intention than I am usually willing to acquire. Changing my process or learning to live with a "weird user interface" is usually a lot easier.
And we share this idea that open source is a collaborative effort. The whole model of modern open source is built around forks that briefly diverge and then converge again: fork to create a pull request.
Structural Friction
I believe this friction to forking has three structural reasons.
First, a lot of repositories aren't even set up to be forked. An extreme example here is the rust-lang/rust repository that I contributed to as a member of the Rust project. When you fork it, you will immediately get notifications that GitHub Actions on the default branch are failing. The reason is that the project's continuous integration pipeline is so big that it has been hard-coded for that specific organization. User accounts don't have access to the number of parallel runners needed for Rust's CI or to its caches in S3.
There is no malicious intent behind this setup or the friction to cleanly fork the project. It grew over time out of the needs that the project had, and maintainers learned to live with it by working in the rust-lang/rust repository directly.
Other large repositories on GitHub have similar issues. The expectation is that contributors fork the repository, push a branch to their fork, and then open a pull request to run CI in the original repository. The fork is just a temporary means for collaboration.
Which brings me to the second point: the friction comes from the platforms we are using. GitHub has made the pull request flow the default way to contribute to other projects, and everything in its user interface is designed to make that as smooth as possible. Maintaining a long-lived fork, on the other hand, is surprisingly difficult. As demonstrated by the fact that GitHub added the "Sync fork" button to the user interface only in 2021, 13 years after launch.
And lastly, forking only copies the code and leaves a lot of the systems around it behind. This matters less for small projects, but as soon as projects grow they tend to be supported by a wide range of external tools and integrations. It is up to the person forking the code to figure out what secrets, apps, servers, and other external systems are necessary to get things fully working.
None of this is broken. These things make for a world in which forks are only short-lived means to contribute to an upstream repository.
Hyper-Personalized Software
I believe there is a strong argument to be made that this premise of a single upstream repository and many short-lived forks might change due to the rise of AI. As the costs of software modification fall, it becomes cheaper to prompt an agent to change your personal copy of some software than to contribute a patch upstream. I wrote about this here:
Jan David Nose
In this world, there is suddenly a strong incentive to maintain a long-lived fork: It can be customized cheaply with an AI agent to adapt the software to the user's specific needs. Instead of the compromises that are necessary to develop software together, users can make it their own and add whatever features they want.
If long-lived forks become the norm, I expect three problems to get worse.
The Update Problem
The first problem goes back to the friction around keeping a fork up-to-date. This is already annoying for forks that don't diverge from upstream that much. But once there are substantial changes between the upstream and the fork, a lot of the automation starts failing and updates have to be applied manually. That's when it usually becomes too costly to maintain the fork.
Linux distributions are an interesting example of the complexity of this problem. They maintain patch stacks against thousands of upstream repositories, and have people, tooling, and processes dedicated to patching and packaging open source software.
That is what it takes right now to maintain a long-lived fork.
The average user doesn't have access to any of that. They do have access to AI agents, and there is the possibility that agents will be able to handle this work in the future. But then it will still be necessary to teach the agents which parts of a fork diverge on purpose. And how to evolve those changes as the underlying foundations change.
The Sharing Problem
The second problem is about sharing features across forks.
Right now, everything operates under the assumption that there is one single source of truth: the upstream. It is the one codebase to which new features get added, and users can pull them down into their personal forks. There is usually no reason to pull in changes from someone else's fork.
With hyper-personalized software and long-lived forks, this is changing. Other users will independently add new features to their version of the software, but we have no tools to discover them. And even if we had, pulling in specific features from other forks is not supported in any tool today.
The reason for this is simple: Resolving merge conflicts between an upstream and a fork is difficult enough as is. Resolving merge conflicts between multiple forks that all diverged from the shared upstream at different times is probably impossible. Tools will need a deeper understanding of the changes and their motivation so that they can figure out what to keep and what to change.
Which brings me back to this question that I've been pondering: What will we collaborate on in the future?
The Social Problem
Today, almost all open source projects have one definitive upstream version. Everyone understands that the rust-lang/rust repository is the authoritative source for Rust releases, no matter how many forks exist.
If we move to a world where there are many long-lived forks of a particular piece of software, it becomes less clear what the authoritative version is. And eventually, there might not even be one anymore. Especially if we figure out ways to share features freely across the network, the need for and value of a single source of truth diminishes sharply.
But that introduces a new problem: Large open source projects are only sustainable when they have healthy communities. And right now, these communities organize around a single repository to which everyone contributes. If communities are spread across a network of repositories and not a single one anymore, how are they going to coordinate, communicate, and collaborate?
Conclusion
Today's model for collaboration in open source is built around one specific workflow: fork to open a pull request. But it does not support either of the two workflows that I believe will become more important in the future: maintaining a personal long-lived fork, and pulling from multiple forks in a network.
Some of the problems we will be able to address with better tools and AI. But the fundamental problem that our current tooling is built for a different paradigm will remain. A lot of things are happening with source control and forges right now, and I am curious if anyone is building for this new paradigm yet.