Open-Source vs. Closed AI Models

Governance Implications

This article is part of my series on AI safety and governance. If you haven't read the pillar article, I'd start there for the broader context.

Why I Don't Think There's a Clean Answer Here

I said in the pillar article that this is a debate I find myself returning to often, and I want to be upfront about why: I think both sides of this debate are arguing from real values, not just from self-interest, and I think those values genuinely point in different directions. This isn't a case where I think one side is obviously right and the other is making excuses. I think the governance implications of open versus closed AI models cut in both directions, sometimes within the same sentence.

What I Mean by Open and Closed

I want to define these terms carefully, because I think the line between them is blurrier than it's sometimes presented.

A fully closed model is one where the underlying weights, the actual trained parameters that define the model's behavior, aren't released. Access happens through an API or a product, controlled by the company that built it. An open-weight model is one where those parameters are released publicly, allowing anyone to download, run, modify, and build on the model themselves, often under licenses that vary in how permissive they are.

I think it's worth noting that "open source" in the traditional software sense, where the training code, training data, and methodology are also fully open, is rarer than "open-weight," where the trained model itself is released but the process that produced it may not be fully disclosed. I think this distinction matters for governance because a lot of the transparency benefits people associate with openness depend on which of these you're actually talking about.

The Case for Openness

I think the strongest argument for open models is about distributing power and enabling scrutiny. If a small number of companies control access to the most capable AI systems, I think that concentrates an enormous amount of influence over how those systems get used, what they're allowed to do, and who benefits from them. Open models, by contrast, let a much wider range of people, researchers, smaller companies, governments, individuals, actually examine, modify, and build on these systems directly.

I've seen this framed explicitly in terms of what's sometimes called AI sovereignty: the idea that as AI becomes critical infrastructure, organizations and even nations have a stake in not being dependent on a small number of external providers for capabilities that matter to them. I think there's something to this. A government agency or a defense contractor operating under strict data handling requirements may simply not be able to use a closed model that requires sending data to an external API, regardless of how good that model is, whereas an open-weight model that can be run entirely on infrastructure they control sidesteps that problem entirely.

I also think there's a real research and safety argument for openness. The interpretability work I discussed in my article on explainability and transparency often depends on being able to examine a model's internals directly, and I think open models make this kind of research accessible to a much broader community than would otherwise have the access to do it. I've seen open development described as fostering more rapid, collaborative improvement, and I think there's something to the idea that more eyes on a system, including more eyes specifically looking for problems, can surface issues that a closed development process might not.

The Case for Keeping Models Closed

But I think the governance case for keeping the most capable models closed is also genuinely substantive, and I don't think it's just about protecting commercial advantage, even though that's obviously part of it for the companies involved.

The core argument, as I understand it, is about controllability after release. Once a model's weights are public, the organization that built it loses essentially all ability to update its safeguards, restrict its use, or respond to misuse. If a closed model is found to have a serious safety issue after deployment, the company can patch it, restrict access, or roll it back. If an open-weight model with the same issue has already been downloaded by millions of people and is running on infrastructure the original developer has no visibility into, I don't think there's a practical way to undo that.

I think this matters most for the kinds of capability risks I discussed in my article on risk assessment. The capability evaluations and red-teaming I described in that article and in my piece on red-teaming are largely premised on the idea that if something concerning is found, there's a chance to address it before, or even after, deployment. With an open-weight model, I think that window effectively closes the moment the weights are released, which raises the stakes of getting pre-release evaluation right in a way that I think doesn't apply in quite the same way to closed models.

I also think there's a meaningful difference in who bears the consequences of misuse. With a closed model, the company that built it retains some ongoing relationship with how it's used, and arguably some responsibility for it. With an open-weight model, that relationship effectively ends at release, and I think this creates a real question about accountability that I don't think gets resolved just by saying the model is "more transparent."

Why I Think the Performance Gap Matters for This Debate

I think one thing that's changed the shape of this debate, even just in the time I've been following it, is that the performance gap between open and closed models has narrowed considerably. For a while, I think there was a fairly clear hierarchy, with the most capable models being closed and open alternatives lagging meaningfully behind. That gap appears to have narrowed substantially, with open-weight models now matching or exceeding proprietary alternatives for a lot of practical use cases, even if the absolute frontier, the most capable systems overall, still tends to be closed.

I think this matters for governance because a lot of the safety arguments around openness implicitly assumed that the most concerning capabilities would remain primarily in closed systems, where pre-release evaluation could catch problems before they reached a wide audience. If highly capable systems are increasingly available as open weights too, I think that assumption becomes harder to rely on, and I think it raises the stakes of the controllability argument I described above.

I've also seen this framed in terms of global influence: the open-source layer of the AI ecosystem is where a huge amount of real-world adoption happens, and where leadership can shift quickly between different actors and countries, sometimes within a single generation of models. I think this is a dimension that a lot of governance frameworks, which tend to focus on frontier training runs by a handful of major developers, may not be well set up to address, since the open ecosystem operates by genuinely different dynamics.

The Hybrid Reality I Think Most Organizations Are Actually Living In

I want to push back gently on framing this as a binary choice, because I think that's not how most organizations actually experience it in practice. I've seen a pattern where organizations route different kinds of requests to different kinds of models: open, smaller models handling routine, high-volume tasks, with more capable closed models reserved for the harder cases that need top-tier reasoning. I think this kind of layered approach reflects something true about how these systems actually get used, which is that "open vs. closed" is often less a single strategic choice and more a set of trade-offs made differently for different parts of a system.

I think this matters for governance because it means the relevant question often isn't "should this organization use open or closed models," but something more granular: for this specific use case, with this specific risk profile, which set of trade-offs makes sense? A use case involving sensitive data that can't leave an organization's own infrastructure might point toward open models regardless of the broader debate. A use case requiring the most advanced reasoning available, where the organization is comfortable with an external API, might point toward closed models. I think most real deployments end up being a mix, which I'd argue reflects an underlying truth: the governance considerations I've described above don't all point the same direction even within a single organization.

Where I Land on This

I think the honest summary is that openness and closedness each solve real problems and create real risks, and I don't think those risks are symmetric in a way that makes one approach categorically safer than the other. Openness distributes power, enables scrutiny, and supports sovereignty, but it also means losing the ability to respond after release, at exactly the moment when a serious issue might be discovered. Closedness preserves the ability to patch, restrict, and respond, but concentrates power and access in ways that I think raise their own legitimate governance concerns.

I also think this is a debate where the ground keeps shifting under the people having it, as the performance gap between open and closed systems changes and as the open ecosystem becomes a more significant part of the overall picture, including in ways that involve actors outside the handful of countries and companies that current governance frameworks tend to focus on. I don't think there's a stable answer to this question waiting to be discovered. I think it's more likely that the right balance, if there is one, will keep being renegotiated as the technology and the landscape around it continue to change, and I'd be skeptical of anyone, on either side, who presents this as a settled question.