AI Safety and Governance: A Complete Guide
When people ask me what I actually mean by "AI safety and governance," I usually start by separating the two halves. Safety is the technical question: how do we build AI systems that behave the way we intend, even as they become more capable? Governance is the human question: who decides what "safe enough" means, who enforces it, and how do we hold organizations accountable when something goes wrong? In my experience, most confusion in this space comes from collapsing these two ideas into one conversation when they really need to be tackled separately, even though they're deeply connected.
I wrote this guide as the entry point to a broader series I'm building on this topic. Below, I'll walk through why this subject matters right now, the key concepts you need to understand, the major frameworks shaping the field, and where I think things are heading. Throughout, I'll point you toward the more focused articles I've written on specific pieces of this puzzle.
What's in this guide:
Core concepts (the "what" and "why")
The global landscape (the "who" and "what rules")
Practice (the "how")
- AI Risk Assessment
- Red Teaming for AI Safety
- AI Incident Reporting
- Responsible AI Development Best Practices
Operationalizing it (putting it together)
Why I Think This Matters Right Now
I've watched AI capabilities accelerate faster than most institutions can adapt to them. Models that seemed impressive a few years ago now look primitive next to what's available today, and the gap between "what AI can do" and "what rules exist for AI" keeps widening.
This isn't abstract for me. I think about it in three buckets:
Capability risk. As systems become more capable, the consequences of getting things wrong scale with them. A flawed recommendation algorithm is one thing; a flawed system making decisions about healthcare, hiring, or critical infrastructure is another.
Trust risk. I've noticed that public trust in AI systems is fragile and easily damaged by high-profile failures, even when those failures are rare. Without credible safety and governance practices, I think adoption stalls or happens in ways that aren't accountable.
Coordination risk. Different countries, companies, and research labs are moving at different speeds with different rules. I worry that without some baseline coordination, we end up with a patchwork that's either too weak to matter or so fragmented that compliance becomes its own industry.
The Core Concepts I Keep Coming Back To
Before diving into frameworks and policy, I want to define a few terms I use throughout this series, because I've found that people often use them interchangeably when they shouldn't.
Alignment is about whether an AI system's goals and behaviors match what its developers and users actually intend. I go deeper on this in my piece on AI alignment, but the short version is that alignment is fundamentally a technical challenge with philosophical roots.
Robustness is about whether a system behaves reliably across situations it wasn't explicitly trained for, including adversarial conditions. This is where I find a lot of the technical safety work lives.
Transparency and explainability are about whether humans can understand why a system produced a given output. I've written a dedicated article on this because I think it's one of the most misunderstood parts of the field.
Governance is the broader umbrella: the laws, regulations, organizational policies, and institutional structures that determine how AI is developed, deployed, and monitored. This is where I spend most of my time in this guide.
How I See the Global Regulatory Landscape
One of the things that strikes me most when I look at AI regulation is how differently major jurisdictions are approaching it.
The European Union has taken what I'd describe as the most comprehensive regulatory approach, with a risk-tiered framework that imposes different obligations depending on how an AI system is classified. The United States has historically favored a lighter-touch, sector-specific approach, though I've seen this shift through executive actions and agency guidance. China has developed its own regulatory apparatus, particularly around generative AI and algorithmic recommendation systems, with requirements that reflect different priorities than Western frameworks.
I cover this comparison in much more depth in my guide to global AI governance frameworks, but the takeaway I want to leave you with here is that there's no single "AI law" you can point to. Instead, there's an evolving patchwork, and I think anyone building or deploying AI systems needs to understand which pieces apply to them.
Who I Think Is Actually Doing the Regulating
When I started researching this space, I assumed there'd be one or two obvious regulatory bodies to watch. Instead, I found a sprawling set of institutions, including national regulators, standards bodies, sector-specific agencies, and international coordination efforts.
I've put together a separate guide to the agencies and bodies involved in AI oversight, because I think this is one of the areas where people most need a map. Understanding who has jurisdiction over what is often the first step toward understanding what compliance actually requires.
How Companies Actually Evaluate Risk
I've spent time looking at how organizations assess whether an AI system is safe enough to deploy, and I've found that risk assessment in this field looks different from traditional software risk assessment. It's not just about bugs or security vulnerabilities. It's about evaluating capabilities the developers themselves may not fully understand yet.
In my piece on AI risk assessment, I walk through the methodologies I've seen used, including capability evaluations, red-teaming exercises, and ongoing monitoring after deployment. I think this is one of the most practically useful topics in the whole series if you're working inside an organization trying to build responsible practices.
Why I Think Explainability Deserves Its Own Conversation
I'll be honest, this is one of my favorite topics in the series because I think it's where the technical and governance sides of AI safety meet most directly. A system can be technically robust and still fail the trust test if nobody can explain why it does what it does.
My article on explainability and transparency goes into the different techniques used to make AI systems more interpretable, and why I think this matters as much for regulators and end users as it does for the engineers building these systems.
Bias, Fairness, and the Work I Think Gets Overlooked
I've found that bias and fairness in AI systems often get treated as a checkbox exercise, when in reality I think it's some of the hardest and most ongoing work in the field. Bias can creep in through training data, through the way a problem is framed, or through how a system's outputs get used downstream.
I dedicated a full article to bias and fairness because I think this deserves more than a passing mention, and because I've seen organizations get this wrong in ways that cause real harm.
What Responsible Development Looks Like to Me
I get asked a lot about what "responsible AI development" actually means in practice, beyond the buzzword. In my piece on responsible AI best practices, I lay out the practices I think matter most: documentation standards, internal review processes, staged rollouts, and feedback loops that actually get acted on.
Why I Think Red Teaming Deserves More Attention
Red teaming, the practice of deliberately trying to break or misuse a system before it's deployed, is something I think is underappreciated outside of security circles. I've written a dedicated piece on red teaming for AI safety because I think it's one of the most concrete, actionable practices an organization can adopt, and because I've seen it catch problems that no amount of internal review would have found.
Incident Reporting: The Part Nobody Wants to Talk About
I think incident reporting is the unglamorous cousin of AI safety, but I'd argue it's one of the most important pieces. When something goes wrong with an AI system, what happens next? Is it documented? Is it shared with regulators or the public? Does the organization learn from it?
My article on AI incident reporting covers the systems I've seen emerging to track and respond to AI failures, and why I think this is an area that needs much more investment.
Open Source vs. Closed Models: My Take on the Governance Trade-offs
This is a debate I find myself returning to often. Open-source AI models offer transparency and broad access, which I think has real value for research and accountability. But they also raise governance questions that closed models don't face in the same way, particularly around misuse.
I explore this tension in my piece comparing open-source and closed AI models, and I'll admit upfront that I don't think there's a clean answer here. I think the governance implications cut in both directions.
The Research Organizations I Pay Attention To
Throughout my research, I kept running into the same handful of organizations doing foundational work on AI safety. I've compiled a guide to the key research organizations I follow, along with what I think each one contributes to the broader conversation.
Building Internal Governance: My Practical Guide
For organizations that want to get serious about this but aren't sure where to start, I wrote a step-by-step guide to building an AI ethics board. This is the most operational piece in the series, and I tried to write it as something you could actually use as a starting template.
Where I Think This Is All Heading
I'll close with my honest take: I think AI safety and governance is going to remain a moving target for the foreseeable future. The technology is evolving faster than most regulatory processes can keep pace with, and I don't think that's going to change soon.
What gives me some optimism is that I've seen a growing recognition, across companies, governments, and research institutions, that this work matters. The frameworks I discuss in this guide are imperfect and incomplete, but they represent a starting point that didn't exist a few years ago.
My hope with this series is to give you a grounded, practical understanding of where things stand today, so that whether you're a policymaker, a builder, or just someone trying to make sense of the headlines, you have a map to work from. I'll keep updating these articles as the landscape shifts, because I think it will keep shifting for a long time.