Conway's Law in the Age of AI Teams

Conway's Law Restated for AI

The original insight is simple: the structure of a software system reflects the social structure of the organization that built it. Teams that don't talk to each other produce modules that don't integrate well. Teams under one manager produce tightly-coupled systems.

For AI, this has specific implications:

If your ML team is centralized and separate from product teams, your AI features will have a "separate module" feel. Users will experience them as add-ons rather than native capabilities. The recommendation engine will feel disconnected from the search experience. The AI assistant will feel separate from the workflow it's supposed to assist. Because organizationally, it is separate.

If data engineering reports to a different VP than ML engineering, your training pipelines will have interface friction. The data team will produce datasets in formats that require transformation before ML can use them. Schema changes will break training pipelines because nobody coordinated. Data freshness SLAs won't match model training cadences because the two teams have different priorities.

If prompt engineers sit in platform rather than product, prompts will be optimized for technical correctness rather than user experience. The prompts will be well-structured but won't reflect how actual users phrase requests, what context they have, or what outcomes they need. Good engineering, wrong product.

The Inverse Conway Maneuver for AI

The Inverse Conway Maneuver says: instead of letting your org structure accidentally determine your architecture, deliberately design your org structure to produce the architecture you want.

For AI teams, this means answering a design question first: what AI architecture do you want? Then structuring teams to produce it.

If You Want Native AI Features per Product

Put applied AI engineers (prompt engineers, applied ML, AI product engineers) inside product teams. They attend product standups. They talk to users. They iterate at product speed. They have context on user problems that a centralized team never will.

The tradeoff: some duplication across teams. Two product teams might solve similar prompt engineering problems independently. Accept this. The alternative (centralized coordination) is slower and produces generic solutions that don't fit either product perfectly.

If You Want a Clean Platform Layer

Keep ML infrastructure separate. Model serving, training pipelines, evaluation frameworks, model registry, inference optimization. This team produces APIs that product teams consume. They optimize for reliability, consistency, and cost efficiency.

The boundary is: product teams own what the AI does (features, prompts, user experience). Platform owns how the AI runs (serving, scaling, monitoring, cost). Clean separation, clear ownership, minimal coordination cost.

If You Want Both

Most orgs of 50+ engineers need both. The pattern: a small ML platform team (4-8 people) owns shared infrastructure. Applied AI engineers embed in product teams (1-2 per product). The platform team has internal SLAs to product teams. Product teams have autonomy over feature decisions.

This produces two things Conway's Law predicts: a clean platform API (because the platform team is separate and has consistent interfaces) and native-feeling AI features (because applied AI is part of the product team's daily context).

Examples: Org Structure to Architecture

Prompt Engineers: Product or Platform?

This is the highest-leverage Conway's Law decision for most AI-adopting orgs in 2026. Where you put prompt engineers determines whether your AI features serve users or serve infrastructure.

Prompt engineers in product teams produce prompts shaped by user research, product context, and rapid iteration. They know what users actually ask, what errors frustrate them, and what "good" looks like for that specific product. They iterate daily because they're part of the sprint cycle. The prompts they write feel like the product talking to the user.

Prompt engineers in platform produce prompts optimized for consistency, reuse, and technical correctness. They build prompt libraries. They standardize patterns. They optimize token cost. But they lack product context. Their prompts work correctly but don't feel right for any specific user journey.

My recommendation: prompt engineers in product teams, with a shared prompt engineering guild (a community of practice, not a reporting line) that shares patterns and prevents wheel-reinvention. The guild coordinates without owning. Product teams execute with full context.

The Reporting Line Test

A quick diagnostic for Conway's Law problems in your AI org:

Question 1: Who does your data engineering team report to? If it's a different VP than ML, predict friction at the data handoff point. You'll find schema mismatches, freshness SLA gaps, and "it works on my data" problems.

Question 2: Do your AI engineers attend product standups? If no, predict AI features that don't match user needs. The information about what users want lives in product conversations. AI engineers who don't hear those conversations build for technical elegance, not user value.

Question 3: Who decides what AI features to build: the ML team or the product team? If the ML team decides, predict solutions looking for problems. If the product team decides without ML input, predict technically infeasible commitments. The right answer: product defines the problem, AI engineers propose the solution, together they scope what's buildable.

Question 4: How many teams must coordinate to ship an AI feature? If the answer is three or more (data + ML + product + infra), predict 3-6 month timelines for features that should take 3-6 weeks. Every team boundary adds a coordination tax. Minimize boundaries for your most common workflow.

Making the Change

Restructuring teams is expensive and disruptive. Don't do it lightly. But if your AI features consistently feel disconnected from the product, ship slowly, or have quality problems at the seams, the root cause is likely organizational, not technical. No amount of better code will fix an architecture problem caused by org structure.

The lowest-risk move: embed one AI engineer in your highest-priority product team for a quarter. Don't restructure. Don't announce. Just move one person. If the AI features that product team ships feel qualitatively different (more native, faster iteration, better user fit), you have evidence that the org structure was the bottleneck. Then restructure deliberately based on what worked.