The default answer is wrong
In traditional software, the default advice is clear: buy commodity infrastructure, build what differentiates your product. In AI, this heuristic breaks down because the boundaries between commodity and differentiation are shifting faster than any team can track.
A vector database that was cutting-edge a year ago is now a commodity. An evaluation framework you built in-house might be superseded by an open-source tool that's better maintained. The model provider you chose might change their pricing, deprecate the model you depend on, or ship a competitor to your product.
The build vs. buy decision in AI isn't a one-time choice — it's a continuous portfolio management problem.
The four quadrants
We find it useful to evaluate each component of an AI system on two axes:
- Differentiation: Does this component directly contribute to what makes your product unique?
- Rate of change: How quickly is the best available option in this space improving?
This creates four quadrants:
High differentiation, low rate of change → Build and own. This is your core IP. The algorithms, prompts, and data pipelines that encode your unique value. No vendor will build this for you.
High differentiation, high rate of change → Build with escape hatches. You need control over this, but the landscape is evolving fast. Build abstractions that let you swap underlying components without rewriting your application.
Low differentiation, low rate of change → Buy and forget. Logging, monitoring, basic infrastructure. Don't reinvent wheels that are perfectly round.
Low differentiation, high rate of change → Buy but stay portable. This is the trap zone. The tooling is improving rapidly, so building locks you into today's approach. But the vendor landscape is also volatile, so you need the ability to switch.
What to almost always buy
Some AI components have mature vendor options that are almost never worth building in-house:
- Foundation model inference — Unless you're at the scale of a major tech company, self-hosting frontier models is a distraction. Use APIs.
- Basic observability — Logging, tracing, cost tracking. Multiple vendors do this well.
- Vector storage — The managed options (Pinecone, Weaviate Cloud, etc.) have reached a maturity level where self-hosting Postgres with pgvector is only worth it if you have strong operational reasons.
What to almost always build
- Evaluation pipelines — Your quality criteria are specific to your domain, your users, and your product goals. No off-the-shelf evaluation tool fully captures this.
- Prompt management — The specific prompts, the logic for when to use which prompt, and the testing around prompt changes are core product logic.
- Domain-specific data processing — How you chunk documents, extract entities, or transform data for your specific use case is typically where your differentiation lives.
The hidden cost of buying: integration tax
Every vendor tool you add to your stack introduces integration cost: learning the API, handling the edge cases, managing authentication, and dealing with outages and deprecations. For AI tools specifically, the integration tax is often higher than traditional SaaS because:
- The APIs are less stable (the field is young)
- The failure modes are less predictable (LLMs are stochastic)
- The evaluation criteria are less standardized (what does "good" mean?)
A stack of 8 AI vendors creates more operational complexity than most small teams can manage. Be selective.
The hidden cost of building: opportunity cost
Every component you build in-house is engineering time not spent on your core product. In a fast-moving market, the opportunity cost of building commodity infrastructure is measured in features you didn't ship and customers you didn't win.
The teams that navigate this best maintain a strict distinction between "invest" (build things that compound) and "consume" (use the best available tool and move on).
A decision checklist
For each AI component in your stack, ask:
- Does this directly encode our unique product value?
- Do we have the team to build and maintain it?
- Is there a vendor option that's 80%+ of what we'd build?
- How often will we need to change this component?
- What's the switching cost if we need to change vendors?
If the answer to #1 is no and #3 is yes, buy. If the answer to #1 is yes and #2 is yes, build. Everything else requires judgment — and that's where engineering leadership earns its keep.