Custom AI agent development: when to build and when to buy

Before anyone writes code for an AI agent, there's a cheaper question to ask: does a product already do this? For many workflows the answer is yes, or close enough. For a minority it's no, and those are the cases where custom AI agent development earns its cost.

We build custom AI agents for business at Gezora, so read what follows knowing that we have an interest. Our advice when a product fits is to buy it, so most of what follows is about checking the fit.

Check whether you can buy it first

Write the workflow down in one paragraph, with the systems it touches, then look for a product that covers it. Be strict about what "covers it" means. Say a product handles 80 percent of the steps. Your team still does the other 20 percent by hand, and that's often the part that takes the most skill. Ask a vendor to run your ugliest real case through a trial, and skip their sample data.

If a product fits, buy it: you get maintenance, updates and a support line, and you avoid the cost of owning a build. Custom work starts to make sense when at least one of the following is true.

When custom is the right call

• No product covers the workflow, usually because it is specific to your industry or your way of working.
• The process is part of what makes your business different, so a generic tool would flatten it.
• Data cannot leave your environment, and the products that cover the workflow are hosted somewhere else.
• The agent has to work deep inside systems that products do not integrate with, such as an in house platform or an unusual ERP.
• You need particular guardrails: approvals above a certain amount, a record of every decision, actions that are blocked outright.
• Volume is high enough that the per run cost of a general product exceeds the cost of owning something tuned to the job.

If none of these is true, buy the product.

Build, buy or blend

Most real choices fall into one of three patterns, and each fits a different situation.

Buy a product and build the last mile. The product handles the common parts of the workflow, and a thin layer of custom work covers what's specific to you: an integration into an internal system, a rule set unique to your business, a report in the format your auditors want. This is often the cheapest route when a product covers most of the workflow. The risk is that the custom layer depends on the product's API staying stable.

Build the core and buy the components. You build the agent's logic, guardrails and integrations, and buy the parts that are commodities, such as text extraction from scans, speech to text, or a hosted language model. This suits workflows where the decisions are where the value sits. The risk is a growing list of vendors to manage.

Build the whole thing. That's justified when data can't leave your environment, when volume makes per case costs matter, or when the process is the business. It's also the most to own and maintain, so go in with the first agent scoped narrowly and a named person who will keep it running.

What custom AI agent development involves

A build has a consistent shape, whoever does it. It starts with workflow discovery, a session with the person who owns the process to map each step and each system it touches. Decision mapping comes next. For each decision, record who makes it now, what information they use, and what a wrong call costs. That list settles what the agent may do alone, what needs a person's approval, and what's off limits.

Then comes the build itself: prompts and logic written for your process, and authenticated connections into your systems. The guardrails go in at the same time. Approval steps and audit logging belong in the first version; adding them later is possible but gets done badly, because by then people depend on behavior that the guardrails would have prevented.

Rollout happens in stages: prototype, pilot, production. The prototype answers whether this works on your data at all. The pilot runs on real work with a person watching closely. Production comes only after the pilot has held up. A demo that skips the pilot meets its first messy case in production.

At the end, the builder should hand over the prompts, the evaluations (the test cases the agent is scored against), a runbook for operators, and training for the people who will run it. If the agent needs a larger product built around it, such as an internal tool or a customer facing feature, that is custom AI software development, which starts the same way, with discovery.

Who needs to be in the room

A build goes better when five kinds of people are involved early. The process owner explains how the work really runs and signs off on the boundaries. Someone from IT or security approves access and decides where the agent runs. A few of the people who do the work today review outputs and tell you when the agent is wrong. Finance, or whoever owns the budget, cares about cost per case and needs a number they can defend. And if the data is sensitive, someone from legal or compliance should see the data flow before the prototype, while changes are still cheap.

Each of them has to have said yes to the part that's theirs, though attending every meeting is optional for most.

What you need before a build starts

You need four things: a workflow you can describe, access to the systems it touches, a process owner who can answer questions, and representative data. Access is the one that usually stalls projects. A missing API key or an unapproved service account can hold up custom AI agent development for weeks, so start that conversation with your IT and security people in week one.

The process owner is easy to underrate. A build with no owner gets contradictory answers from five people, each of whom is right about their own corner.

Scope the first agent narrowly

Don't scope the first agent as a platform. Compare "an agent for finance" with "an agent that matches supplier invoices to purchase orders for one legal entity and flags the mismatches". Only the second can be finished and measured. A narrow first agent gets you a working result and a real cost per case sooner, and that evidence is what lets you decide about the second agent.

A useful check on any scope is to write the success measure in one sentence, with a baseline: how many invoices are matched by hand today, how long each takes, and what share must match without edits for the agent to count as working. If that sentence needs a paragraph, the scope is still too wide.

An AI proof of concept answers one question cheaply: does this work on our data? Scope it to be thrown away, and say so on the first day. A proof of concept that becomes the production system without anyone deciding it should carries every one of its shortcuts along.

Guardrails in practice

Guardrails come up in most pitches without much detail, so here is what they look like for one invented agent: a system that matches supplier invoices to purchase orders and proposes payment.

Its actions are limited. It can read invoices and purchase orders and write a proposed match; it cannot release a payment, which stays with a person who sees the proposal and the evidence side by side. Amounts are tiered: matches under a set value are approved in bulk after a person has reviewed a sample, and anything above goes to individual review. Scope is narrow: it only touches suppliers on an approved list, and a new supplier always goes to a person. Some actions are blocked outright. It can't edit supplier records, ever. That belongs on a short written list, and the system's permissions enforce the list, so it doesn't depend on a line in the prompt.

Then there's the record. Every decision is logged with the inputs the agent saw, the match it proposed, the reason, the confidence, and who approved it. If an auditor asks why a payment was proposed in March, the answer is a log entry someone can read back. Prompts and rules carry version numbers, and each decision notes which version made it, so a change in behavior can be traced to a change in the agent. Finally there's a switch: someone can turn the agent off in one step, and the process falls back to the manual path it replaced.

All of this comes to a few lists, a permission set and a log, and each is cheap to build into the first version and costly to add after a wrong payment.

Testing an agent before it touches live work

An agent that looks good in a demo has been tested on the cases the builder chose. Before it touches live work, test it on yours.

Start with a set of real past cases, a few hundred if the volume allows, each with the outcome a competent person reached. Include the ugly ones: the missing field, the odd format, the case that took three emails to resolve. Run the agent over all of them without letting it act, and score where it agrees with the person and where it doesn't. Then read the disagreements one by one. Some are the agent's mistakes and some turn out to be the human's, and both are useful.

Add cases it should refuse: a request outside its limits, an email that tries to give it instructions, a document with conflicting numbers. An agent that handles only the happy path hasn't been tested. After that, run it in shadow mode on live work, where it proposes and people act, and compare the two on real volume before it is allowed to do anything.

Track four numbers throughout: how often it completes a case correctly, how often it escalates, how often it takes a wrong action, and what each case costs to run. The wrong action rate is the one with a ceiling, set in advance from the cost of error you worked out during decision mapping. Rerun the whole set after every change to the prompts, the model or the rules, because an improvement in one place can break another.

A plausible first 90 days

Every engagement differs, but the shape is similar. The timings below are illustrative, and access and data tend to move them more than the building does.

Weeks one and two are discovery. A workshop with the process owner maps the workflow, the decisions are listed, systems and access are identified, and the success measure is written down together with the baseline. The most useful output is a one page scope that says what the agent will and won't do.

Weeks three to six are the prototype. A first version runs against historical cases and never live ones. Prompts, logic and the first guardrails get tuned against the test set, the team learns where the data is worse than expected, and integrations are built against a test environment.

Weeks seven to ten are the pilot. The agent runs in shadow mode, then on real work with a person approving every output, and the numbers are tracked weekly. Boundaries get adjusted from what the pilot teaches, and somebody reads the decision log and acts on it.

Weeks eleven to thirteen decide production. If the wrong action rate and the cost per case are within what you set, autonomy widens step by step on the parts that held up. If they aren't, you've spent a quarter learning something specific, which beats spending a year learning it in production. Either way the handover starts here: documentation, runbooks, and training for the people who will operate it.

Keeping it healthy after launch

Launch is the start of the maintenance bill, and four things need a schedule. Read a sample of decisions every week for the first months and monthly after that, because the log only helps if somebody reads it. Watch the exception rate and the cost per case; a slow rise in either is often an early sign that the inputs or the process have changed.

Treat a model upgrade like any other change. Pin the model version where you can, and rerun the whole test set before a newer one goes live, since a newer model can behave differently on the cases that used to work. And review the boundaries with the process owner every quarter, because the process will have moved (a new approval limit, a new supplier category) and the agent only knows what it was told. It also helps to plan the end of an agent's life up front: the day the process changes enough that retiring the agent is cheaper than patching it.

Questions to ask a custom AI development company

If you decide to hire, ask these.

1. What have you put into production, as opposed to prototyped, and can I speak to someone who runs it?
2. Who owns the agent, the prompts, the evaluations and the data afterwards?
3. What happens when the agent gets something wrong? (It will.)
4. How will we know it works before it touches live work?
5. What do I receive at the end, and could my team run it without you?
6. How do you price a first phase, and can I stop after it?

A good custom AI development company has a specific answer to the third question. A vague one often means nobody has thought about the second week in production.

What it costs

Nobody can quote custom AI development fairly from a blog post, and a flat price before anyone has seen your workflow is a guess. The cost depends on how many systems the agent touches, how much judgment it needs, how clean your data is and how much testing the risk calls for.

Where the money goes is easier to describe than the total. Discovery and mapping take mostly your people's time. Integration is often the largest part of the build when several older systems are involved, and it can cost more than the AI itself. Evaluation is the line most often underestimated, and it is what catches a wrong payment before release. Running costs sit apart from the build: model usage per case, hosting, and the time of the people who review exceptions. Ask any builder to price every one of those lines separately, running costs included.

How we approach it

Our custom AI development engagements start with a workshop on one workflow and move through prototype, pilot and production. The guardrails go into the first build and everything is handed over at the end, including the prompts and the evaluations, so your team can run and change the agent without us.

While you're weighing options, run the five questions from our AI readiness assessment framework on the workflow, and use how to automate business processes to work out where the time goes. To talk through a specific workflow, get in touch.