An agent is software with keys to your systems. We build them the way we would want to find them on a test: one identity of their own, the narrowest tool permissions that still do the job, a written set of cases they must pass before release, and a log of every run.
The agents we build read from systems you already run, decide what to do next, and call tools that change state: writing a record, moving a file, issuing a refund, opening a ticket. The interesting engineering is not the conversation. It is the list of things the agent is allowed to call, what happens when a call fails halfway, and who finds out.
Most of the work in a build is unglamorous: the tool manifest, the retry and idempotency rules, the cases it must pass, and the decision about which actions a person still signs off. We would rather spend four weeks there than ship something that looks impressive in a meeting and surprises you in week six.
We will not ship an agent with write access to a system where nobody can tell us what it should refuse to do.
An agent without an eval suite is a demo. It passed once, in front of you, on an input somebody chose. The suite is how you know whether today's change broke last month's behaviour.
We sit with the people who do the task now and take the real inputs: the malformed invoice, the customer who asks two things at once, the claim that should be rejected. A first suite is typically forty to a hundred and twenty cases, and roughly a third of them are cases where the correct answer is to stop.
Exact match where the answer is a value, a checked assertion where it is a tool call with arguments, and a written rubric where it is prose. We state which cases are judged by another model and treat those scores as weaker evidence than the deterministic ones, because they are.
Prompt edits, tool changes, model version changes, retrieval index rebuilds. A drop against the recorded baseline stops the deploy the same way a failing unit test does. Model upgrades are the ones that catch people out, and that is exactly what the baseline is for.
The cases, the harness and the baseline scores are yours at the end of the engagement, readable without us. If you later hire someone else, they inherit the test of record rather than a folder of prompts.
Sometimes the suite says no. If the agent cannot clear the cases at a rate you can live with, we tell you before the build instead of after it, and occasionally the honest recommendation is a few hundred lines of deterministic code with no model anywhere in the loop.
Prompts are instructions. Permissions are enforcement. Anything the agent must not do gets written where the model cannot argue with it.
The agent gets a service principal of its own, never a human's session or a shared key. Its actions are attributable to it, and revoking it costs one admin action rather than a password reset for a person who is on leave.
Each tool gets the narrowest scope that lets it finish: read limited to the records in question, writes limited to specific fields, no standing administrative role because it was quicker during the pilot.
Tool arguments are checked by code before execution: allow-listed destinations, bounded amounts, typed identifiers. The model proposes the call; something deterministic decides whether it runs.
Deleting, paying, sending externally, changing an entitlement. These stop for a named human with the proposed action and the reasoning in front of them, until you have the run history to justify moving a particular one behind the line.
Caps on steps, tool calls, tokens and wall-clock time per run. An agent in a loop is a cost event and a load event at the same time, and the ceiling belongs in the runtime, not in a dashboard you read on Monday.
One switch that halts new runs, one that revokes the agent's credentials, and a rehearsal that proves both work with someone else on the keyboard. We write down how long a drain takes in practice.
The same people who test other companies' LLM applications test ours, against the failure modes that actually show up rather than the ones that sound frightening.
Text in a retrieved document, a support ticket, a filename, a web page the agent fetched. We plant directives in each channel the agent reads and check whether they change its tool calls.
Arguments the schema allows but the business does not, chained calls that assemble a privileged action out of unprivileged steps, and the same destructive call replayed.
System prompt and credential recovery, one tenant's records surfacing in another's answer, and outbound tools used as a channel to send what the agent was shown.
Questions with no supporting source, documents that contradict each other, and records the agent was never given. If the only option encoded is to answer, it will answer.
Who can put a document in front of the agent, whether permissions follow a record into the index, and what a poisoned or stale chunk does to the answer.
Inputs that drive recursive planning, prompts that expand each turn, and the cost curve of a single adversarial request against the caps we set.
Identifiers refer to the OWASP Top 10 for LLM Applications. Citing a public list is not accreditation and we claim none from OWASP; we name the categories so you can check our coverage against something that is not our own marketing.
What started the run, which identity asked, and the exact input it received.
Which documents and records were retrieved, with versions, so the same context can be rebuilt.
Every tool call in order, with arguments, result, and whether validation rejected it.
What the agent returned or changed, and where a human approved or overrode it.
Tokens, tool calls and latency per run, so the unit economics are a number and not a feeling.
How long each of the above is kept, where it lives, and who can read it.
Logs with personal data in them are a disclosure risk of their own. We agree what gets redacted at write time before the first run, rather than discovering the problem during a privacy review.
Agents fail in ways that are hard to walk back. These are the shapes we have decided not to take on.
If nobody in the room can state what the agent must never do, there is nothing to build a boundary against. We will build the read-only version and wait.
Hiring, credit, discipline, benefits, access revocation. A model can prepare the file and surface the evidence; a named person decides and signs.
The first runs happen against a copy, or against production with every write disabled and the proposed actions logged for comparison. Shadow mode first costs a fortnight and settles the argument with evidence.
If the task is deterministic, we will say so and quote the smaller job. A model in the loop adds non-determinism, inference cost and a new attack surface, and it should earn all three.
LLM applications are one of the nine surfaces on our penetration testing scope. The findings we write for other companies are the reason this page talks about identities, tool argument validation and run logs rather than about what AI can do.
It also means the build and the test are not the same pair of eyes when you do not want them to be. Where we built the agent, we will say so plainly in the report, and you are free to have the boundary tested by someone else entirely. We will hand them the tool manifest and the eval suite.
See how we test →Every finding we write about someone else's agent is a line in our own build checklist.
Whichever one your legal and procurement teams have already cleared, hosted wherever your data is allowed to go. The model is a configurable dependency in what we build, not the architecture. The eval suite is what makes swapping one a decision with evidence behind it rather than a leap.
For a single agent with a narrow tool set, expect a week on cases and the tool manifest, three to five weeks of build and evaluation, then shadow mode against real traffic with writes disabled before anything acts. We would rather tell you that number than a shorter one we would then have to defend.
One owner who can say what the agent must refuse, access to a non-production copy of the systems it will touch, real historical examples of the task including the ones that went wrong, and written authorization for any tool that can write. Missing the first of those stops the engagement, not the last.
Either. If you take it, you get the code, the tool manifest, the eval suite, the baselines and the runbook for the stop switch, and we will sit with your on-call team while they rehearse it. If we operate it, the same artifacts exist and you can read them whenever you like.
Retrieval is where injected instructions arrive, and permissions in the index are where one team reads another team's records. A read-only assistant has a smaller blast radius, not a different set of problems. The eval suite is usually shorter and the boundary work is mostly about who can put a document in front of it.
Often. The first week is an assessment: the tool permissions it actually holds, whether runs are reconstructable, and an eval suite built from your own history so there is a baseline to improve against. Sometimes the answer is that the boundary has to be rebuilt before the behaviour is worth tuning.
The systems the agent calls, and the agent itself: prompt injection, tool abuse, and whether it can exceed its own authorization.
The APIs and platforms the agent acts through, including the ones that need idempotency before anything automated touches them.
When an automated action is disputed, the investigation that reconstructs what ran, on whose instruction, and what it changed.
CobaltLoom