---
title: How do we test whether an AI agent is doing what it should? | Cimbiotic
description: An agent acts, it does not just answer. How to test what an AI agent actually does with test records, what to ask your vendor, and what a clean result means.
image: https://www.cimbiotic.ai/hubfs/cimbiotic-og.png
---

[Skip to content](https://www.cimbiotic.ai/insights/how-do-we-test-whether-an-ai-agent-is-doing-what-it-should#main)

[Cimbiotic](https://www.cimbiotic.ai/)

Menu

[Home](https://www.cimbiotic.ai/)[The Assessment](https://www.cimbiotic.ai/ai-assessment)[CimAssure](https://www.cimbiotic.ai/cimassure)[Framework](https://www.cimbiotic.ai/framework)[Advisory](https://www.cimbiotic.ai/advisory)[About](https://www.cimbiotic.ai/about)[Insights](https://www.cimbiotic.ai/insights) [Free Scorecard](https://www.cimbiotic.ai/scorecard)

---

Insights

# How do we test whether an AI agent is doing what it should?

Oct 8, 2026, 1:23:07 PM · Rick Bawcum

[← All insights](https://www.cimbiotic.ai/insights)

**The short answer:** test what it does, not what it says. Send it realistic requests through the channel it actually uses, with test records instead of real members, and check what happened: whether the record changed, whether the email went out, whether the refund was issued. Then repeat the test whenever the agent, its tools or its instructions change. A vendor's assurance or a read through the logs is not a test.

Most associations that put an AI agent to work, for example one that sorts the member-services inbox, updates records or drafts and sends replies, checked it once before launch, on the cases the project team expected. That is a reasonable start. It does not show how the agent behaves with real requests, after the vendor's model and your own processes have changed.

## Why an agent is different from a chatbot

A chatbot says things. An agent does things. A wrong answer can be corrected with a follow-up message. A sent email, a changed membership record or a refund that went out usually cannot be quietly taken back. That is why testing an agent is mostly about what it did, and only partly about what it said. We wrote separately about [who answers when an agent gets it wrong](https://www.cimbiotic.ai/insights/who-is-liable).

## What is worth testing

1. **Ordinary requests.** Does it do the right thing on the everyday cases it was built for: route the message correctly, update the right record, leave the rest alone?
2. **Its limits.** Ask it for something outside its role: a refund over its limit, another member's details, a message to an outside address. It should refuse or hand off. Write down what it did.
3. **Strange or hostile input.** Put an instruction inside an email or an attachment, the way a careless sender or a bad actor might. Check whether the agent follows it.
4. **Stopping and review.** Can a named person stop it quickly? Can someone see afterwards what it did and why? An agent nobody can stop or review is a governance gap, whatever its test results.

## How to run the tests safely

- Use test records you create, not real members.
- Use email addresses you own, so nothing reaches an outside person.
- Start with tests that only read or ask. Do not run anything that could change real data.
- Get written authorization before you test, and decide beforehand who can reverse a change if one slips through.
- Record the exact request and what happened each time. A result without the exact request cannot be repeated.

## "Our vendor is supposed to test this"

Good vendors do test their products, and you should ask to see what they test. But vendor testing and your testing answer different questions.

- **A vendor tests the product in general.** It does not test your setup: your permissions, your inbox rules, your records, the limits you set, or the requests your members actually send.
- **You usually cannot see what was covered.** A vendor's statement that a product is tested rarely says what was tested, on which version, or when.
- **Things change after the test.** The vendor updates a model or a tool, or you change the instructions. Your own testing is how you notice.
- **The responsibility does not move with the contract.** If an agent acts badly on your members' behalf, your members, your board and your insurer are likely to come to you first. What the vendor owes you depends on your contract.

Testing your agent does not replace the vendor's testing. It checks the part the vendor cannot see. Four questions are worth putting to the vendor in writing:

1. What did you test, on which version, and when?
2. Can you show results for a setup like ours?
3. How will you tell us when the model, the tools or the defaults change?
4. What can the agent do today that we have not limited?

## Common questions

**Do we need access to the vendor's tools?** No. You can test through the same channel the agent uses, such as the inbox it reads. Where a test could not be run on your setup, the report should say which one and why.

**Can we do this ourselves?** Yes, and it is much better than not testing. The weak point is consistency and independence. The people who run an agent tend to try the cases it handles well, and internal testing tends to lapse when staff are busy.

**Does a clean result mean the agent is safe?** No. It means the agent passed the tests that were run, at that time. It is not a certification, and it does not predict how the agent will behave later.

## What we do not know

We have no reliable data on how often association agents act outside their limits, and nobody should quote you a number. The field is new, tools change quickly, and any set of tests can miss a case. That is a reason to test your own agent on a schedule, and to keep the tests honest about what they did not cover.

## Where to start

Write down every tool in your association that can take an action without a person prompting each step, what it is allowed to do, and who can stop it. That list is the first piece of evidence, and many associations find it is longer than they expected.

If you want an independent outside review, our Agent Controls Assessment looks at one agent's safeguards. Agent Assurance, which tests what an agent actually does, is open to a small founding pilot group. [See how Agent Assurance works](https://www.cimbiotic.ai/cimassure#agent).

*This article is general information, not legal advice. Testing is assurance, not a guarantee: it does not warrant that an agent is safe or that it complies with any law or regulation.*

## Where does your organization actually stand?

Seventeen questions, about five minutes, no sign-up to see your result.

[Take the free Scorecard](https://www.cimbiotic.ai/scorecard) [Book a 30-minute call](https://calendly.com/rbawcum)

Cimbiotic

Independent AI assurance for professional and trade associations, and the wider non-profit sector.

Humans Leading Change

#### Cimbiotic

- [The AI Assessment](https://www.cimbiotic.ai/ai-assessment)
- [CimAssure](https://www.cimbiotic.ai/cimassure)
- [Framework & approach](https://www.cimbiotic.ai/framework)
- [Advisory](https://www.cimbiotic.ai/advisory)
- [About](https://www.cimbiotic.ai/about)
- [Insights](https://www.cimbiotic.ai/insights)

#### Start

- [Free Trust Readiness Scorecard](https://www.cimbiotic.ai/scorecard)
- [Book a 30-minute call](https://calendly.com/rbawcum)
- [rick@cimbiotic.ai](mailto:rick@cimbiotic.ai)

© 2026 Cimbiotic. Assurance, not a guarantee. Aligning to ISO/IEC 17065. Not currently accredited. [Privacy](https://www.cimbiotic.ai/privacy) Cookie settings

```json
{
  "@context" : "https://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Rick Bawcum",
    "url" : "https://www.cimbiotic.ai/insights/author/rick-bawcum"
  },
  "dateModified" : "2026-10-08T17:23:07.548Z",
  "datePublished" : "2026-10-08T17:23:07.000Z",
  "headline" : "How do we test whether an AI agent is doing what it should?",
  "mainEntityOfPage" : {
    "@id" : "https://www.cimbiotic.ai/insights/how-do-we-test-whether-an-ai-agent-is-doing-what-it-should",
    "@type" : "WebPage"
  },
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "url" : "https://www.cimbiotic.ai/hubfs/cimbiotic.svg"
    },
    "name" : "Cimbiotic LLC"
  }
}
```