Insights

The Day 2 file: six things an association can show its Board about its AI

Nov 10, 2026, 6:30:00 AM · Rick Bawcum

← All insights

The question behind the file

Buying AI is Day 1. Day 2 is every day after. At some point a Board member, an auditor or a member company will ask a plain question: how do you know your AI still does what you approved?

The best answer is not a conversation. It is a file you can hand over. Here is what is in it, and what each part is for.

The six parts

  1. The guardrails, written down. What the assistant must never say, which member data it must not touch, and what goes to a person. If these exist only in people's heads, they cannot be tested.
  2. A dated baseline for each product, as it runs today. The model and version, the instructions, the content sources, the tools switched on, and how it answered a fixed set of questions on that date. This is what every later result is compared with.
  3. A fixed set of real questions, run from the outside. The questions members actually ask, plus the ones you would least want answered badly: eligibility, fees, deadlines, anything touching money, health or legal rights. Run them the way a member would, on a regular schedule.
  4. A re-test whenever anything changes. A vendor announces an update, staff edit the instructions, a complaint arrives, new content is loaded. A calendar alone misses these. A list of triggers catches them.
  5. Grading by someone who neither runs the assistant nor sells it. A result that can come back red is worth more than one that cannot. If the people who built or operate the assistant also grade it, the file shows effort but not independence.
  6. A record, and one named owner. Dated results kept in one place, and a single person responsible for the schedule and for acting on a bad result. A folder with an owner beats a recollection.

Which parts tend to lapse

A small team can run a modest version of all six itself, and that is far better than nothing. In practice, the first two parts get done because they feel like launch work. The parts that quietly lapse are the regular testing, the re-test after changes, and the independent grading. They take time that nobody has been assigned, and nothing breaks visibly when they stop.

That is the useful way to read the list. Ask which of the six you could show a Board member today. The gaps are where Day 2 is thin.

Start with what you have

You do not need all six to begin. Write the guardrails for your most member-facing assistant. Record today's baseline. Choose twenty questions. The first pass takes an afternoon, and it turns an open-ended worry into a short list of things to do.

Where Cimbiotic fits

Parts three to six are the ones organizations most often want help with. CimAssure runs a versioned set of tests against your chatbots and agents from the outside, on a regular schedule and when the AI changes, and reports the result as a dated Assurance Statement: Green, Amber, Red or Not rated. It provides assurance, not a guarantee, and it is not a certification, audit or attestation. Cimbiotic is independent of the vendors it tests.

Choose your next step:

Where does your organization actually stand?

Seventeen questions, about five minutes, no sign-up to see your result.

Take the free Scorecard Book a 30-minute call