Insights | Cimbiotic

A member says your assistant and your website disagree. What do you do first?

Written by Rick Bawcum | Nov 9, 2026, 11:30:00 AM

An email that could arrive on any Tuesday

Picture this message. It is hypothetical, but nothing about it is far-fetched.

"Your assistant told me the registration deadline was the 15th. Your website says the 22nd. Which is right?"

The date is the easy part. Someone checks the calendar, replies, and apologizes. The harder part is the question the email raises about the assistant itself, and it is the one a Board member or a member company will ask next: how did that happen, and how do we know it is not happening to other people?

Four questions you would need to answer

Before you could reply with confidence, you would need to know:

  1. Which assistant said it? Many associations now have more than one: the one on the website, the one inside the event platform, the one in the help desk. Each may run on a different vendor's model.
  2. Which version was it? Hosted AI is a service, not an install. The model, the instructions around it and the content it draws on can all change without a notice reaching you.
  3. What changed since anyone last checked? A vendor update, a staff edit to the instructions, a revised deadline on the website that the assistant never picked up. If nobody recorded a baseline, there is nothing to compare against.
  4. Is this one wrong answer, or one of many? A single complaint tells you where to look. It does not tell you how far the problem goes.

Most organizations can answer the first question and struggle with the other three. That is not carelessness. Nobody was asked to keep that record.

Buying AI is Day 1. Day 2 is every day after.

Launch is the visible event: a decision, a contract, a demo that goes well. Day 2 is the long stretch afterward, when vendors update their models, staff adjust settings, and the content the assistant relies on moves. We call the discipline of showing that your AI is still on the guardrails you set the Day 2 challenge.

It matters most for small teams, because a small team cannot read every conversation. What it can do is decide in advance what the assistant must never say, which member data it must not touch, and what goes to a person. Then it can check, regularly, that the assistant still behaves that way, and keep the dated results.

A five-minute test of your own readiness

You do not need a project to find out how exposed you are. Ask yourself how long it would take to learn what happened if the email above landed in your inbox today. If the honest answer is "a few days, and a lot of guessing," that is useful to know before a member asks.

The Trust Readiness Scorecard has 17 questions covering inventory, human review, member data, disclosure and monitoring. It takes about five minutes and needs no sign-up to see your result.

Where Cimbiotic fits

Cimbiotic is an independent third party. CimAssure tests the chatbots and agents your members rely on, from the outside, keeps every reply as evidence, and reports a dated Assurance Statement your Board can check. Tests repeat on a regular schedule and again when the AI changes. It provides assurance, not a guarantee, and it is not a certification, audit or attestation.

Choose your next step:
View full post