Cimbiotic Blog

AI is already inside your certification program. The question is where.

rick@cimbiotic.ai (Rick Bawcum) — Wed, 16 Sep 2026 20:17:33 GMT

Most writing about AI risk in associations is about communications. A newsletter goes out with an error in it. A chatbot says something wrong. Embarrassing, fixable, forgotten in a quarter.

Credentialing is a different category. If your organization certifies people, the credential is not a programme you run — it is the asset the organization is built on. Members pay for it, employers rely on it, and in some professions a regulator points at it. Its entire value rests on a belief: that the standard is real and consistently applied.

That belief is not robust to surprises.

AI is probably already in the process

Not as a decision anyone announced. As features that arrived inside tools you already use, and as sensible shortcuts staff adopted because the work is hard and the deadlines are real.

Five places worth checking, in rough order of how quietly they tend to appear:

Item development. Draft exam items generated or rewritten with AI assistance, then reviewed by a subject-matter expert. The review is real; the provenance is often unrecorded.
Item analysis and forms assembly. Statistical tooling that now ships with AI-assisted recommendations about which items to retain, retire or place.
Scoring support. Anything beyond a straight key — essay scoring, performance rubrics, practical assessments — where a model contributes to or triages a human judgement.
Eligibility screening. Reviewing applications for experience, education or supervised hours. This is where AI is most useful and least visible, because it looks like admin rather than assessment.
Candidate preparation. Adaptive study tools, often licensed from a vendor, sometimes trained on your own item bank.

The last one deserves particular attention, because the exposure runs in an unexpected direction. If preparation material is derived from live items, the integrity question is not about the AI at all — it is about whether your item bank leaked into a training set.

Why this is different from other AI risk

An error in a member newsletter damages a relationship. An unexplained irregularity in a certification decision damages the thing the relationship is based on.

Consider how a challenge actually unfolds. A candidate fails, appeals, and asks a reasonable question: was any part of this decision automated, and if so, how was it checked? For most credentialing bodies today, answering that requires a scramble — and the scramble itself is the finding. If you cannot describe your own process confidently, the appeal stops being about one candidate and starts being about the programme.

A credential is worth exactly what the market believes about its rigour. That belief does not survive a story about an algorithm deciding who passed — even if the algorithm did no such thing.

Note the last clause. The reputational damage does not require the AI to have made the decision. It only requires that you cannot demonstrate it did not.

The legal edge is closer than most realize

Emerging US state law is converging on the idea of a consequential decision — one that materially affects someone's access to employment, education, credit, housing or similar. Where such decisions are made or substantially assisted by automated systems, the pattern in recent legislation is a set of duties around disclosure, explanation and human review.

Colorado replaced its 2024 AI Act in May 2026 with SB 26-189, the Automated Decision-Making Technology Act, effective 1 January 2027, and it centres on exactly those duties. California's CCPA automated decision-making provisions arrive in January 2027. The EU AI Act's high-risk obligations land in August 2027, which matters if you certify anyone in Europe.

A certification decision that determines whether someone can practise, or be hired, looks a great deal like a consequential decision. Whether any specific statute reaches any specific credentialing programme is a question for your counsel, not for a blog post, and the answer will depend on your jurisdictions, your candidates and how your process actually works.

But the direction is not ambiguous, and the practical implication is the same either way: you will be expected to describe where automation touches decisions about individuals, and to show that a person remains accountable. Organizations that can already answer that will find this straightforward. Organizations that cannot will be doing archaeology under time pressure.

What accreditors and boards will ask

Your board is likely to get there first. NACD found roughly 72% of directors say their boards lack sufficient expertise to oversee AI effectively — and in our experience that gap does not produce silence. It produces a written question to staff, usually phrased broadly, usually with a deadline.

Meanwhile the general state of practice offers little cover. In ISACA's 2026 poll of more than 3,400 professionals, only around 18% of organizations both require and enforce disclosure of AI use, while roughly a third have no requirement at all. We wrote about that gap here. Being typical is not a defence when the asset at stake is your credential's authority.

Four things worth doing

None of these require a policy project.

Map where AI touches the candidate. Walk your certification lifecycle end to end — application, eligibility, item development, form assembly, delivery, scoring, appeals, recertification — and mark every point where an automated system contributes. Include vendor tooling. This is a two-hour exercise and it is usually the whole finding.

Record the human decision. Wherever a determination is made about an individual, capture who made it and on what basis. Not a new process in most cases — just writing down what already happens. This is the single artifact that answers an appeal, a regulator or a board.

Ask your exam vendors two questions in writing. Where does AI operate in the services you provide us, and is any of our item bank or candidate data used to train models? Get it in writing, because the answer sometimes turns out to be a configurable setting rather than a contractual commitment.

Draft the appeal answer now. Write the paragraph you would send a candidate who asks whether their result was automated. Writing it while nothing is contested is considerably easier than writing it afterwards, and the difficulty tells you which of the other three to do first.

What we do not know

There is no good survey of how credential holders specifically respond to AI in certification processes. The evidence cited here comes from professionals reporting on their own organizations, from directors, and from general consumer research. It is directionally consistent, but the precise figures should not be treated as census data, and nobody should claim to know what your candidates think.

The legal position is also genuinely unsettled. We have described the direction of travel, not a determination about your programme.

What is not uncertain is the asymmetry. The cost of mapping where AI touches your credentialing process is a couple of hours. The cost of not being able to describe it, at the moment somebody insists you do, is the authority of the credential itself.

Where to start

Our AI Trust Readiness Scorecard includes the question about automated decisions affecting individuals, alongside fifteen others. It takes about five minutes and there is no sign-up to see your result. If you would rather see how we assess this in depth, the framework is published.

Almost everyone expects AI disclosure. Almost nobody enforces it.

rick@cimbiotic.ai (Rick Bawcum) — Wed, 16 Sep 2026 20:16:23 GMT

Sooner or later a member emails and asks how you use AI. Not as a complaint — just as a question. Which tools, what happens to their data, whether a person reads the thing before it reaches them.

Most associations cannot answer it. That is not a failure of diligence. It is what happens when software you already pay for starts shipping AI features without asking, and staff quite reasonably adopt tools that make their week easier. The tools arrived faster than anyone's ability to describe them.

What makes this worth attention now is that the question has stopped being hypothetical.

The numbers

The clearest evidence comes from inside the association world itself. ISACA — a professional association of digital trust practitioners — surveyed more than 3,400 members for its 2026 AI Pulse Poll. These are the people best equipped to govern AI, reporting on their own organizations. Asked whether their organization requires disclosure when AI has been used to create or substantially assist a work product:

Around 18% both require disclosure and enforce it
About 20% require it but do not enforce consistently
Roughly 32% have no disclosure requirement at all

Set that against what people expect. Fractl's Q2 2026 survey of 1,008 US consumers found roughly 84% want AI-assisted written content labelled, rising for audio, images and video. Association members are consumers too, and they do not suspend that expectation when they open a message from their professional body.

So: roughly four in five organizations cannot stand behind a disclosure practice, while roughly four in five people expect one.

Requiring is not the same as enforcing

The middle number is the one worth sitting with. Around a fifth of organizations have a disclosure requirement that is not consistently applied.

Under scrutiny, that is a weaker position than having no policy at all. A missing policy is a gap. A policy that exists on paper but is not operating implies a control that is not working — and it invites exactly the follow-up question you least want. If a board member asks whether you disclose AI use and the answer is "we have a policy", the next question is how you know it is being followed.

This is the difference between having a rule and being able to show the rule works. For an organization whose authority rests on standards it sets for others, that distinction is not academic.

The trap: disclosure on its own can backfire

The obvious response to a gap this wide is to start disclosing. Done without care, the evidence suggests that makes things worse.

The same Fractl research found the share of people who say heavy AI use would reduce their trust in an organization they favour roughly doubled between 2025 and 2026 — from about 20% to around 40%. Only about 14% said it would increase their trust. The effect was strongest among younger respondents, at roughly 54% of Gen Z, and stronger among women than men.

Those are the cohorts most associations are working hardest to recruit and retain.

Disclosure on its own reads as an admission. Disclosure backed by evidence reads as competence. Same sentence, opposite effect.

An organization that says "we use AI" invites the penalty and offers nothing to weigh against it. An organization that says "we use AI, here is what it touches, here is where a person decides, here is who is accountable" is making a materially different claim. The second one is harder to write, which is precisely why it carries weight.

What generic guidance misses about associations

Most AI governance writing is aimed at companies. Four things are specific to membership organizations, and each raises the stakes.

Credentialing. AI now touches exam item generation, adaptive preparation, scoring support and eligibility screening. A credential is worth exactly what the market believes about its rigour, and that belief does not survive a story about an algorithm deciding who passed.

Members whose careers AI threatens. National polling in March 2026 found around 70% of Americans expect AI to reduce job opportunities, rising to roughly 81% among Gen Z. Using AI carelessly on the very members whose livelihoods feel exposed is a particular kind of misstep, and they read it that way.

Member-company data. Trade associations hold pricing, volume and wage data from competing companies under long-established antitrust discipline. Whether that data reaches an AI tool, and whether a vendor trains on it, is a question with no clean equivalent in the general market.

Chapters and components. Local chapters adopt tools independently, under the national brand, usually with no governance capacity at all. The risk is federated. The reputation is not.

Four things worth doing

None of these require budget or a consultant.

Build the inventory. One spreadsheet: tool, owner, what data it touches, whether a person reviews the output. Go system by system through your AMS, LMS, certification platform, CRM and email tools and list what AI is switched on. Most organizations find something they did not know about. Half a day of work.

Decide what always gets human review. Write down which categories of output a person must approve before they go out, and which may go automatically. Ambiguity here is where incidents come from.

Write the paragraph you would send. Draft the reply you would give if a member asked tomorrow. If it is hard to write, that difficulty is the finding — and it tells you exactly which of the other three to do first.

Name someone accountable. One person, written into a role description, responsible for answering to leadership on AI use. Not who operates the tools — who answers for them. This costs nothing and changes how every other question gets answered.

What this evidence does not tell you

Worth being straight about the limits, since this piece is arguing for candour.

None of the research above asks association members directly how their organization's AI use affects their view of it. ISACA surveyed professionals about their own organizations. Fractl surveyed consumers in general. The closest stakeholder research sits in the charitable sector, and members are not donors — dues are not gifts, and the relationship is different.

The direction of travel is consistent across sources and well corroborated. The precise percentages should not be treated as census data, and we try not to quote them that way.

That gap is itself worth naming. Nobody has published credible research on how members respond to their association's AI use. Until someone does, everyone in this conversation — including us — is reasoning from adjacent evidence.

Where to start

If you want a structured view of where your organization actually sits, our AI Trust Readiness Scorecard walks the same five areas in sixteen questions. It takes about five minutes, there is no sign-up to see your result, and if the answer is that you are in reasonable shape, it will say so.

You did not choose most of the AI in your association. Your vendors did.

rick@cimbiotic.ai (Rick Bawcum) — Wed, 16 Sep 2026 20:14:23 GMT

Ask an association executive which AI tools their organization uses and you will usually get a short, confident list. Ask which AI features are running inside the software they already pay for, and the answer changes shape.

That second category is almost always larger. Your AMS, your LMS, your certification platform, your email tool, your community software and your events system have all shipped AI capability over the past two years. Most of it arrived in a release note. Some of it arrived switched on.

None of that was a decision your organization made. It was a decision your vendors made, and then told you about — in a changelog, in a webinar you could not attend, in a footer of an email about something else.

This is not a story about bad vendors

It is worth being fair here, because the vendor-bashing version of this argument is both wrong and useless.

Your software vendors are not being careless. They are doing exactly what their incentives and their customers demand: shipping useful capability quickly, in a market where being six months behind on AI is an existential problem. Defaults are set to on because a feature nobody discovers is a feature that does not renew.

The asymmetry is not about competence or good faith. It is that your vendor's job is to ship value fast, and your job is to be able to answer for what runs on your members' data. Both are legitimate. They simply do not produce the same answer about what should be enabled, when, and with what disclosure.

Which means the gap is yours to close. Nobody else in that relationship is incentivised to close it for you.

"We take security seriously" is not evidence

The standard response to a governance question is a trust page: a security overview, a compliance badge, a paragraph about responsible AI. These are not worthless — a vendor that has thought about the question is better than one that has not.

But they are self-attestation. The vendor is describing its own practice, against criteria it selected, with no independent party checking and no consequence for overstating. That is a marketing artifact, not assurance.

A vendor cannot credibly certify itself. Not because vendors are dishonest, but because an assessment nobody can fail is not an assessment.

This matters more than usual right now because there is no established floor. In ISACA's 2026 poll of more than 3,400 professionals, only around 18% of organizations both require and enforce disclosure of AI use. When general practice is that thin, a confident trust page tells you very little about what is actually happening inside the product.

Four questions worth putting in writing

Not a procurement overhaul. Four questions, by email, to each vendor that touches member data. The written answer is the point — it creates a record, and it tends to produce more careful responses than a call.

1. Where does AI operate in the services you provide us? Ask for the specific features, not a philosophy. You are building an inventory, and this is the fastest way to fill in the part you cannot see from your own admin console.

2. Is any of our data used to train your models, or any third party's? Is that a contractual commitment or an account setting? This distinction is the one that catches people out. A great many vendors will tell you truthfully that your data is not used for training — because a toggle in your account is currently set that way. A setting can be changed by anyone with admin access, or reset in a migration. A contractual term cannot.

3. Which model providers and subprocessors are involved, and where does the data go? Your vendor is usually not the party running the model. The answer determines who actually holds your members' data, and under whose terms.

4. How will you notify us before enabling a new AI feature on our instance? The most useful question of the four, because it is forward-looking. If the answer is a release note, you now know that your governance depends on someone reading release notes.

The question almost nobody asks: can you turn it off?

Incident response is where vendor-run AI gets genuinely uncomfortable. If a tool starts producing wrong output at scale — an inaccurate answer repeated across thousands of member queries, a misfiring automated message — who can stop it, and how fast?

The ISACA poll found roughly 56% of professionals are unsure how long it would take to halt an AI system during a security incident, and around 39% do not know whether a documented shutdown or override process exists at all. Those are respondents whose profession is digital trust, reporting on systems their own organizations run.

When the system belongs to a vendor, the answer usually involves a support ticket. That is worth knowing before you need it, not after. It is a reasonable thing to ask, and a reasonable thing to have in writing.

A specific warning for trade associations

If you are a trade association, one of these questions carries weight the others do not.

You hold pricing, volume, wage and market data submitted by companies that compete with each other, under antitrust discipline that predates all of this by decades. The controls around that data were designed for a world where the risk was a person seeing something they should not.

An AI feature that summarises, searches or surfaces patterns across your member data operates differently. Whether that data reaches a model, whether it is retained, and whether it could inform output shown to another member company are questions your existing information-sharing policies almost certainly do not address. This is a competition-law question as much as a privacy one, and it is worth raising with the counsel who advises you on information sharing rather than treating it as an IT matter.

What this does not tell you

We have no systematic data on how association software vendors actually handle AI defaults, training rights or notification — nobody has published it, ourselves included. The pattern described here comes from what organizations find when they look, not from a survey of vendors.

It is also entirely possible that your vendors answer all four questions well. Several will. The point is not that the answers will be bad; it is that you currently do not have them in writing, and the asking costs an afternoon.

We have written before about the gap between expecting disclosure and enforcing it, and about where AI has already entered certification programmes. This is the same problem approached from the supply side: you cannot disclose what you do not know, and you do not know what your vendors have not told you.

Where to start

Two of the sixteen questions in our AI Trust Readiness Scorecard deal specifically with vendor-enabled features and model-training terms. It takes about five minutes, and there is no sign-up to see your result. If you would rather see how we assess this properly, the framework is published.