Product

AI AgentsAI SafetyHelpdeskSelf-Service PortalSocial Media AIReturns & Exchanges

Results

Customer Stories50% Performance GuaranteeTrust Center

More

PricingMigrationPartnersLog in
For the person who signs off before AI talks to customers

“The AI decided” is not an answer. Here is what you govern instead.

A tribunal has already rejected the argument that a company's chatbot is “a separate legal entity that is responsible for its own actions.” Since 2 August 2026, the EU AI Act requires you to tell people when they are talking to an AI. Neither problem is solved by a policy document. This is the operating framework underneath one: five controls you can actually enforce, the caps and approval gates that go inside them, the failure mode most spend caps miss, and the ten questions that reveal whether a vendor's platform enforces your policy or merely documents it.

By Amit RG, Founder, Richpanel Published 2026-08-04 Updated 2026-08-04 ~16 min read
View as Markdown →
AR
Amit RG is the founder of Richpanel, the AI-native helpdesk serving 3,000+ brands. He sits on the receiving end of the reviews described here several times a month: Richpanel is SOC 2 Type II audited, HIPAA-audited and GDPR-audited, and answers these questions in writing for security and procurement committees. The buyer quotes below are from recorded evaluation calls, anonymized. On X: @realamitrg.
Start here

The five controls to sign off before AI talks to a customer

1. The permission model. Every action sorted into autonomous, approval-gated, or prohibited, with monetary caps that apply across the whole conversation rather than per tool, and identity verification as a precondition on any account action. 2. The data boundary. What the AI sees on the customer record, where that data sits, which model providers process it by name, and how long any of it is kept. 3. The audit trail. Every action attributable to a named identity with the authorization that permitted it, and every answer reconstructable months later. 4. Escalation as policy. A written list of categories that always reach a human, owned by legal and CX, not tuned by a vendor. 5. Change control. Anything that alters what the AI says is a reviewed, versioned, testable, reversible change.

The test that separates a real control from a written one: can the platform block the thing your policy forbids, with nobody watching? This guide is informational and not legal advice; your obligations depend on your jurisdictions, sectors, and contracts.

Why the policy document is not the control. Most AI governance material aimed at customer service is written for a different reader: a data science function standing up model risk management, or a committee drafting responsible-AI principles. Both are useful, and neither answers the question you were actually asked in the review. That question is narrow and operational. What can this thing do on its own, what stops it doing something expensive or illegal, and when it does, who is accountable and how do we prove what happened?

What this guide is not: it is not a hallucination guide, which is the accuracy problem and is covered in AI hallucination defense. It is not a vendor scoring rubric, which is the 40-question AI RFP template. It is not sector compliance, which for health data is HIPAA-compliant AI customer service. It is the layer those three sit on: the authority model, which survives whichever vendor you pick.

The distinction most programs miss

Governance is a question about authority. Accuracy is a different program.

Teams conflate these constantly, and the conflation is expensive because the two risks scale differently.

The accuracy program asks whether the AI will be right. Its tools are grounding, test cases run before launch, quality review of what was sent, and handing off when confidence is low. An agent that fails here writes a wrong answer.

The governance program asks what the AI is permitted to do, who owns the outcome when it is wrong, and whether you can reconstruct what happened. An agent that fails here moves money, exposes data, or commits you to something.

A perfectly accurate agent with no permission model is still one crafted message away from an unbudgeted refund run. An agent that occasionally writes a wrong sentence but can only send text is a customer-satisfaction problem. The moment you connect that same agent to a payments API, it becomes a balance-sheet problem, and the controls that were adequate an hour earlier are not.

Prompts are instructions, not permissions. If the only thing standing between your policy and the action is a sentence in a system prompt, you do not have a control. You have a preference that the model is usually willing to honor.

A restriction written in the prompt is negotiated with the model on every turn, by a customer who is free to argue. A restriction enforced in the tool layer is not negotiable, because the model never gets to make the call: it can request an action, and something deterministic decides whether the request executes. When you evaluate any platform in this category, the first thing worth establishing is which of those two you are being sold.

The accountability question is already settled

In Moffatt v. Air Canada (2024 BCCRT 149), a customer asked the airline's chatbot about bereavement fares after his grandmother died. The bot told him he could book now and claim the discount retroactively within 90 days. The airline's actual policy required approval beforehand. When Air Canada refused the refund, it argued in front of the British Columbia Civil Resolution Tribunal that the chatbot was, in the tribunal's paraphrase, “a separate legal entity that is responsible for its own actions.”[1]

The tribunal called that “a remarkable submission” and held: “While a chatbot has an interactive component, it is still just a part of Air Canada's website. It should be obvious to Air Canada that it is responsible for all the information on its website.” It found negligent misrepresentation and awarded damages of $650.88.

The money is trivial. The holding is not. Your agent's output is your output, the mechanism that produced it is not a defense, and a company that cannot say who authorized an action is not in a better position for having automated it. Every control below exists because that liability is not delegable to a vendor, a model, or a prompt.

And since 2 August 2026, disclosure is law in the EU

Article 50(1) of the EU AI Act (Regulation (EU) 2024/1689) requires that AI systems intended to interact directly with people are designed so that a person is informed they are interacting with an AI, unless it is obvious from the circumstances. Those transparency obligations became applicable on 2 August 2026, and the Act's penalty tier for this class of breach runs to 15 million euro or 3% of total worldwide annual turnover.[2] You do not need an EU entity to be in scope. Reaching EU users is enough.

The operational consequence is small and easy to miss: the disclosure line in your chat widget has stopped being a copy decision and become a compliance artifact. It belongs inside change control (Control 5), where a growth team cannot quietly test it away to lift a conversion rate. That is a one-line change to your process and it closes a real exposure.

Control 1 of 5

The permission model: what it may do alone, and what it must never do.

Sort every action into three tiers, written as specific actions rather than as risk levels. Vague tiers do not survive contact with an integration, because the integration has specific endpoints and somebody has to decide about each one.

Action Tier The parameter that makes it enforceable
Order status, tracking, delivery estimate Autonomous Read-only scope on the order record
Address change before fulfillment Autonomous Fulfillment-status precondition, plus verified identity
Refund inside published policy Autonomous to a cap Per action, per conversation, and per customer per rolling window
Store credit, goodwill discount, free replacement Autonomous to the same cap Drawn from the same value budget as refunds, not a separate one
Subscription pause, skip, swap Autonomous Verified identity, plus a plan-change allowlist
Anything financial above the cap Approval-gated One-click human queue with full context attached
Chargebacks, fraud flags, legal threats Prohibited Hard block that routes to a named team
Policy exceptions, timeframe promises, price commitments Prohibited The agent may quote published policy. It may never create one.

That last row is the Air Canada row, and it is the one teams forget because it does not look like an action. Inventing a policy requires no API call and no permission. It is a sentence. The control for it is not a spend cap, it is grounding plus the escalation categories in Control 4, plus a change-controlled source of truth for what your policy actually says.

Authentication is a precondition, not a step

The rule that holds up: the agent may answer from published policy without verifying anybody, and the moment it reads or writes anything account-specific it must have verified the person. The trap is treating verification as something the AI performs during the conversation. An order number is not a secret; it appears in forwarded emails, on packing slips, and in screenshots. A customer typing one into an anonymous chat widget has not authenticated.

Verification belongs on the tool, not in the prompt: the tool refuses to return account data unless a verified-identity token is present, which means no amount of conversational persuasion produces the data. Email threads carry a usable signal already, because the reply lands in the mailbox that started the thread. Anonymous widgets and social DMs generally do not, and that difference should show up in your permission table channel by channel.

Cap the economic effect, not the tool

This is the control that most implementations get wrong, and it is worth reading twice if you are the person signing off.

A $150 cap on the refund tool is not a $150 cap. An agent that is refused a $150 refund can, entirely within its granted permissions, offer three $50 store credits. Or approve a free replacement worth more than the refund. Or waive shipping on the next four orders. Or extend a subscription by two months. Every one of those actions is individually permitted, individually under the cap, and individually unremarkable in the log. The aggregate is exactly the outcome your policy meant to prevent.

This is not a hypothetical about a rogue model. It is what a well-behaved, helpful agent does when it is trying to satisfy a customer and one door is closed. A June 2026 reference architecture for runtime governance of production AI agents makes the same point at the systems level: the risk in agentic systems is not the single forbidden action, it is a sequence of individually permitted ones that composes into a business outcome nobody authorized.[3]

The fix is a budget, not a list. Every tool that can move money or goods draws down one shared value budget. The budget is capped per conversation and per customer per rolling window, and the approval gate fires on the running total, not on the individual call. Ask a vendor whether caps are enforced per tool call or across the conversation. Most will answer per tool call. A good number of them will not have considered the question, and their answer to it tells you more about the platform's governance maturity than any slide in the deck.

How to test this in a trial: ask for a refund above the cap, let the agent decline, then ask for store credit instead, then a free replacement, then a discount on the next order. If it grants any of them, your caps are per tool and your real exposure is the sum of every value-moving tool the agent can reach.
Control 2 of 5

The data boundary: what it sees, where that sits, and who else gets a copy.

On a recorded evaluation call, a buying committee at a consumer-hardware brand asked it in one breath: “where does PII sit, SOC 2, SOC 3 stuff, how sustainable are you guys?” Three separate questions arriving as one. Pull them apart, because they have different answers and different owners.

Least privilege on the customer record

An agent answering a delivery question needs the order. It does not need the lifetime purchase history, the internal fraud notes, the support tags a human wrote about the customer's temperament, or the payment instrument. Scope the context by intent, and treat “we pass the full customer profile into the prompt” as the default that it usually is, rather than as a decision somebody made.

The reason this matters more with AI than with a human agent is that the context window is a copy. A human reads the screen; the data stays. A model call transmits whatever is in the context to an inference provider, so an over-broad context is not just an access decision, it is a disclosure decision.

Redact in two places, not one

Most teams redact in the log. Fewer redact in the prompt, and the prompt is the one that leaves the building. If a customer pastes a card number or a medical detail into chat, the question is not only whether it is masked in your stored transcript, it is whether it was masked before the message was sent for inference. Ask where redaction runs in the pipeline and what happens to the raw message.

The three questions about the model layer that buyers skip

Residency, retention, and whether deletion actually deletes

Which region, what the default retention is, whether you can shorten it, and whether deletion propagates. That last part is where most answers get thin. An erasure request has to reach the conversation transcript, any derived embeddings or search indexes built from it, and any evaluation or test set the transcript was copied into. The first is standard. Ask specifically about the second and third, because a transcript that was pulled into a regression suite six months ago is a copy that no deletion job is looking at.

How to test this: ask for four artifacts, not four adjectives. The sub-processor register, the model providers by name and jurisdiction, the default retention with the process to change it, and the notice period for adding a sub-processor.
Control 3 of 5

The audit trail: every action attributable, every answer reconstructable.

These are two different requirements and they are almost always conflated. Attribution answers who did what. Reconstruction answers why it said that. You need both, and a conversation transcript delivers neither, because the transcript shows the words and not the authority.

What one log line has to carry

Run that list against three real situations. A chargeback representment needs the authorization and the parameters, in a format your payment processor accepts. A regulator or a security reviewer asks who saw what, and expects per-record attribution. A customer dispute asks what was promised and by whom. None of the three is answerable from a chat transcript, and all three are answerable from the line above.

Reconstruction is the requirement people discover too late

Four months after the fact, someone will ask why the AI told a customer their order was covered. Answering that means pinning, at the time of the answer, the sources retrieved, the instruction version in force, the tool results returned, and the model version used. If your platform overwrites its knowledge base in place, that answer becomes permanently unavailable the next time somebody edits the article, and you will not find out until you need it.

That is a governance defect rather than a missing feature, and it is worth failing a vendor over. Ask the question directly, because it is not usually on the security questionnaire and the honest vendors will tell you.

Retention on the log is a separate number

Audit-log retention is frequently shorter than conversation retention and is rarely mentioned unprompted. It should outlast your longest dispute window. Card-network chargeback timelines commonly run past 120 days, and regulatory and contractual windows run considerably longer, so a 30-day audit log is a control that expires before the risk does. Ask for the number, in writing, and ask whether the export is machine-readable.

How to test this: ask to see the log entry for an AI-issued refund and its export format. Then edit a help-center article and ask them to show you an answer given before the edit, with the source it used at the time.
Control 4 of 5

Escalation is a policy decision. Not a tuning parameter.

Most teams treat escalation as a quality bar: hand off when the customer sounds frustrated. That part is product design. A subset of it is governance, and the difference is that a governance category is not something anyone gets to tune down to improve an automation number.

The always-human list

Write it as named categories with a named owner. The recurring set:

Two properties make this a control rather than a list. It has to be enumerable, meaning you can read it as configuration rather than infer it from behavior. And it has to be testable, meaning you can send a message in each category and watch the handoff happen.

Fail closed on uncertainty

The default when the agent is unsure must be to escalate, not to attempt. A CX lead at a performance auto-parts brand, where a wrong answer can be a safety issue, put the requirement better than any framework does on a recorded call: any AI they use has to be “willing to give up easily, if it's reached the bound of its legitimate knowledge.” That instinct is correct, and it is directly testable. Ask something adjacent to but outside the documented knowledge and watch whether the agent reaches or retreats.

Add loop detection to the same control. A repeated non-answer, or a turn limit crossed without resolution, has to force a handoff. A customer stuck in a loop is an escalation the system failed to make, and it will not appear in your escalation metrics because no escalation happened.

Handoff fidelity, and who owns the list

An escalation that arrives without the reasoning, the tool calls, and the sources is a control that fires and then destroys its own evidence. The human picking it up is now re-investigating from the customer's last message, which is both slower and unauditable. Handoff context is part of the control, not a nicety.

Finally, write down who owns the categories and how often they are reviewed. Legal and CX jointly is the arrangement that works: legal decides what must never be automated, CX knows which intents actually appear. If the honest answer to “who set these?” is that the vendor tuned them during onboarding, you have outsourced a legal decision to an implementation consultant.

How to test this: ask to see the escalation categories as a list you can edit yourself, the audit record of the last change to them, and what happens to a conversation that hits a turn limit without resolving.
Control 5 of 5

Change control: the knowledge base is production code now.

Ask who can change what the AI says. At most companies the honest answer on launch day is “anyone with a login to the help center.” Which means a new hire's tidy-up of a returns article changes the refund policy the AI quotes to every customer that night, with no review, no test, and no way back.

That is an unreviewed production deploy. It just does not look like one, because the interface is a text editor and the artifact is prose.

Four questions every change must answer

Who approved it. What changed. What was tested. How we roll it back. If any of the four is unanswerable for a given change, that change was not governed, whatever the policy document says.

Tier the review by blast radius

Regression before ship, and who owns the test cases

You need a set of test cases drawn from your own historical tickets that runs on every change, plus a visible diff of what the change moved. The ownership question underneath it is the one that matters in a governance review: if the vendor authors and holds the test set, you can be told about a regression but you cannot demonstrate one to your own auditor. Insist that your CX team authors the cases, that every regression is visible to you, and that prompt and policy changes are auditable on your side of the line.

Rollback is a number, not a capability

“We can redeploy” is not an answer. You want a named previous version you can restore and a measured time to restore it. Ask what that time is. Then ask them to do it while you watch, because the gap between the architecture diagram and the runbook is usually discovered during an incident.

How to test this: change a policy sentence in a trial workspace, then ask for the approval record, the test run, the diff, and a restore of the previous version, in that order, on the call.
For your procurement review

Ten questions your vendor should answer in writing.

Governance answers should be artifacts, not adjectives. Take these into the security review. The first four are disqualifying if the answer is vague, and they are ordered so you find that out before you invest in a feature evaluation.

1. Are permissions enforced in the platform, or written in the prompt?

You want a configuration screen and a log of rejected actions. “It is in the system prompt” describes an instruction, not a control.

2. Is the spend cap enforced per tool call, or across the conversation and the customer?

Ask them to run the decline-then-ask-for-credit sequence live. If nobody has considered it, you have found the gap before it found you.

3. Show me the audit log entry for an AI-issued refund, and the export format.

Acting identity, parameters, the authorizing policy version, the downstream outcome, a timestamp that survives export.

4. Name your model providers, their jurisdictions, and your no-training commitment.

A subscribable sub-processor register with a notice period. Not a sentence about leading AI providers.

5. Who authors the test cases, and can we author ours?

If the vendor owns the evaluation set, you can be told about a regression but you cannot prove one to your own auditor.

6. Can we edit escalation categories ourselves, and is that edit logged?

What must never be automated is a legal decision. It should live in your hands with a visible change history.

7. What is your rollback procedure, and your measured time to restore?

A named previous version and a number in minutes. Then ask them to demonstrate it during the trial.

8. What is the retention on the audit log, separately from conversations?

It is usually shorter and rarely volunteered. It has to outlast your longest dispute window.

9. Which compliance claims are independently audited, by whom, on what dates, with what scope?

A named auditor, a framework, a period, and findings. A “certification” for a regime with no certifying body is a yellow flag worth one follow-up question.

10. Connect us with a customer at our scale whose security team ran this review.

The most reliable signal available, and more reliable than any badge or demo.

Disclosure

How Richpanel answers its own checklist, including where the answer is no.

I wrote this framework because I answer these questions for buying committees several times a month. It would be dishonest to publish the checklist and skip our own row, and equally dishonest to publish a row where we pass everything. Here is both halves.

Where the answer is yes

Where the answer is no, or not yet

One economic note, because it is a governance risk

Richpanel charges once per conversation, at about $0.20 for an AI-resolved one, rather than metering an AI resolution on top of a ticket you already paid for. That is a commercial point everywhere else on this site. It belongs here for a narrower reason: an AI that costs more per resolution than the alternative creates steady pressure to widen the caps and thin the approval gates, and governance that has to fight the budget every quarter tends to lose. Cost structure is a governance variable, not just a procurement one.

The permission model, in production

Aeons, a premium UK supplements brand, runs AI on 63% of every inbound customer message at 4.39 / 5 CSAT, above its own team-wide 4.33, split across two permission tiers rather than one.

Routine conversations close end to end with no human touch, and the higher-risk cases are drafted by the AI and reviewed by a human before sending, because those carried legal, financial or complaint risk. That split is not a product feature. It is the permission model in Control 1, running in production, and it is what a governed deployment looks like from the inside. Read the full case study →

Limitations

Five things this framework will not save you from.

Frequently asked

AI governance, in plain English.

What is AI governance in customer service?

It is the set of enforceable controls that decide what an AI agent may do on its own, what it must get a human to approve, what it must never do, and how you prove after the fact what happened and who authorized it. It is distinct from accuracy work such as test sets, grounding, and quality review, which answers whether the AI will be right. Governance answers who holds the authority and who owns the outcome when it is wrong. In practice it comes down to five controls: a permission model with monetary caps, a data boundary, an audit trail that is attributable and reconstructable, an escalation policy, and change control over anything that alters what the AI says.

What should an AI customer service agent never be allowed to do without a human?

Draw the line at categories rather than at a risk score. The common always-human list is legal threats and regulatory complaints, safety and self-harm and product-injury reports, discrimination and harassment and accessibility complaints, chargebacks and fraud flags and law-enforcement requests, any financial action above your cap, identified minors and customers flagged as vulnerable, bereavement, and anything with press or public amplification. Separately, an agent should never create policy. It can quote your published policy; it must not invent an exception, promise a timeframe you do not offer, or commit to a price. That last boundary is the one that produced the leading chatbot liability case.

Is it a defense to say the AI made a mistake?

No, and this has already been tested. In Moffatt v. Air Canada (2024 BCCRT 149) the airline argued that its chatbot was a separate legal entity responsible for its own actions, after the bot described a refund policy the airline did not offer. The British Columbia Civil Resolution Tribunal called that a remarkable submission and held that a chatbot, however interactive, is still part of the company's own website and the company is responsible for the information on it. The damages were small. The principle is not: your agent's output is your output, and the mechanism that produced it is not a defense. Design the controls on the assumption that you own every sentence the AI sends.

What does the EU AI Act require for customer service chatbots?

Article 50(1) of Regulation (EU) 2024/1689 requires that AI systems intended to interact directly with people are designed so that a person is informed they are interacting with an AI, unless that is obvious from the circumstances. Those transparency obligations apply from 2 August 2026, and the Act's penalty tier for this class of breach runs to 15 million euro or 3% of total worldwide annual turnover. You do not need an EU entity to be in scope; reaching EU users is enough. The practical governance consequence is that the disclosure line in your widget stops being a copy decision and becomes a compliance artifact, so it belongs inside change control where nobody can quietly test it away.

What has to be in an audit log for an AI customer service action?

One line per action, carrying the acting identity (a named agent identity, not the word bot), the human approver and approval time where a gate applied, the action and its parameters such as amount and order, the record it touched, the authorization that permitted it including which policy version, the outcome the downstream system returned, and a timestamp that survives export. A conversation transcript is not an audit log: it shows the words and not the authority. You also need reconstruction, meaning you can show months later which sources and which instruction version produced a given answer. If a vendor overwrites its knowledge base in place, that question becomes permanently unanswerable after the next edit.

Who owns AI governance, CX or legal or IT?

Split it by control rather than handing the whole thing to one function. CX owns the permission tiers and the escalation categories in practice, because they know which intents are safe. Legal owns the always-human list and signs off on the disclosure language. Security and IT own the data boundary, the sub-processor review, and the audit-log retention. Finance owns the monetary caps, since they are a spend authority question. One named person should own the change-control process itself and the review cadence. The failure mode is nobody owning the caps, which is how they end up set by whoever configured the integration.

Sources & references

Where the claims come from.

Legal and regulatory points trace to the primary sources below. Richpanel's audit facts come from the reports named in the text, available under NDA. Buyer quotes are verbatim from recorded evaluation calls and are anonymized, because those companies have not cleared their names for publication.

  1. Moffatt v. Air Canada, 2024 BCCRT 149. British Columbia Civil Resolution Tribunal, February 2024. Cited for the “separate legal entity” submission, the tribunal's rejection of it, the finding of negligent misrepresentation, and the $650.88 award. canlii.org, 2024 BCCRT 149
  2. Regulation (EU) 2024/1689 (the AI Act), Article 50. Transparency obligations for AI systems that interact directly with natural persons, applicable from 2 August 2026 under the Act's transitional timetable, with penalties for this class of breach under Article 99. The European Commission published guidelines on the Article 50 obligations in 2026. artificialintelligenceact.eu, Article 50 and European Commission FAQ
  3. Tallam, K. (2026). “A Five-Plane Reference Architecture for Runtime Governance of Production AI Agents.” arXiv:2606.12320, June 2026. Cited for the systems-level statement of the composition problem: that production agents execute sequences of individually permitted actions capable of transforming a business process without explicit authorization. arxiv.org/abs/2606.12320
  4. NIST AI Risk Management Framework 1.0 (NIST AI 100-1), January 2023. The voluntary US framework whose GOVERN function is the layer the controls in this guide operationalize for a customer service context. nist.gov, AI RMF
  5. ISO/IEC 42001:2023, Artificial intelligence management system. The certifiable AI management system standard your security reviewer may reference. Noted for completeness; Richpanel does not hold it and does not claim it. iso.org, ISO/IEC 42001
  6. Richpanel third-party audit reports and sub-processor register. SOC 2 Type II by Percilchofe CPA LLC (1 May to 31 October 2025, unqualified opinion, zero exceptions across approximately 90 controls); HIPAA audit by Scrut Automation against NIST SP 800-66 Rev 2 (December 2025, zero non-compliant findings); GDPR audit by Scrut Automation (December 2025, 55 controls across 10 domains, zero non-compliant findings). Reports under NDA on request via the trust portal; the live register is at /trust/sub-processors.

Version history, v1.0 (2026-08-04): initial publication. Covers the five-control framework (permissions and caps, data boundary, audit trail, escalation, change control), the ten-question vendor checklist, and the Richpanel disclosure including where the answer is no. Regulatory positions and audit dates are a snapshot as of August 2026. This guide is informational and not legal advice.

Keep reading

The three layers this one sits on.

Bring the checklist. We will answer it on the call.

Ten questions, one 30-minute call. We will show you where permissions and caps are configured, what an audit log entry looks like for a real refund, and how a change gets reviewed and rolled back. Bring your security reviewer.

Book my 30-min demo →