AI Agent Governance: Check the Sequence, Not Each Click

AI agent governance for a small business is checking the sequence, not each click. Five refunds of $900 can sit under a $1,000 cap and still empty the cash drawer for the day. Watch the whole run against the job you assigned. A unique login tells you who acted. The next job is making sure the path still matches that assignment.

If you make a person click allow on every tiny step, they get tired and click through. You have seen the same thing when a login keeps asking for another code. After the fifth prompt, the sixth one is cheap. Bill Fisher and Ryan Galluzzo at NIST (27 August 2026) warned that overusing that loop recreates the same failure.

I run Kief Studio with Brian, from Shrewsbury, Massachusetts. We are two people. We treat an agent like a junior clerk: they can pull files, they cannot empty the drawer without a named person at the gate. The actor still needs a name. Start with what a non-human identity for an AI agent is. If you and I were on a Zoom with last night's log open, I would not start with the model name. I would start with the one-sentence assignment, the dollar total, and the tool list.

Why checking each click still misses

A policy engine that asks whether this one transfer is under $1,000 will say yes five times. A person watching the same screen will feel busy and useful. The cash drawer still drops $4,500. That is how limits work when they are per action instead of per goal. The walkthrough on the vendor call will show one clean refund. Looking good in a meeting is not the same as being ready to refund a customer's card overnight.

Imagine we are looking at the petty-cash tin on a Friday afternoon, with the refund drawer next to the tickets. Policy says no single refund over $1,000. The agent issues five refunds of $900 to five ticket numbers that all look valid. Each click is inside policy. The sequence still spent a day's float. Sequence governance asks whether that set still matches processing ordinary returns, or whether the job has quietly become emptying the drawer. You do not need a new binder for that question. You need a run total sitting next to the assignment.

NIST's identity post is the foundation: unique agent credentials, no shared human passwords, no long-lived keys in a config file. A commenter on that post put the leftover gap cleanly. Authorization is granted at a point in time, then execution switches tools. The sequence can leave the original authority even when every hop looks valid. The office version is simple. The agent did what you said, just not what you meant.

A worked example: the printer invoice

You tell an agent: pay the overdue printer invoice. It finds three line items, three vendors, and a rush fee. Each payment is under your per-click cap. The rush fee was never in the PDF on the desk. Per-click governance approves all four. Sequence governance asks whether this set still matches the printer invoice on the desk. If the assignment said pay invoice 4411 and the log shows four vendors, the per-click engine can still look clean. The path is the problem.

Same pattern for a public post. Each sentence can pass a brand check. The thread as a whole can still announce a price you have not shipped. Approve the finished artifact. Rubber-stamping tokens is how a safe draft becomes a live price. People already fail this test without software. We judge fairness moment by moment. We feel the next click more than the total. Agents inherit that if we encode only moments. Encode the run. Store the assignment next to the trace. If the trace no longer matches the assignment, stop. Do not add a tenth permission prompt.

What to encode instead of more prompts

Intent in one sentence stored with the run. Example: pay invoice 4411. Do not store pay vendors. If you need and also to describe it, the job is still too big. That is the same cut I use when we write a first job for an agent: one sentence you could read out loud on a call without hedging.

A budget for the run, not only per tool. Dollars, records touched, posts published. Five in-policy refunds of $900 fail a $1,000 run cap even when they pass a $1,000 per-click cap. Write the cap on the same sticky note as the assignment. If the sticky note is longer than that, you are still designing per-click theater.

A flight plan of allowed tools. Fisher and Galluzzo point at approved plans so you are not asked for a password mid-task. Search the invoice folder. Read invoice 4411. Create a payment draft. Stop. If the agent reaches for the CRM export or the public social tool, the run ends.

A stop when the path leaves the plan. One stop. Not a new are you sure on every file. You review the log. You do not type your password into the chat to just finish. A named person still belongs on irreversible steps: wire, delete, public send. Looking things up can run on its own. The person does not belong on every search. Fatigue makes the irreversible click cheap. That is how five in-policy refunds get through before lunch.

Name the actor, then watch the chain

Without a non-human identity, the sequence log will say a person did the path. You cannot govern what you cannot name. A non-human identity is a login that belongs to the agent, the way a badge belongs to a contractor. If the refund log shows your name, auditors will ask you. If it shows refund-agent-01, you can revoke that badge without rotating the founder's password.

Without data governance, the agent will pick whichever copy of the invoice is easiest. Sequence control on dirty data is theater. Data governance here means you know which folder is the source of truth. The desk PDF, not last year's export in a shared drive named final-final.

CISA's 1 May 2026 Five Eyes paper told operators to start with low-risk, non-sensitive tasks and to keep agentic AI inside the existing security model. Sequence checks sit on that floor. The five-move translation is Five Eyes guidance for a small business. If the task is refund production orders, you are already past low-risk. Start with work where a mistake is easy to fix. Leave the drawer alone until the smaller job is boring.

When the agent turns around and asks you a question

The Model Context Protocol can ask a human for extra input mid-task. MCP is a way for an agent to call tools, the way a clerk might walk to another desk. Elicitation is the agent turning around to ask you a question. Useful when the agent is lost. Dangerous when it asks for a password.

NIST notes that MCP's own spec warns against using elicitation for secrets. If your small-team setup still pastes keys into the chat to just finish, you have left sequence governance. Put secrets in a vault the agent cannot read. Put irreversible tools behind a named person. Pin the tool list so it cannot grow under you. The office version of that pin is MCP security for a small business. Public utilities for lockfiles and similar checks live on kief.dev.

What a Monday morning review looks like

Open yesterday's agent log under the agent's name. Count refunds. Sum dollars. Compare the sum to the run budget. Compare the tools used to the flight plan. If five in-policy refunds emptied the drawer, the per-click engine did its job and the sequence failed. Write the stop rule that would have fired after refund two, or after $1,000 total, whichever comes first.

That review is how you audit what AI is actually doing. A demo of one click will not show the drawer emptying. Overnight production is a chain. The overnight version of this lesson is agents at 3am versus the demo. Whether the chain is even allowed is why AI pilots stall before production. After ten minutes of a polished walkthrough, people hand the agent a real job. Do not. Give it an internal FAQ first.

LTFI is how we hire a department that already treats tools as a stack with owners. You do not install a binary. You keep name, content, and data. Architecture notes for the control plane live on briansgagne.com. Vendor theater, if you are still being sold a dashboard of allow clicks, is covered in how to evaluate AI tools without getting sold. We do not add a seventh person to watch the clicks. We encode the run so two people can still be accountable for it.

Write a first rule on a sticky note. One sentence of intent. A dollar or record cap for the whole run. A human gate on send, delete, and pay. Log the chain under the agent's name. Practice the off switch once, the way you practice a fire drill, not the way you write a policy nobody opens. If the assignment said pay invoice 4411 and the log shows four vendors, write the stop that would have fired after the second payment, then practice turning the agent off on a quiet afternoon, before you need to.

Have it draft an internal FAQ, or summarize last week's tickets for the team. Give that job its own login. Cap the run. Put a person on send, pay, and delete. That is the version of AI agent governance a five-person shop can actually run this month, without a binder and without a prompt on every hop. Five Eyes already told you to start low-risk. Sequence governance makes that start honest. Leave the refund drawer alone until the smaller job is boring.

Related reading

• What is a non-human identity for an AI agent
• Five Eyes agentic AI guidance for a small business
• AI agents at 3am versus the demo
• Why AI pilots stall before production
• How to audit what AI is actually doing
• MCP security for a small business
• How to evaluate AI tools without getting sold

Frequently Asked Questions

Is per-click approval enough for AI agent governance?

No. Per-click limits miss totals and goal drift. Encode a run budget and a stored intent. Keep humans on irreversible steps only.

Will more human prompts make us safer?

Usually the opposite. Fisher and Galluzzo at NIST flag consent fatigue, the same pattern as too many codes on a login. Fewer, better stops beat a prompt on every tool call.

Do we need a unique login for the agent on a five-person team?

Yes. A non-human identity lets you revoke the agent without rotating a founder's password. Shared human logins make the sequence log lie.

How do five in-policy refunds get through?

Each refund sits under the per-action cap, so the engine says yes five times. Sequence governance caps the run, or stops after a count, so the drawer cannot empty on in-policy clicks.

What is a simple first rule for a small team?

One sentence of intent, a dollar or record cap for the whole run, and a human gate on send, delete, and pay. Log the chain under the agent's name.