The in-app AI agent launch checklist
Twenty go-live checks for an in-app AI agent, grouped into identity, actions, testing, data and running it.
Quick answer
Before an AI agent goes live inside your product, check five areas. Identity: it acts as the signed-in user, never an admin key. Actions: only the ones you chose, each described clearly, and every change confirmed with the real values. Testing: on a test copy of your app, not production data. Data and abuse: what is stored, for how long, and limits per user. Running it: logs of every action, alerts on failures, and a way to switch it off in one step.
Twenty checks, in five groups, for the week before an AI agent goes live inside your product. Copy them into your launch document and tick them off. Each one comes from something that goes wrong when it is skipped.
1. Identity
- The agent acts as the signed-in user, with their own session or token, never a shared admin key.
- The user's identity comes from your backend, never from anything typed into the chat.
- Signing out ends the agent's access and clears the conversation on that device.
- A user with the fewest permissions asking for an admin-only action is refused by your own API.
2. Actions
- The agent can use only the actions you chose, listed somewhere you can read.
- Each action has a plain description of what it does and when to use it.
- Every action that changes data stops on a confirmation showing the real values.
- You can switch off one action, or the whole agent, without a deploy.
3. Testing
- It has run on a test copy of your app with test data, through the same API it will use live.
- You have tried the ten requests users ask most, and each one does the right thing.
- You have tried requests it should refuse, and requests worded to trick it.
- When it cannot do something, it says so rather than guessing.
4. Data and abuse
- You know what is stored, where, for how long, and who can see it.
- Your privacy policy names the AI processors involved.
- Each user and each visitor has a request limit, and the chat box is protected from bots.
- One user's data can be deleted on request.
5. Running it
- Every action is logged with who asked, what ran and the result.
- Failed actions alert someone, rather than only appearing in a log.
- Someone owns reading what users asked for that it could not do.
- You have decided how you will measure whether it is working, before launch.
Where to read more
- Can an AI agent act as the signed-in user?
- Confirming destructive actions
- Testing an AI agent without production data
- Protecting the agent on your site from abuse
- Are in-app AI agents GDPR compliant?
- Measuring an AI agent pilot
- Who approves an AI agent going live
Verb, which I build, is designed to pass this list out of the box: it acts as the signed-in user, uses only the actions you allow, confirms every change, runs on a test copy first and logs every action.
Frequently asked questions
What is the most important check before launching an AI agent?
That it acts as the signed-in user with their own permissions. Almost every serious failure, one customer seeing another's data or an action nobody was allowed to take, comes from an agent running with more access than the person asking.
Should an AI agent ask before every action?
Before every action that changes data, yes, showing the real values it will use. Reading and looking things up can run straight away, which keeps it fast for questions.
How do I test an AI agent before launch?
On a test copy of your app with test data, through the same API it will use in production. Try the requests your users really make, the ones it should refuse, and requests worded to trick it, and check every action in the log.

Vishnu, founder of AskVerb @itsvishnups
Building the AI agent that lives inside a SaaS product and does the work its users ask for. Writing down what I learn along the way, including the parts I got wrong first.