Release Checklist

Before you take an agent to production, review the settings that protect your users, your data, and your budget. This guide walks through them section by section, with a short checklist at the end of each step.¹ Most settings live in the Agent Design Studio; a few are organisation-wide, under Settings or Billing.

1. Access and permissions

Review the whole Usage & Permissions tab. See Usage and Permissions for the detail.

• API keys — give each key the minimum permission it needs, and remove unused ones. Prefer agent-scoped keys over global keys such as main.
• User and group restrictions — for sensitive agents, enable the Administrator/Auditor/Executor toggles and assign only the people or groups that need access.
• Restricted domains — for embedded agents, limit the origins allowed to call the agent.
• Execution limits — set an executions-per-minute cap and a monthly usage limit. For Assistant agents, also consider per-conversation and message limits.
• Log retention — confirm the retention period matches your compliance requirements.

Checklist:

• API keys reviewed; minimum permission each, unused keys removed
• User and group restrictions configured for sensitive agents
• Allowed domains restricted (for embedded agents)
• Execution limits set
• Log retention period confirmed

2. Content safety

Decide whether the agent needs Guardrails or PII obfuscation, both in the Content Safety tab. This matters most for external-facing agents or anything handling personal data.

• Guardrails — enable if the agent should not process or produce certain content categories (hate, illicit acts, self-harm, and similar) for user input and/or responses.
• PII obfuscation — enable if users may share personal or financial data, so sensitive details are masked before reaching the model.

Checklist:

• Guardrails decided (and configured if needed)
• PII obfuscation decided (and configured if needed)

3. Reliability

Keep the agent running under model failures and traffic peaks.

• Fallback model — a fallback model keeps your agent running when the primary model fails or is unavailable. In the agent's Model section, set a fallback model where your plan allows it. We recommend an equivalent model from a different vendor, so a provider-wide outage does not take both offline at once.
• Rate limits — your plan sets organisation-wide rate limits per second, minute, hour, and 12 hours. Check they are sufficient for your expected traffic. High-resource endpoints share the same counter. View your limits under Billing > Your Plan; if they are too low, upgrade the plan or contact support before launch. See Rate Limiting.

Checklist:

• Fallback model configured (where the plan allows)
• Fallback is an equivalent model from a different vendor where possible
• Plan rate limits reviewed against expected traffic
• Plan upgraded or support contacted if limits are too low

4. Cost controls

Services are pay-as-you-go, so agents consume credit while they run.

• Auto recharge — enable Auto Recharge to top up automatically below a threshold, or agree a manual top-up process.
• Usage limits and alerts — under Settings > Usage Limits, set daily/monthly/annual caps, add scoped limits where you need finer control, and configure alerts (e.g. at 80%) so admins are warned before a cap is reached.

Checklist:

• Auto Recharge enabled, or a manual top-up process agreed
• Organisation usage limits configured
• Scoped limits added where needed
• Threshold alerts configured

5. Analytics and observability

These settings are optional but recommended for visibility into how the agent is used.

• User and channel identifiers — send a userIdentifier (and groupIdentifier for AI Proxy agents) and a channel with your requests. These attribute usage and cost, drive scoped usage limits, and segment your statistics.
• Insights — turn on Insights to automatically analyse a sample of conversations and extract structured data (sentiment, topics, custom fields), each linked back to its source conversation.
• User feedback — when using the Chat Widget, enable feedback collection from the channel configuration to track response quality once the agent is live.

Checklist:

• userIdentifier / groupIdentifier and channel sent with requests
• Insights enabled with the fields and sample rate you need
• Feedback collection enabled in the Chat Widget channel (if used)

Before launch

• Agent tested end to end with production-like inputs
• Agent validated with test plans in Agent Quality Studio
• Verify all intended changes have been applied to the agent's Published version

Full checklist

Copy the list below to track your review in one place.

Access and permissions
- [ ] API keys reviewed; minimum permission each, unused keys removed
- [ ] User and group restrictions configured for sensitive agents
- [ ] Allowed domains restricted (for embedded agents)
- [ ] Execution limits set
- [ ] Log retention period confirmed

Content safety
- [ ] Guardrails decided (and configured if needed)
- [ ] PII obfuscation decided (and configured if needed)

Reliability
- [ ] Fallback model configured (where the plan allows)
- [ ] Fallback is an equivalent model from a different vendor where possible
- [ ] Plan rate limits reviewed against expected traffic
- [ ] Plan upgraded or support contacted if limits are too low

Cost controls
- [ ] Auto Recharge enabled, or a manual top-up process agreed
- [ ] Organisation usage limits configured
- [ ] Scoped limits added where needed
- [ ] Threshold alerts configured

Analytics and observability
- [ ] userIdentifier / groupIdentifier and channel sent with requests
- [ ] Insights enabled with the fields and sample rate you need
- [ ] Feedback collection enabled in the Chat Widget channel (if used)

Before launch
- [ ] Agent tested end to end with production-like inputs
- [ ] Agent validated with test plans in Agent Quality Studio
- [ ] Verify all intended changes have been applied to the agent's Published version

Footnotes

1. Configurations may vary depending on the agent type. Adjust accordingly. ↩