Agent demonstrations often centre on planning, memory and tool use. Once the system can update a record or send a message, it occupies a role in an operating process. Review the action beyond the chat window: what changed, who can see it and how it can be undone.

An agent that can contact a customer, run code or approve a refund has authority. The investment case should specify that authority action by action, including the handoff when the system reaches a limit or leaves a task half-complete.

The operating envelope

Start with one purpose, a named set of tools and approved data sources. Mark which actions can be undone and which require confirmation. Define where uncertain cases go. The operator also needs a usable history of what the agent attempted before the handoff.

  • Which records may the agent read, and for how long may it retain them?
  • Which actions can run without a person confirming them?
  • What amount, destination or volume triggers a second check?
  • Which information stays inside the approved system boundary?
  • How does an operator restore state after a partial action?
  • Who receives an exception during and outside business hours?

These answers shape the technical build and its running cost. Human approval may contain the riskiest action, though every approval adds labour and delay. Narrow permissions reduce the initial feature set and leave a clearer incident trail for support staff.

Evaluation beyond the staged conversation

A fluent conversation reveals little about an unavailable tool or a half-completed action. Test ordinary tasks alongside missing context, conflicting instructions and cases where the agent should stop. Record whether staff detect the error, how much state changed and how long restoration takes.

The budget pays for the full workflow, including its controls. Recalculate the projected benefit with confirmation steps, exception queues and the support burden observed during the test.

Named ownership

Assign authority over permissions, incidents and suspension. Providers and implementation partners may support the system, but someone inside the operating organisation still needs the right and the time to change or stop it.

A proposal for a general digital colleague leaves the first approval almost unbounded. An agent limited to drafting replies and creating unsubmitted tickets gives the decision owner a defined scope. The first live review can then examine the draft quality, exception queue and staff correction time.