AI workflow testing playbook

Seven Lessons for Testing AI Agents in Real Workflows

AI agent testing is moving beyond the demo. The question is no longer only whether an assistant can produce a convincing answer. Teams need to know whether it can complete a defined job, use the right sources and tools, follow policy, stop safely, and improve without creating new failures.

Two product announcements made that shift especially visible this week. On July 22, OpenAI introduced Presence, describing production agent deployments built around specific jobs, simulations, evaluations, approved actions, escalation rules, and controlled updates after launch. On July 21, Salesforce announced deeper Agentforce observability, including session context, custom quality scores, source-linked traces, error locations, and multi-agent workflow views.

Those releases come from vendors describing their own products, so they should not be treated as independent proof that any particular agent will work for your organization. They are useful evidence of where production tooling is concentrating: workflow-specific evaluation, traceability, escalation, and ongoing review. A July 2 practitioner discussion on Reddit echoed the same buyer concern in less formal language. That discussion is an audience signal, not factual authority.

The practical conclusion is simple: test the workflow, not just the model. These seven lessons turn that principle into an operating plan.

For a broader systems view, review the ServingIntel integrated operations platform.

1. Define one job and its acceptable outcome

Start with a job narrow enough to describe in one sentence. "Resolve a billing question using approved account fields and company policy" is testable. "Help customers" is not.

Write the successful outcome before choosing test cases. Include what the agent should produce or change, which system may be touched, what evidence must support the action, and when a person must take over. This prevents the team from moving the goalposts after seeing a polished demo.

OpenAI's Presence announcement says deployments begin with a specific job and only the knowledge and system access required for it. That is also a good evaluation boundary. If the workflow cannot be defined clearly, a pass rate will not mean much because the team has not agreed on what counts as a pass.

2. Build the test set from real work, not ideal prompts

A useful test set should resemble the messy work the assistant will actually receive. Pull representative cases from approved, de-identified historical examples when possible. Include normal requests, incomplete requests, unusual wording, conflicting records, stale policy references, and requests that fall outside scope.

For neutral background and operating context, see NIST AI Risk Management Framework.

Do not let the vendor or builder select every example. The team that performs the work should contribute cases that are easy to mishandle. Frontline staff often know which exceptions create rework, customer frustration, or risky shortcuts.

Keep each case tied to an expected outcome and an explanation. That explanation matters when two reviewers disagree. A test set without a decision key becomes a collection of anecdotes rather than a repeatable evaluation.

For restaurant or senior living operations, use cases should reflect actual service timing, role boundaries, resident or guest sensitivity, and escalation paths. ServingIntel's overview of support operations is useful context for mapping who owns the next step when an automated workflow cannot continue.

Related infrastructure planning is available in ServingIntel POS hardware guidance.

3. Score the path, not only the final answer

An agent can reach a plausible answer through the wrong path. It may use an outdated source, call the wrong tool, repeat steps unnecessarily, omit an approval, or take an action before checking a required condition. A final-answer score will miss those failures.

Score the workflow in separate dimensions:

• Did it identify the user's intent correctly?
• Did it use the approved source and the current policy?
• Did it call only the tools required for the job?
• Did it preserve the correct sequence of checks and actions?
• Did it request approval when required?
• Did it produce the right outcome in the required format?
• Did it avoid unnecessary side effects?

Salesforce's July 21 observability release highlights this need for deeper context. Its session model captures user input, planner decisions, prompt and action flows, gateway inputs and outputs, errors, and final output. The vendor-specific implementation will vary, but the broader testing lesson is portable: the path contains evidence that the result alone cannot provide.

4. Test data, tools, and permissions as separate failure points

When an agent fails, the model is not always the cause. The source may be missing, the connected system may return an error, a permission may be too broad or too narrow, a field name may have changed, or the workflow may have handed the agent contradictory instructions.

A complementary portfolio perspective is available in the ServingIQ AI decision-log guide.

Design tests that isolate each layer. Give the agent a valid request with a missing source. Give it a valid source with a failed tool call. Confirm it behaves differently when it has read-only access versus permission to write. Change one input field and see whether the failure is visible or silently absorbed.

This separation makes remediation faster. If the problem is a stale knowledge article, changing the prompt is unlikely to solve it. If the problem is an overbroad permission, adding more examples will not reduce the access risk.

Record the system, data source, permission set, prompt or policy version, and model version used for each evaluation run. Without that configuration record, a later pass may not be comparable with the original failure.

5. Exercise edge cases and human escalation deliberately

An escalation route is not tested merely because it exists on a diagram. Trigger it.

Use the following resource when assigning escalation and recovery ownership: ServingIntel support resources.

Test missing required information, conflicting policies, uncertain identity, an unavailable system, a request outside scope, a proposed high-impact action, and a case where the correct response is to stop. Verify that the assistant explains why it stopped and passes useful context to the human queue.

OpenAI says Presence simulations and graders check whether an agent reached the right outcome, followed policy, used tools correctly, and escalated when appropriate. That list is a good starting point for any platform. The test should also confirm that the receiving person gets the original request, sources checked, action attempted, failure reason, and recommended next step.

For resident-facing or guest-facing workflows, the fallback must match the operating environment. A generic "contact support" message is not enough when timing and ownership matter. Review the handoff against the real senior living dining workflow or other service process the assistant is meant to support.

For additional independent reference material, review NIST AI Resource Center.

6. Keep traces that a reviewer can actually investigate

Logs should help a reviewer reconstruct one session without guessing. At minimum, capture:

• the request and relevant context;
• the sources retrieved;
• the decisions or classifications made;
• the tools called and their results;
• approvals requested and received;
• the final action or output;
• errors, retries, and escalation; and
• the configuration version.

Restrict access to those traces because they may contain the same sensitive information used in the workflow. Set a retention period based on the consequence and investigation needs of the use case.

Salesforce's release describes source-linked session views, error locations, and multi-agent traces. Those features reinforce an important distinction: a dashboard that reports volume is not automatically an investigation record. Reviewers need enough context to see what happened, where it went wrong, and whether the same condition is recurring.

7. Compare changes against a stable baseline

Agent behavior changes when the model, prompt, policy, source data, tool, permission, or workflow changes. Treat each material update as a new version.

For another practical workflow in the portfolio, read the SI Receipt control checklist.

Keep a stable baseline test set and run it before and after the change. Add the new failure case without removing older cases. Compare results by dimension, not only by an aggregate score, so an improvement in tone does not hide a decline in policy compliance.

Then roll out in stages. Start with a limited audience or low-consequence workflow, watch production sessions and escalations, and expand only when the evidence supports it. OpenAI's announcement describes teams testing proposed changes against the production version before approving a controlled rollout. The same pattern applies even when the tools are different: propose, test, compare, release narrowly, observe, and retain a rollback path.

A minimum viable workflow test brief

Before launch, a one-page brief should identify:

For additional restaurant and senior-living technology context, consult ServingIntel News & Insights.

• the single job being tested;
• the owner and reviewers;
• the approved systems, sources, and permissions;
• the real-case and edge-case test set;
• the expected outcome for each case;
• the path-level scoring dimensions;
• the required escalation triggers;
• the trace location and retention period;
• the baseline configuration and pass criteria; and
• the staged rollout and rollback plan.

The brief does not replace detailed technical testing. It makes the decision visible to operators, security partners, knowledge owners, and leaders who need to agree on what "ready" means.

The standard is explainable, repeatable evidence

No finite test set can prove that an AI agent will never fail. The goal is to replace confidence based on a demonstration with evidence based on the real workflow.

A production-ready evaluation should be explainable: reviewers can see why a case passed or failed. It should be repeatable: the same cases can be run after a change. It should be operational: tool use, permissions, escalation, traces, and rollback are tested alongside the answer.

That is the difference between testing whether an agent looks capable and testing whether a team can operate it responsibly.

For a final neutral reference point, consult CISA artificial-intelligence guidance.

Sources

• OpenAI, July 22, 2026
• Salesforce, July 21, 2026
• Reddit r/AI_Agents discussion, July 2, 2026