Implementing AI in business workflows

Key takeaways

• Discover requirements through cases and people. Procedures, prior outputs, system records and conversations reveal different parts of the work. Resolve missing rules with the people authorized to decide them.

• Define completion first. Specify the business result, records that must change, information the customer needs and responsibility for unfinished work.

• Allocate model discretion deliberately. Interpreting language, selecting the next step and authorizing a business action are separate design decisions.

• Build the application around durable state. Model outputs, tool requests, committed changes and outstanding obligations need distinct representations.

• Require evidence for additional flexibility. Compare architectures on the same cases, permitted actions and outcome criteria, including the human work each leaves behind.

• Evaluate the whole obligation. A valid response, a correct transaction and a resolved case establish different things. Measure their costs and failure modes separately.

Customer-support staff combine a customer’s account of a problem with order records, company policy and the actions available in several systems. Language models can help interpret the request, gather evidence and communicate the result. Completing the work also requires identifying the intended purchase, applying the rule and recording the remedy.

A convincing refund confirmation with no recorded refund leaves the customer waiting. A correctly recorded refund whose confirmation is lost leaves the customer and the next operator uncertain. The application needs to handle both the business action and the communication that follows it.

Implementing AI therefore requires decisions about the surrounding application: what information it can access, which choices software makes, where models contribute judgment, how actions change records and how interrupted work resumes. Those decisions determine whether useful model capabilities become a dependable business process.

This paper follows the work from discovering its requirements to implementing and evaluating a system. A hypothetical support application and a shorter document-review example show how incomplete rules become design decisions, how predefined orchestration compares with model-selected execution, and what evidence supports a completed outcome. The objective is to choose an architecture whose flexibility earns its operating cost.

Discover requirements from procedures, completed work and unresolved cases

A procedure rarely answers every question an implementation needs to resolve. Start by examining the policy documents, forms, system records and outputs already used for a bounded piece of work. Include cases completed normally, cases corrected or reopened, and cases still awaiting a decision. An accepted report can reveal the evidence a recipient expects; an unresolved case can reveal a dependency that the written procedure omits. Historical acceptance is itself something to examine, not an automatic reference label.

Walk through those cases with the people who perform the work. Ask what they inspected, what made them seek another source, which alternatives they rejected and who could authorize the eventual action. Observe the work where possible. The manager can clarify priorities, capacity and accountability, while the people receiving the output can identify omissions that merely move effort into their queue. Asking everyone for an ideal process diagram would miss these differences.

Treat the sources as evidence that can disagree. A policy may assign a decision to one role while records show another person routinely making it. A timestamp may record data entry rather than the underlying event. The Process Mining Manifesto emphasizes event quality and meaning, and identifies incomplete records and incompatible identifiers as obstacles to reconstructing a process. A recorded path therefore needs interpretation; it does not automatically establish the required or authorized path.

Distinguish information that has not been recorded from judgment that has not been articulated. A clerk who recognizes an unusual quantity may be applying a missing unit conversion that can become explicit data. Deciding which conflicting account of a delivery deserves further investigation may require examples and continued expert review. Ask the person to demonstrate the next check. Do not assume that a fluent description has captured all of their expertise or that every judgment can become a rule.

Turn an ambiguity into a requirement and a test

In the hypothetical support application developed below, suppose the written refund policy describes eligibility but does not explain what happens when a customer changes the selected order after confirming a remedy. This is a constructed discovery exercise, not an observed client interview.

The question for the process owner is whether the earlier confirmation still authorizes an action against the newly selected purchase. For this design, the answer is no: changing the order invalidates the proposal, requires another eligibility check and requires fresh confirmation. That decision changes the application. It needs an identifiable request revision, a proposal bound to one order, and execution that rejects a stale proposal. The evaluation includes a customer changing the order between confirmation and execution; the old purchase must remain unchanged.

Keep that chain inspectable: the source of the ambiguity, the accountable decision, its scope, the affected records or permissions, and a case that can demonstrate a violation. For document review, the corresponding questions concern which document versions govern, which outputs the recipient needs and who can resolve missing evidence. A later filename is not an authority rule, and a generated summary cannot supply an absent amendment.

The discovery output should identify required information and sources, accepted output examples, prohibited actions, exception ownership and unresolved decisions. Use it to construct the initial case set and rubric. Keep ordinary workload sampling separate from deliberately selected difficult cases, and retain cases not used to refine the implementation. Discoveries during testing can change the requirements, with the responsible person’s decision recorded rather than silently absorbed into a prompt.

Define the result before assigning work to a model

Begin with a case entering the existing process. Identify the requested result, available evidence, decisions, records to change and exceptions requiring another person’s involvement. Translate that investigation into a completion contract: facts that must hold when the obligation has been discharged, together with the owner and status of anything still pending.

This exercise can remove the apparent need for AI. If customers contact support because delivery information arrives late, repairing the data feed may eliminate those contacts. If staff repeatedly copy identifiers between applications, a direct integration may remove the task. A clearer order-selection screen can resolve ambiguity before any model reads a message. These changes alter the workload itself and should remain candidates throughout implementation.

Conventional software belongs wherever the rule is explicit and the inputs are dependable. It can calculate refundable amounts, apply date windows, validate account ownership and update records. A predictive model can estimate which cases risk missing a service deadline and help allocate scarce reviewer capacity. Its probabilities need evaluation against observed outcomes, and the queueing policy must account for capacity and the cost of delaying other customers.

A retention decision raises a different question. A model predicting cancellation does not by itself identify customers whose behavior would change after an offer. Choosing interventions requires evidence about their effects, potentially from randomized experiments and causal estimation. Eva Ascarza’s retention research illustrates this distinction through two field experiments. A language model may explain the resulting offer, while a separate decision procedure determines eligibility and expenditure. These are substantive architectural alternatives to asking one model to interpret, predict and decide everything.

Human handling also remains a sensible design for infrequent, poorly specified disputes where formalizing the process would cost more than the work saved. Implementation should make those cases easier to investigate and resume. A named queue, a concise evidence packet and a visible deadline can contribute more than another inference call.

Separate execution control from authority and persistence

Anthropic’s Building effective agents distinguishes workflows using predefined orchestration from agents whose models direct processes and tool use. Its examples also include an orchestrator-workers workflow that dynamically allocates subtasks. The distinction concerns the allocation of execution control; it does not imply that every intermediate choice in a workflow is fixed. The article presents engineering experience from Anthropic and its customers.

For an implementation, describe the actual decisions. Software may call a model to extract a request and then follow a prescribed sequence. It may route among predefined branches or repeat a check until a condition or iteration limit is reached. Alternatively, the model may choose tools within one phase, or plan and revise work across several phases. These are design choices that can be combined. They are not maturity stages.

Fixed orchestration does not make model outputs deterministic. The same programmed step can produce different classifications, proposed arguments or wording. A routing error can select the wrong predefined branch even when the branch itself executes exactly as implemented.

Execution control also leaves several independent questions unanswered. Action authority determines which records an application may change and under what conditions. Persistence determines what survives a restart. Task horizon determines how long work may continue and what resources it can consume. Human intervention determines when a person supplies information, approves an exception or takes ownership. A fixed workflow can automatically perform an authorized write; an agent that plans a lengthy investigation can remain read-only.

Andrew Ng’s 2024 series offers a complementary vocabulary: reflection, tool use, planning and multi-agent collaboration. These patterns describe ways of organizing model work. Reflection obtains criticism and revises an output; tool use connects the model to application functions; planning selects steps toward a goal; collaboration divides work among roles, potentially using the same underlying model. They can operate within different allocations of execution control.

Their value depends on the work. A response critic can compare a draft with a confirmed request and recorded result. Tool access can supply facts unavailable in the conversation. Planning can adapt information gathering to what has been found. Separate document-reading roles may help partition a large evidence set. For a short, explicit refund rule, several debating model roles add little obvious value. Agreement among them supplies no independent evidence that a transaction occurred.

Compare architectures on one support requirement

Consider a hypothetical local business application using synthetic records. Its refunds are entries in a fictional sandbox ledger; no money moves. The following sequence is a proposed design, not an observed model execution.

An authenticated customer writes, “The kettle arrived cracked. Please refund it.” The account has two kettle orders. O1042 was delivered 12 days ago; O0991 was delivered 45 days ago. Each contains one item with a paid price of $60 and no prior refund.

Fictional policy P3 permits a full item refund for reported damage within 30 calendar days of delivery, after the customer confirms the order and remedy. It requires no return in this example. Older claims require a support lead’s exception decision. The application calculates the amount from the order record.

For an approved refund, completion requires one $60 ledger entry against the correct item, no remaining refundable amount, an accurate confirmation available in the customer’s case view and a closed case with its supporting records. An exception awaiting a person remains open. The requested support outcome, permitted effects and completion checks are identical across the architectures below.

Architecture	How work is selected
Fixed orchestration with an LLM component	The model extracts the request within a simple prescribed sequence. Software presents an order selector, applies policy, obtains confirmation and executes the result. Unhandled variations go to staff.
Predefined routing and iterative checks	Software includes branches for ambiguous orders, changed requests and policy exceptions. Clarification or response revision follows explicit conditions and bounded loops.
Bounded model-selected tools	Within identification and clarification, the model chooses order lookups and questions. A typed proposal then enters a fixed policy, confirmation and execution stage.
Broader planning and execution	The model chooses and revisits subgoals across identification, policy investigation, proposal and communication. The application retains the same action rules and transaction checks.

Table 1. Four illustrative implementation designs for the same support obligation. The first two use software-directed control. Their tradeoffs with the model-directed designs are hypotheses to evaluate.

The first design may be sufficient when the interface captures the few required facts. The second can accommodate all the exceptions specified here if those branches are implemented. Both provide explicit transition rules, but adding new case types requires maintaining those rules.

Bounded tool selection becomes attractive when customers describe their orders in varied ways and the useful next question depends on previous answers. It may reduce rigid dialogue and staff clarification. It can also introduce unnecessary searches, repeated questions and variable response times. Broader planning offers more room to reorganize work, with a larger set of possible paths to evaluate and diagnose.

This narrow refund family gives little reason to prefer broad planning initially. A routed workflow is a reasonable starting hypothesis. Evidence favoring greater discretion would be a reduction in unresolved cases or staff effort on representative conversational variation, at acceptable latency and operating cost. If most work arrives through an order-specific form, conventional automation with optional language assistance may be sufficient.

Build the successful path through application components

Give the model usable information and a clear interface

The customer interface supplies an authenticated account and a durable case identifier. Account access belongs to the application. The model receives relevant order records through an adapter to the order system. An existing ticketing interface can expose the conversation, selected order, current status and next owner to support staff.

The adapter should return fields needed for the decision, including identifiers, dates, paid amounts, existing refunds and record versions. Policy access should return the applicable rule and its version. Unstructured policy explanations may require retrieval, but explicit refund conditions should be encoded in application logic where feasible. Store which records and policy version informed the proposal so a reviewer can reconstruct it.

The distinction between source data and instructions belongs in this interface. A customer message can request a remedy; it cannot redefine the refund policy. Retrieved material should retain its provenance and access restrictions. Give the model enough relevant context to resolve the case without exposing unrelated customer records.

Convert interpretation into an inspectable proposal

The initial phrase “the kettle” matches two orders. The application shows their dates and identifiers, and the model can phrase a short clarification. The customer selects O1042. The application checks that selection against the account, establishes eligibility under P3 and prepares the $60 remedy for confirmation.

The model’s structured output might identify the requested action, candidate order, reported problem and missing information. Schema validation can ensure those fields have acceptable forms. It cannot establish that the chosen order is the intended one. OpenAI’s Structured Outputs announcement explicitly separates schema conformity from mistakes in the values generated. Refusals, incomplete outputs and invalid proposals need application paths of their own.

Before asking for confirmation, the application stores an execution proposal with an identity and revision. It contains the selected item, amount, reason, policy version, current request revision, relevant record versions and a stable operation identifier. The customer sees the order and remedy, and confirmation is recorded against that specific proposal. Staff can inspect the complete record. Execution checks that the confirmation still applies to the current proposal and request revision.

Execute a business operation and retain its result

Expose a narrow operation such as “record full damaged-item refund” through the application’s service layer. Within the authorized account, its handler first matches the operation identifier and immutable arguments. If that operation already committed, it returns the stored result. A new operation requires current ownership, policy eligibility or a valid approved exception, customer confirmation and no prior refund. The handler calculates the amount from authoritative records. The same handler serves programmed workflows and model-selected requests, giving either route the same business meaning.

For this local design, one database transaction records the refund, updates the item’s refundable balance, stores the operation result and creates a pending confirmation event. Either the required records commit together or the transaction rolls back. A separate reader can then inspect the resulting refund and balance.

The confirmation event preserves the remaining communication obligation. A worker uses it to create the customer-visible case message and records its completion. This is an application of the transactional outbox pattern: record the change and required follow-up together, then process committed follow-up work separately. Duplicate delivery still needs handling, for example by allowing one confirmation message per event identifier.

The response can now accurately state that the sandbox has recorded a $60 refund for O1042. A case-completion check reads the ledger, item balance and confirmation record before closing the case. In an application that actually moves funds, acceptance by a payment provider, settlement and customer notification would require separately defined states and evidence.

With independently operated order, payment and ticketing services, establish which system owns each authoritative record and what evidence establishes each operation’s result. For every consequential write, identify its authorization requirements, request identifier, retry guarantees and status-query interface, including how long duplicate requests are recognized. Persist the intended action before calling the service, retain the returned identifiers and results, and reconcile uncertain outcomes against the authoritative service. A local outbox can preserve the obligation to send a request; it does not establish that a remote payment completed. The retry contract must come from the receiving service. If its interface cannot support safe retries or verification, limit automated actions and assign unresolved cases to an operator. Pending payment, order and customer-communication updates remain durable work until their separate completion conditions are met.

A small application can use a relational database, a background worker and explicit case transitions. As long waits and recovery paths accumulate, a durable execution engine may reduce orchestration work. Temporal’s execution documentation illustrates recovery from recorded event history; its activity documentation still calls for retry-safe side effects. Durable execution preserves progress under its operating contract. Business operations retain their own correctness requirements.

Preserve intent when requests change and outcomes are uncertain

Two branches expose why conversation history alone is insufficient application state.

A changed request invalidates the prepared action

Before execution, the customer says, “I meant the older order.” The application records a new request revision and invalidates the O1042 proposal. O0991 is outside the 30-day rule, so the case enters an exception review with the two order references, the current request, the applicable policy and a specific decision for the support lead.

The reviewer can approve a $60 exception or decline it with a reason. That decision is bound to the current order and request revision. Approval returns the case to confirmation and execution; a decline requires an accurate explanation before closure. Merely assigning the case to the reviewer is a successful handoff, not resolution of the customer’s claim.

Request updates and execution must be coordinated. Otherwise a worker can act on the old proposal while another process records the correction. Within the write transaction, locks or conditional version updates protect the case revision and item state; request changes follow the same coordination protocol. A stale proposal returns a conflict for fresh evaluation. Closure also checks the current request revision so a later message cannot be silently closed. PostgreSQL’s transaction isolation documentation explains the concurrent-update problem. The transaction and constraint design must protect invariants across all affected records.

If the earlier operation may already have committed, the system first establishes that outcome. A later message cannot retroactively cancel a recorded effect. Any correction becomes explicit follow-up work with an owner.

A lost response requires reconciliation

In a separate branch of the ordinary O1042 case, the refund commits but its response is lost. The caller records a timeout observation and looks up the existing operation identifier. That observation cannot overwrite a committed result or a newer case state. It must not trigger a second refund request merely because the first call timed out.

The Amazon Builders’ Library account of idempotent APIs explains the mechanism: the receiver recognizes repeated attempts at one intended operation, and records the identity together with the effects atomically. Applying that contract here lets a retry with the same identifier and arguments recover the saved result. Reusing the identifier for a changed order must be rejected.

Two independently created operations could still target the same item. The sandbox therefore also enforces one full refund per purchased item inside the transaction. Operation deduplication handles repeated attempts; this business constraint handles duplicate intent arriving through separate cases or workers.

If an authoritative lookup cannot resolve the outcome, the case remains pending reconciliation. Recovery retries use bounded delays and an attempt limit, after which an assigned operator investigates. The customer sees the pending status. A restart resumes from persisted case, operation and follow-up records, rather than asking the model to infer what probably happened from its last message.

Design document review around evidence coverage

A separate hypothetical review asks whether supplier documents establish round-the-clock incident support for a proposed service. A brochure advertises continuous support, a service schedule specifies weekday coverage and a referenced amendment is missing.

The application inventories the supplied documents, extracts text while retaining page and document references, and records versions and service scope. Retrieval locates relevant passages; the model organizes claims, supporting evidence, apparent conflicts and unresolved questions. A useful output lets the reviewer open each passage and see which service and effective period it covers.

Parsing is one component of that workflow. Evaluate whether it preserves the fields, table relationships, qualifications and source locations needed for the agreed review. A high text-similarity score can coexist with a lost exception or a value attached to the wrong period. Reference transcription or reference passages can help isolate an extraction defect from a retrieval or interpretation defect. The completed review still needs its own evaluation against the original packet and review requirements.

Flexible searching may locate a misplaced amendment or reveal that the documents concern different service tiers. If the amendment remains unavailable, the review must preserve that evidence gap. An unspecified hierarchy of authority requires the responsible person’s decision. The review should identify what evidence or decision is needed next.

Here, the central checks concern retrieval coverage, source-to-claim support and completeness against the review questions. The ALCE research paper evaluates answer correctness separately from citation quality, including whether statements are supported and citations are relevant. A supported statement can still omit another document that would change the conclusion. Citation presence therefore cannot establish a complete review.

A small fixed packet may suit an explicit document-by-document procedure. A dispersed collection with cross-references may justify bounded searching or planning. The system can retain read-only access to business records while producing an editable evidence report. It still needs access controls, provenance, saved progress and an accountable reviewer, but payment-style transaction recovery is not its organizing problem.

Evaluate completed obligations and the work left behind

Evaluate components to locate defects. Check request extraction, order identification, retrieval coverage, schema validity, policy calculations and tool behavior. Evaluate complete cases to decide whether the design is useful.

For the support application, preserve three distinctions. A valid output meets its format contract. A correct business action affects the intended item for the permitted amount and follows the required confirmation or exception decision. A completed obligation also leaves the customer with accurate status and all required follow-up completed. Unresolved work remains open with an owner. These require evidence from different records.

The original tau-bench specification provides a useful precedent: it checks final database state and required response information, and measures consistency across repeated trials. It also explicitly acknowledges that a passing result can miss a policy violation such as acting without customer confirmation. Check both the resulting records and whether execution followed the required customer confirmation or exception approval.

For a proposed comparison, use the same initial records, policy, allowable actions and case distribution across architectures. Include ordinary cases, changed references, policy exceptions, concurrent requests and lost responses. Retain a held-out set of cases and vary conversational wording across repeated trials. Separate results by case type so improvements on common cases do not hide deterioration on exceptions. Inspect a sample of judgments against source evidence, including any model-based grading.

A checker should independently read the changed records and confirm the intended target, amount, uniqueness and completed communication. It should also detect unrelated changes. For cases requiring a person, record whether the handoff was correct, how long the case waited and the staff time needed to finish it. Report the backlog at the measurement cutoff; transferring difficult work cannot count as automatic resolution.

Human review needs an explicit purpose. Approval before a consequential write can prevent an unauthorized effect; sampling completed work can estimate residual errors but cannot prevent an effect that already occurred. Use the severity and reversibility of errors, applicable obligations, observed error patterns and the reviewer’s ability to detect them to choose the operating arrangement. An overall accuracy score or uncalibrated model confidence does not determine a safe review rate.

An audit should include accepted and withheld cases, not only cases the system flags. If difficult cases are oversampled, preserve their selection probabilities before estimating workload-wide rates. Check reviewer disagreements and errors after correction as well as the initial machine output. Mandatory human decisions and unresolved source authority remain constraints even where automation appears cheaper.

Compare outcome quality with model and tool costs, latency, staff clarification, review, recovery and attributable maintenance. The research argument in AI Agents That Matter supports evaluating accuracy jointly with cost. Business evaluation extends that comparison to the full case cohort, including spending on unfinished cases and the consequences of incorrect or delayed resolutions. Refund principal itself is a policy outcome, separate from the operating cost of handling the case.

As a synthetic illustration, suppose 1,000 cases require $300 of model and tool spending, 20 staff hours at $40 per hour, and $400 of allocated operating maintenance. Total handling cost is $1,500. If 900 cases are correctly completed by the cutoff, cost per correct completion is $1.67, with 100 cases still open. That figure is not lifetime cost or evidence of savings. A comparison needs the same workload, cutoff, quality standard and remaining-work accounting for the existing process.

A demonstration can make mechanisms inspectable and reveal specific defects. A benchmark can compare systems under its task distribution and grader. Neither establishes a company’s achievable savings, performance on its own customers or readiness for its operating environment. Those conclusions require evidence from the intended workload and integration.

Before limited live use, name the process owner who accepts the evaluation evidence and authorizes the initial case types and permitted actions. Match that scope to the staff actually available to review exceptions, reconcile uncertain writes and complete follow-up, with explicit responsibility when work transfers. In the integrated workflow, observe correct completions, errors, staff effort, waiting times and open queues against the agreed criteria. Define conditions for pausing automated actions or returning cases to manual handling, such as an unexplained payment result, an unauthorized action or a review queue exceeding staffed capacity. Use those observations to decide whether to change the design, continue within the initial scope or extend it. The appropriate initial scope depends on the workflow’s consequences and available recovery paths.

Maintain the system as models and business conditions change

Version model configurations, prompts, schemas, tools, policy rules and retrieval indexes together with evaluation results. A model change can alter tool selection or interpretation while leaving the API shape intact. Rerun representative cases and failure scenarios before extending its use, and retain a practical rollback or manual handling path.

Operational monitoring should connect failures to cases: unresolved operation status, growing review queues, repeated clarification, missed deadlines, unsupported explanations and rising cost per completed obligation. A useful incident record contains the relevant inputs, tool arguments, results, policy versions and state transitions. It should let the owner diagnose and resume work without exposing unrelated customer data.

These responsibilities continue established ML systems engineering. Sculley and colleagues’ Hidden Technical Debt in Machine Learning Systems explains how data dependencies and changes in the surrounding world create maintenance work beyond model testing. AI-supported workflows add conversational and action-selection behavior to that operating burden.

An implementation is ready to expand when its completed outcomes, residual human effort and failure handling justify the additional workload. That decision may favor a direct integration, a predictive model, a routed language workflow or a bounded agent. The architecture should make its contribution visible in the business result it delivers.