In this website guide
An AI strategy should explain which business results an organization will pursue, which changes could produce them, and what it will need to deliver those changes. Its commitments include applications, the people and systems that support them, and the evidence required before further investment. A useful strategy connects these choices closely enough that a discovery about the work can change the investment plan.
This connection matters in both directions. An attractive application may require data access, evaluation skills, or operating capacity that the organization does not yet have. Developing those capabilities may also make other applications feasible. Funding projects without their supporting work leaves delivery exposed; funding capabilities without identifiable uses makes their value difficult to assess.
The approach in this guide starts with business objectives and the current operation, compares plausible alternatives, and investigates the uncertainties that could change the choice. Requirements discovery then identifies what people and systems must be able to do. Responsibilities, delivery arrangements, and staged funding follow from that evidence. The same approach can review an existing strategy, preserving useful investments while revising assumptions and commitments that no longer hold.
The scope includes predictive machine learning, generative assistance, and bounded agent workflows, alongside process improvements, conventional software, purchased products, and continued human handling. Each can be an appropriate destination. Increasing autonomy is a separate decision about delegated authority, justified by the work and its consequences.
1 Review the strategy through its decisions
Establish the business result and the current arrangement
Start with a business objective that identifies whose result should improve. Reducing the effort required to resolve a customer problem, improving replenishment decisions, and making a new advisory service viable imply different investments. "Adopt AI across operations" provides little basis for choosing among them. Clarify the decision or process through which the proposed change would produce the result, including what lies outside the team's control.
For an established process, reconstruct how work enters, reaches a decision, and leaves. Record the output the recipient needs, the people who authorize consequential actions, and the exceptions that remain unresolved. Examine volumes, active effort, elapsed time, quality, downstream rework, and existing costs. A queue can be slow because staff lack information, because authority is concentrated in one person, or because the underlying task is difficult. Each diagnosis points toward different remedies.
For a new offering, the baseline may be the customer's current alternative and the organization's decision to forgo the opportunity. Investigate demand, willingness to rely on the output, distribution, and fulfillment before assuming technical feasibility will create a business. An internally impressive prototype has limited strategic value if no one needs the proposed result at a sustainable price.
Make constraints explicit: available funding, delivery capacity, access rights, service obligations, unacceptable harms, and the authority the organization can legitimately delegate. Distinguish the business objective from the acceptance conditions that must hold while pursuing it. A shorter resolution time can be desirable while an unauthorized payment remains unacceptable. Financial attractiveness cannot offset a binding legal prohibition or an unavailable permission.
Separate portfolio choices from application choices
At the portfolio level, leaders decide which objectives deserve resources, how much uncertainty to investigate, and which capabilities to develop across initiatives. They balance immediate improvement, more substantial process change, and opportunities whose commercial value is still uncertain. No fixed allocation among these categories follows from the technology itself. The appropriate mix depends on the organization's objectives, resources, existing commitments, and ability to absorb failure or delay.
Within an application, the questions become more specific: what task should change, which method is suitable, which information may be used, and what output is acceptable? Model selection belongs here, after the decision has been defined. A stronger model does not resolve an unclear objective; equally, clear ownership and good data do not establish that a model can meet the required accuracy, latency, or cost. Technical feasibility remains an empirical question.
The portfolio must also account for scarce people. Several individually attractive initiatives can become infeasible together if they depend on the same reviewers, data owners, or engineers. Compare the work that the organization could actually deliver and sustain within its capacity. An initiative's priority should reflect the other work it delays and the capabilities it leaves available afterward.
Review existing commitments before replacing them
An existing strategy provides a starting set of assumptions to inspect. For each material commitment, identify the intended business result, the current alternative, the causal argument for improvement, the supporting evidence, and the next resource decision. Record whether its dependencies and responsible people are still available. Distinguish an idea awaiting investigation from an application already delivering useful work.
Then examine why performance differs from the original expectation. A disappointing pilot may have uncovered a technical limit, an unsuitable workload, or an omitted cost. Low use may reflect poor access or an output that does not help the intended recipient. Conversely, a useful application may need little more than continued support. These findings justify different actions: retain the plan, narrow or redesign it, investigate a consequential uncertainty, or stop further investment.
Evaluate future spending against future benefits while preserving assets that remain useful, such as data contracts, tested integrations, case collections, and staff expertise. Prior expenditure is not a reason to continue an unattractive project. It can, however, leave capabilities that reduce the cost of another initiative. The review should show which assumptions changed and what those changes imply for the next commitment, so that a new strategy does not discard accumulated knowledge merely to produce a new roadmap.
2 Compare alternatives and sequence the portfolio
Comparison requires a common result and an explicit scope. "Use an agent" and "improve invoice processing" cannot be compared until each becomes an operating proposal. Specify the incoming work, the steps being changed, the people who remain involved, and how exceptions will reach an accountable decision. Include the current arrangement on the same terms, with its delays, errors, and support costs.
Consider a hypothetical distributor whose unresolved supplier invoices delay reconciliation and sometimes threaten continuity of supply. Its objective is to resolve discrepancies with less total effort while preserving purchasing and payment authority. The initial candidate is gathering evidence and preparing discrepancies for review; changing bank details and authorizing payment are outside its scope. The following options illustrate different ways to address that result. They are constructed alternatives, not an account of a deployment.
| Approach | Proposed change | Question that could determine the choice |
|---|---|---|
| Process improvement | Require approved purchase amendments to be recorded where reviewers can find them. | Does missing documentation cause enough of the delay to justify changing the process? |
| Conventional software | Match dependable identifiers and amounts, and route explicit exceptions. | Are the inputs and matching rules reliable enough for the intended cases? |
| Predictive ML | Prioritize unresolved cases by the risk of missing a resolution deadline. | Does the resulting queue improve outcomes within available reviewer capacity? |
| Generative assistance | Extract proposed fields and draft an evidence summary for a reviewer. | Can reviewers check the evidence and complete the case with acceptable effort and quality? |
| Bounded agent workflow | Retrieve approved records and assemble an evidence packet using limited tools. | Does adaptive retrieval help enough to justify its access, evaluation, and support requirements? |
| Vendor product | Configure a purchased invoice or case-management product for the process. | Does the product meet the same requirements with acceptable integration, operating, and exit costs? |
| Retain human handling | Continue the existing process for uncommon or disputed cases. | Would formalizing these cases cost more than the improvement available? |
A vendor product is a sourcing choice and may contain several of the methods in the table. Likewise, an application can combine ordinary validation with prediction or generative assistance. Compare complete designs rather than assuming that choosing a method settles procurement, or that buying a product removes the need to change the surrounding process.
The target must match the decision. A model that predicts a late invoice resolution could help organize a queue, but its usefulness depends on which delays reviewers can influence. In customer analytics, predicting cancellation poses a different question from estimating whether a retention offer will change behavior. Ascarza's field experiments in prepaid wireless and membership renewal illustrate why high predicted churn risk need not identify the best recipients of a particular retention intervention. The practical implication is to establish the required evidence for the action: a forecast, a ranking policy under capacity constraints, or an estimate of an intervention's effect. The complementary customer-analytics paper and customer-intervention resource develop these distinctions further.
Identify dependencies and reusable investments
Dependencies change which alternatives can be pursued together and in what order. In the distributor example, making approved amendments accessible could support conventional matching, assisted review, and a purchased product. An evaluation case set could help compare those alternatives. Those investments have identifiable consumers and can be scoped accordingly.
Some dependencies are mandatory: the application cannot use records without an authorized access path. Others improve economics or reliability without being prerequisites: a shared retrieval service may become useful once several applications need the same sources. Keep that distinction visible. A broad platform should have a case based on credible demand, maintenance ownership, and the costs of serving different users. It need not precede every bounded application.
Assess shared investments at portfolio level, then use incremental costs when comparing applications that would consume them. Do not charge the full common investment to each project or credit every project with the same downstream benefit. If the first application must fund a reusable component alone, show that exposure and identify what would remain valuable if later applications do not proceed. An economical local solution may be appropriate while demand is uncertain; a well-scoped shared service may be justified where reuse is already clear.
Learning can also connect initiatives. A small extraction investigation may reveal the quality of records needed by a larger reconciliation project. State which decision that learning could change. Calling every pilot a foundation for future work would make it impossible to distinguish a useful investigation from an experiment whose results have no prospective user.
3 Investigate readiness for a bounded application
Readiness concerns a particular application, a proposed scope, and the next commitment. An organization can be capable of operating a forecasting service while lacking the permissions or reviewers needed for document assistance. It can also be ready to investigate a candidate without being ready to deploy it. Assessing the whole organization with one maturity label obscures these differences.
Define the candidate before assessing it. In the distributor example, "invoice automation" includes evidence gathering, reconciliation, negotiation, approval, and payment. Those activities have different requirements. Preparing a discrepancy packet for a purchaser is a more useful unit: it has identifiable inputs, a recipient, an output, and a boundary around permitted actions. That boundary can change if the investigation reveals a better design.
Inspect the conditions that could change the decision
Begin with the data the proposed application would actually receive. Determine whether relevant records exist, whether the team can lawfully and practically access them, and whether they can be associated with the correct case. Inspect document quality, missing information, timestamps, and conflicting versions. For predictive work, ask whether the intended outcome is observable and whether historical labels reflect the decision being studied. For document assistance, ask which sources are authoritative and how a reviewer can inspect them.
Next, examine task repeatability and variation. Frequency does not establish that correctness can be assessed. Experts may agree about routine cases while disagreeing about remedies, uncertain evidence, or unusual circumstances. Some disagreements can be resolved through explicit policy; others require continued professional judgment. An application can still support that judgment if the required contribution is clear. Open-ended work may need a rubric covering several defensible outcomes rather than one reference answer.
Test technical performance on relevant material. Existing benchmarks or supplier demonstrations can identify candidates worth investigating, but the proposed workload may contain different languages, document quality, event patterns, or error consequences. Requirements can exceed available model capability. When they do, the choice may be a narrower task, a different method, further research, or continued human handling.
Inspect integration and information authority together. Determine which systems must supply or receive information, which interfaces exist, which access is read-only, and which actions require approval. A reliable extract does not establish that the correct record can be updated safely. Identify the owner of each dependency and the effort needed to make it usable. Keep implementation details in the engineering plan; the strategic question is whether the necessary integration can be delivered and maintained at a worthwhile cost.
Revisit value after this examination. The benefit belongs to the work that actually improves, including downstream quality or timeliness. Review, correction, and escalation may add effort while also improving the result. Their contribution must be measured within the full process. If assisted drafting lets an expert spend more time on judgment and less on assembly, retaining expert review can be central to the business case.
Finally, examine consequences, authority, and human capacity. Ask what happens when an error passes the checks, whether exposure can be limited, and who can accept the remaining risk. Establish whether people have time to review, resolve exceptions, learn the new process, and support it. A design that assumes immediate expert attention for every difficult case depends on that attention being available.
Separate uncertainty from a constraint
Record what is established for the proposed stage, what remains uncertain, and what prevents progression. An uncertain extraction quality can be investigated on authorized historical cases. An incomplete data mapping may be a remediable engineering gap. A binding prohibition on the intended use or an access request that the responsible owner has rejected blocks that design unless the constraint legitimately changes. Record who can resolve each issue and what evidence or decision would do so.
Use the smallest investigation capable of changing the choice. This might be an authorized data sample, a walkthrough of difficult cases, an interface test, or a comparison of reviewer effort. Select ordinary cases to understand the workload and deliberately difficult cases to expose weaknesses. Keep their purposes separate: a collection enriched for difficult cases cannot establish ordinary failure prevalence without accounting for its selection.
A rubric can make judgments easier to compare, provided its categories have written meanings and supporting evidence. It remains a judgment aid. Require each assessment to identify the proposed scope, its evidence, the assessor, and the unresolved issue. Do not convert it into a probability of success, or average a binding constraint against attractive value. Numerical scores are optional; clear evidence and consequences are essential.
Independent assessments can expose disagreements worth investigating. Suppose the data owner considers amendment records complete, while purchasers report that concessions frequently remain in email. Inspect cases in which those accounts differ. The decision may change from "build automated reconciliation" to "repair amendment recording and test assisted evidence gathering." The useful output is the revised scope and the reason for it. Date that finding and revisit it when the records, permissions, workload, or proposed authority change.
4 Turn working knowledge into requirements
Readiness inquiries show where evidence is missing. Requirements discovery determines what the application must do and who can settle the unresolved choices. Begin with written procedures and the artifacts already used to perform the work: forms, records, correspondence, accepted outputs, corrected outputs, failed cases, and unresolved cases. Include the work passed to downstream recipients, since an apparently completed task can leave substantial checking or repair elsewhere.
Walk through cases with practitioners, then discuss priorities and authority with managers, process owners, data owners, and others accountable for the consequences. Ask practitioners to demonstrate the next check, explain why they consulted another source, and identify what would make them stop. A manager can clarify staffing and escalation responsibilities; a recipient can explain why an output that looks adequate is unusable. These perspectives answer different questions and should remain distinguishable when they disagree.
Treat records as evidence requiring interpretation. The Process Mining Manifesto identifies event meaning, missing records, and incompatible identifiers as obstacles to understanding processes. A recorded approval may show who entered a status without showing who made the decision. The requirement therefore needs an explicit authority rule; frequency in historical data alone cannot establish one.
Tacit knowledge also takes different forms. A unit conversion can become an explicit calculation. An experienced purchaser's suspicion that a concession is incomplete may require further evidence and human adjudication. Preserve that distinction when designing the application. Documenting examples can make expert judgment easier to communicate and evaluate without claiming to reduce all expertise to rules.
Resolve an ambiguity and define its acceptance check
Extend the hypothetical distributor example. Its written procedure says reviewers should reconcile invoices against the purchase order, but completed cases show negotiated price changes appearing in email. Some staff use the latest email; others require an approved amendment. A model cannot settle which practice the business authorizes.
For this illustration, the purchasing process owner decides that a price change is authoritative only after it is approved and recorded as an amendment linked to the purchase order. An email can supply evidence for review, but it cannot itself authorize reconciliation at the changed price. If records conflict or the approval is missing, a named purchaser must resolve the case. This is a constructed policy decision for the example, not a claim about a universal procurement rule.
The requirement is now specific. Each discrepancy packet must identify the purchase order, the governing approved amendment if one exists, the source of each proposed price, and any unresolved conflict. When approval is absent, the output must show that reconciliation is pending and assign the exception to the purchaser. The application may prepare the packet; it may not approve the concession, release payment, or alter bank details.
The acceptance check follows from that decision. Present a case containing an original purchase order and a later email proposing a lower price, with no approved amendment. The application must preserve the distinction between the authorized price and the proposed change, surface the email for review, leave the case unresolved, and route it to the purchaser. It must not report the invoice as reconciled. A separate case with an approved, linked amendment checks that the packet shows the authorized price and its approval evidence without flagging a missing approval. Conflicting approvals test the explicit exception path.
This chain links observed ambiguity, accountable decision, requirement, and evidence of compliance. It also reveals work the strategy must fund: clearer amendment recording, access to the approval record, case preparation for evaluation, and reviewer capacity. If those costs undermine the proposal, revise the application or retain human handling under the clarified approval rule.
Keep unresolved requirements visible instead of silently embedding assumptions in prompts or configuration. Preserve the decision, its owner, the examples that explain it, and the circumstances that would require reconsideration. Retain evaluation cases that were not used to refine the implementation, and change requirements through the responsible owner when testing reveals a new issue. The complementary workflow implementation paper develops the technical treatment of execution, permissions, and reliable completion; this guide uses discovery to determine the commitments that implementation needs.
5 Develop the capabilities each commitment needs
Capability development should be attached to work people will have to perform. A proposed application creates concrete obligations: selecting a useful target, obtaining appropriate data, establishing acceptance criteria, building or integrating software, reviewing results, and sustaining the service. Map those obligations to the people who can perform them now, those who need supported practice, and those whose expertise must be obtained elsewhere.
This mapping prevents training from becoming a parallel activity with no delivery consequence. A course can build understanding, but the relevant test is whether participants can use it to make the decisions or complete the work their roles require. Tool usage and attendance provide evidence of participation. Demonstrated performance, with suitable support, provides evidence of capability.
Help executives make and revise commitments
Executives need enough understanding of the methods to distinguish a plausible application from an attractive demonstration. They should be able to ask how a proposed change affects a business result, what comparison supports that expectation, which constraints remain, and what the next investment would establish. They also need to understand the implications of approving access, changing a process, or delegating an action.
An executive program can combine technical concepts, implementation cases, and discussion of projects relevant to the participants' responsibilities. Stefan Jansen has delivered programs with this combination. In a strategy process, the practical output might be a revised initiative brief: an explicit incumbent, feasible alternatives, a justified next investigation, and a named decision owner. A later review can assess whether the resulting decisions improved, without assuming that attendance converted directly into projects or financial returns.
Use cases that expose the interaction of technology and organization. A prediction can be accurate yet fail to improve an allocation decision. A useful assistant can require more information access than the organization can grant. An intervention can affect a different group from the one a risk model prioritizes. Working through these choices gives leaders a basis for questioning proposals and revising assumptions as evidence arrives.
Give domain experts and users time to practice
Practitioners contribute the knowledge needed to define acceptable work, identify material exceptions, and recognize when a result needs further investigation. Develop this contribution through real or appropriately prepared cases. Ask people to compare outputs, explain disagreements, and demonstrate how they would check or escalate a result. Assess whether they can do this under the time and information conditions the proposed operation will provide.
For the distributor, a reviewer needs to distinguish a proposed concession from an authorized amendment and resolve the packet accordingly. Training should include missing approval, contradictory evidence, and a correct routine case. A session that only demonstrates successful extraction cannot establish that the reviewer understands the authority boundary. Support should remain available when unfamiliar cases appear, with a clear route to a process decision rather than an expectation that users improvise policy.
Build practice into normal work through time allowances, accessible guidance, and a place to report problems. Discuss how roles and workloads will change, including what will happen to released capacity. If staff are judged only by throughput, they may lack the time to investigate a doubtful output. If they must maintain both old and new procedures, that transition burden belongs in the plan. These are design decisions about work and incentives, not defects to be corrected by exhorting people to adopt AI.
Low use should prompt investigation of the intended users and cases. It can reveal poor fit, duplicated entry, inaccessible tools, unreliable output, or a task that occurs infrequently. A useful specialist tool may have a small user base. Measure whether it improves the relevant work before setting adoption targets or rewarding activity counts.
Build engineering and evaluation capability together
Technical capability includes data preparation, software integration, deployment, and the ability to diagnose failures. Predictive applications require attention to label quality, the information available at decision time, and validation that represents future use. Generative applications require evaluation of outputs against evidence and task requirements. Applications that use tools also need enforceable limits on information access and action. The particular combination follows from the design.
Evaluation is shared work between technical specialists and people accountable for the outcome. Domain experts establish material distinctions; engineers make them testable and traceable; an appropriately independent reviewer can challenge the assumptions and interpretation. Teams need to understand which results concern ordinary workload performance, which concern difficult cases, and which uncertainties remain. A favorable aggregate metric should not obscure a failure class that makes the proposed use unacceptable.
Develop these skills through delivery tasks: reconstruct a baseline, improve a data contract, adjudicate disputed examples, compare alternative methods, or investigate a failed check. Pairing, technical review, and targeted instruction can close different gaps. Hiring or specialist advice may be necessary where the organization cannot develop the required expertise within the decision window. Internal development and external expertise can complement each other.
Retain knowledge and provide usable support
Retain the artifacts that explain how the work should be done: approved requirements, annotated cases, data definitions, evaluation criteria, integration decisions, user guidance, and known limitations. Assign owners who will update them as the process changes. This makes learning available to new staff and future applications, while reducing dependence on one enthusiast or supplier.
Provide an approved route for experimentation and a clear way to request tools or access. Reports of unsanctioned use can reveal unmet needs, but they also require decisions about permitted information and activities. Assess the actual product, account, contract, configuration, and data involved. A deployment label alone does not settle the organization's obligations. Where restrictions are necessary, explain the applicable rule and provide a workable alternative when one exists.
Support capacity should be explicit: who answers questions, who resolves disputed outputs, and who can change the implementation or the process? A local champion can help colleagues practice and surface problems, provided the role has time and support. Enthusiasm does not confer authority to approve access or accept risk. The next step is to make those responsibilities clear enough that learning and delivery can lead to decisions.
6 Assign responsibility and support decisions
Responsibilities should follow the decisions the initiative requires. A smaller organization may assign several to the same person; a larger or more consequential application may require separation and independent review. The important property is that each decision has someone with the authority, information, and capacity to make it, including a route for resolving disagreement.
| Responsibility | Decision or obligation |
|---|---|
| Investment | Select objectives, fund the next stage, and decide whether to expand, revise, or stop. |
| Process and outcomes | Define acceptable work, resolve policy ambiguities, and own the business result. |
| Implementation | Design, build, integrate, and demonstrate that the proposed system meets its requirements. |
| Information access | Authorize appropriate data use and permissions, and resolve source ownership. |
| Risk acceptance | Assess remaining exposure and approve it within the relevant authority and constraints. |
| Adoption and work design | Provide practice, support, staffing, and changes to how people perform the work. |
| Ongoing operation | Maintain the service, handle unresolved work, and exercise the authority to limit or stop it. |
NIST's AI RMF 1.0 provides a useful reference for this allocation: GOVERN 2 addresses documented responsibilities, relevant training, and executive responsibility for risk decisions. Its framework is voluntary and is not presented as an ordered checklist. The allocation above is a practical design aid; it does not certify readiness or substitute for the obligations applicable to a particular use.
Keep technical decisions within their proper authority. An engineer can explain the consequences of a proposed threshold or permission, but implementing it does not authorize the underlying business policy. A sponsor can fund a project, but cannot waive a constraint outside their authority. Document consequential decisions at the point where they affect scope, acceptance, or exposure, and identify who can reconsider them.
Organizational structure should support those decisions with manageable coordination. An internal team may own a complete application. A small central function may maintain shared data access or specialist evaluation support while business teams own their processes. A federated arrangement may be useful where several units share infrastructure but differ in operating needs. None requires a universal center of excellence or a large permanent team.
Watch for the capacity constraints created by the arrangement itself. Central review can provide scarce expertise but become a bottleneck if every minor change requires the same meeting. Distributed delivery can move quickly but duplicate work or leave responsibilities unclear. Decide which decisions require common standards, which can be delegated, and what evidence the receiving person needs. Match review effort to the consequence and reversibility of the change.
An external specialist can hold an implementation or advisory role, while the client retains decisions about business priorities, process authority, information access, and acceptable exposure. Agree on this boundary before delivery starts. Confidentiality also belongs in the arrangement: public explanations of a method can use constructed examples without identifying which client received a particular implementation or exposing their internal choices.
7 Choose delivery and ownership arrangements
The delivery arrangement should keep information moving between people who understand the process and people who can test and change the implementation. Discovery continues as real cases reveal missing assumptions. Construction produces evidence about feasible performance, review effort, and integration. The arrangement needs a way to turn that evidence into authorized changes to requirements and scope.
Embedded engineering can make this feedback direct. An engineer working closely with practitioners can examine cases, test a bounded implementation, and bring emerging decisions to the process owner. This arrangement is useful when the work is difficult to specify in advance and access to people and systems is available. It also concentrates context in a small number of people, so availability, knowledge transfer, and eventual operation need explicit provision. Embedding an engineer does not transfer business authority to them.
An internal team can provide continuity and already understand the organization's systems. It may need specialist help with evaluation, a particular model class, or an unfamiliar integration. Vendor integration can be appropriate where a product fits the required workflow and the organization can configure and support it. Specialist advice can resolve a decision about feasibility, procurement, architecture, or risk without requiring the adviser to deliver the whole application.
Strategy-only work is legitimate when the immediate need is a portfolio decision, an independent review, or a specification for an established delivery team. A handoff can work when it carries adequate evidence and provides a route for revising assumptions. Its quality depends on what the receiving team can use and what happens when new facts appear. Continuous employment by the same person is one way to preserve context, not a requirement of successful delivery.
Decide what to own and what to obtain
Make build and buy decisions at the level of components and obligations. An organization might buy model access and case-management software, retain its business rules and evaluation cases, and commission a narrow integration. Strategic differentiation, access requirements, achievable quality, delivery capacity, and the cost of dependency all matter. Neither proprietary data nor a desire for control automatically justifies developing every component internally.
Procurement should test the product against requirements that matter to the application. Ask how the organization can evaluate representative cases, inspect relevant evidence, enforce permissions, respond to product changes, and obtain the support it needs. Examine data use, retention, subcontracting where relevant, commercial terms, and the practical ability to export records and move to another arrangement. The review must concern the actual offering and contract being considered.
Price the remaining client work. Configuration, access approval, integration, evaluation, staff practice, and exception handling can remain substantial even when the software is purchased. Conversely, a custom build entails maintenance and staffing obligations that a license comparison can miss. Compare credible operating designs over the same horizon, including transition and exit, before treating either option as cheaper.
Client ownership should be explicit in the agreement. Specify rights to custom code and configuration, access to repositories and deployment accounts, ownership or permitted use of data and evaluation assets, documentation, and responsibilities after delivery. Where a supplier retains proprietary components, document the resulting dependency and the information the client needs to operate or replace the service. Preserve credentials through appropriate access arrangements rather than depending on an individual's account.
Make the handoff testable
A usable handoff gives the receiving people the approved requirements and their rationale, representative and difficult cases, evaluation results, known limitations, integration information, operating instructions, and unresolved decisions with owners. It also states which changes need renewed approval and how the client can obtain help. The detail should match the work being transferred.
Test the transfer through practical work. The receiving team should be able to process a relevant case, investigate a failed acceptance check, and follow the agreed fallback with the access it will actually have. A document being delivered does not establish that these actions are possible. Where a future operator is not yet appointed, that remains a delivery dependency rather than an assumption that support will appear after launch.
These arrangements can be combined over the life of an initiative. Specialist advice may narrow a candidate, embedded work may resolve uncertain requirements, an internal team may take over implementation, and a vendor may supply a stable component. Preserve the decision record across those transitions so that commercial boundaries do not sever the feedback between evidence, requirements, and investment.
8 Fund evidence before expansion
Investment decisions should compare complete alternatives over a stated horizon. Include initial work, transition, integration, evaluation, training, review, exception handling, maintenance, and the operation of excluded cases. Include the consequences of errors that survive checking in both alternatives, using scenarios or exposure limits when their frequency or severity cannot be estimated credibly. Consider benefits in quality, capacity, timeliness, and risk as well as expenditure. State which estimates come from observation, supplier information, or assumptions, and identify the quantities that could reverse the choice.
Released time has value when it can be used productively. It becomes a cash saving only through a credible change in expenditure, such as avoided overtime, reduced external work, or a staffing decision. Additional sales should be valued with the relevant costs of serving them. Net benefit is the incremental benefit less incremental cost for the chosen comparison and period; a reported ROI also needs a defined investment denominator. These distinctions prevent activity or gross revenue from being presented as a return on investment.
Human review belongs on both sides of the comparison. It consumes resources and can improve correctness, interpretation, and the usefulness of the result. Its value depends on the reviewer, the evidence available, the task, and the conditions of work. Measure the complete assisted route against the incumbent, including corrections and downstream consequences. A design can be worthwhile while retaining substantial expert judgment.
The complementary economics white paper develops the comparison in more detail. Here the strategic task is to decide which uncertainty merits further spending and what evidence would justify the next commitment.
Give each investigation a decision to resolve
An early investigation should have a specific purpose. It might determine whether authorized records can be assembled, whether an existing product meets a crucial requirement, or whether reviewers can use a generated packet without recreating it. Identify the decision threshold: for example, the review effort above which assistance loses its economic advantage, or the demand below which a shared service cannot justify its fixed cost. Investigate quantities whose plausible range crosses that threshold. Define what finding would support progression, what would cause redesign, and what would end the investigation. If the answer will not change a decision, the work needs a different justification.
Evidence requirements should reflect consequence and reversibility. A low-cost change that can be withdrawn easily may justify a modest trial. Delegating a consequential action or committing to a costly integration requires stronger support. Consider the cost of the investigation itself, including delayed benefits and staff attention. Staging is useful when it resolves relevant uncertainty while preserving a real ability to change course.
For the distributor, the next question might be whether evidence packets reduce total reviewer effort while preserving the approved-amendment requirement. An initial test can use authorized historical cases without granting payment authority. If missing approval records dominate the work, the useful decision may be to improve recordkeeping before increasing automation. If a conventional integration achieves the desired result, a more elaborate workflow may not warrant further investment.
Specify what a bounded pilot must establish
A bounded pilot needs a defined workload, a comparison route, a budget and exposure limit, an owner, and a decision at completion. Identify which cases are eligible, who receives the output, what actions are permitted, and where unresolved cases go. Establish acceptance conditions before interpreting the results, including required quality, service level, review capacity, and constraints that cannot be traded against financial benefit.
Measure the whole route under comparable conditions. Track incoming work, accepted outcomes, justified rejection or no action, pending cases, corrections, active effort, elapsed time, and relevant downstream use. Include all attempts and exceptions in the economic picture. Audit whether apparently accepted outputs contain defects and whether rejected outputs were needlessly discarded. A pilot that counts only successful automated cases cannot establish the cost of serving the workload.
Where practical, assign comparable work to alternatives in a way that supports a causal comparison. The unit might be a team or period when reviewers, learning, or shared queues create interactions between cases. When using a before-and-after comparison, examine changes in staffing, workload, case mix, and learning that could explain the difference. The evidence can still inform a decision, with its limitations attached to the inference.
Use ordinary work to estimate ordinary performance and difficult cases to test consequential failure modes. Keep their results distinguishable. Preserve evaluation material outside the examples used to improve the system, and have disputed labels adjudicated by appropriate people. A short trial with no observed severe failures cannot establish that a rare severe event is acceptably unlikely. Exposure limits, enforceable restrictions, and a narrower scope may be necessary where measurement alone cannot provide the required assurance. The complementary evaluation white paper develops the design and interpretation of those checks.
Cost the fallback and test the next scale
The fallback needs an operating plan and a cost. Specify who takes the work, what evidence they receive, how quickly they can respond, and what happens if their queue fills. Estimate the staff time, retained systems, support, and delay involved in normal exception handling and in an interruption affecting many cases. Maintaining an alternative may be essential even when it is seldom used. A nominal "human fallback" without available people is an unresolved capacity assumption.
Before expansion, test whether the proposed level of demand can be served under the intended staffing. Immediate help from a specialist during a pilot may disappear after handoff. Review time can grow with the mix of exceptions, and downstream teams may be unable to absorb faster output. Either fund the conditions that made the pilot workable or evaluate the arrangement that will actually operate at the larger scale.
The investment decision should identify the preferred design, the acceptance conditions it meets, the incremental resources required, and the assumptions that could reverse the choice. An expansion can be limited to a case class or volume for which the evidence is adequate. A revision can retain useful components while reducing scope. Stopping can preserve findings and reusable assets without funding further delivery. Each is a legitimate outcome of a stage that answered its assigned question.
9 Use operational evidence to revise the strategy
Once an application is in use, its evidence should return to the portfolio decision. Compare realized results with the assumptions that justified investment: eligible volume, accepted quality, review burden, exception frequency, integration effort, available staff, and the use of released capacity. Examine whether changes come from the application, better inputs, staff learning, or a different workload. Operational detail supports the strategic review when it can change scope, resources, or priority.
Set review points around material decisions and changing conditions. Revisit the plan when a new case class is proposed, delegated authority expands, permissions or contracts change, a supplier changes behavior, or the people required for review and support become unavailable. Persistent rework, a serious escaped error, or a sustained increase in exceptions can invalidate the operating assumptions. Changes in business demand or the quality and price of alternatives can also change the preferred investment.
Review dependencies across the portfolio. A shared investment may have fewer users than expected and require a narrower remit. A reliable data source or a more capable internal team may make a previously deferred application feasible. Several projects drawing on the same experts may require sequencing even if each still has a positive individual case. These findings should change allocations and capability development together.
Keep the current strategy concise enough to use: the objectives being pursued, initiatives and alternatives under consideration, evidence behind the next commitments, dependencies, accountable people, and conditions for reconsideration. Its supporting records should preserve why those choices were made. The next review can then begin with what the organization has learned and what it can now deliver.
References
Ascarza, E. (2018). "Retention Futility: Targeting High-Risk Customers Might Be Ineffective." Journal of Marketing Research, 55(1), 80-98. https://doi.org/10.1509/jmr.16.0163. Author-hosted full text. Relevant evidence concerns the studied retention interventions and does not establish a universal targeting rule.
van der Aalst, W. M. P., et al. (2012). "Process Mining Manifesto." In Business Process Management Workshops: BPM 2011, LNBIP 99, 169-194. Official task-force edition: https://www.tf-pm.org/upload/1580737614108.pdf. Guiding Principle 1 and Challenge C1 concern event quality and the difficulties of reconstructing processes from records.
National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1, January 2023. https://doi.org/10.6028/NIST.AI.100-1. See GOVERN 2 for responsibilities and training, and section 5 for the nonprescriptive organization of the framework.