Applied AI Insights
Stefan Jansen on machine learning, language models, agents and the design of AI-assisted work.
Articles and research
Research and practical guidance on choosing AI methods, evaluating results and implementing systems. Full articles will appear on Substack.
Articles by topic
Long-form articles grouped by the question they address.
Models and decisions
Choosing methods, predicting customer behavior, explaining model outputs and connecting them to business decisions.
-
When to use generative AI, a task-specific model, or ordinary code
How to choose between ordinary software, task-specific machine learning and generative AI, accounting for development, verification and operating costs.
In preparation
-
Customer analytics: prediction, experimentation and business decisions
How prediction, experiments and customer value inform acquisition, retention and lifecycle decisions, including when an intervention changes an outcome.
In preparation
-
Understanding model decisions and the limits of explanations
What explanations reveal about a model, why they do not establish causality, and how they relate to business decisions.
In preparation
-
What AI decision models do, and when to use them
How models score, classify and rank options, and how to connect their predictions to rules for taking action.
In preparation
Evaluation and reliability
Judging whether AI-assisted work is complete and correct, and what external benchmark scores do and do not show.
-
Evaluating an AI-assisted workflow
How to define a successful outcome, test the evaluators and measure performance across completed requests, human handoffs and unfinished work.
In preparation
-
What LLM and agent benchmark scores actually tell you
What benchmark tasks, graders and repeated trials tell us about model and agent performance, and what needs testing in a business workflow.
In preparation
System design and operation
Implementing AI in business workflows and document systems, redesigning the work around them, and working with coding agents.
-
Implementing AI in business workflows
How to discover requirements, choose between fixed workflows and agent-directed execution, and define system permissions, human decisions and acceptance tests.
In preparation
-
Building AI systems for business documents and knowledge
How to build document workflows that connect extraction and retrieval to source evidence, permissions, coverage checks and human review.
In preparation
-
Redesigning workflows for AI: tasks, exceptions and responsibility
How to map tasks, exceptions and responsibilities before deciding what to automate and which decisions should remain with people.
In preparation
-
Using coding agents: from a written task to a tested change
A worked example of turning a written task into a tested code change, including an incomplete repair and the regression checks that catch it.
In preparation
Economics and outlook
The full costs and benefits of AI-assisted work, measured against the process it changes.
-
Measuring the costs and benefits of AI-assisted work
How development, integration, checking and correction change the business case, and when saved time translates into lower costs or additional output.
In preparation