Introduction to Decision Models in AI
How JEV, Microsoft-Decision-1, and decision-centric architectures are changing the way we build AI agents.
Introduction
The first wave of generative AI focused on models that could understand and generate language. The next challenge is building AI systems that can make reliable, fast, structured decisions inside real-world applications.
Consider an AI agent processing 100,000 customer support tickets. It may need to classify intent, assess urgency, select a workflow, determine whether a human should intervene, and decide whether an AI-generated response is safe to send.
A general-purpose large language model (LLM) can perform these tasks. However, repeatedly invoking a large model for small, constrained decisions may introduce unnecessary latency, cost, output variability, and integration complexity.
Decision models offer an alternative: given a defined context and a set of possible outcomes, they return structured decisions and associated scores or probabilities that software can consume directly.
Recent examples include JEV from TypeSafe AI and Microsoft-Decision-1. Alongside these emerging models, research such as Decision Transformer demonstrates a broader history of learning to make decisions using machine learning.
Why AI needs a decision layer ?
Most modern AI applications combine perception, reasoning, generation, and action. Yet these capabilities are not equally necessary at every step.
Imagine a procurement agent evaluating an incoming purchase request. It might need to answer:
Is this request related to procurement?
Does it exceed the configured approval threshold?
Which workflow should process it?
Is additional documentation required?
Should the request be escalated to a human?
These are not necessarily open-ended generation problems. They are decisions over a
defined set of outcomes.
A conventional architecture may send the entire context to a general-purpose LLM, ask it to reason about the request, generate JSON, and then validate that JSON before executing the next step.
A decision-oriented architecture can instead delegate the constrained judgment to a specialized decision component.
The architectural shift
Traditional generative pattern
Context → LLM reasoning and generation → Parse output → Validate → Execute action
Decision-centric pattern
Context + defined choices → Decision model → Structured scores → Policy and threshold checks → Execute, defer, or escalate
The second pattern can simplify the execution path when the task is genuinely constrained. It does not eliminate the need for reasoning, validation, or safety controls across the whole application.
What is a decision model?
A decision model is a machine learning model designed to select, classify, rank, or score outcomes based on an input state and a defined decision task.
A simple decision model can be expressed as:
Score vector
The model produces a score for each candidate action:
where si represents the score assigned to action ai
These scores are not necessarily probabilities. They may be arbitrary ranking scores or logits, depending on the model.
Conditional probability of an action
If the model produces probabilities over mutually exclusive candidate actions, the probability of selecting action ai, given context x and candidate set A is:
For a normalized probability distribution:
with:
Optimal action selection
A simple decision policy selects the action with the highest predicted probability:
Here, a* denotes the selected action. The arg max operator returns the action whose probability is highest.
Optional: Converting scores into probabilities
If the model produces logits si, a softmax function can convert them into a normalized probability distribution:
This conversion is appropriate when the scores are logits for mutually exclusive classes. It does not, by itself, guarantee that the resulting probabilities are calibrated.
However, choosing the highest-scoring option is not always the correct production policy. The application may instead enforce business rules, require a minimum confidence level, or escalate ambiguous cases.
Example: Customer support routing
Suppose a support system receives the following ticket:
“I was charged twice for my subscription, and I have not received a response to my previous ticket.”
The model is given three predefined routing choices.
Candidate decision | Illustrative score |
Billing | 0.91 |
Technical support | 0.06 |
Sales | 0.03 |
These numbers are hypothetical and are provided to illustrate the concept, not as measured results from a particular model.
A routing policy could select Billing if its score exceeds a validated threshold and no other rule blocks the action. The key distinction is that the output is directly usable by the application. The system does not need to infer the routing choice from a paragraph.
What makes a decision model different?
A decision model is not defined simply by returning JSON. General-purpose LLMs can also produce structured JSON outputs. The important distinction is the model's intended role and training or inference setup: it is optimized for constrained judgments rather than primarily for open-ended text generation.
A taxonomy of decision models
The term decision model can refer to several related but technically distinct approaches. Understanding the differences is important because they solve different problems.
Approach | Primary objective | Typical output | Example |
Classification model | Assign an input to a category | Label or class probabilities | Spam detection |
Ranking model | Order candidates by relevance or utility | Scores or ranked items | Search ranking |
Decision-scoring model | Evaluate predefined outcomes or criteria | Option scores or probabilities | Jev, Microsoft-Decision-1 |
Reinforcement learning policy | Select actions to maximize long-term reward | Action or action distribution | Robot control |
Decision Transformer | Learn action sequences conditioned on states and desired returns | Next action or action tokens | Offline reinforcement learning |
Rule engine | Apply explicit, deterministic rules | Rule-defined outcome | Approval limits |
These categories overlap in some implementations, but they should not be treated as interchangeable. For example, a classification model can be used as a decision component. A reinforcement learning policy may choose actions without presenting a closed list of business labels. A rule engine can make reliable decisions without machine learning at all.
The right choice depends on the problem, available data, required behavior, and consequences of errors.
JEV: A model built for decisions
JEV is a decision model introduced by TypeSafe AI in September 2026. Its documented interface takes a state and a collection of questions, then returns typed answers and associated probabilities or scores.
Official documentation: Jev AI documentation.
The design principle is straightforward: rather than asking a model to generate a response that software must interpret, ask it a defined question and receive a result in a known format.
How JEV works ?
Consider an incoming support ticket. The application can provide the ticket as a state and ask several questions about it in one request.
For example:
Does the ticket need human attention?
Which team should own it?
How frustrated is the customer?
JEV's documented API supports question types called noul, choice, and score. These represent different forms of judgments, including yes/no-style questions, choices among alternatives, and scoring against a scale.
A simplified representation of the request is:
{
"state": "The customer was charged twice and has not received a reply.",
"questions": {
"queue": {
"type": "choice",
"instructions": "Choose the team that should handle the ticket.",
"criteria": {
"billing": "Invoices, refunds and duplicate charges",
"technical": "API errors and integration problems",
"sales": "Plan and pricing questions"
}
},
"needs_human": {
"type": "noul",
"instructions": "Does this ticket require human attention?"
}
}
}This is a simplified illustration of the documented request format, not a complete production integration. The application receives structured answers instead of having to extract a decision from free-form prose.
Why this interface matters ?
The interface has three important properties:
Typed outputs: The application knows whether it is receiving a choice, a score, or another supported answer type.
Multiple judgments per request: Related questions can be evaluated against the same state in one call.
Explicit uncertainty information: Scores and probability distributions can inform downstream policies, provided they have been validated for the actual task.
These properties can reduce the amount of glue code required between model output and business logic. However, a structured answer is not automatically correct, and a returned probability is not automatically well calibrated. Both need empirical evaluation.
Example: Procurement triage
Suppose a procurement agent receives this request:
“We need 50 additional laptops for a new engineering team. The purchase is outside our approved quarterly budget.”
A decision layer might evaluate the following:
Decision question | Possible outcomes |
Request category | Hardware, software, services, other |
Budget status | Within budget, over budget, unknown |
Required workflow | Standard purchase, exception approval, additional information |
Human intervention | Required, not required, uncertain |
The model evaluates the request against the defined categories. The application then applies the actual procurement policy.
For example, a model may identify that the request is over budget, but only the policy engine should determine the mandatory approval route from the company's configured thresholds and delegation rules.
Important architectural principle: The model interprets the situation; the application enforces the governing policy.
Microsoft-Decision-1: Structured decision scoring at scale
Microsoft introduced Microsoft-Decision-1 on October 9, 2026, positioning it as a model for fast, structured decision-scoring in AI applications.
Official announcement: Introducing Microsoft-Decision-1.
Model catalog: Microsoft-Decision-1 in Microsoft Foundry.
Microsoft's documentation describes the model as built on Alibaba's open-weight Qwen3.5-9B and post-trained for single-pass decision scoring. Given a situation and a fixed set of answer options, it returns probability scores for those options.
Supported workloads
Microsoft identifies several intended use cases:
Classification and intent analysis
Model and workflow routing
Scoring and prioritization
AI-output evaluation
Groundedness and relevance checks
Agent action verification
Safety screening
Confidence-aware automation
The model is designed for constrained judgments, not open-ended conversation, summarization, or general-purpose text generation.
Example: Evaluating an AI-generated answer
Imagine a legal research assistant has generated an answer based on a supplied set of documents. Before returning the answer to the user, the application needs to assess whether the answer is supported by the evidence.
The evaluation task could be framed as:
Question: Is the answer adequately supported by the supplied evidence?
Possible choices:
Supported
Partially supported
Unsupported
Cannot determine
The decision model evaluates the answer against the evidence and returns scores for the available outcomes.
The application can then implement a policy such as:
Accept answers that pass the validated evidence threshold.
Send borderline answers for additional verification.
Reject or regenerate answers that fail the check.
This example illustrates a useful architectural pattern. It does not mean a decision model can establish legal correctness on its own; the evidence, rubric, and validation process remain critical.
What Microsoft's published benchmarks tell us ?
In its announcement, Microsoft reported that Microsoft-Decision-1 achieved the highest accuracy in its 36-benchmark comparison, covering nearly 150,000 questions, and was the fastest measured model in that comparison. Microsoft also reported a 1.3% average decision-flip rate across its eight input perturbations.
These are vendor-reported results, not an independent guarantee of performance on every application. Comparisons should be interpreted in the context of the benchmark selection, tested models, hardware or service conditions, task distribution, and evaluation methodology.
Production considerations
Microsoft explicitly states that its model is not designed or evaluated to be the sole automated decision-maker for consequential decisions about people, such as employment, credit, housing, education, healthcare, or legal rights. Its documentation also makes clear that the integrating application is responsible for answer options, confidence thresholds, escalation, human oversight, and safeguards.
That distinction matters: a fast decision component should not become an unchecked authority over high-impact decisions.
Decision models versus Decision Transformer
The names sound similar, but Microsoft-Decision-1 and Jev should not be confused with Decision Transformer.
Decision Transformer is a research architecture introduced by Chen and colleagues in 2021. It frames offline reinforcement learning as conditional sequence modeling. Instead of directly learning a conventional reinforcement learning policy through value-function bootstrapping or policy gradients, it trains a causal Transformer on sequences of returns, states, and actions. Original paper
The difference in one example
Suppose an autonomous warehouse robot needs to navigate around obstacles. A constrained decision-scoring model might choose among:
Move forward
Turn left
Turn right
Stop
A Decision Transformer, trained on suitable trajectory data and conditioned on the desired return, predicts actions as part of a sequence of behavior. The first is a natural fit for selecting among explicit alternatives at a decision point. The second is designed to learn behavior from sequences of states, actions, and returns.
Dimension | Jev / Microsoft-Decision-1 | Decision Transformer |
Core task | Structured judgments over defined alternatives | Sequential action generation |
Input | State, question, and answer choices or criteria | Desired return, past states, and actions |
Output | Labels, scores, or option probabilities | Predicted next action in a sequence |
Main application | Routing, evaluation, classification, workflow control | Offline reinforcement learning and sequential control |
Time horizon | Often a local decision, though embedded in a larger workflow | Explicitly models action sequences |
Main evaluation | Accuracy, calibration, robustness, latency, cost | Return, task success, and policy performance |
The shared theme is learning to make decisions. The mechanisms and objectives are different.
How decision models fit into an agentic AI architecture
The most compelling application is not necessarily replacing an LLM. It is combining different models according to the work each one does best.
An agent may need a reasoning model to interpret a complex objective, a retrieval component to gather evidence, a decision model to select among defined outcomes, and a policy layer to authorize an action.
Reference architecture
Why use a hybrid architecture?
Different components have different responsibilities.
The LLM interprets ambiguous requests, synthesizes evidence, and generates plans or responses.
The decision model performs repeatable judgments over defined options.
The policy engine enforces hard constraints such as approval limits, permissions, and mandatory checks.
The orchestration layer coordinates the components, handles failures, and decides when to retry or escalate.
The evaluation layer measures whether decisions are correct, stable, safe, and useful.
This separation creates clearer interfaces and makes the overall system easier to test. It also makes failures easier to diagnose: a wrong action may result from bad evidence, a poor model judgment, an incorrect threshold, or a faulty policy implementation.
A concrete example: Decision models in an HR Agent
Consider a talent acquisition agent that helps recruiters process applications. The system needs to identify job-related information, route applications, check whether required information is present, and determine whether a case needs human attention.
A responsible architecture could use:
Document extraction to retrieve candidate-provided information.
A general-purpose model to summarize relevant evidence against explicit job criteria.
A decision model to classify administrative workflow states, such as whether required information is missing.
Deterministic rules to enforce process requirements.
Human review for consequential candidate assessments and decisions.
An example decision schema might look like this:
{
"question": "Is required application information missing?",
"options": [
"complete",
"missing_information",
"cannot_determine"
]
}The model evaluates the supplied application information against the defined criteria. The application can then request additional information or flag an incomplete record. This is different from using a model to decide whether a person should be hired. Employment decisions are consequential, and Microsoft-Decision-1 is explicitly not intended to be the sole automated decision-maker for such decisions.
The broader lesson is to use constrained automation for appropriate administrative tasks while preserving meaningful human oversight for high-impact judgments.
The economics of decision models
One of the strongest reasons to introduce decision models into an AI architecture is economics. In an agentic workflow, not every step requires a large language model capable of complex reasoning or open-ended generation.
Consider an AI agent that processes invoices. It may use an LLM to interpret an unusual invoice or resolve an ambiguous discrepancy, but thousands of routine decisions—such as classifying invoice types, routing requests, or checking document completeness—may be handled by a specialized decision model.
The objective is not simply to replace an expensive model with a cheaper one. It is to allocate each task to the least expensive model that meets the required accuracy, latency, and safety standards.
The cost of making decisions at scale
The basic cost of inference can be estimated as:
For a system that makes one million decisions each month, even a small difference in the cost per decision can become significant.
Consider the following illustrative comparison.
These are hypothetical values for demonstrating the economics, not actual prices or benchmark results for Jev or Microsoft-Decision-1.
Under these assumptions, the specialized model reduces inference expenditure by 90%.
The savings come from the assumed difference in cost per decision—not from a guarantee that every specialized decision model will be cheaper.
Understanding the cost reduction
The percentage reduction in inference cost can be calculated as:
Using the illustrative monthly costs above:
This means inference costs are 90% lower under the stated assumptions.
However, the total economic benefit depends on more than inference pricing. Model integration, infrastructure, evaluation, monitoring, and human review also contribute to operating costs.
Latency: Why faster decisions matter
Cost is only one part of the equation. Decision latency can also affect the performance of an AI agent. Suppose an agent executes 20 sequential decision steps. If each step becomes 100 milliseconds faster, the theoretical reduction in cumulative model latency is:
That is a potential reduction of two seconds in the model-processing portion of the workflow.
In practice, the improvement depends on whether the steps are sequential or parallel, network overhead, retries, model loading, and other orchestration costs. Faster individual decisions do not necessarily translate into proportionally faster end-to-end execution.
The hidden cost: Incorrect decisions
A cheaper model is not necessarily the cheaper solution if it produces more errors.
For example, an incorrect invoice-routing decision might trigger additional processing, delay payment, and require manual intervention. Those costs can outweigh the original inference savings.
A more realistic cost model is:
Each term represents a different component of the system's operating cost. The economic comparison should therefore account for the quality of decisions and the cost of recovering from mistakes, rather than comparing model prices alone.
When does a decision model make economic sense?
Decision models are particularly attractive when:
The application performs a high volume of repeated, well-defined decisions.
The available outcomes can be expressed as a finite set of choices or measurable scores.
The specialized model meets the required quality threshold.
Low latency is important to the overall workflow.
The task does not require extensive open-ended reasoning at every step.
Conversely, if a task requires deep contextual reasoning or has a high cost of error, a general-purpose LLM—or a hybrid workflow combining both models—may provide better overall value.
The difficult problem: Confidence calibration
A decision model that returns probabilities introduces an important question: what do those probabilities mean?
Suppose a model assigns a probability of 0.90 to a particular decision. If the model is well calibrated on representative data, predictions assigned approximately 90% probability should be correct roughly 90% of the time over the relevant population.
Calibration is different from accuracy.
Accuracy measures how often predictions are correct.
Calibration measures whether predicted probabilities align with observed frequencies.
Robustness measures whether harmless changes in the input cause inappropriate changes in the decision.
Coverage measures how many cases the system handles without abstaining or escalating.
A model can have high accuracy but poor calibration. It can also be well calibrated on average while performing badly for a specific class or subgroup.
Example: Confidence-based routing
Consider a hypothetical policy for an invoice-processing workflow:
Model confidence | Illustrative action |
At least 0.95 | Continue automatically, if all other checks pass |
0.75–0.95 | Run a secondary verification |
Below 0.75 | Escalate for review |
These are example thresholds, not recommended universal settings.
The correct thresholds depend on the cost of false positives, false negatives, and human review. A harmless support-ticket label may tolerate more automation than a financial transaction or access-control decision.
A production evaluation should include:
Reliability diagrams
Brier score or log loss
Expected calibration error, with its binning limitations understood
Per-class precision and recall
False-accept and false-reject rates
Abstention and escalation rates
A confidence score should be treated as an input to a policy, not as a guarantee of correctness.
How to evaluate a decision model before deployment
Evaluating a decision model requires more than testing whether it returns valid JSON or selecting the model with the highest benchmark accuracy.
A useful evaluation framework should measure at least six dimensions.
Dimension | Key question | Example metric |
Decision quality | Does it choose the correct outcome? | Accuracy, macro-F1 |
Calibration | Do probabilities reflect observed correctness? | Brier score, reliability curves |
Robustness | Does paraphrasing or reordering change decisions unexpectedly? | Decision-flip rate |
Efficiency | Can it meet latency and cost targets? | p50/p95 latency, cost per decision |
Safety | Does it refuse or escalate inappropriate cases? | Policy-violation rate, false-refusal rate |
Operational value | Does it improve the full workflow? | Task success, human-review rate, total cost |
A practical test strategy
Step 1: Build a representative test set. Include ordinary examples, edge cases, ambiguous inputs, rare classes, and examples from the expected production distribution.
Step 2: Define the ground truth. Have qualified reviewers establish labels or acceptable decisions. For rubric-based tasks, document the rubric and measure reviewer agreement where appropriate.
Step 3: Establish baselines. Compare against a deterministic rules engine, a conventional classifier, and a general-purpose LLM where those baselines are relevant.
Step 4: Measure calibration and robustness. Test equivalent paraphrases, changed option ordering, formatting differences, missing fields, and out-of-distribution inputs.
Step 5: Run a shadow deployment. Let the model generate decisions without executing them. Compare its outputs with the current system and human judgments.
Step 6: Introduce bounded automation. Automate only the cases that meet validated quality, confidence, safety, and policy requirements. Keep an explicit fallback path.
Step 7: Monitor drift. Track changes in input distributions, class frequencies, error rates, confidence distributions, and downstream business outcomes.
The most important metric is end-to-end task success
A model can perform well on an isolated classification benchmark while reducing the success rate of the overall agent.
For example, an incorrect routing decision may send a case to the wrong team, create a delay, trigger retries, and eventually require human intervention.
Therefore, model-level accuracy should be complemented by workflow-level metrics such as resolution time, failure rate, cost per successfully completed task, and escalation quality.
When should you use a decision model?
Decision models are especially promising when a system makes a large number of repeated, bounded judgments.
For difficult tasks, a decision model can still contribute a narrow sub-decision within a larger reasoning pipeline. The objective should be to assign each component a role that matches its strengths and limitations.
An implementation blueprint for an AI decision layer
For teams building multiple agents across finance, HR, procurement, compliance, and customer operations, the most reusable approach is to create a common decision service rather than implementing model calls independently in every agent.
A possible service contract is:
decision_request = {
"task_id": "support_ticket_routing",
"context": {
"ticket_text": (
"The customer was charged twice "
"and has not received a response."
)
},
"options": [
"billing",
"technical",
"sales",
"unknown"
],
"policy": {
"allow_abstention": True,
"require_audit_log": True
}
}The service can normalize the request, invoke the selected decision model, validate its response, apply the application-defined policy, and emit a structured result.
A conceptual response might be:
{
"decision": "billing",
"confidence": 0.91,
"status": "accepted_for_routing",
"model_version": "example-version",
"policy_version": "routing-policy-v3",
"requires_human_review": false
}This response is an illustrative application-level schema, not the native output schema of Jev or Microsoft-Decision-1. The confidence value is hypothetical, and the application should record the underlying score distribution when available.
Essential engineering controls
A production decision service should include:
Strict input and output schema validation
Versioned decision definitions and policy rules
Model and prompt/configuration version tracking
Request and response audit logs, subject to privacy requirements
Timeouts, retries, and fallback behavior
Thresholds selected from empirical validation
Monitoring for calibration drift and class imbalance
Human-review paths for ambiguous or consequential cases
Protection against prompt injection and untrusted input
Access control and authorization outside the model
For agentic applications, the decision model should not be the sole enforcement mechanism for permissions or irreversible actions. The application must independently verify whether the proposed action is authorized.
The limitations and open questions
Decision models are a useful architectural development, but several questions remain important.
First, specialization does not guarantee superiority. A decision model should be compared with simpler classifiers, rule-based systems, and general-purpose LLMs on the actual workload. The simplest system that meets the requirements may be the best choice.
Second, probabilities need validation. Calibration can change when the production distribution differs from the test data. A score that was reliable during evaluation may become unreliable after a major change in inputs.
Third, closed-set decisions can hide missing options. If the correct answer is not in the predefined choice set, a model may still select an incorrect option. Systems need an explicit abstention or “cannot determine” path where appropriate.
Fourth, benchmarks are not production guarantees. Results reported by a vendor are useful evidence, but teams should reproduce relevant comparisons using their own data, constraints, and costs.
Fifth, deterministic policies remain essential. A model can interpret evidence, but a hard business rule should be enforced by code or a dedicated policy system where deterministic enforcement is required.
Finally, decision models are not a substitute for broader reasoning. Agents still need to gather evidence, understand objectives, construct plans, and recover from unexpected states. Decision models are best understood as specialized components in a larger system.
The future: From AI assistants to decision-centric systems
The evolution of AI applications can be understood as three overlapping stages.
Stage 1 — Generative AI: Models generate text, code, images, and other content.
Stage 2 — Agentic AI: Models use tools, plan tasks, and coordinate multi-step workflows.
Stage 3 — Decision-centric AI systems: Applications combine reasoning models, specialized
decision components, deterministic policies, and evaluation systems to make workflows more measurable and controllable.
These stages are not replacements for one another. A mature agentic system may use all three.
For example, an enterprise finance agent might use a reasoning model to understand a complex request, retrieval to obtain relevant policy documents, a decision model to classify the requested operation, and deterministic authorization logic to decide whether the operation is permitted.
The result is not necessarily a smarter single model. It is a better-designed system in which each component has a clear responsibility.
Conclusion
Decision models represent an important direction in AI engineering: separating the generation of information from the selection of actions. Jev and Microsoft-Decision-1 illustrate the emerging category of specialized models that return structured judgments over defined options. Decision Transformer illustrates a related but distinct research direction in which Transformers learn sequential behavior from states, actions, and returns.
For developers building agentic AI, the key question is not whether a decision model should replace an LLM. It is whether a specific part of the workflow can be made faster, cheaper, more consistent, and easier to evaluate by using a specialized decision component.
The most robust architecture combines model-based judgment with explicit policies, empirical evaluation, authorization checks, and human oversight where necessary.
The next step in AI engineering may not be making every model reason about everything. It may be learning how to assign each decision to the right component.
References
TypeSafe AI / Jev documentation. Introduction to the Jev decision API. Describes state-based requests, typed question formats, and structured answers.
Microsoft. Introducing Microsoft-Decision-1: Our model for fast decision-making, October 9, 2026. Describes Microsoft's approach, reported benchmark results, use cases, and deployment considerations.
Microsoft Foundry. Microsoft-Decision-1 model catalog. Documents model capabilities, limitations, and intended use cases.
Chen et al. Decision Transformer: Reinforcement Learning via Sequence Modeling, NeurIPS 2021. The original research paper on conditional sequence modeling for offline reinforcement learning.
Comments