Model Routing, Prompt Libraries and Cost in Microsoft Foundry

AB-100 links model selection to the business decision

An architect planning an AI-powered business solution must decide which model capabilities are necessary, what quality constraints cannot be compromised and how to measure the total cost of a completed business task. Microsoft’s AB-100 blueprint includes model routing, custom or small language models, prompt guidance, a prompt library and return-on-investment analysis. These are related decisions but not identical technical controls: a routing system selects a model, a prompt library governs instructions, and a value model determines whether their combined benefit justifies cost and risk.

The official AB-100 study guide gives the plan domain a 25–30 percent range and expects architects to assess build/buy/extend choices. It should not be read as requiring one specific current Foundry model version for every exam question. Vendor product names and pools evolve, so compare the architecture principles with up-to-date deployment documentation before implementing.

As of October 13, 2026, the English exam objectives are scheduled for an update on October 14. The advance outline already names model routing and cost-versus-value reasoning. That scheduling fact should be reflected in historical course materials without suggesting the updated English exam has already started today.

Understand what Microsoft Foundry model router actually does

Microsoft’s model router architecture documentation describes a trained routing layer that analyzes the incoming request and chooses an eligible underlying model. A request can include system instructions, recent dialogue and tool definitions; the router considers task difficulty and a configured mode before forwarding it. The router is not itself a replacement for every generative model, nor does it guarantee the answer is correct.

The exposed modes are Balanced, Cost and Quality. Balanced favors a cost-and-quality tradeoff, Cost leans toward less expensive eligible models, and Quality favors higher-rated models for each prompt. Those modes are decision controls rather than promises of particular percentages, tokens or model selections. Routing results depend on the available pool, configuration, deployment region and the actual prompt.

A single chat can contain many requests whose complexity differs. A short FAQ classification may be assigned to a smaller model while a multi-step exception explanation may need stronger reasoning. This routing flexibility can reduce unnecessary use of expensive models, but it also means quality, latency and response style must be evaluated across a realistic multi-turn workload, not just a single friendly prompt.

Compare routing with a fixed model deployment

A fixed deployment offers predictable model identity and may simplify validation when the workload has a narrow, stable task. A router can adapt among available models but introduces another managed selection decision. Choosing between them requires evidence about the request mix, model capabilities, latency limits, data residency, rate limits, costs and the failure impact of poor answers. Neither choice should be made solely from a provider’s marketing benchmark.

Consider a fictional internal procurement assistant that answers simple policy questions, extracts structured invoice details, and occasionally evaluates contradictory supplier terms. The first two tasks may suit lower-cost models; the third could need deeper reasoning and more review. A router might allocate model capability dynamically, but the business still needs approval gates for contractual or financial decisions. The model selection cannot authorize an invoice payment by itself.

The Foundry model router concepts discuss available models, modes and supported deployment boundaries. An architect should use current region and model-subset documentation for the actual tenant. Do not assume every model, feature or provider is available in every geography or that one historical product screenshot accurately represents today’s catalog.

Small language models and custom models need a specific use case

A customized small language model can be attractive when a domain task is narrow, high volume and measured against reliable reference data. However, small size alone does not make it safe, compliant or inexpensive in operation. A specialist model still needs data quality, model lifecycle, performance evaluation, access controls and a plan for input cases that lie outside its competence. A larger general model may be the simpler solution if training and maintenance overhead outweigh any inference savings.

A build-versus-buy-versus-extend decision should compare a prebuilt Copilot capability, a prompt-and-retrieval layer around a managed model, a router, and a custom model trained for a justifiable requirement. For example, if the objective is to answer an internal HR-policy question, a knowledge-grounded agent may be enough; tuning a small model on confidential records introduces additional data-governance work that may not improve quality. Evaluate risk as well as functionality.

The Microsoft agent-stack architecture frames which teams must operate each layer. Operations, security and business ownership affect the choice as much as model scores do. An architecture that is cheap to demonstrate but difficult to update or audit may have a worse total cost than a managed alternative.

Maintain a prompt library as governed architecture

A prompt library should not be an untracked folder of text fragments copied between apps. Each reusable prompt needs a purpose, accountable owner, applicable model or agent versions, data classification, permitted input schema, expected output format, safety rules and tests. Its changes can affect operational behavior even when the underlying model remains the same. Review prompts as release artifacts rather than casual wording improvements.

A finance-policy prompt might require a citation to the current approved policy and a refusal to invent facts when retrieval fails. A service-summary prompt might require a structured, concise answer without disclosing another customer’s record. These are distinct prompt contracts. The library should identify which contract an application uses and preserve a review history so an unexpected answer can be traced to a specific version.

Do not assume that a prompt which works on one model produces equivalent results on every model selected by a router. If the routing pool changes, re-run the applicable evaluation set, including tool-calling and refusal cases, across realistic contexts. Prompt standardization and model flexibility can work together only if there is evidence that the contract remains satisfied.

Evaluate quality and cost against the full transaction

Build a benchmark from actual business task categories, without exporting restricted records to an unauthorized evaluation environment. Include routine and complex cases, incomplete input, misleading retrieved material, tool errors and cases requiring a human. For each scenario, record the expected task outcome, model or routing mode, latency, observed quality, exception handling and how much human rework followed. The combined evidence supports a defensible model selection.

Microsoft describes using Azure Monitor to observe model-routing distributions. That telemetry can reveal which underlying models handled requests, but a count of smaller-model selections is not itself a business saving. Measure cost per accepted resolved ticket or completed review, not just cost per generated token. A lower inference bill may be offset by more correction time, security incidents, retries or expensive escalations.

The AI agent value and ROI framework connects this evidence to outcome, risk, adoption and total ownership cost. Decide which quality thresholds prevent release, and keep a comparable baseline. A successful cost experiment preserves required business quality and compliance; a reduction in price without that evidence is not a validated efficiency gain.

Plan failure handling when a routing choice is unsuitable

A router may return an answer that fails a task’s quality threshold, encounters a timeout, or cannot satisfy a required tool-call contract. A resilient design defines how the application recognizes those cases and what happens next: ask a clarifying question, retry safely with a supported mode, fall back to a fixed model, or escalate to a human. Blind repeated requests can increase cost and perform the same bad action several times, so retries must be bounded and compatible with downstream idempotency.

A deployment with data-residency restrictions must use approved geographic and deployment boundaries. Model subsets, supported models and compliance requirements can constrain which choices are eligible. If a specific long-context or multimodal capability is essential, validate it against the router’s documented limits rather than assuming every underlying model shares a maximum context window or tool behavior.

Monitoring should connect routing outcomes with user reports and quality issues. When a business unit complains about a change, the team should be able to identify the deployed routing configuration, applicable prompt version, relevant model if recorded, request category and tested recovery procedure without exposing sensitive conversation data in ordinary logs.

What an AB-100 candidate should be able to recommend

Given a business workload, the candidate should explain why a managed prompt, retrieval, fixed model, model router or customized smaller model is appropriate. The answer should mention specific business constraints such as throughput, cost, reliability, security, data availability and the operations team that will maintain the solution. A technically possible architecture is not necessarily commercially worthwhile.

Practise with two fictional workloads: high-volume low-risk knowledge lookups and low-volume high-stakes decisions. Compare their quality thresholds, routing modes, human approval needs and failure behavior. Explain what extra evidence would change your recommendation. The goal is architecture judgment, not predicting which model a real router would choose for a request you have not submitted.

An AB-100 practice scenario is useful when it reveals an unclear build/buy/extend or cost-quality tradeoff, but it is a third-party learning aid rather than a guaranteed copy of Microsoft’s examination. The source examples here are conceptual; no live model-router deployment, tenant charge or benchmark measurement was performed.