{"id":25194,"date":"2026-10-11T17:20:17","date_gmt":"2026-10-11T17:20:17","guid":{"rendered":"https:\/\/www.exam-labs.com\/blog\/?p=25194"},"modified":"2026-10-11T17:20:17","modified_gmt":"2026-10-11T17:20:17","slug":"image-generation-and-editing-microsoft-foundry","status":"publish","type":"post","link":"https:\/\/www.exam-labs.com\/blog\/image-generation-and-editing-microsoft-foundry","title":{"rendered":"Image Generation and Editing in Microsoft Foundry"},"content":{"rendered":"

Image generation and image understanding solve different problems. A vision model may classify an object, describe a photograph, or answer a question about what is already visible. A generation model creates a new visual asset, while an editing model changes an existing one under instructions and, sometimes, a mask. Choosing between these operations is the first engineering decision. An application that needs the exact dimensions of a manufactured part should not use a generative model to invent those measurements, even when its image looks convincing. A marketing workflow that needs several approved background treatments for the same product has a more plausible use for controlled generation and editing.<\/p>\n

Microsoft Foundry exposes image-oriented models and tools, but a model appearing in a catalog is not proof that the application can deploy it in every region or invoke it with every editing option. Image APIs, supported models, deployment types, quotas, and preview conditions differ. The goal is to design a workflow that chooses a supported capability, protects source media, controls the changes being requested, and can distinguish an acceptable visual result from an attractive but incorrect one.<\/p>\n

Choose the image task before the model<\/h2>\n

Start with the intended relationship between source material and output. Text-to-image is suitable when the brief describes an asset that does not yet exist. Image-to-image editing is suitable when a known photograph, rendering, or graphic should be transformed but still retain recognisable source characteristics. Mask-based editing narrows the desired change to a selected area, such as replacing a distracting background while preserving the object in front of it. These tasks share some generative techniques, but the acceptance tests differ sharply.<\/p>\n

Suppose a retail team wants an approved backpack photograph placed against several seasonal backgrounds. Generating a completely new backpack from the product description risks changing its logo, pockets, seams, color, or shape. An image edit using the original photograph is better aligned with the requirement to preserve the actual item. A mask can express which region should change, but it does not create a contractual guarantee that every unmasked pixel or small detail will remain identical. The system still needs a comparison and review step.<\/p>\n

Now consider a team creating concept art for a fictional environment. There may be no reference photograph that needs faithful preservation. The important requirements are composition, aspect ratio, tone, audience appropriateness, and whether the result communicates the intended scene. Here a text-driven generation path may be more suitable. The distinction is not that one model is categorically better: the inputs, fidelity needs, and review burden make the choice.<\/p>\n

Use the Microsoft Foundry model catalog<\/a> to narrow candidates according to task, modality, supported interfaces, deployment constraints, and documented model behavior. Current Microsoft documentation includes GPT-image-family options and MAI image models, with some MAI offerings explicitly labeled preview. Treat that preview label as an operational condition, not a footnote. A preview may have limited regions, features, support terms, and service commitments, and it should not be presented as equivalent to a generally available production feature.<\/p>\n

Design a controlled generation request<\/h2>\n

Microsoft’s documented MAI image preview endpoint provides a concrete way to examine this contract. In an authorized subscription with a supported region, deployed image model, and appropriate role permissions, the following illustrative Python client sends a text-to-image request. It uses environment variables rather than hard-coding credentials; install the requests<\/code> package first. The MAI endpoint and width<\/code>\/height<\/code> fields are specific to this model family, not a universal Foundry image API.<\/p>\n

import base64\nimport os\nimport requests\n\nendpoint = os.environ[\"AZURE_ENDPOINT\"].rstrip(\"\/\")\napi_key = os.environ[\"AZURE_API_KEY\"]\ndeployment = os.environ[\"DEPLOYMENT_NAME\"]\n\npayload = {\n    \"model\": deployment,\n    \"prompt\": \"Minimalist illustration of a lighthouse in fog\",\n    \"width\": 1024,\n    \"height\": 1024,\n}\nresponse = requests.post(\n    f\"{endpoint}\/mai\/v1\/images\/generations\",\n    headers={\"api-key\": api_key, \"Content-Type\": \"application\/json\"},\n    json=payload,\n    timeout=120,\n)\nresponse.raise_for_status()\nitems = response.json().get(\"data\", [])\nif not items or \"b64_json\" not in items[0]:\n    raise ValueError(\"No encoded image returned\")\nwith open(\"result.png\", \"wb\") as output:\n    output.write(base64.b64decode(items[0][\"b64_json\"]))<\/code><\/pre>\n
This example is derived from Microsoft’s MAI image generation documentation and has been syntax-reviewed offline, not executed against Azure. Preview MAI models have limited availability and no preview service-level agreement; do not treat the snippet as proof that a particular account can deploy the model. A corresponding supported MAI image edit uses a different endpoint, \/mai\/v1\/images\/edits<\/code>, with multipart file upload rather than the JSON generation body. A GPT-image masking workflow has its own constraints\u2014for supported GPT-image edits, a mask must be a same-dimension PNG, and fully transparent mask pixels identify the region to change. Verify the selected model’s reference before transferring those rules to a different image family.<\/p>\n
An image request should be a traceable application operation rather than a raw prompt passed from an anonymous browser directly to a model endpoint. The application needs to know the requesting identity, why a particular model is permitted, which input assets can be used, what dimensions or output format are acceptable, and where the result may be stored. Those decisions belong in ordinary software controls around the model. A prompt should not be the only place where permission to use a customer photograph or branded asset is expressed.<\/p>\n
For a new asset, store the task brief as structured application data: intended subject, composition, output use, permitted reference materials, requested format, and review criteria. Generate the model prompt from that approved data. This does not require turning every creative preference into a rigid schema, but it allows the team to distinguish a changed business requirement from a changed prompt-writing tactic. Record the chosen model and version, deployment, operation type, important request settings, response identifier, and the application’s decision about the output.<\/p>\n
For an edit, the source image needs additional provenance. Keep its storage identifier, owner or license basis, original dimensions, checksum where appropriate, and the version that was approved for transformation. If the user submits a mask or selection, treat it as a separate input with its own validation. The upload pipeline should reject unexpected formats and excessive files before model processing, and it should avoid exposing the media to broader users or systems than the workflow requires.<\/p>\n
Authentication and authorization are separate checks. The service credential authenticates the workload to Foundry or the underlying Azure resource; the application must still authorize the human or system that requested the job to work with a particular image. For Azure-hosted applications, managed identity<\/a> can reduce long-lived secret handling where the selected service and deployment support the relevant identity path. Do not assume the same identity configuration works unchanged across every model provider, endpoint, or preview API. Confirm the actual service documentation and perform a permissions test with a minimally scoped application identity.<\/p>\n
Understand masks, reference fidelity, and editing boundaries<\/h2>\n
A mask specifies an intended edit region, not a promise of pixel-level conservation. In supported masked-edit APIs, transparent areas of a compatible mask commonly designate where changes may occur, with other areas indicating content to retain. The precise alpha-channel convention, file format, dimensions, model compatibility, and behavior must be checked against the selected API. Some image families offer edits without the same masking features, so it is incorrect to write one universal request shape and apply it to every Foundry image deployment.<\/p>\n
The backpack example gives a useful validation plan. Preserve the original photograph and ask the model to change only the background. Then inspect the boundary around handles, straps, and small openings where foreground and background overlap. Compare the logo, zippers, edge stitching, product proportions, and color against the original. A result that looks more polished but silently changes product details fails the task, even if a casual reviewer prefers the new picture. For commercial imagery, fidelity to the actual product may matter more than aesthetic novelty.<\/p>\n
Reference-image strength, output fidelity, and generation controls are model-specific. A setting exposed by one API should not be assumed to exist in another, and a model update can change how closely it follows a reference. Where possible, keep a small, licensed reference suite of easy, difficult, and deliberately ambiguous images. Run those through a staging workflow when a deployment is changed. This exposes edge effects, object mutation, text rendering problems, and subtle shifts in prompt adherence before the application rolls forward broadly.<\/p>\n
Not every visual task needs a generative edit. If the required operation is a precise crop, exposure adjustment, standard background replacement, or deterministic compositing step, a conventional image-processing library may be more repeatable and easier to verify. Image generation is useful when creative synthesis is genuinely required; it is not an excuse to surrender exactness that ordinary software can preserve.<\/p>\n
Write prompts that support review rather than wishful control<\/h2>\n
Strong image prompts communicate the desired outcome in visual terms: subject, setting, composition, lighting, viewpoint, style, and constraints tied to the final use. A vague instruction such as \u201cmake this professional\u201d leaves too much of the requirement to the model. For the product-background edit, a better brief might describe a neutral studio wall, soft shadows consistent with the existing product lighting, no extra objects, and preservation of the approved backpack’s distinguishing marks. Those details create testable expectations.<\/p>\n
Separate creative variation from non-negotiable properties. The color of the background may vary, while product labeling and dimensions must remain faithful. If the output includes a factual diagram, technical label, compliance mark, or safety instruction, generated text and geometry should not be treated as verified merely because they appear legible. An application can generate a visual concept and still use deterministic typesetting for facts, measurements, or text that must be exact. This hybrid workflow often offers a better balance of flexibility and reliability.<\/p>\n
Negative constraints help communicate unwanted outcomes but should not be treated as a guaranteed enforcement mechanism. A prompt can ask the model not to add extra hands, brand symbols, objects, or written claims, yet the result still needs inspection. Different model families interpret negative instructions differently, and some interfaces expose dedicated content controls while others do not. The implementation should test the controls the selected model actually supports rather than copying settings from another provider’s examples.<\/p>\n
Use a small set of difficult real tasks when revising a prompt. Include a low-contrast object against a busy background, a product with thin straps or reflective parts, an image with existing small text, and a composition that might require the model to invent details outside the source. Evaluate whether improvements to one image cause regressions on the others. Prompt changes are versioned application changes when customers depend on consistent output.<\/p>\n
Handle safety, rights, privacy, and untrusted media<\/h2>\n
Generated-image safety is broader than a content filter. The application needs clear rules for who can upload images, what people or brands may be depicted, whether personal media can be retained, and which outputs require human review. Some model deployments impose restrictions on particular categories of people, likenesses, reference photographs, or copyrighted material. Those restrictions may differ by product, region, and preview status. The current service policy must be checked, and requests should not be constructed to evade it.<\/p>\n
Reference images can also carry untrusted instructions. A screenshot, poster, or document embedded in an image might contain text telling an AI system to ignore its original task or reveal unrelated data. If the workflow performs OCR or multimodal reasoning before an edit, that detected text is data from an untrusted input, not a new system instruction. Preserve the instruction hierarchy and prohibit an image from granting new tool permissions. This is the same trust-boundary question encountered in other multimodal and tool-using applications, although the attack surface has a distinct visual entry point.<\/p>\n
Keep the privacy boundary clear. Uploading a customer image to a model service is a data-processing action, not merely a cosmetic browser operation. Evaluate residency, retention, access logs, and onward sharing for the actual deployment. An output may itself expose sensitive information that was present in a source image or inferred by the model. Storage encryption, role permissions, expiration rules, and deletion procedures must apply to both the source and generated versions.<\/p>\n
Moderation outcomes should be handled predictably. If a request is blocked, the user should receive an appropriate explanation without revealing internal detection rules or retrying automatically with progressively weaker safeguards. Log sufficient structured information to diagnose whether the failure was policy, authorization, unsupported format, or service availability. Avoid logging entire sensitive source images or prompts into general-purpose telemetry systems just because debugging is convenient.<\/p>\n
Evaluate visual quality against the actual task<\/h2>\n
Visual evaluation needs more than asking whether the image looks good. Start with the decision the application must make about the output: accept it, send it for review, request a revision, or reject it. For a product-background edit, criteria include object preservation, boundary artifacts, correct lighting, brand accuracy, no invented product features, and a usable composition at the intended crop or screen size. A creative-art workflow may value atmosphere and variation more heavily but still require safety and output-format checks.<\/p>\n
Build a representative test collection with permission to use every reference image. Store the task instructions and expected properties separately from the model output. Include normal examples and adversarial cases: busy edges, transparent objects, low-quality source media, text that must remain exact, or ambiguous instructions. A simple pass\/fail decision may be appropriate for brand integrity; visual preference can be judged on a graded rubric. Do not combine all these measures into one impressive-looking score that hides a failure on a critical requirement.<\/p>\n
Automation can help screen output dimensions, file validity, prohibited empty responses, known logo regions, or obvious missing assets. It cannot reliably establish every property of semantic fidelity or legal suitability. Human review may be required for customer-facing imagery or sensitive categories. When people evaluate the output, give them the original and the task requirements, not only the generated file. Reviewers need to know what was supposed to be preserved.<\/p>\n
Test the workflow across model and prompt versions. If an image edit that once preserved the backpack’s logo begins subtly changing it after a deployment update, the regression should be visible in stored evaluation results. Track both acceptance rate and reasons for rejection. That feedback distinguishes model-choice problems from weak prompts, poor source imagery, incorrect masks, and ambiguous product policy.<\/p>\n
Operate the service with observable failures and costs<\/h2>\n
An image operation can fail before the model generates anything. Unsupported regions, unavailable deployment types, incorrect endpoint configuration, authorization failures, bad file types, content filtering, rate limits, and transient service problems call for different responses. Retrying an invalid mask will not fix it. Increasing quota will not authorize an image that violates policy. The application should classify failures before it decides whether and when another attempt is worthwhile.<\/p>\n
Requests must be bounded by budget and concurrency. Image operations can produce larger inputs and outputs than a text-only request, and some model options or resolutions affect latency and price. Record the selected deployment, number of attempts, approximate workload, latency, and estimated service cost using the usage details the actual API supplies. If a request is rejected for capacity, a controlled queue or a user-visible retry window may be better than many simultaneous requests that deepen the overload.<\/p>\n
Observe the workflow as a complete transaction: upload, input validation, rights\/permission check, model invocation, moderation outcome, result storage, human review, and delivery. The telemetry principles<\/a> used in AI agent systems also apply here: use a correlation identifier across components and keep diagnostic detail proportional to its privacy risk. An apparently successful model response is not a successful user task if the output never reaches storage or the reviewer cannot retrieve it.<\/p>\n
Keep generated artifacts versioned with the request metadata needed for future explanation, while respecting the organization’s retention and deletion policy. Image endpoints and available model versions can change. Maintain an upgrade path that includes checking product documentation, verifying actual deployment support, rerunning the representative evaluation set, and being able to roll back application settings where the service permits. Preview features particularly need explicit lifecycle review rather than an assumption of indefinite compatibility.<\/p>\n
What AI-103 candidates should be able to explain<\/h2>\n
For the current AI-103 computer vision domain, it is not enough to memorize that Microsoft Foundry has image models. A learner should distinguish generation from understanding; choose between a new image and a constrained edit; explain the role and limitations of masks and reference media; identify model- and region-specific controls; and design a workflow that handles access, rights, filtering, evaluation, failures, and cost. Those decisions are more durable than memorizing a particular interface label or assuming today’s preview options will remain unchanged.<\/p>\n
A useful practical exercise is to take a licensed, non-sensitive product image and define two legitimate tasks: create an entirely new promotional visual, and edit only its background while preserving the source product. Write acceptance criteria before selecting a model. Verify the current supported API and model documentation, prepare an identity with appropriate permissions, and run the permitted requests only in an environment where the learner has actual access. Then compare results against the acceptance criteria and record what failed. This is an exercise design, not a claim that the particular requests have been executed or that any given model is available in a specific region.<\/p>\n
The lesson is that image generation belongs inside a controlled software workflow. Model creativity can be valuable, but the product team remains responsible for choosing the right task, protecting media, assessing visual truthfulness, and deciding what the output can safely be used for. A reliable system makes those responsibilities visible rather than hiding them in one prompt.<\/p>\n","protected":false},"excerpt":{"rendered":"
A practical look at Foundry image generation and editing: model and mask selection, protected reference images, visual fidelity, versioning, and evaluation.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1111],"tags":[],"class_list":["post-25194","post","type-post","status-publish","format-standard","hentry","category-ai-machine-learning"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t