Conversation state is what lets an agent behave as though the second turn belongs to the first. It sounds simple until the application has to decide which messages belong together, how long they should persist, what happens when two requests arrive at once, and whether the conversation is allowed to contain sensitive data. In Microsoft Foundry, the Responses protocol can manage conversation history around a conversation identifier, but the application still needs an explicit state model.<\/p>\n
That state model should separate conversational continuity from application truth. A conversation can remember that a user asked for a refund, but it should not be the authoritative place where refund approval is stored. It can remember a document selected earlier in the session, but the document\u2019s ownership and access rules still belong to the application and identity systems.<\/p>\n
Within Microsoft AI agents<\/a>, the practical question is not whether the platform can persist conversation data. It can. The question is which state should be platform-managed, which state should be customer-managed, and which state should be reloaded from business systems every time because freshness matters.<\/p>\n The Responses API can associate model responses with a conversation. A client can create or reference a conversation and let the runtime hydrate the relevant history on subsequent turns. That means the application does not have to resend every previous message manually just to preserve continuity. It also means a conversation identifier becomes a security-sensitive reference: the wrong association can attach one user\u2019s context to another user\u2019s request.<\/p>\n Applications should therefore treat conversation IDs the way they treat other scoped resource identifiers. They belong with the authenticated user or tenant, should not be accepted blindly from an untrusted client, and should be checked against ownership before use. If a client can substitute someone else\u2019s conversation ID, the system has a cross-user data problem even if the model itself behaves perfectly.<\/p>\n This connects directly to agent session isolation<\/a>. Conversation state explains what should continue; isolation explains who is allowed to continue it.<\/p>\n Hosted agents introduce a separate session lifecycle. Under the Responses protocol, the platform can manage conversation history, while the hosted runtime also has session-level compute and persistent session storage such as the sandbox home directory and files area. Under the Invocations protocol, the application owns more of the state because Foundry passes through an arbitrary request instead of hydrating conversation history automatically.<\/p>\n This difference matters when a team moves code from one protocol to another. A workflow that relied on Responses conversation hydration can become accidentally stateless when rebuilt on Invocations. Conversely, an application that manually persists history may duplicate state if it later adopts a managed conversation surface without revisiting ownership.<\/p>\n The Microsoft Foundry agent architecture<\/a> should document these boundaries explicitly. \u201cStateful agent\u201d is too vague for production operations. Teams need to know whether they mean conversation items, sandbox files, vector stores, workflow checkpoints, or business records.<\/p>\n Foundry can use Microsoft-managed storage, or organizations can use their own Azure resources for stronger ownership and compliance control. Standard setups can place conversation and response state in Azure Cosmos DB, files in Azure Storage, and vector stores in Azure AI Search. This helps with data residency, customer-managed access, and enterprise governance, but it also creates resources the organization must size, monitor, back up, and secure.<\/p>\n Owning the database does not mean application code should edit Foundry\u2019s internal containers directly. The service still manages the data model. The value of bring-your-own storage is control over location, access, encryption, and operational policy. Application business state should use its own schema and lifecycle rather than borrowing the agent runtime\u2019s storage layout.<\/p>\n That separation is especially important for retention. A conversation may persist until it is deleted, while a business process may require a different legal or operational retention period. Data governance should define the lifecycle of both instead of assuming one deletion policy can satisfy every category of information.<\/p>\n Persisting every turn does not mean every turn should be sent back to the model forever. Long conversations eventually pressure model context windows, increase token cost, and make irrelevant history compete with current evidence. The runtime may preserve history, but the application still needs a strategy for what the model should actually see when the conversation becomes large.<\/p>\n That strategy can include compacting prior responses, summarizing resolved threads, reloading durable facts from a database, and retrieving only the evidence relevant to the current question. The planned Azure OpenAI Responses API<\/a> article addresses the mechanics of the response surface, while retrieval topics under this hub address how external knowledge should be brought into the prompt without turning conversation history into a dumping ground.<\/p>\n The design goal is not \u201cmaximum memory.\u201d It is enough trustworthy context to make the next decision well. Old instructions, superseded facts, and stale tool results can be actively harmful if the model treats them as current.<\/p>\n Conversation systems often assume a human sends one message, waits for the answer, and then sends another. Real applications break that assumption. A user can double-submit, two browser tabs can share an identity, a background workflow can append state while the user is active, or an integration can retry a request after a timeout.<\/p>\n The application should decide whether concurrent turns are serialized, branched, rejected, or reconciled. There is no universal answer. A support chat may prefer sequential turns. A research workspace may intentionally fork a conversation. A transaction assistant may need to reject overlapping operations because ordering affects the result.<\/p>\nConversation IDs are references to durable interaction history<\/h3>\n
Conversation state and hosted-agent session state are not identical<\/h3>\n
Customer-managed conversation storage changes operational responsibility<\/h3>\n
Context windows make state selection an active process<\/h3>\n
Concurrency needs a rule for conflicting turns<\/h3>\n