Retry settings in Microsoft Fabric Data Factory are simple to configure and easy to misuse. Most pipeline activities can retry automatically, and Fabric now supports fixed or increasing-delay intervals plus conditional retry logic for selected activities. The hard part is deciding which failures are genuinely transient and whether repeating the activity is safe.<\/p>\n
This article sits under the Microsoft Fabric engineering<\/a> hub because retry behavior is part of production pipeline design, not a late operational toggle. The existing Fabric data pipelines<\/a> article covers the wider orchestration model.<\/p>\n A good retry policy should reduce manual recovery without turning one failure into duplicate data or a long sequence of useless attempts.<\/p>\n Fabric supports retries because pipelines interact with networks, APIs, databases, Spark jobs, and services that can fail temporarily. Throttling, a brief outage, or an intermittent connection can clear without code changes. Authentication failures, invalid schemas, and broken business rules generally do not.<\/p>\n By default, an activity can retry on any failure if retries are enabled. Conditional retries can narrow the behavior for supported activity types using fields such as error code, message, or failure type. That is useful when the team wants to retry 429 throttling but fail fast on a user error.<\/p>\n Selective retry shortens incident detection. A pipeline that spends an hour retrying a permanent configuration error only delays the alert.<\/p>\n A fixed interval waits the same amount of time between attempts. It can work for brief, predictable faults where the dependency typically recovers quickly. Increasing delay uses an exponential-backoff pattern with a randomized interval that grows up to a configured maximum.<\/p>\n Increasing delay is more appropriate for overloaded or recovering services because repeated clients are less likely to hammer the dependency at the same moment. The randomness also reduces synchronized retry storms when several pipeline runs fail together.<\/p>\n The interval should match the dependency. Waiting ten minutes between retries for a five-second network blip wastes recovery time; retrying every few seconds during a service outage creates noise and load.<\/p>\n A high retry count looks resilient until it pushes the run far beyond the business freshness window. Retry count, retry interval, activity timeout, and the overall pipeline schedule should be designed together.<\/p>\n If an hourly pipeline can spend forty minutes retrying one activity, downstream loads may overlap with the next scheduled run. That creates concurrency and duplication problems that did not exist in the original failure. The recovery budget should leave enough time for the rest of the workflow or fail cleanly and trigger escalation.<\/p>\n Operational dashboards should show time lost to retries, not only whether the final run succeeded. A \u201csuccessful\u201d pipeline that needed dozens of attempts may be a capacity or connector problem in disguise.<\/p>\n Retrying a read is generally safer than retrying a write. If a copy, stored procedure, notebook, or API call can partially commit before returning a failure, the second attempt may duplicate data or repeat a business action.<\/p>\n Activities with side effects should use idempotent write patterns where possible: deterministic merge keys, transaction boundaries, checkpoint tables, operation IDs, or delete-and-replace strategies that make a replay produce the same result. If the target API supports idempotency keys, preserve the same key across retries for one logical operation.<\/p>\n This is the same principle used in agent workflows and distributed systems. A retry policy is only safe when the work can be repeated without changing meaning.<\/p>\n Fabric\u2019s retry conditions can inspect error code, error message, and failure type for supported activities. That makes it possible to encode an operational rule such as \u201cretry rate limiting, but not authentication failure.\u201d The policy becomes clearer than a blanket attempt count.<\/p>\n Teams should avoid building a fragile library of text-message matches when a stable error code or failure type is available. Error messages can change between connectors or service versions. Where only message matching is possible, the condition should be tested against real failure samples.<\/p>\n Retry-condition changes should move through the same deployment path as pipeline logic because they change production behavior.<\/p>\n A pipeline that succeeds on the fourth attempt can still meet its freshness SLO, but frequent retries may indicate a deteriorating source or capacity constraint. Monitoring should track attempt count and retry reasons over time.<\/p>\nRetries are for transient failure, not every failure<\/h3>\n
Fixed and increasing delay serve different failure patterns<\/h3>\n
Retry count should fit the pipeline deadline<\/h3>\n
Side-effecting activities require idempotency<\/h3>\n
Conditional retries should encode known failure classes<\/h3>\n
Retries can hide source instability if monitoring stops at final status<\/h3>\n