Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 54
Question 530
An Azure Databricks architect reviews the following problem: A scheduled notebook contains both transformation SQL and a long-running external API loop that occasionally hangs. What should the team implement FIRST?
- Route deterministic validation failures to quarantine rather than retrying them as transient infrastructure errors.
- Write to a staging target, validate row totals and invariants, then perform the approved publication step.
- Design a compatible atomic publication or versioned readiness contract so consumers see consistent results.
- Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages.
Explanation
Correct Answer
D:The most appropriate action is: Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages. It addresses the stated condition: A scheduled notebook contains both transformation SQL and a long-running external API loop that occasionally hangs. The technical basis is: Task boundaries make duration limits and operational ownership explicit. Without this control, A stalled API call can hold an entire pipeline indefinitely and consume compute. A direct acceptance check is: A simulated hung request times out, alerts the owner and avoids publishing incomplete data.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A pipeline automatically retries malformed source records indefinitely and consumes its entire compute budget. The corresponding measure there would be: Route deterministic validation failures to quarantine rather than retrying them as transient infrastructure errors. The stated situation instead calls for: Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages.
B:This alternative primarily addresses a different Azure Databricks problem: A finance transformation writes its output directly into a published table before reconciling totals against the source. The corresponding measure there would be: Write to a staging target, validate row totals and invariants, then perform the approved publication step. The stated situation instead calls for: Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages.
C:This alternative primarily addresses a different Azure Databricks problem: A regulated pipeline needs consistent transaction boundaries across multiple table writes but updates are published one at a time. The corresponding measure there would be: Design a compatible atomic publication or versioned readiness contract so consumers see consistent results. The stated situation instead calls for: Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages.
Question 531
A pipeline owner declines the action: Inspect driver collection and query planning behavior before simply enlarging every worker. The business situation is: A SQL transformation consumes large amounts of memory on the driver despite executor tasks processing most source partitions. What specific failure can follow?
- An untrusted library or unrestricted cluster edit can affect other users' sessions or access to protected data.
- Unmeasured tuning changes can hide the original bottleneck or increase compute costs.
- Ignoring startup overhead can make small jobs miss a narrow delivery window even when their Spark tasks are fast.
- Blindly increasing worker nodes leaves driver out-of-memory failures unchanged and increases cost.
Explanation
Correct Answer
D:The direct residual exposure is: Blindly increasing worker nodes leaves driver out-of-memory failures unchanged and increases cost. The triggering condition is: A SQL transformation consumes large amounts of memory on the driver despite executor tasks processing most source partitions. The omitted safeguard is: Inspect driver collection and query planning behavior before simply enlarging every worker. This matters because: Driver-side collection and planning bottlenecks cannot be corrected solely by scaling executor count. A corrective test would show: Spark UI and driver metrics show bounded driver memory after correcting the collection pattern.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A multi-team interactive cluster permits users to attach arbitrary unapproved libraries and run notebooks under broad shared privileges. Its preventative action would be: Apply suitable compute access modes, policies and entitlements that restrict who can attach or modify shared compute. The most direct exposure in the current scenario is: Blindly increasing worker nodes leaves driver out-of-memory failures unchanged and increases cost.
B:This failure mode belongs mainly to another situation: A team turns on every performance setting for a workload without benchmarking whether disk spills or network shuffle dominate runtime. Its preventative action would be: Collect Spark stage profiles and test a targeted compute or Photon setting against comparable data. The most direct exposure in the current scenario is: Blindly increasing worker nodes leaves driver out-of-memory failures unchanged and increases cost.
C:This failure mode belongs mainly to another situation: A scheduled workflow launches dozens of short-lived classic clusters and loses several minutes each time waiting for virtual machines. Its preventative action would be: Evaluate compatible instance pools or supported serverless startup behavior rather than keeping unused all-purpose clusters awake. The most direct exposure in the current scenario is: Blindly increasing worker nodes leaves driver out-of-memory failures unchanged and increases cost.
Question 532
An Azure Databricks architect reviews the following problem: A governance team grants dataset access solely by a table's displayed name while records are moved to a new schema. What should the team implement FIRST?
- Configure supported Databricks audit logging and route relevant events to a secured retention destination.
- Run a scheduled ownership and grant recertification process using catalog metadata and audit events.
- Reconcile catalog object identity, lineage and current grants after schema or owner changes.
- Publish a restricted derived dataset through Delta Sharing after privacy and freshness validation.
Explanation
Correct Answer
C:The most appropriate action is: Reconcile catalog object identity, lineage and current grants after schema or owner changes. It addresses the stated condition: A governance team grants dataset access solely by a table's displayed name while records are moved to a new schema. The technical basis is: Permission reviews should follow actual securable objects and their dependencies rather than name assumptions. Without this control, A migrated table can retain unintended access or lose required controls after namespace changes. A direct acceptance check is: An access matrix tests current schema objects and verifies intended principals after migration.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A compliance auditor requests a record of who changed Unity Catalog grants but audit logs are not collected. The corresponding measure there would be: Configure supported Databricks audit logging and route relevant events to a secured retention destination. The stated situation instead calls for: Reconcile catalog object identity, lineage and current grants after schema or owner changes.
B:This alternative primarily addresses a different Azure Databricks problem: A company needs to review table-owner changes and permissions after organizational restructuring. The corresponding measure there would be: Run a scheduled ownership and grant recertification process using catalog metadata and audit events. The stated situation instead calls for: Reconcile catalog object identity, lineage and current grants after schema or owner changes.
D:This alternative primarily addresses a different Azure Databricks problem: A customer wants to share only a daily refreshed aggregate, not raw personally identifiable transactions. The corresponding measure there would be: Publish a restricted derived dataset through Delta Sharing after privacy and freshness validation. The stated situation instead calls for: Reconcile catalog object identity, lineage and current grants after schema or owner changes.
Question 533
An operator introduces: Add representative unit and integration tests asserting transformation results and schema contracts. The situation is: A release modifies transformation logic but tests only whether Python files compile. Which validation should be required before signoff?
- Rebuilding the prior release resolves identical pinned versions and passes the saved regression suite.
- A seeded test dataset returns documented expected outputs and intentionally bad inputs fail.
- A production run references an approved immutable commit and a prior version can be restored.
- A staging dashboard loads its documented fields and a breaking-change simulation is rejected.
Explanation
Correct Answer
B:The strongest acceptance proof is: A seeded test dataset returns documented expected outputs and intentionally bad inputs fail. It specifically verifies: Add representative unit and integration tests asserting transformation results and schema contracts. The original issue was: A release modifies transformation logic but tests only whether Python files compile. The underlying mechanism is: Syntax checks do not validate business semantics or interaction with external tables. Without it, A logically incorrect aggregation can pass compilation while corrupting metrics.
Incorrect Answers
A:This result would validate a different measure: Pin package and runtime versions in the governed release artifact and record its dependency manifest. That is relevant to the distinct problem: A team cannot reproduce a prior release because package versions are resolved dynamically during deployment. Here the acceptance criterion is: A seeded test dataset returns documented expected outputs and intentionally bad inputs fail.
C:This result would validate a different measure: Store deployment source in Git with reviewed branches and pull requests before promoting changes. That is relevant to the distinct problem: Engineers edit production notebooks directly and lose the ability to identify which code change broke a pipeline. Here the acceptance criterion is: A seeded test dataset returns documented expected outputs and intentionally bad inputs fail.
D:This result would validate a different measure: Run end-to-end contract tests for dependent readers and obtain owner approval for breaking changes. That is relevant to the distinct problem: A sensitive SQL view is renamed in the release but the team does not test downstream dashboards before promotion. Here the acceptance criterion is: A seeded test dataset returns documented expected outputs and intentionally bad inputs fail.
Question 534
A solution applies: Use a supported Spark Structured Streaming or Lakeflow streaming source with an Event Hubs connection and checkpoint. It addresses: A product emits append-only telemetry to Azure Event Hubs and dashboards need near-real-time processing. Which operational outcome MOST directly verifies success?
- Two full snapshots produce one current inventory row per item and expected updates.
- A shuffled CDC test results in the correct highest-version value after replay.
- An event batch survives a controlled consumer restart and downstream totals match the expected sample.
- Injected duplicate and malformed source samples follow the intended dedupe or quarantine rules.
Explanation
Correct Answer
C:The strongest acceptance proof is: An event batch survives a controlled consumer restart and downstream totals match the expected sample. It specifically verifies: Use a supported Spark Structured Streaming or Lakeflow streaming source with an Event Hubs connection and checkpoint. The original issue was: A product emits append-only telemetry to Azure Event Hubs and dashboards need near-real-time processing. The underlying mechanism is: Message-bus streams require offset progress tracking and a durable state boundary. Without it, an uncheckpointed consumer can replay or omit events after a restart.
Incorrect Answers
A:This result would validate a different measure: Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records. That is relevant to the distinct problem: A daily file load uses append-only mode for periodic snapshots of the same full inventory. Here the acceptance criterion is: An event batch survives a controlled consumer restart and downstream totals match the expected sample.
B:This result would validate a different measure: Apply ordered CDC sequencing using source change version or timestamp and consistent delete handling. That is relevant to the distinct problem: A transactional feed produces out-of-order CDC events and a naive MERGE overwrites a newer status with an older update. Here the acceptance criterion is: An event batch survives a controlled consumer restart and downstream totals match the expected sample.
D:This result would validate a different measure: Define Lakeflow Spark Declarative Pipelines source options, incremental behavior and expectations with a durable target. That is relevant to the distinct problem: A declarative pipeline loads cloud object files continuously but its configuration omits how duplicate files and bad records are handled. Here the acceptance criterion is: An event batch survives a controlled consumer restart and downstream totals match the expected sample.
Question 535
A pipeline owner declines the action: Define and validate environment-specific catalog, workspace and identity targets in deployment configuration. The business situation is: A multi-environment setup uses the same catalog for test and production because bundle target variables were omitted. What specific failure can follow?
- Duplicate jobs may run on the same schedule and load data twice.
- A wrong schedule can duplicate loads or miss critical reporting windows.
- A faulty last-minute edit can create an untested release with no accountable reviewer.
- A staging test might overwrite production assets when default variables point to shared locations.
Explanation
Correct Answer
D:The direct residual exposure is: A staging test might overwrite production assets when default variables point to shared locations. The triggering condition is: A multi-environment setup uses the same catalog for test and production because bundle target variables were omitted. The omitted safeguard is: Define and validate environment-specific catalog, workspace and identity targets in deployment configuration. This matters because: Deployment isolation depends on correct values for each environment's object references. A corrective test would show: A rendered staging plan references only staging catalogs and refuses production resource paths.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A REST deployment script creates duplicate jobs every time CI reruns after a timeout. Its preventative action would be: Use idempotent deployment methods or reconcile existing resource IDs before creating replacements. The most direct exposure in the current scenario is: A staging test might overwrite production assets when default variables point to shared locations.
B:This failure mode belongs mainly to another situation: A data engineer updates a bundled job trigger but another environment's deployment silently uses the wrong schedule. Its preventative action would be: Validate per-target bundle variable resolution and compare rendered job definitions before deployment. The most direct exposure in the current scenario is: A staging test might overwrite production assets when default variables point to shared locations.
C:This failure mode belongs mainly to another situation: A release process merges unreviewed notebook changes directly into the main branch minutes before a production run. Its preventative action would be: Protect the main branch with peer review, required tests and controlled promotion windows. The most direct exposure in the current scenario is: A staging test might overwrite production assets when default variables point to shared locations.
Question 536
The following control has NOT been implemented: Set task and job timeout limits with notifications and a documented retry and recovery route. In this case: A long-running job occasionally hangs but there is no configured runtime ceiling or escalation. Which exposure remains?
- Mixed processing dates across tasks can create unreconciled reports and repeated records.
- A missed overnight job can remain unnoticed until business users complain.
- A hung task can consume cluster hours indefinitely and prevent expected downstream refresh.
- Temporarily editing the main schedule may skip normal deliveries or replay wrong dates.
Explanation
Correct Answer
C:The direct residual exposure is: A hung task can consume cluster hours indefinitely and prevent expected downstream refresh. The triggering condition is: A long-running job occasionally hangs but there is no configured runtime ceiling or escalation. The omitted safeguard is: Set task and job timeout limits with notifications and a documented retry and recovery route. This matters because: Timeouts and incident routing bound the cost and impact of stalled workflow execution. A corrective test would show: A simulated stall exceeds the threshold, stops correctly and alerts the support team.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A workflow needs to pass a date parameter from a scheduled trigger into all its downstream notebook tasks. Its preventative action would be: Define validated job parameters and propagate them through supported task parameter bindings. The most direct exposure in the current scenario is: A hung task can consume cluster hours indefinitely and prevent expected downstream refresh.
B:This failure mode belongs mainly to another situation: A scheduled Lakeflow Job has no owner alert when its final task fails overnight. Its preventative action would be: Configure job failure notifications to approved recipients and validate incident routing. The most direct exposure in the current scenario is: A hung task can consume cluster hours indefinitely and prevent expected downstream refresh.
D:This failure mode belongs mainly to another situation: An ingestion job requires a one-time backfill without changing the future daily production schedule. Its preventative action would be: Run a controlled parameterized manual invocation for the historical interval while retaining the regular trigger. The most direct exposure in the current scenario is: A hung task can consume cluster hours indefinitely and prevent expected downstream refresh.
Question 537
An operator introduces: Maintain a governed schema contract and integration tests across both notebook and declarative pipeline paths. The situation is: An engineer changes the expected file schema in one notebook but forgets to update the Lakeflow pipeline definition. Which validation should be required before signoff?
- Injected critical and noncritical bad rows follow their respective approved actions and emit metrics.
- A controlled negative-age record triggers the configured expectation event without being silently published.
- Quality dashboards separate optional-null rates from prohibited null keys and meet agreed thresholds.
- Both pipeline paths validate the same additive and incompatible change samples consistently.
Explanation
Correct Answer
D:The strongest acceptance proof is: Both pipeline paths validate the same additive and incompatible change samples consistently. It specifically verifies: Maintain a governed schema contract and integration tests across both notebook and declarative pipeline paths. The original issue was: An engineer changes the expected file schema in one notebook but forgets to update the Lakeflow pipeline definition. The underlying mechanism is: Schema governance must span all ingestion paths that write to a common table. Without it, inconsistent schema assumptions create intermittent casts, quarantines or dropped columns.
Incorrect Answers
A:This result would validate a different measure: Configure expectations with an explicit fail, drop or retention behavior aligned to business severity. That is relevant to the distinct problem: A Lakeflow pipeline has a quality rule but nobody has decided whether invalid transactions should stop publishing. Here the acceptance criterion is: Both pipeline paths validate the same additive and incompatible change samples consistently.
B:This result would validate a different measure: Define a range expectation with metrics and the approved drop, fail or quarantine disposition. That is relevant to the distinct problem: A bronze stream sometimes includes ages below zero or above 120 yet the transformation casts every value to integer. Here the acceptance criterion is: Both pipeline paths validate the same additive and incompatible change samples consistently.
C:This result would validate a different measure: Profile source missingness and define field-specific required versus optional expectations. That is relevant to the distinct problem: A new customer feed is missing optional contact information for 8% of rows and the team treats every null as pipeline failure. Here the acceptance criterion is: Both pipeline paths validate the same additive and incompatible change samples consistently.
Question 538
A platform architect adopts: Use a Delta MERGE keyed by SKU with appropriate matched-update and not-matched-insert logic. For this requirement: A product catalog receives corrections for existing SKUs plus entirely new items on each incremental load. Which rationale MOST directly supports it?
- Filter placement and join predicates determine which rows contribute to a grouped measure.
- Aggregation alone cannot distinguish valid repeat enrollments from accidental duplicates.
- MERGE combines inserts and updates atomically against a keyed target.
- Append is generally simpler for immutable events than key-by-key mutation logic.
Explanation
Correct Answer
C:The decisive mechanism is: MERGE combines inserts and updates atomically against a keyed target. That explains the selected action: Use a Delta MERGE keyed by SKU with appropriate matched-update and not-matched-insert logic. In the original scenario: A product catalog receives corrections for existing SKUs plus entirely new items on each incremental load. The specific unmanaged risk is: Blind append creates multiple current versions and breaks consumer joins. Validate with: Two consecutive loads yield one current row per SKU, including corrected price and new items.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: An analyst calculates total revenue grouped by country but a join includes countries not approved for reporting. Its targeted response is: Apply the required filters and join conditions before aggregation and validate authorized output groups. The current case is governed more directly by: MERGE combines inserts and updates atomically against a keyed target.
B:This principle concerns a separate data-engineering condition: A student dataset uses SQL GROUP BY with unhandled duplicate enrollment rows before reporting course completion. Its targeted response is: Validate business keys and choose a deterministic latest-enrollment record before aggregation. The current case is governed more directly by: MERGE combines inserts and updates atomically against a keyed target.
D:This principle concerns a separate data-engineering condition: An append-only telemetry feed includes newly observed events but engineers use MERGE with expensive full-table scans on every insert. Its targeted response is: Use an incremental append pattern with suitable deduplication when source events cannot change. The current case is governed more directly by: MERGE combines inserts and updates atomically against a keyed target.
Question 539
The design team proposes: Review and revoke stale grants and any linked service credentials, preserving required internal ownership. The situation is: A vendor's service principal was granted global access for a one-month integration and the project ended. Which technical principle justifies the decision?
- Managed identity authentication avoids exposing reusable static secrets and ties authorization to a governed identity.
- Data access delegation need not transfer administrative ownership of data objects.
- Object discovery or BROWSE is not a grant to read its underlying data.
- Least privilege includes removing access when the business purpose expires.
Explanation
Correct Answer
D:The decisive mechanism is: Least privilege includes removing access when the business purpose expires. That explains the selected action: Review and revoke stale grants and any linked service credentials, preserving required internal ownership. In the original scenario: A vendor's service principal was granted global access for a one-month integration and the project ended. The specific unmanaged risk is: A retired vendor workload can continue reading business data through unchanged credentials. Validate with: After revocation the vendor principal cannot list or query protected objects while internal jobs remain healthy.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A scheduled cloud-data ingestion needs an Azure storage identity but engineers embed a reusable account key inside notebooks. Its targeted response is: Use a supported Azure managed identity and Unity Catalog storage credential or equivalent approved access path. The current case is governed more directly by: Least privilege includes removing access when the business purpose expires.
B:This principle concerns a separate data-engineering condition: A data steward wants to grant temporary read access without handing over Unity Catalog ownership of a schema. Its targeted response is: Issue time-bounded group membership or an approved scoped SELECT grant with an access review. The current case is governed more directly by: Least privilege includes removing access when the business purpose expires.
C:This principle concerns a separate data-engineering condition: An analyst can view the names of confidential tables and assumes browsing metadata implies permission to query records. Its targeted response is: Separate discoverability privileges from USE and SELECT privileges in the catalog permissions model. The current case is governed more directly by: Least privilege includes removing access when the business purpose expires.
Question 540
A team omits the recommended measure: Test the identity-aware filter function, its permissions and supported execution context on representative queries. Its environment shows: An administrator attaches a row-filter function that returns errors for unauthorized query contexts and treats the issue as data corruption. What is the MOST direct residual risk?
- Manual per-table rules can miss newly created columns and expose sensitive values.
- A broken policy function may deny intended users or expose inconsistent query results if bypassed.
- An object may be available to overprivileged identities in unintended contexts if either boundary is ignored.
- A retired vendor workload can continue reading business data through unchanged credentials.
Explanation
Correct Answer
B:The direct residual exposure is: A broken policy function may deny intended users or expose inconsistent query results if bypassed. The triggering condition is: An administrator attaches a row-filter function that returns errors for unauthorized query contexts and treats the issue as data corruption. The omitted safeguard is: Test the identity-aware filter function, its permissions and supported execution context on representative queries. This matters because: Fine-grained security functions are part of the query execution path and can affect availability. A corrective test would show: Approved and disallowed identities receive the expected records without filter execution errors.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A data compliance team needs consistent redaction of columns tagged as personal data across hundreds of catalog tables. Its preventative action would be: Use centrally governed tags and compatible Unity Catalog ABAC masking policies where supported. The most direct exposure in the current scenario is: A broken policy function may deny intended users or expose inconsistent query results if bypassed.
C:This failure mode belongs mainly to another situation: A team restricts access through a catalog-to-workspace binding but treats it as a replacement for data privileges. Its preventative action would be: Apply both the workspace binding and appropriate Unity Catalog object grants, then test each dimension. The most direct exposure in the current scenario is: A broken policy function may deny intended users or expose inconsistent query results if bypassed.
D:This failure mode belongs mainly to another situation: A vendor's service principal was granted global access for a one-month integration and the project ended. Its preventative action would be: Review and revoke stale grants and any linked service credentials, preserving required internal ownership. The most direct exposure in the current scenario is: A broken policy function may deny intended users or expose inconsistent query results if bypassed.