Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 89
Question 880
During a lakehouse implementation review, engineers observe: A long-running job using an unsupported custom system dependency is moved to serverless without checking feature compatibility. What is the BEST next action?
- Record and pin supported package versions in the workload deployment specification and test reproducibility.
- Review current serverless limitations and select compatible classic jobs compute when necessary.
- Configure bounded autoscaling using measured workload demand and verify maximum cost and startup behavior.
- Select stream-appropriate compute and lifecycle settings so monitoring and availability targets are met.
Explanation
Correct Answer
B:The most appropriate action is: Review current serverless limitations and select compatible classic jobs compute when necessary. It addresses the stated condition: A long-running job using an unsupported custom system dependency is moved to serverless without checking feature compatibility. The technical basis is: Serverless and classic compute have different configuration surfaces and supported dependency patterns. Without this control, A serverless migration may fail when a required library or environment customization cannot be installed. A direct acceptance check is: A compatibility matrix and staging execution demonstrate every necessary dependency on the chosen compute.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A notebook workflow imports a Python package without pinning its version, producing different results after an environment update. The corresponding measure there would be: Record and pin supported package versions in the workload deployment specification and test reproducibility. The stated situation instead calls for: Review current serverless limitations and select compatible classic jobs compute when necessary.
C:This alternative primarily addresses a different Azure Databricks problem: A data engineering job sometimes requires twice its usual executor capacity during month-end loads while weekdays are much smaller. The corresponding measure there would be: Configure bounded autoscaling using measured workload demand and verify maximum cost and startup behavior. The stated situation instead calls for: Review current serverless limitations and select compatible classic jobs compute when necessary.
D:This alternative primarily addresses a different Azure Databricks problem: A long-running streaming pipeline is configured with aggressive auto-termination during brief periods without arriving events. The corresponding measure there would be: Select stream-appropriate compute and lifecycle settings so monitoring and availability targets are met. The stated situation instead calls for: Review current serverless limitations and select compatible classic jobs compute when necessary.
Question 881
For this Microsoft DP-750 case: A long-running ingestion job persists its checkpoint in the same temporary folder that daily cleanup deletes. Which design or operational response BEST fits?
- Use Spark-native distributed read and transformation APIs or partition-safe processing.
- Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle.
- Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records.
- Evaluate supported Lakeflow Connect ingestion rather than hardcoding ad hoc database extraction.
Explanation
Correct Answer
B:The most appropriate action is: Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle. It addresses the stated condition: A long-running ingestion job persists its checkpoint in the same temporary folder that daily cleanup deletes. The technical basis is: Checkpoint loss removes the stream's reliable progress and state record. Without this control, deleting stream state can cause full reprocessing or skipped incremental data. A direct acceptance check is: A maintenance simulation retains the checkpoint and resumes ingestion from the expected offset.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. The corresponding measure there would be: Use Spark-native distributed read and transformation APIs or partition-safe processing. The stated situation instead calls for: Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle.
C:This alternative primarily addresses a different Azure Databricks problem: A daily file load uses append-only mode for periodic snapshots of the same full inventory. The corresponding measure there would be: Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records. The stated situation instead calls for: Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle.
D:This alternative primarily addresses a different Azure Databricks problem: A batch integration needs a connector-managed path from a supported operational source into the lakehouse. The corresponding measure there would be: Evaluate supported Lakeflow Connect ingestion rather than hardcoding ad hoc database extraction. The stated situation instead calls for: Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle.
Question 882
A pipeline owner declines the action: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements. The business situation is: An external sharing integration needs interoperable columnar read files but not native Delta updates. What specific failure can follow?
- Unnecessary overwrites increase compute cost and can cause data loss during partial reruns.
- Inner joining only currently known dimension rows can undercount legitimate transactions.
- Duplicate dimension keys can inflate sums when each fact joins more than one dimension row.
- Choosing Delta-specific mutation assumptions for ordinary Parquet files can break consumers.
Explanation
Correct Answer
D:The direct residual exposure is: Choosing Delta-specific mutation assumptions for ordinary Parquet files can break consumers. The triggering condition is: An external sharing integration needs interoperable columnar read files but not native Delta updates. The omitted safeguard is: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements. This matters because: File and table format choices depend on transactional features and consumer interoperability. A corrective test would show: The external reader loads approved columns and a documented compatibility test passes.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A raw event lake contains append-only records but a developer chooses daily full overwrite of the curated Delta table. Its preventative action would be: Choose incremental append or merge based on immutable versus mutable source behavior. The most direct exposure in the current scenario is: Choosing Delta-specific mutation assumptions for ordinary Parquet files can break consumers.
B:This failure mode belongs mainly to another situation: A late-arriving purchase refers to a customer identifier whose dimension record has not yet been ingested. Its preventative action would be: Define a deliberate unknown-member or late-dimension reconciliation pattern without silently dropping facts. The most direct exposure in the current scenario is: Choosing Delta-specific mutation assumptions for ordinary Parquet files can break consumers.
C:This failure mode belongs mainly to another situation: A finance dashboard needs monthly revenue by customer and must join customer dimensions without multiplying facts. Its preventative action would be: Design a conformed star schema with explicit fact grain and unique dimension keys. The most direct exposure in the current scenario is: Choosing Delta-specific mutation assumptions for ordinary Parquet files can break consumers.
Question 883
An implementation skips: Include performance and end-to-end tests using representative scale and data distributions. Engineers are observing: An engineering test suite passes on miniature in-memory data yet fails in production on skewed billion-row joins. What consequence is MOST likely?
- A release may finish successfully while critical reports cannot resolve their expected view.
- An exposed or disabled personal token can compromise or halt all production deployments.
- Skew and memory limits may remain invisible until a full-scale run misses its SLA.
- A numerically correct but semantically changed field can mislead business users.
Explanation
Correct Answer
C:The direct residual exposure is: Skew and memory limits may remain invisible until a full-scale run misses its SLA. The triggering condition is: An engineering test suite passes on miniature in-memory data yet fails in production on skewed billion-row joins. The omitted safeguard is: Include performance and end-to-end tests using representative scale and data distributions. This matters because: Correctness under small inputs does not establish workload performance at production cardinality. A corrective test would show: A load test includes known hot keys and verifies latency, spills and output totals within targets.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A sensitive SQL view is renamed in the release but the team does not test downstream dashboards before promotion. Its preventative action would be: Run end-to-end contract tests for dependent readers and obtain owner approval for breaking changes. The most direct exposure in the current scenario is: Skew and memory limits may remain invisible until a full-scale run misses its SLA.
B:This failure mode belongs mainly to another situation: A CI pipeline deploys Databricks assets under an engineer's personal token stored in repository variables. Its preventative action would be: Use a secure service identity and protected credential handling for bundle or REST API deployment. The most direct exposure in the current scenario is: Skew and memory limits may remain invisible until a full-scale run misses its SLA.
D:This failure mode belongs mainly to another situation: A developer changes a transformation with a user-visible business effect but omits acceptance testing by the data product owner. Its preventative action would be: Add user acceptance tests for the relevant reporting contract before production promotion. The most direct exposure in the current scenario is: Skew and memory limits may remain invisible until a full-scale run misses its SLA.
Question 884
A production data team leaves out: Use an incremental append pattern with suitable deduplication when source events cannot change. Its scenario is: An append-only telemetry feed includes newly observed events but engineers use MERGE with expensive full-table scans on every insert. Which failure mode is MOST relevant?
- Conflating zero with missing data hides legitimate stockouts and distorts service metrics.
- Duplicate orders inflate sales totals and can trigger incorrect downstream payouts.
- An unrestricted join can multiply keys and exaggerate customer counts.
- Unnecessary MERGE operations can add substantial cost and contention without correctness benefit.
Explanation
Correct Answer
D:The direct residual exposure is: Unnecessary MERGE operations can add substantial cost and contention without correctness benefit. The triggering condition is: An append-only telemetry feed includes newly observed events but engineers use MERGE with expensive full-table scans on every insert. The omitted safeguard is: Use an incremental append pattern with suitable deduplication when source events cannot change. This matters because: Append is generally simpler for immutable events than key-by-key mutation logic. A corrective test would show: A timed append comparison meets the freshness and duplicate controls with lower compute consumption.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A team filters out null quantities but needs to preserve rows with zero as valid inventory states. Its preventative action would be: Implement explicit null predicates rather than treating zero and null as equivalent. The most direct exposure in the current scenario is: Unnecessary MERGE operations can add substantial cost and contention without correctness benefit.
B:This failure mode belongs mainly to another situation: An incoming orders table contains repeated order IDs because a vendor retransmits the same daily extract. Its preventative action would be: Apply a deterministic deduplication rule based on stable order identity and source version before aggregation. The most direct exposure in the current scenario is: Unnecessary MERGE operations can add substantial cost and contention without correctness benefit.
C:This failure mode belongs mainly to another situation: A team wants only customer IDs present in both systems but uses an inner join that multiplies repeated rows. Its preventative action would be: Use an appropriate intersect or semi-join operation based on set and duplicate requirements. The most direct exposure in the current scenario is: Unnecessary MERGE operations can add substantial cost and contention without correctness benefit.
Question 885
A solution applies: Publish a restricted derived dataset through Delta Sharing after privacy and freshness validation. It addresses: A customer wants to share only a daily refreshed aggregate, not raw personally identifiable transactions. Which operational outcome MOST directly verifies success?
- Authorized privacy reviewers see full test values while ordinary report roles see only permitted redactions.
- A test recipient reads the expected daily aggregate and cannot enumerate underlying detailed transaction records.
- An access matrix tests current schema objects and verifies intended principals after migration.
- A data steward can find the certified revenue definition and verify the correct field in Catalog Explorer.
Explanation
Correct Answer
B:The strongest acceptance proof is: A test recipient reads the expected daily aggregate and cannot enumerate underlying detailed transaction records. It specifically verifies: Publish a restricted derived dataset through Delta Sharing after privacy and freshness validation. The original issue was: A customer wants to share only a daily refreshed aggregate, not raw personally identifiable transactions. The underlying mechanism is: Data sharing scope should minimize disclosed fields and rows and specify a refresh contract. Without it, sharing raw transaction tables can disclose protected customer attributes unnecessarily.
Incorrect Answers
A:This result would validate a different measure: Develop an approved role-aware column mask and test its outputs across supported query clients. That is relevant to the distinct problem: A data product stores PII in a shared Delta table but several reports require different masking for each audience. Here the acceptance criterion is: A test recipient reads the expected daily aggregate and cannot enumerate underlying detailed transaction records.
C:This result would validate a different measure: Reconcile catalog object identity, lineage and current grants after schema or owner changes. That is relevant to the distinct problem: A governance team grants dataset access solely by a table's displayed name while records are moved to a new schema. Here the acceptance criterion is: A test recipient reads the expected daily aggregate and cannot enumerate underlying detailed transaction records.
D:This result would validate a different measure: Add reviewed Unity Catalog table and column comments, owners and business definitions for consumer discovery. That is relevant to the distinct problem: A business glossary has table names but no indication of which field is net revenue versus gross sales. Here the acceptance criterion is: A test recipient reads the expected daily aggregate and cannot enumerate underlying detailed transaction records.
Question 886
A data engineering team faces this Azure Databricks requirement: A finance transformation writes its output directly into a published table before reconciling totals against the source. Which action is MOST appropriate?
- Write to a staging target, validate row totals and invariants, then perform the approved publication step.
- Parameterize environment-specific paths, catalogs and identities through governed deployment configuration.
- Design a task DAG with parallel independent ingestion and an aggregation dependency after both complete.
- Propagate nonrecoverable exceptions to Lakeflow Jobs and attach actionable error metadata.
Explanation
Correct Answer
A:The most appropriate action is: Write to a staging target, validate row totals and invariants, then perform the approved publication step. It addresses the stated condition: A finance transformation writes its output directly into a published table before reconciling totals against the source. The technical basis is: Staging plus validation separates construction of data from its declaration as trusted. Without this control, A bad transformation may make a broken dataset immediately visible to consumers. A direct acceptance check is: A seeded mismatch prevents cutover and the prior certified table remains queryable.
Incorrect Answers
B:This alternative primarily addresses a different Azure Databricks problem: A multi-team notebook uses hard-coded workspace names and credentials, making the same pipeline fail in staging. The corresponding measure there would be: Parameterize environment-specific paths, catalogs and identities through governed deployment configuration. The stated situation instead calls for: Write to a staging target, validate row totals and invariants, then perform the approved publication step.
C:This alternative primarily addresses a different Azure Databricks problem: A pipeline uses a single notebook to read four sources sequentially even though two can be processed independently. The corresponding measure there would be: Design a task DAG with parallel independent ingestion and an aggregation dependency after both complete. The stated situation instead calls for: Write to a staging target, validate row totals and invariants, then perform the approved publication step.
D:This alternative primarily addresses a different Azure Databricks problem: A workflow uses a notebook task that catches exceptions and returns success even when a write step fails. The corresponding measure there would be: Propagate nonrecoverable exceptions to Lakeflow Jobs and attach actionable error metadata. The stated situation instead calls for: Write to a staging target, validate row totals and invariants, then perform the approved publication step.
Question 887
A Databricks engineer recommends: Use idempotent writes or MERGE keyed by stable order identity for repeatable task execution. It responds to: A nightly pipeline rerun appends the same orders to the target and doubles totals after an operator presses Retry. What underlying mechanism BEST explains why?
- A recoverable pipeline should produce the same logical state after an equivalent retry.
- DAG-based orchestration coordinates data readiness and avoids race conditions across dependent tasks.
- Notebooks and declarative pipelines serve different orchestration and transformation patterns.
- Task-level repair reduces unnecessary recomputation while preserving dependency correctness.
Explanation
Correct Answer
A:The decisive mechanism is: A recoverable pipeline should produce the same logical state after an equivalent retry. That explains the selected action: Use idempotent writes or MERGE keyed by stable order identity for repeatable task execution. In the original scenario: A nightly pipeline rerun appends the same orders to the target and doubles totals after an operator presses Retry. The specific unmanaged risk is: At-least-once task execution can duplicate side effects when outputs are append-only. Validate with: Two executions of the same input batch yield one approved current record per order.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. Its targeted response is: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. The current case is governed more directly by: A recoverable pipeline should produce the same logical state after an equivalent retry.
C:This principle concerns a separate data-engineering condition: An ad hoc machine-learning preparation task has complex custom Python orchestration and no declarative table graph. Its targeted response is: Select a notebook task within Lakeflow Jobs when procedural control and custom libraries fit the requirements. The current case is governed more directly by: A recoverable pipeline should produce the same logical state after an equivalent retry.
D:This principle concerns a separate data-engineering condition: A bulk repair executes all pipeline tasks again even when only the final reporting notebook failed. Its targeted response is: Separate tasks and use the supported repair-run workflow at the failed stage when dependencies remain valid. The current case is governed more directly by: A recoverable pipeline should produce the same logical state after an equivalent retry.
Question 888
The following control has NOT been implemented: Compare workload contention and resource reservations, then isolate or resize the critical job compute. In this case: A production notebook intermittently times out only when many other jobs share the same classic cluster. Which exposure remains?
- Excessive false alerts can obscure serious freshness incidents.
- A critical SLA may depend on unrelated teams' scheduling and resource usage.
- A warehouse sized to average load may breach dashboard p95 latency during business opening.
- Ignoring heavy spills can cause cascading I/O latency and missed pipeline deadlines.
Explanation
Correct Answer
B:The direct residual exposure is: A critical SLA may depend on unrelated teams' scheduling and resource usage. The triggering condition is: A production notebook intermittently times out only when many other jobs share the same classic cluster. The omitted safeguard is: Compare workload contention and resource reservations, then isolate or resize the critical job compute. This matters because: Shared compute competition can cause unpredictable task latency independent of input correctness. A corrective test would show: The isolated benchmark meets its percentile duration under simultaneous unrelated workload pressure.
Incorrect Answers
A:This failure mode belongs mainly to another situation: An alert triggers on every harmless microbatch delay while real hour-long ingestion stalls go unnoticed. Its preventative action would be: Define meaningful latency, error and backlog thresholds with alert routing based on the business SLO. The most direct exposure in the current scenario is: A critical SLA may depend on unrelated teams' scheduling and resource usage.
C:This failure mode belongs mainly to another situation: An interactive warehouse has high query queues during the morning but low CPU utilization after lunch. Its preventative action would be: Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages. The most direct exposure in the current scenario is: A critical SLA may depend on unrelated teams' scheduling and resource usage.
D:This failure mode belongs mainly to another situation: A large aggregation has repeated shuffle spills to disk even though source scans finish quickly. Its preventative action would be: Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage. The most direct exposure in the current scenario is: A critical SLA may depend on unrelated teams' scheduling and resource usage.
Question 889
A production data team leaves out: Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle. Its scenario is: A long-running ingestion job persists its checkpoint in the same temporary folder that daily cleanup deletes. Which failure mode is MOST relevant?
- Append-only full extracts can multiply current records and leave deleted source rows active.
- Appending full inventories multiplies items and distorts on-hand quantities.
- Deleting stream state can cause full reprocessing or skipped incremental data.
- Daily batches cannot satisfy short update deadlines even when their processing is correct.
Explanation
Correct Answer
C:The direct residual exposure is: Deleting stream state can cause full reprocessing or skipped incremental data. The triggering condition is: A long-running ingestion job persists its checkpoint in the same temporary folder that daily cleanup deletes. The omitted safeguard is: Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle. This matters because: Checkpoint loss removes the stream's reliable progress and state record. A corrective test would show: A maintenance simulation retains the checkpoint and resumes ingestion from the expected offset.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A SaaS database updates and deletes customer records but a pipeline appends every nightly full extract to a Delta table. Its preventative action would be: Use a supported change-data-capture feed and apply insert, update and delete semantics to the target. The most direct exposure in the current scenario is: Deleting stream state can cause full reprocessing or skipped incremental data.
B:This failure mode belongs mainly to another situation: A daily file load uses append-only mode for periodic snapshots of the same full inventory. Its preventative action would be: Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records. The most direct exposure in the current scenario is: Deleting stream state can cause full reprocessing or skipped incremental data.
D:This failure mode belongs mainly to another situation: A team selects batch processing for customer account changes though downstream applications require minute-scale freshness. Its preventative action would be: Use supported streaming or sufficiently frequent incremental CDC with measured end-to-end latency. The most direct exposure in the current scenario is: Deleting stream state can cause full reprocessing or skipped incremental data.
Question 890
A data engineering team faces this Azure Databricks requirement: A Spark workload repeatedly fails with executor out-of-memory errors on a high-cardinality groupBy. Which action is MOST appropriate?
- Inspect the Spark UI DAG and skewed-key distribution before changing cluster size.
- Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages.
- Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change.
- Use run-level compute utilization, cost and stage metrics to identify unused capacity.
Explanation
Correct Answer
C:The most appropriate action is: Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change. It addresses the stated condition: A Spark workload repeatedly fails with executor out-of-memory errors on a high-cardinality groupBy. The technical basis is: Wide aggregations redistribute data and can create memory-intensive hot partitions. Without this control, blindly restarting the cluster may reproduce the same failure at the same stage. A direct acceptance check is: A controlled representative load finishes without OOM and output group counts reconcile.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A production Spark query is slow and its stage graph shows one task running far longer than all peers. The corresponding measure there would be: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size. The stated situation instead calls for: Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change.
B:This alternative primarily addresses a different Azure Databricks problem: An interactive warehouse has high query queues during the morning but low CPU utilization after lunch. The corresponding measure there would be: Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages. The stated situation instead calls for: Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change.
D:This alternative primarily addresses a different Azure Databricks problem: A pipeline experiences escalating costs after a team increases node counts but execution duration barely changes. The corresponding measure there would be: Use run-level compute utilization, cost and stage metrics to identify unused capacity. The stated situation instead calls for: Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change.