Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 65
Question 640
A solution applies: Design a separate failure-handling or completion branch with the supported run-if logic for alerts and cleanup. It addresses: A source task fails yet the monitoring task never executes because all downstream tasks require upstream success. Which operational outcome MOST directly verifies success?
- Invalid input blocks publication while the previous approved dataset remains available.
- A staged failure triggers the documented alert and cleanup while skipping dependent publication.
- A simulated hung request times out, alerts the owner and avoids publishing incomplete data.
- A delayed-ingestion test prevents gold refresh until bronze and silver are completed successfully.
Explanation
Correct Answer
B:The strongest acceptance proof is: A staged failure triggers the documented alert and cleanup while skipping dependent publication. It specifically verifies: Design a separate failure-handling or completion branch with the supported run-if logic for alerts and cleanup. The original issue was: A source task fails yet the monitoring task never executes because all downstream tasks require upstream success. The underlying mechanism is: Orchestration may need error paths that execute regardless of normal success dependencies. Without it, an upstream failure can remain invisible and leave temporary resources without cleanup.
Incorrect Answers
A:This result would validate a different measure: Place validations before publishing and use an atomic cutover or controlled target write. That is relevant to the distinct problem: A workflow writes to production tables before a validation task examines null keys and duplicates. Here the acceptance criterion is: A staged failure triggers the documented alert and cleanup while skipping dependent publication.
C:This result would validate a different measure: Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages. That is relevant to the distinct problem: A scheduled notebook contains both transformation SQL and a long-running external API loop that occasionally hangs. Here the acceptance criterion is: A staged failure triggers the documented alert and cleanup while skipping dependent publication.
D:This result would validate a different measure: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. That is relevant to the distinct problem: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. Here the acceptance criterion is: A staged failure triggers the documented alert and cleanup while skipping dependent publication.
Question 641
For this Azure Databricks issue: A team selects batch processing for customer account changes though downstream applications require minute-scale freshness. Engineers choose: Use supported streaming or sufficiently frequent incremental CDC with measured end-to-end latency. What acceptance evidence is BEST?
- A changed customer field reaches the approved target within the measured freshness SLO.
- A deliberately delayed file is ingested exactly once within the allowed arrival window.
- Replaying the same microbatch produces one target effect per business key.
- Readers observe a complete validated target before or after cutover, not partial data.
Explanation
Correct Answer
A:The strongest acceptance proof is: A changed customer field reaches the approved target within the measured freshness SLO. It specifically verifies: Use supported streaming or sufficiently frequent incremental CDC with measured end-to-end latency. The original issue was: A team selects batch processing for customer account changes though downstream applications require minute-scale freshness. The underlying mechanism is: Freshness requirements should determine batch versus streaming design. Without it, daily batches cannot satisfy short update deadlines even when their processing is correct.
Incorrect Answers
B:This result would validate a different measure: Use incremental file ingestion with explicit late-arrival cutoffs and a measurable source-to-target watermark. That is relevant to the distinct problem: A Databricks job receives daily structured extracts but an upstream schedule occasionally delivers files late. Here the acceptance criterion is: A changed customer field reaches the approved target within the measured freshness SLO.
C:This result would validate a different measure: Implement idempotent foreachBatch or compatible transactional sink logic keyed by epoch or business identity. That is relevant to the distinct problem: A stream writes into an external sink without stable batch identities and causes duplicate side effects on retry. Here the acceptance criterion is: A changed customer field reaches the approved target within the measured freshness SLO.
D:This result would validate a different measure: Choose a controlled CREATE OR REPLACE TABLE or compatible atomic table-replacement strategy and validate schema/permissions. That is relevant to the distinct problem: An existing table must be rebuilt from corrected historical files while its readers require a consistent atomic replacement. Here the acceptance criterion is: A changed customer field reaches the approved target within the measured freshness SLO.
Question 642
An Azure Databricks architect reviews the following problem: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. What should the team implement FIRST?
- Persist necessary source metadata and checkpoint lineage for troubleshooting and replay.
- Coordinate ADF orchestration with the appropriate Databricks task and Unity Catalog access model.
- Use Spark-native distributed read and transformation APIs or partition-safe processing.
- Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options.
Explanation
Correct Answer
C:The most appropriate action is: Use Spark-native distributed read and transformation APIs or partition-safe processing. It addresses the stated condition: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. The technical basis is: Pulling distributed data to a single driver defeats parallel execution and memory limits. Without this control, driver out-of-memory can fail the load despite adequate executor resources. A direct acceptance check is: The workload completes without driver collection and Spark metrics show distributed processing.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A pipeline ingests Event Hubs messages into a bronze table but never captures the partition, offset or event timestamp. The corresponding measure there would be: Persist necessary source metadata and checkpoint lineage for troubleshooting and replay. The stated situation instead calls for: Use Spark-native distributed read and transformation APIs or partition-safe processing.
B:This alternative primarily addresses a different Azure Databricks problem: A data product needs to copy a supported Azure Data Factory pipeline output into governed Databricks tables. The corresponding measure there would be: Coordinate ADF orchestration with the appropriate Databricks task and Unity Catalog access model. The stated situation instead calls for: Use Spark-native distributed read and transformation APIs or partition-safe processing.
D:This alternative primarily addresses a different Azure Databricks problem: An integration performs a one-off initial import of an existing CSV folder into a Delta table. The corresponding measure there would be: Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options. The stated situation instead calls for: Use Spark-native distributed read and transformation APIs or partition-safe processing.
Question 643
A solution applies: Define and validate environment-specific catalog, workspace and identity targets in deployment configuration. It addresses: A multi-environment setup uses the same catalog for test and production because bundle target variables were omitted. Which operational outcome MOST directly verifies success?
- A test UI change is flagged and the approved deployment restores the intended configuration.
- A rendered staging plan references only staging catalogs and refuses production resource paths.
- A seeded test dataset returns documented expected outputs and intentionally bad inputs fail.
- An unauthorized direct push is rejected and a reviewed pull request completes all required gates.
Explanation
Correct Answer
B:The strongest acceptance proof is: A rendered staging plan references only staging catalogs and refuses production resource paths. It specifically verifies: Define and validate environment-specific catalog, workspace and identity targets in deployment configuration. The original issue was: A multi-environment setup uses the same catalog for test and production because bundle target variables were omitted. The underlying mechanism is: Deployment isolation depends on correct values for each environment's object references. Without it, A staging test might overwrite production assets when default variables point to shared locations.
Incorrect Answers
A:This result would validate a different measure: Use drift detection or a release reconciliation that compares deployed objects against the versioned bundle. That is relevant to the distinct problem: A Databricks bundle deployment differs from what appears in the repository because someone edits the job in the UI afterward. Here the acceptance criterion is: A rendered staging plan references only staging catalogs and refuses production resource paths.
C:This result would validate a different measure: Add representative unit and integration tests asserting transformation results and schema contracts. That is relevant to the distinct problem: A release modifies transformation logic but tests only whether Python files compile. Here the acceptance criterion is: A rendered staging plan references only staging catalogs and refuses production resource paths.
D:This result would validate a different measure: Protect the main branch with peer review, required tests and controlled promotion windows. That is relevant to the distinct problem: A release process merges unreviewed notebook changes directly into the main branch minutes before a production run. Here the acceptance criterion is: A rendered staging plan references only staging catalogs and refuses production resource paths.
Question 644
A platform architect adopts: Implement idempotent foreachBatch or compatible transactional sink logic keyed by epoch or business identity. For this requirement: A stream writes into an external sink without stable batch identities and causes duplicate side effects on retry. Which rationale MOST directly supports it?
- Streaming source metadata supports provenance and exact-range incident investigation.
- Declarative ingestion still requires explicit quality and replay assumptions.
- Structured Streaming may retry microbatches, so external effects need idempotency.
- Watermarks define the tolerated late-event horizon for stateful stream processing.
Explanation
Correct Answer
C:The decisive mechanism is: Structured Streaming may retry microbatches, so external effects need idempotency. That explains the selected action: Implement idempotent foreachBatch or compatible transactional sink logic keyed by epoch or business identity. In the original scenario: A stream writes into an external sink without stable batch identities and causes duplicate side effects on retry. The specific unmanaged risk is: Retries can duplicate invoices or notifications even if the source checkpoint is correct. Validate with: Replaying the same microbatch produces one target effect per business key.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A pipeline ingests Event Hubs messages into a bronze table but never captures the partition, offset or event timestamp. Its targeted response is: Persist necessary source metadata and checkpoint lineage for troubleshooting and replay. The current case is governed more directly by: Structured Streaming may retry microbatches, so external effects need idempotency.
B:This principle concerns a separate data-engineering condition: A declarative pipeline loads cloud object files continuously but its configuration omits how duplicate files and bad records are handled. Its targeted response is: Define Lakeflow Spark Declarative Pipelines source options, incremental behavior and expectations with a durable target. The current case is governed more directly by: Structured Streaming may retry microbatches, so external effects need idempotency.
D:This principle concerns a separate data-engineering condition: A stream must group clicks into five-minute event-time windows while some mobile devices transmit data several minutes late. Its targeted response is: Configure event-time windowing and an appropriate watermark based on observed lateness. The current case is governed more directly by: Structured Streaming may retry microbatches, so external effects need idempotency.
Question 645
The design team proposes: Recreate or update supported external table registration using an approved location and correct access controls. The situation is: A managed Delta table is migrated to external storage but the team does not update table registration or grant scopes. Which technical principle justifies the decision?
- Metadata definitions, locations and privileges must align after a table lifecycle transition.
- Reading a governed object does not imply permission to create or administer peer objects.
- Catalog separation and workspace access controls can help enforce organizational and geography boundaries.
- The object type should reflect whether data is computed on read or persisted and maintained.
Explanation
Correct Answer
A:The decisive mechanism is: Metadata definitions, locations and privileges must align after a table lifecycle transition. That explains the selected action: Recreate or update supported external table registration using an approved location and correct access controls. In the original scenario: A managed Delta table is migrated to external storage but the team does not update table registration or grant scopes. The specific unmanaged risk is: Data may remain readable by direct path while the intended catalog table fails authorization. Validate with: Catalog lookup and controlled read work from the registered target while unapproved principals cannot bypass it.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A team expects every analyst to CREATE SCHEMA simply because they can SELECT tables in an existing catalog. Its targeted response is: Grant precise container-use and object-creation privileges based on actual job roles. The current case is governed more directly by: Metadata definitions, locations and privileges must align after a table lifecycle transition.
C:This principle concerns a separate data-engineering condition: A platform architect creates one catalog for subsidiaries with incompatible regional data-residency permissions. Its targeted response is: Split catalogs or governance boundaries along residency and ownership requirements and validate workspace bindings. The current case is governed more directly by: Metadata definitions, locations and privileges must align after a table lifecycle transition.
D:This principle concerns a separate data-engineering condition: A data pipeline creates a physical table for a reporting transformation that would be better maintained as a query definition. Its targeted response is: Choose a view, materialized view or stored table based on freshness, cost and refresh responsibilities. The current case is governed more directly by: Metadata definitions, locations and privileges must align after a table lifecycle transition.
Question 646
A data engineering team faces this Azure Databricks requirement: A data owner builds a masked view but analysts can still query the underlying sensitive table directly. Which action is MOST appropriate?
- Grant a dedicated writer identity the minimum supported modification permissions and revoke broad user write grants.
- Configure a tested Unity Catalog row filter or suitable ABAC policy based on the authorized identity.
- Use a supported Azure managed identity and Unity Catalog storage credential or equivalent approved access path.
- Revoke inappropriate base-table access and publish only the authorized view or policy-governed object.
Explanation
Correct Answer
D:The most appropriate action is: Revoke inappropriate base-table access and publish only the authorized view or policy-governed object. It addresses the stated condition: A data owner builds a masked view but analysts can still query the underlying sensitive table directly. The technical basis is: Security of a view depends on the complete authorization path, not merely its SELECT projection. Without this control, A user may bypass the masked presentation by reading the unmasked source table. A direct acceptance check is: Direct source queries are denied for analysts while the governed view produces the approved masked data.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: An engineering pipeline attempts to insert records into a production Delta table using an account with SELECT only. The corresponding measure there would be: Grant a dedicated writer identity the minimum supported modification permissions and revoke broad user write grants. The stated situation instead calls for: Revoke inappropriate base-table access and publish only the authorized view or policy-governed object.
B:This alternative primarily addresses a different Azure Databricks problem: A user must see only the East region rows of a common sales table while managers see all territories. The corresponding measure there would be: Configure a tested Unity Catalog row filter or suitable ABAC policy based on the authorized identity. The stated situation instead calls for: Revoke inappropriate base-table access and publish only the authorized view or policy-governed object.
C:This alternative primarily addresses a different Azure Databricks problem: A scheduled cloud-data ingestion needs an Azure storage identity but engineers embed a reusable account key inside notebooks. The corresponding measure there would be: Use a supported Azure managed identity and Unity Catalog storage credential or equivalent approved access path. The stated situation instead calls for: Revoke inappropriate base-table access and publish only the authorized view or policy-governed object.
Question 647
For this Microsoft DP-750 case: A team attempts to apply both ZORDER and liquid clustering to the same Delta table. Which design or operational response BEST fits?
- Choose and document a supported format and catalog integration with explicit read/write compatibility tests.
- Implement the required SCD Type 1 overwrite with stable business keys and validated updates.
- Choose one compatible data layout strategy based on table format, platform support and query evidence.
- Design event-time fields, ingestion timestamps and late-data behavior suited to replay and temporal queries.
Explanation
Correct Answer
C:The most appropriate action is: Choose one compatible data layout strategy based on table format, platform support and query evidence. It addresses the stated condition: A team attempts to apply both ZORDER and liquid clustering to the same Delta table. The technical basis is: Liquid clustering and traditional ZORDER/partitioning have compatibility constraints and different maintenance paths. Without this control, mixing incompatible layout mechanisms can fail operations or leave the team relying on nonexistent optimizations. A direct acceptance check is: A supported optimization command completes and query plans show the chosen layout works.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A cross-engine consumer expects a Delta table but source is plain Iceberg and engineers assume identical mutation semantics. The corresponding measure there would be: Choose and document a supported format and catalog integration with explicit read/write compatibility tests. The stated situation instead calls for: Choose one compatible data layout strategy based on table format, platform support and query evidence.
B:This alternative primarily addresses a different Azure Databricks problem: A reference table stores only the latest airport name and business users explicitly do not need historical values. The corresponding measure there would be: Implement the required SCD Type 1 overwrite with stable business keys and validated updates. The stated situation instead calls for: Choose one compatible data layout strategy based on table format, platform support and query evidence.
D:This alternative primarily addresses a different Azure Databricks problem: A lakehouse receives sensor observations with event timestamps that arrive out of order. The corresponding measure there would be: Design event-time fields, ingestion timestamps and late-data behavior suited to replay and temporal queries. The stated situation instead calls for: Choose one compatible data layout strategy based on table format, platform support and query evidence.
Question 648
A data product owner adopts: Choose the appropriate retry and repair-run functionality based on failed-task state and valid dependencies. The reason is: A team assumes that automatic job retries are the same as recovering a failed multi-step run at its last unsuccessful task. What observed result BEST proves effectiveness?
- A controlled dual-trigger test respects the run limit and produces one correct final target.
- Repeated injected failures hit the configured stop threshold and create a human-actionable incident.
- A simulated stall exceeds the threshold, stops correctly and alerts the support team.
- A failed downstream task is repaired without repeating successful non-idempotent upstream work.
Explanation
Correct Answer
D:The strongest acceptance proof is: A failed downstream task is repaired without repeating successful non-idempotent upstream work. It specifically verifies: Choose the appropriate retry and repair-run functionality based on failed-task state and valid dependencies. The original issue was: A team assumes that automatic job retries are the same as recovering a failed multi-step run at its last unsuccessful task. The underlying mechanism is: Repair runs and per-task retries address different recovery granularities. Without it, replaying an entire DAG without need can duplicate side effects and increase cost.
Incorrect Answers
A:This result would validate a different measure: Configure appropriate maximum concurrent runs and idempotent target writes to control overlap. That is relevant to the distinct problem: A daily data job runs twice during an incident and both executions write to the same target. Here the acceptance criterion is: A failed downstream task is repaired without repeating successful non-idempotent upstream work.
B:This result would validate a different measure: Configure bounded restarts and alerting with root-cause investigation for repeated failures. That is relevant to the distinct problem: A continuously running pipeline is scheduled to restart automatically but restart storms consume a growing compute budget. Here the acceptance criterion is: A failed downstream task is repaired without repeating successful non-idempotent upstream work.
C:This result would validate a different measure: Set task and job timeout limits with notifications and a documented retry and recovery route. That is relevant to the distinct problem: A long-running job occasionally hangs but there is no configured runtime ceiling or escalation. Here the acceptance criterion is: A failed downstream task is repaired without repeating successful non-idempotent upstream work.
Question 649
The following control has NOT been implemented: Write to a staging target, validate row totals and invariants, then perform the approved publication step. In this case: A finance transformation writes its output directly into a published table before reconciling totals against the source. Which exposure remains?
- At-least-once task execution can duplicate side effects when outputs are append-only.
- Infinite retries on bad records can block valid work and multiply costs.
- One late-stage error may force a costly replay of all preceding successful processing.
- A bad transformation may make a broken dataset immediately visible to consumers.
Explanation
Correct Answer
D:The direct residual exposure is: A bad transformation may make a broken dataset immediately visible to consumers. The triggering condition is: A finance transformation writes its output directly into a published table before reconciling totals against the source. The omitted safeguard is: Write to a staging target, validate row totals and invariants, then perform the approved publication step. This matters because: Staging plus validation separates construction of data from its declaration as trusted. A corrective test would show: A seeded mismatch prevents cutover and the prior certified table remains queryable.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A nightly pipeline rerun appends the same orders to the target and doubles totals after an operator presses Retry. Its preventative action would be: Use idempotent writes or MERGE keyed by stable order identity for repeatable task execution. The most direct exposure in the current scenario is: A bad transformation may make a broken dataset immediately visible to consumers.
B:This failure mode belongs mainly to another situation: A pipeline automatically retries malformed source records indefinitely and consumes its entire compute budget. Its preventative action would be: Route deterministic validation failures to quarantine rather than retrying them as transient infrastructure errors. The most direct exposure in the current scenario is: A bad transformation may make a broken dataset immediately visible to consumers.
C:This failure mode belongs mainly to another situation: A multi-step notebook workflow handles source extraction and final reporting in one cell sequence with no recoverable checkpoints. Its preventative action would be: Split ingestion, validation and publishing into explicit orchestrated tasks with durable intermediate outputs. The most direct exposure in the current scenario is: A bad transformation may make a broken dataset immediately visible to consumers.
Question 650
The design team proposes: Run a controlled parameterized manual invocation for the historical interval while retaining the regular trigger. The situation is: An ingestion job requires a one-time backfill without changing the future daily production schedule. Which technical principle justifies the decision?
- Backfills should be separated from ordinary schedules to avoid accidental duplicate processing.
- Concurrency management prevents competing executions from corrupting shared mutable state.
- Job task state transitions follow dependency and conditional execution rules.
- Consistent parameter passing ensures dependent tasks operate on the same business interval.
Explanation
Correct Answer
A:The decisive mechanism is: Backfills should be separated from ordinary schedules to avoid accidental duplicate processing. That explains the selected action: Run a controlled parameterized manual invocation for the historical interval while retaining the regular trigger. In the original scenario: An ingestion job requires a one-time backfill without changing the future daily production schedule. The specific unmanaged risk is: Temporarily editing the main schedule may skip normal deliveries or replay wrong dates. Validate with: The historical batch completes with reconciled counts and next daily scheduled run remains unchanged.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A daily data job runs twice during an incident and both executions write to the same target. Its targeted response is: Configure appropriate maximum concurrent runs and idempotent target writes to control overlap. The current case is governed more directly by: Backfills should be separated from ordinary schedules to avoid accidental duplicate processing.
C:This principle concerns a separate data-engineering condition: A job owner cannot explain why a task skipped execution after a predecessor failed. Its targeted response is: Review explicit dependency and run-if settings and validate skip behavior through a staged failure. The current case is governed more directly by: Backfills should be separated from ordinary schedules to avoid accidental duplicate processing.
D:This principle concerns a separate data-engineering condition: A workflow needs to pass a date parameter from a scheduled trigger into all its downstream notebook tasks. Its targeted response is: Define validated job parameters and propagate them through supported task parameter bindings. The current case is governed more directly by: Backfills should be separated from ordinary schedules to avoid accidental duplicate processing.