Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 76
Question 750
An implementation skips: Use an appropriate intersect or semi-join operation based on set and duplicate requirements. Engineers are observing: A team wants only customer IDs present in both systems but uses an inner join that multiplies repeated rows. What consequence is MOST likely?
- Refund amounts treated as sales inflate net revenue even when record counts match.
- Orders near midnight may be reported under the wrong business day.
- Unknown statuses may be dropped or duplicated if pivot categories are assumed without validation.
- An unrestricted join can multiply keys and exaggerate customer counts.
Explanation
Correct Answer
D:The direct residual exposure is: An unrestricted join can multiply keys and exaggerate customer counts. The triggering condition is: A team wants only customer IDs present in both systems but uses an inner join that multiplies repeated rows. The omitted safeguard is: Use an appropriate intersect or semi-join operation based on set and duplicate requirements. This matters because: Set operations and joins differ in duplicate behavior and result cardinality. A corrective test would show: A test with repeated IDs produces the expected unique or multiset intersection by policy.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A daily revenue report double-counts refunds because both debit and credit rows are summed as positive numbers. Its preventative action would be: Normalize transaction sign semantics before grouping by business date and customer. The most direct exposure in the current scenario is: An unrestricted join can multiply keys and exaggerate customer counts.
B:This failure mode belongs mainly to another situation: A date field mixes UTC timestamps and local store times but a pipeline groups all values by the raw date string. Its preventative action would be: Normalize time zone semantics before extracting business-local reporting dates. The most direct exposure in the current scenario is: An unrestricted join can multiply keys and exaggerate customer counts.
C:This failure mode belongs mainly to another situation: A quality report needs one column for each product status category but stores status as separate rows. Its preventative action would be: Apply a controlled pivot with a defined set of expected categories and aggregation rule. The most direct exposure in the current scenario is: An unrestricted join can multiply keys and exaggerate customer counts.
Question 751
A pipeline owner declines the action: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys. The business situation is: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. What specific failure can follow?
- Rigid directory partitioning can cause imbalance and require costly redesign as filters change.
- Assuming indefinite history can fail once physical file retention expires.
- Migrating data into a managed table can violate external storage governance agreements.
- Poorly chosen keys increase full scans and maintenance overhead as traffic grows.
Explanation
Correct Answer
D:The direct residual exposure is: Poorly chosen keys increase full scans and maintenance overhead as traffic grows. The triggering condition is: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. The omitted safeguard is: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys. This matters because: Efficient layout keys are driven by actual access patterns and file statistics. A corrective test would show: A measured workload benchmark shows scan reduction across the highest-value query templates.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A large Delta fact table is filtered primarily on high-cardinality account IDs with changing query patterns. Its preventative action would be: Enable supported liquid clustering with appropriate keys and run compatible optimization. The most direct exposure in the current scenario is: Poorly chosen keys increase full scans and maintenance overhead as traffic grows.
B:This failure mode belongs mainly to another situation: A business needs to compare two point-in-time table versions during a financial reconciliation. Its preventative action would be: Use supported Delta table version history and time travel within the documented retention window. The most direct exposure in the current scenario is: Poorly chosen keys increase full scans and maintenance overhead as traffic grows.
C:This failure mode belongs mainly to another situation: A dataset has a strict requirement that Azure storage ownership and lifecycle remain under a separate platform team. Its preventative action would be: Use an external Unity Catalog table with an approved registered location instead of assuming managed storage ownership. The most direct exposure in the current scenario is: Poorly chosen keys increase full scans and maintenance overhead as traffic grows.
Question 752
A platform architect adopts: Evaluate compatible instance pools or supported serverless startup behavior rather than keeping unused all-purpose clusters awake. For this requirement: A scheduled workflow launches dozens of short-lived classic clusters and loses several minutes each time waiting for virtual machines. Which rationale MOST directly supports it?
- SQL warehouses are optimized for SQL analytics and isolate query serving from other compute activities.
- Pool capacity can shorten classic cluster startup but reserved idle capacity must still be budgeted.
- Ephemeral task-oriented compute can avoid idle capacity costs while retaining reproducible batch execution.
- End-to-end task time includes compute startup and orchestration overhead, not just transformation CPU time.
Explanation
Correct Answer
B:The decisive mechanism is: Pool capacity can shorten classic cluster startup but reserved idle capacity must still be budgeted. That explains the selected action: Evaluate compatible instance pools or supported serverless startup behavior rather than keeping unused all-purpose clusters awake. In the original scenario: A scheduled workflow launches dozens of short-lived classic clusters and loses several minutes each time waiting for virtual machines. The specific unmanaged risk is: Ignoring startup overhead can make small jobs miss a narrow delivery window even when their Spark tasks are fast. Validate with: Job run traces show lower startup latency and the combined pool or serverless cost stays within budget.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. Its targeted response is: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. The current case is governed more directly by: Pool capacity can shorten classic cluster startup but reserved idle capacity must still be budgeted.
C:This principle concerns a separate data-engineering condition: A nightly batch transformation uses a continuously running all-purpose cluster even though no analyst interacts with the workload between runs. Its targeted response is: Move the scheduled transformation to appropriately configured jobs compute or supported serverless jobs and release resources after execution. The current case is governed more directly by: Pool capacity can shorten classic cluster startup but reserved idle capacity must still be budgeted.
D:This principle concerns a separate data-engineering condition: A data team uses a high-power job cluster for a low-volume data-quality check that spends most runtime starting the cluster. Its targeted response is: Separate the small validation into an eligible lightweight or serverless task and measure total end-to-end runtime. The current case is governed more directly by: Pool capacity can shorten classic cluster startup but reserved idle capacity must still be budgeted.
Question 753
The team implements: Define validated job parameters and propagate them through supported task parameter bindings. The original problem is: A workflow needs to pass a date parameter from a scheduled trigger into all its downstream notebook tasks. Which test MOST convincingly demonstrates correctness?
- A staged predecessor failure shows the approved downstream skip and cleanup path in run history.
- The historical batch completes with reconciled counts and next daily scheduled run remains unchanged.
- Run history records one intended business date and every task publishes that same interval.
- A simulated stall exceeds the threshold, stops correctly and alerts the support team.
Explanation
Correct Answer
C:The strongest acceptance proof is: Run history records one intended business date and every task publishes that same interval. It specifically verifies: Define validated job parameters and propagate them through supported task parameter bindings. The original issue was: A workflow needs to pass a date parameter from a scheduled trigger into all its downstream notebook tasks. The underlying mechanism is: Consistent parameter passing ensures dependent tasks operate on the same business interval. Without it, mixed processing dates across tasks can create unreconciled reports and repeated records.
Incorrect Answers
A:This result would validate a different measure: Review explicit dependency and run-if settings and validate skip behavior through a staged failure. That is relevant to the distinct problem: A job owner cannot explain why a task skipped execution after a predecessor failed. Here the acceptance criterion is: Run history records one intended business date and every task publishes that same interval.
B:This result would validate a different measure: Run a controlled parameterized manual invocation for the historical interval while retaining the regular trigger. That is relevant to the distinct problem: An ingestion job requires a one-time backfill without changing the future daily production schedule. Here the acceptance criterion is: Run history records one intended business date and every task publishes that same interval.
D:This result would validate a different measure: Set task and job timeout limits with notifications and a documented retry and recovery route. That is relevant to the distinct problem: A long-running job occasionally hangs but there is no configured runtime ceiling or escalation. Here the acceptance criterion is: Run history records one intended business date and every task publishes that same interval.
Question 754
An operator introduces: Use supported Delta table version history and time travel within the documented retention window. The situation is: A business needs to compare two point-in-time table versions during a financial reconciliation. Which validation should be required before signoff?
- A controlled historical version query returns the expected prior balance before maintenance removes eligible old files.
- A measured workload benchmark shows scan reduction across the highest-value query templates.
- A controlled late event appears in the correct historical period according to the approved policy.
- A backdated invoice joins to the address valid on its transaction date while the current record shows the latest address.
Explanation
Correct Answer
A:The strongest acceptance proof is: A controlled historical version query returns the expected prior balance before maintenance removes eligible old files. It specifically verifies: Use supported Delta table version history and time travel within the documented retention window. The original issue was: A business needs to compare two point-in-time table versions during a financial reconciliation. The underlying mechanism is: Delta version history can provide auditable historical snapshots while underlying files are retained. Without it, assuming indefinite history can fail once physical file retention expires.
Incorrect Answers
B:This result would validate a different measure: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys. That is relevant to the distinct problem: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. Here the acceptance criterion is: A controlled historical version query returns the expected prior balance before maintenance removes eligible old files.
C:This result would validate a different measure: Design event-time fields, ingestion timestamps and late-data behavior suited to replay and temporal queries. That is relevant to the distinct problem: A lakehouse receives sensor observations with event timestamps that arrive out of order. Here the acceptance criterion is: A controlled historical version query returns the expected prior balance before maintenance removes eligible old files.
D:This result would validate a different measure: Model a slowly changing dimension Type 2 with effective dates, current flags and correctly managed surrogate keys. That is relevant to the distinct problem: A retail customer table changes delivery address repeatedly but analysts must reconstruct the address known at any prior invoice date. Here the acceptance criterion is: A controlled historical version query returns the expected prior balance before maintenance removes eligible old files.
Question 755
The team implements: Select a notebook task within Lakeflow Jobs when procedural control and custom libraries fit the requirements. The original problem is: An ad hoc machine-learning preparation task has complex custom Python orchestration and no declarative table graph. Which test MOST convincingly demonstrates correctness?
- A controlled late-event test updates the interval before its approved closeout and leaves later events for correction.
- A controlled consumer test sees only one approved publication version across dependent datasets.
- A job executes the procedural notebook under controlled dependencies and produces the validated output.
- Two executions of the same input batch yield one approved current record per order.
Explanation
Correct Answer
C:The strongest acceptance proof is: A job executes the procedural notebook under controlled dependencies and produces the validated output. It specifically verifies: Select a notebook task within Lakeflow Jobs when procedural control and custom libraries fit the requirements. The original issue was: An ad hoc machine-learning preparation task has complex custom Python orchestration and no declarative table graph. The underlying mechanism is: Notebooks and declarative pipelines serve different orchestration and transformation patterns. Without it, forcing arbitrary procedural logic into a declarative table pipeline may complicate testing and maintenance.
Incorrect Answers
A:This result would validate a different measure: Specify the stream watermark and an explicit closeout readiness condition before running downstream aggregation. That is relevant to the distinct problem: A streaming process depends on a downstream batch closeout but no one defines whether it should wait for the latest watermark. Here the acceptance criterion is: A job executes the procedural notebook under controlled dependencies and produces the validated output.
B:This result would validate a different measure: Design a compatible atomic publication or versioned readiness contract so consumers see consistent results. That is relevant to the distinct problem: A regulated pipeline needs consistent transaction boundaries across multiple table writes but updates are published one at a time. Here the acceptance criterion is: A job executes the procedural notebook under controlled dependencies and produces the validated output.
D:This result would validate a different measure: Use idempotent writes or MERGE keyed by stable order identity for repeatable task execution. That is relevant to the distinct problem: A nightly pipeline rerun appends the same orders to the target and doubles totals after an operator presses Retry. Here the acceptance criterion is: A job executes the procedural notebook under controlled dependencies and produces the validated output.
Question 756
The team chooses: Set a measured maintenance cadence or evaluate supported predictive optimization for eligible managed tables. Its observed issue is: A new Delta table maintenance job combines repeated OPTIMIZE and VACUUM at every microbatch even though changes are tiny. What is the strongest technical basis?
- Different failure layers require different diagnostic paths instead of generic cluster restarts.
- Task-level repair can reduce replay cost while preserving successful work.
- Maintenance operations themselves consume compute and should correspond to data growth and read patterns.
- Caching trades execution time for memory and may increase pressure when datasets are not reused.
Explanation
Correct Answer
C:The decisive mechanism is: Maintenance operations themselves consume compute and should correspond to data growth and read patterns. That explains the selected action: Set a measured maintenance cadence or evaluate supported predictive optimization for eligible managed tables. In the original scenario: A new Delta table maintenance job combines repeated OPTIMIZE and VACUUM at every microbatch even though changes are tiny. The specific unmanaged risk is: Over-maintenance can spend more resources rewriting files than it saves in queries. Validate with: A benchmark shows lower total maintenance-plus-query cost without compromising retention.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: An operations team sees Dataflow-like ingestion failures but cannot identify whether the problem is a Unity Catalog grant or a Spark stage. Its targeted response is: Correlate error messages, job run IDs, catalog authorization logs and Spark execution evidence before remediation. The current case is governed more directly by: Maintenance operations themselves consume compute and should correspond to data growth and read patterns.
B:This principle concerns a separate data-engineering condition: A job fails during a temporary Azure storage outage but operators rerun every successful upstream task. Its targeted response is: Use Lakeflow Jobs repair at the failed task when dependency outputs remain valid. The current case is governed more directly by: Maintenance operations themselves consume compute and should correspond to data growth and read patterns.
D:This principle concerns a separate data-engineering condition: A team enables Spark cache for a one-time ETL dataset and never unpersists it before processing the next large table. Its targeted response is: Use caching only for beneficial reuse and release persisted state when no longer needed. The current case is governed more directly by: Maintenance operations themselves consume compute and should correspond to data growth and read patterns.
Question 757
A Databricks engineer recommends: Use Lakeflow Jobs repair at the failed task when dependency outputs remain valid. It responds to: A job fails during a temporary Azure storage outage but operators rerun every successful upstream task. What underlying mechanism BEST explains why?
- Concurrent demand peaks can drive queue delays independently of average compute utilization.
- Centralized telemetry supports durable troubleshooting when ephemeral compute terminates.
- Alert quality depends on actionable thresholds and distinguishing transient variation from sustained failure.
- Task-level repair can reduce replay cost while preserving successful work.
Explanation
Correct Answer
D:The decisive mechanism is: Task-level repair can reduce replay cost while preserving successful work. That explains the selected action: Use Lakeflow Jobs repair at the failed task when dependency outputs remain valid. In the original scenario: A job fails during a temporary Azure storage outage but operators rerun every successful upstream task. The specific unmanaged risk is: Unnecessary replay may duplicate writes and extend incident recovery time. Validate with: A controlled transient outage is repaired without rerunning completed expensive ingestion.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: An interactive warehouse has high query queues during the morning but low CPU utilization after lunch. Its targeted response is: Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages. The current case is governed more directly by: Task-level repair can reduce replay cost while preserving successful work.
B:This principle concerns a separate data-engineering condition: An Azure Databricks pipeline has recurring errors but its logs remain only in a short-lived cluster session. Its targeted response is: Export relevant logs and metrics through a supported Log Analytics or Azure Monitor integration with controlled retention. The current case is governed more directly by: Task-level repair can reduce replay cost while preserving successful work.
C:This principle concerns a separate data-engineering condition: An alert triggers on every harmless microbatch delay while real hour-long ingestion stalls go unnoticed. Its targeted response is: Define meaningful latency, error and backlog thresholds with alert routing based on the business SLO. The current case is governed more directly by: Task-level repair can reduce replay cost while preserving successful work.
Question 758
An implementation skips: Run supported OPTIMIZE or predictive optimization and validate file-size distribution and query cost. Engineers are observing: A Delta table contains millions of tiny files after frequent microbatch updates and SQL scans spend time opening files. What consequence is MOST likely?
- Premature deletion may invalidate time travel or a lagging reader's file references.
- Collecting billions of rows can crash the driver despite idle executors.
- Unused cached data can cause eviction, spilling and unexpected failures for later stages.
- A growing small-file count can dominate query planning and scan overhead.
Explanation
Correct Answer
D:The direct residual exposure is: A growing small-file count can dominate query planning and scan overhead. The triggering condition is: A Delta table contains millions of tiny files after frequent microbatch updates and SQL scans spend time opening files. The omitted safeguard is: Run supported OPTIMIZE or predictive optimization and validate file-size distribution and query cost. This matters because: File compaction reduces metadata and small-file overhead without changing logical table records. A corrective test would show: Before-and-after file statistics show fewer small files with identical row counts and faster queries.
Incorrect Answers
A:This failure mode belongs mainly to another situation: An engineer runs VACUUM with a zero-hour retention on a shared Delta table used for historical audit queries. Its preventative action would be: Respect documented safe retention windows and coordinate VACUUM with readers, streams and time-travel needs. The most direct exposure in the current scenario is: A growing small-file count can dominate query planning and scan overhead.
B:This failure mode belongs mainly to another situation: A Spark join reads large inputs but the driver reports failures after an analyst calls collect() on the result. Its preventative action would be: Keep large results distributed and replace driver collection with bounded sampling or distributed writes. The most direct exposure in the current scenario is: A growing small-file count can dominate query planning and scan overhead.
C:This failure mode belongs mainly to another situation: A team enables Spark cache for a one-time ETL dataset and never unpersists it before processing the next large table. Its preventative action would be: Use caching only for beneficial reuse and release persisted state when no longer needed. The most direct exposure in the current scenario is: A growing small-file count can dominate query planning and scan overhead.
Question 759
The team implements: Resolve Git merge conflicts with tests and reviewed integration before merging the release branch. The original problem is: Two engineers change the same ingestion function on separate branches and one silently overwrites the other's fix. Which test MOST convincingly demonstrates correctness?
- A merged branch passes regression tests for both bug fixes and shows reviewed conflict decisions.
- A seeded test dataset returns documented expected outputs and intentionally bad inputs fail.
- An integration fixture with historical updates yields exactly the intended target versions.
- A production run references an approved immutable commit and a prior version can be restored.
Explanation
Correct Answer
A:The strongest acceptance proof is: A merged branch passes regression tests for both bug fixes and shows reviewed conflict decisions. It specifically verifies: Resolve Git merge conflicts with tests and reviewed integration before merging the release branch. The original issue was: Two engineers change the same ingestion function on separate branches and one silently overwrites the other's fix. The underlying mechanism is: Conflict resolution must preserve intended behavior from both change sets. Without it, unreviewed overwrites can remove validation or idempotency logic from the deployed pipeline.
Incorrect Answers
B:This result would validate a different measure: Add representative unit and integration tests asserting transformation results and schema contracts. That is relevant to the distinct problem: A release modifies transformation logic but tests only whether Python files compile. Here the acceptance criterion is: A merged branch passes regression tests for both bug fixes and shows reviewed conflict decisions.
C:This result would validate a different measure: Include an integration test using actual Delta table semantics and representative source changes. That is relevant to the distinct problem: A test harness mocks out Delta writes but the real job uses MERGE and unexpectedly changes historical rows. Here the acceptance criterion is: A merged branch passes regression tests for both bug fixes and shows reviewed conflict decisions.
D:This result would validate a different measure: Store deployment source in Git with reviewed branches and pull requests before promoting changes. That is relevant to the distinct problem: Engineers edit production notebooks directly and lose the ability to identify which code change broke a pipeline. Here the acceptance criterion is: A merged branch passes regression tests for both bug fixes and shows reviewed conflict decisions.
Question 760
A data product owner adopts: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. The reason is: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. What observed result BEST proves effectiveness?
- A backdated invoice joins to the address valid on its transaction date while the current record shows the latest address.
- File-count and query-profile comparisons show fewer fragments and faster targeted scans.
- Storage metrics show a manageable file distribution and point lookups meet query SLAs.
- A controlled late event appears in the correct historical period according to the approved policy.
Explanation
Correct Answer
C:The strongest acceptance proof is: Storage metrics show a manageable file distribution and point lookups meet query SLAs. It specifically verifies: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. The original issue was: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. The underlying mechanism is: Partition pruning benefits can be outweighed by extremely fragmented high-cardinality directory layouts. Without it, one-file-per-customer patterns can overwhelm metadata and worsen query startup.
Incorrect Answers
A:This result would validate a different measure: Model a slowly changing dimension Type 2 with effective dates, current flags and correctly managed surrogate keys. That is relevant to the distinct problem: A retail customer table changes delivery address repeatedly but analysts must reconstruct the address known at any prior invoice date. Here the acceptance criterion is: Storage metrics show a manageable file distribution and point lookups meet query SLAs.
B:This result would validate a different measure: Use coarse suitable partitioning or liquid clustering selected from filter patterns and cardinality. That is relevant to the distinct problem: An event table is partitioned by the unique transaction identifier and creates millions of tiny partitions. Here the acceptance criterion is: Storage metrics show a manageable file distribution and point lookups meet query SLAs.
D:This result would validate a different measure: Design event-time fields, ingestion timestamps and late-data behavior suited to replay and temporal queries. That is relevant to the distinct problem: A lakehouse receives sensor observations with event timestamps that arrive out of order. Here the acceptance criterion is: Storage metrics show a manageable file distribution and point lookups meet query SLAs.