Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 3
Question 20
A team omits the recommended measure: Implement explicit null predicates rather than treating zero and null as equivalent. Its environment shows: A team filters out null quantities but needs to preserve rows with zero as valid inventory states. What is the MOST direct residual risk?
- Blindly broadcasting a large table can exhaust executor memory while unnecessary shuffles slow joins.
- Blind append creates multiple current versions and breaks consumer joins.
- Refund amounts treated as sales inflate net revenue even when record counts match.
- Conflating zero with missing data hides legitimate stockouts and distorts service metrics.
Explanation
Correct Answer
D:The direct residual exposure is: Conflating zero with missing data hides legitimate stockouts and distorts service metrics. The triggering condition is: A team filters out null quantities but needs to preserve rows with zero as valid inventory states. The omitted safeguard is: Implement explicit null predicates rather than treating zero and null as equivalent. This matters because: SQL NULL indicates absent or unknown values while numeric zero is a valid value. A corrective test would show: A seeded dataset retains zero-quantity items while rows with null quantities follow the documented rule.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A pipeline joins a huge fact table with a tiny dimension and a normal shuffle join creates excessive network traffic. Its preventative action would be: Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans. The most direct exposure in the current scenario is: Conflating zero with missing data hides legitimate stockouts and distorts service metrics.
B:This failure mode belongs mainly to another situation: A product catalog receives corrections for existing SKUs plus entirely new items on each incremental load. Its preventative action would be: Use a Delta MERGE keyed by SKU with appropriate matched-update and not-matched-insert logic. The most direct exposure in the current scenario is: Conflating zero with missing data hides legitimate stockouts and distorts service metrics.
C:This failure mode belongs mainly to another situation: A daily revenue report double-counts refunds because both debit and credit rows are summed as positive numbers. Its preventative action would be: Normalize transaction sign semantics before grouping by business date and customer. The most direct exposure in the current scenario is: Conflating zero with missing data hides legitimate stockouts and distorts service metrics.
Question 21
An Azure Databricks architect reviews the following problem: A team expects dropping a managed table to preserve all physical data as though it were an external table. What should the team implement FIRST?
- Create an authorized connection and supported foreign catalog for the external source.
- Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics.
- Register the structured dataset as an appropriate managed Delta table rather than treating the volume itself as a SQL table.
- Create an eligible materialized view or managed derived table with appropriate refresh and access rules.
Explanation
Correct Answer
B:The most appropriate action is: Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics. It addresses the stated condition: A team expects dropping a managed table to preserve all physical data as though it were an external table. The technical basis is: Managed versus external tables differ in who manages data paths and deletion behavior. Without this control, A drop operation may remove managed table data that administrators expected to retain. A direct acceptance check is: A controlled create-drop rehearsal confirms the expected metadata and physical data lifecycle for the table type.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A workspace needs to query a foreign transactional source without first loading every row into a Delta table. The corresponding measure there would be: Create an authorized connection and supported foreign catalog for the external source. The stated situation instead calls for: Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics.
C:This alternative primarily addresses a different Azure Databricks problem: A team exposes structured transaction data as files in a volume but wants SQL analytics and ACID MERGE behavior. The corresponding measure there would be: Register the structured dataset as an appropriate managed Delta table rather than treating the volume itself as a SQL table. The stated situation instead calls for: Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics.
D:This alternative primarily addresses a different Azure Databricks problem: A BI group needs to query a materialized aggregate while the source transaction Delta table changes hourly. The corresponding measure there would be: Create an eligible materialized view or managed derived table with appropriate refresh and access rules. The stated situation instead calls for: Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics.
Question 22
For this Microsoft DP-750 case: An integration performs a one-off initial import of an existing CSV folder into a Delta table. Which design or operational response BEST fits?
- Apply ordered CDC sequencing using source change version or timestamp and consistent delete handling.
- Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle.
- Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options.
- Coordinate ADF orchestration with the appropriate Databricks task and Unity Catalog access model.
Explanation
Correct Answer
C:The most appropriate action is: Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options. It addresses the stated condition: An integration performs a one-off initial import of an existing CSV folder into a Delta table. The technical basis is: One-time or incremental file loading can be expressed with SQL ingestion commands without building a persistent streaming service. Without this control, A bespoke long-running stream for a one-time load adds complexity and idle cost. A direct acceptance check is: The import reads the intended input files once and reconciles the expected row count and schema.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A transactional feed produces out-of-order CDC events and a naive MERGE overwrites a newer status with an older update. The corresponding measure there would be: Apply ordered CDC sequencing using source change version or timestamp and consistent delete handling. The stated situation instead calls for: Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options.
B:This alternative primarily addresses a different Azure Databricks problem: A long-running ingestion job persists its checkpoint in the same temporary folder that daily cleanup deletes. The corresponding measure there would be: Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle. The stated situation instead calls for: Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options.
D:This alternative primarily addresses a different Azure Databricks problem: A data product needs to copy a supported Azure Data Factory pipeline output into governed Databricks tables. The corresponding measure there would be: Coordinate ADF orchestration with the appropriate Databricks task and Unity Catalog access model. The stated situation instead calls for: Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options.
Question 23
The team chooses: Evaluate dimensional normalization against query complexity, maintenance and expected cardinality. Its observed issue is: A customer table includes both region hierarchy and customers; teams disagree whether to snowflake descriptive attributes. What is the strongest technical basis?
- Data models balance redundancy and query simplicity according to workload requirements.
- Partition pruning benefits can be outweighed by extremely fragmented high-cardinality directory layouts.
- CTAS populates a table at execution time and does not imply continuous synchronization with source.
- Fact grain determines which analyses can be reconstructed without information loss.
Explanation
Correct Answer
A:The decisive mechanism is: Data models balance redundancy and query simplicity according to workload requirements. That explains the selected action: Evaluate dimensional normalization against query complexity, maintenance and expected cardinality. In the original scenario: A customer table includes both region hierarchy and customers; teams disagree whether to snowflake descriptive attributes. The specific unmanaged risk is: Unneeded normalization can add joins while denormalization without governance can produce inconsistent descriptions. Validate with: Representative business queries return consistent region totals with an acceptable plan.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. Its targeted response is: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. The current case is governed more directly by: Data models balance redundancy and query simplicity according to workload requirements.
C:This principle concerns a separate data-engineering condition: A pipeline uses CREATE TABLE AS SELECT for a one-time derived dataset but later assumes the result refreshes automatically. Its targeted response is: Select CTAS for an explicitly materialized snapshot or a scheduled pipeline for ongoing refresh. The current case is governed more directly by: Data models balance redundancy and query simplicity according to workload requirements.
D:This principle concerns a separate data-engineering condition: A facts table groups product sales at daily level although auditors must inspect individual transaction adjustments. Its targeted response is: Define fact-table grain at the required transaction or line-item level before aggregation. The current case is governed more directly by: Data models balance redundancy and query simplicity according to workload requirements.
Question 24
A platform architect adopts: Move the scheduled transformation to appropriately configured jobs compute or supported serverless jobs and release resources after execution. For this requirement: A nightly batch transformation uses a continuously running all-purpose cluster even though no analyst interacts with the workload between runs. Which rationale MOST directly supports it?
- Spark and runtime upgrades can change supported APIs, dependency versions and execution behavior.
- Ephemeral task-oriented compute can avoid idle capacity costs while retaining reproducible batch execution.
- Pool capacity can shorten classic cluster startup but reserved idle capacity must still be budgeted.
- SQL warehouses are optimized for SQL analytics and isolate query serving from other compute activities.
Explanation
Correct Answer
B:The decisive mechanism is: Ephemeral task-oriented compute can avoid idle capacity costs while retaining reproducible batch execution. That explains the selected action: Move the scheduled transformation to appropriately configured jobs compute or supported serverless jobs and release resources after execution. In the original scenario: A nightly batch transformation uses a continuously running all-purpose cluster even though no analyst interacts with the workload between runs. The specific unmanaged risk is: Leaving an interactive cluster running around the clock accumulates charges unrelated to completed pipeline work. Validate with: The scheduled job completes within its service objective and compute billing shows no unattended idle cluster hours.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A developer selects a new Databricks Runtime version for production despite an untested JDBC driver dependency. Its targeted response is: Pin a supported Databricks Runtime version and validate application libraries against it before deployment. The current case is governed more directly by: Ephemeral task-oriented compute can avoid idle capacity costs while retaining reproducible batch execution.
C:This principle concerns a separate data-engineering condition: A scheduled workflow launches dozens of short-lived classic clusters and loses several minutes each time waiting for virtual machines. Its targeted response is: Evaluate compatible instance pools or supported serverless startup behavior rather than keeping unused all-purpose clusters awake. The current case is governed more directly by: Ephemeral task-oriented compute can avoid idle capacity costs while retaining reproducible batch execution.
D:This principle concerns a separate data-engineering condition: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. Its targeted response is: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. The current case is governed more directly by: Ephemeral task-oriented compute can avoid idle capacity costs while retaining reproducible batch execution.
Question 25
The recommended measure is: Validate timezone-aware timestamp types and quarantine ambiguous or unparseable event values. The operational context is: A regulated dataset requires all timestamps to parse as UTC but several rows have ambiguous local values. Which data engineering concept applies?
- Completeness thresholds depend on business contracts and cannot be uniform across all columns.
- Layered quality gates prevent malformed source values from contaminating trusted downstream tables.
- Valid data type does not guarantee semantic range correctness.
- Type checks and temporal semantics must be aligned to prevent incorrect time-series results.
Explanation
Correct Answer
D:The decisive mechanism is: Type checks and temporal semantics must be aligned to prevent incorrect time-series results. That explains the selected action: Validate timezone-aware timestamp types and quarantine ambiguous or unparseable event values. In the original scenario: A regulated dataset requires all timestamps to parse as UTC but several rows have ambiguous local values. The specific unmanaged risk is: Ambiguous timestamp parsing can move financial events across reporting windows. Validate with: A sample of valid UTC timestamps passes and known ambiguous strings produce the expected exception record.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A new customer feed is missing optional contact information for 8% of rows and the team treats every null as pipeline failure. Its targeted response is: Profile source missingness and define field-specific required versus optional expectations. The current case is governed more directly by: Type checks and temporal semantics must be aligned to prevent incorrect time-series results.
B:This principle concerns a separate data-engineering condition: A product data contract requires every transaction amount to use nonnegative decimal values but new raw files include quoted text strings. Its targeted response is: Apply type, nullability and range expectations before promoting bronze rows into the silver target. The current case is governed more directly by: Type checks and temporal semantics must be aligned to prevent incorrect time-series results.
C:This principle concerns a separate data-engineering condition: A bronze stream sometimes includes ages below zero or above 120 yet the transformation casts every value to integer. Its targeted response is: Define a range expectation with metrics and the approved drop, fail or quarantine disposition. The current case is governed more directly by: Type checks and temporal semantics must be aligned to prevent incorrect time-series results.
Question 26
A data product owner adopts: Use Spark-native distributed read and transformation APIs or partition-safe processing. The reason is: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. What observed result BEST proves effectiveness?
- Insert-update-delete test events produce one correct current target record per key.
- Replaying the same microbatch produces one target effect per business key.
- A duplicate-event test yields one current business event while preserving valid distinct events.
- The workload completes without driver collection and Spark metrics show distributed processing.
Explanation
Correct Answer
D:The strongest acceptance proof is: The workload completes without driver collection and Spark metrics show distributed processing. It specifically verifies: Use Spark-native distributed read and transformation APIs or partition-safe processing. The original issue was: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. The underlying mechanism is: Pulling distributed data to a single driver defeats parallel execution and memory limits. Without it, driver out-of-memory can fail the load despite adequate executor resources.
Incorrect Answers
A:This result would validate a different measure: Use a supported change-data-capture feed and apply insert, update and delete semantics to the target. That is relevant to the distinct problem: A SaaS database updates and deletes customer records but a pipeline appends every nightly full extract to a Delta table. Here the acceptance criterion is: The workload completes without driver collection and Spark metrics show distributed processing.
B:This result would validate a different measure: Implement idempotent foreachBatch or compatible transactional sink logic keyed by epoch or business identity. That is relevant to the distinct problem: A stream writes into an external sink without stable batch identities and causes duplicate side effects on retry. Here the acceptance criterion is: The workload completes without driver collection and Spark metrics show distributed processing.
C:This result would validate a different measure: Deduplicate by stable event identity and business time using documented state and watermark rules. That is relevant to the distinct problem: An ingestion pipeline reads cloud events with at-least-once delivery and assumes every event ID appears only once. Here the acceptance criterion is: The workload completes without driver collection and Spark metrics show distributed processing.
Question 27
For this Microsoft DP-750 case: A finance field arrives as a string with currency symbols and the team aggregates it lexicographically. Which design or operational response BEST fits?
- Use a Delta MERGE keyed by SKU with appropriate matched-update and not-matched-insert logic.
- Use a left anti-join or EXCEPT operation with aligned schemas and explicit null semantics.
- Parse and cast monetary values to a suitable decimal type with invalid-value handling.
- Use an incremental append pattern with suitable deduplication when source events cannot change.
Explanation
Correct Answer
C:The most appropriate action is: Parse and cast monetary values to a suitable decimal type with invalid-value handling. It addresses the stated condition: A finance field arrives as a string with currency symbols and the team aggregates it lexicographically. The technical basis is: Correct numeric types prevent sorting and arithmetic errors from text representations. Without this control, string summation or floating rounding may materially misstate financial totals. A direct acceptance check is: A controlled money sample matches expected exact decimal totals and rejects malformed amounts.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A product catalog receives corrections for existing SKUs plus entirely new items on each incremental load. The corresponding measure there would be: Use a Delta MERGE keyed by SKU with appropriate matched-update and not-matched-insert logic. The stated situation instead calls for: Parse and cast monetary values to a suitable decimal type with invalid-value handling.
B:This alternative primarily addresses a different Azure Databricks problem: An analyst must find records present in the source system but absent in a target migration. The corresponding measure there would be: Use a left anti-join or EXCEPT operation with aligned schemas and explicit null semantics. The stated situation instead calls for: Parse and cast monetary values to a suitable decimal type with invalid-value handling.
D:This alternative primarily addresses a different Azure Databricks problem: An append-only telemetry feed includes newly observed events but engineers use MERGE with expensive full-table scans on every insert. The corresponding measure there would be: Use an incremental append pattern with suitable deduplication when source events cannot change. The stated situation instead calls for: Parse and cast monetary values to a suitable decimal type with invalid-value handling.
Question 28
The design team proposes: Correlate error messages, job run IDs, catalog authorization logs and Spark execution evidence before remediation. The situation is: An operations team sees Dataflow-like ingestion failures but cannot identify whether the problem is a Unity Catalog grant or a Spark stage. Which technical principle justifies the decision?
- Different failure layers require different diagnostic paths instead of generic cluster restarts.
- Transient cluster restarts can remove evidence necessary to distinguish dependency, data and resource failures.
- Logical deletes can leave physical layout and scan behavior needing maintenance.
- Cost optimization requires throughput per resource rather than simply maximizing available workers.
Explanation
Correct Answer
A:The decisive mechanism is: Different failure layers require different diagnostic paths instead of generic cluster restarts. That explains the selected action: Correlate error messages, job run IDs, catalog authorization logs and Spark execution evidence before remediation. In the original scenario: An operations team sees Dataflow-like ingestion failures but cannot identify whether the problem is a Unity Catalog grant or a Spark stage. The specific unmanaged risk is: Restarting a healthy compute resource cannot repair a denied storage credential or table grant. Validate with: An injected permission failure maps to the relevant principal and grant while compute remains healthy.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: An operator restarts a failed Databricks cluster before collecting the Spark executor and driver logs. Its targeted response is: Preserve diagnostic events and Spark UI stage evidence before restart where possible. The current case is governed more directly by: Different failure layers require different diagnostic paths instead of generic cluster restarts.
C:This principle concerns a separate data-engineering condition: A query on a Delta table slows after a small percentage of rows are deleted but data files are not optimized. Its targeted response is: Review deletion-vector behavior, table layout and suitable optimization for the supported runtime. The current case is governed more directly by: Different failure layers require different diagnostic paths instead of generic cluster restarts.
D:This principle concerns a separate data-engineering condition: A pipeline experiences escalating costs after a team increases node counts but execution duration barely changes. Its targeted response is: Use run-level compute utilization, cost and stage metrics to identify unused capacity. The current case is governed more directly by: Different failure layers require different diagnostic paths instead of generic cluster restarts.
Question 29
A production data team leaves out: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. Its scenario is: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. Which failure mode is MOST relevant?
- Assuming every transformation accelerates equally may raise costs without reducing job duration.
- Leaving an interactive cluster running around the clock accumulates charges unrelated to completed pipeline work.
- A shared notebook cluster can cause dashboards to stall when unrelated Spark notebooks consume executors.
- Forgotten development clusters can consume cloud resources long after the last useful notebook cell.
Explanation
Correct Answer
C:The direct residual exposure is: A shared notebook cluster can cause dashboards to stall when unrelated Spark notebooks consume executors. The triggering condition is: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. The omitted safeguard is: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. This matters because: SQL warehouses are optimized for SQL analytics and isolate query serving from other compute activities. A corrective test would show: A load rehearsal meets dashboard p95 response targets while notebook resource spikes do not affect query service.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A Spark SQL pipeline spends most of its time executing wide joins and aggregations but the team has not evaluated a compatible vectorized execution engine. Its preventative action would be: Benchmark Photon on a supported runtime with representative queries and compare cost, latency and correctness. The most direct exposure in the current scenario is: A shared notebook cluster can cause dashboards to stall when unrelated Spark notebooks consume executors.
B:This failure mode belongs mainly to another situation: A nightly batch transformation uses a continuously running all-purpose cluster even though no analyst interacts with the workload between runs. Its preventative action would be: Move the scheduled transformation to appropriately configured jobs compute or supported serverless jobs and release resources after execution. The most direct exposure in the current scenario is: A shared notebook cluster can cause dashboards to stall when unrelated Spark notebooks consume executors.
D:This failure mode belongs mainly to another situation: A newly created cluster remains available for hours after manual exploration even though the organization has a strict idle-cost policy. Its preventative action would be: Set appropriate auto-termination or idle controls for interactive compute while excluding workloads requiring continuous availability. The most direct exposure in the current scenario is: A shared notebook cluster can cause dashboards to stall when unrelated Spark notebooks consume executors.
Question 30
A Databricks engineer recommends: Split ingestion, validation and publishing into explicit orchestrated tasks with durable intermediate outputs. It responds to: A multi-step notebook workflow handles source extraction and final reporting in one cell sequence with no recoverable checkpoints. What underlying mechanism BEST explains why?
- Task-level repair reduces unnecessary recomputation while preserving dependency correctness.
- Job status must accurately reflect failed required operations for alerting and dependency logic.
- Retryable transient failures should be distinguished from permanent data or permission errors.
- Separating pipeline stages creates observable failure boundaries and supports targeted recovery.
Explanation
Correct Answer
D:The decisive mechanism is: Separating pipeline stages creates observable failure boundaries and supports targeted recovery. That explains the selected action: Split ingestion, validation and publishing into explicit orchestrated tasks with durable intermediate outputs. In the original scenario: A multi-step notebook workflow handles source extraction and final reporting in one cell sequence with no recoverable checkpoints. The specific unmanaged risk is: One late-stage error may force a costly replay of all preceding successful processing. Validate with: A simulated final-task failure can be repaired without rerunning already validated upstream ingestion.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A bulk repair executes all pipeline tasks again even when only the final reporting notebook failed. Its targeted response is: Separate tasks and use the supported repair-run workflow at the failed stage when dependencies remain valid. The current case is governed more directly by: Separating pipeline stages creates observable failure boundaries and supports targeted recovery.
B:This principle concerns a separate data-engineering condition: A workflow uses a notebook task that catches exceptions and returns success even when a write step fails. Its targeted response is: Propagate nonrecoverable exceptions to Lakeflow Jobs and attach actionable error metadata. The current case is governed more directly by: Separating pipeline stages creates observable failure boundaries and supports targeted recovery.
C:This principle concerns a separate data-engineering condition: A batch enrichment task downloads a remote file and sometimes receives transient HTTP 503 responses. Its targeted response is: Add bounded retries with exponential backoff and observable failure after the retry budget. The current case is governed more directly by: Separating pipeline stages creates observable failure boundaries and supports targeted recovery.