Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 20
Question 190
A team omits the recommended measure: Apply type, nullability and range expectations before promoting bronze rows into the silver target. Its environment shows: A product data contract requires every transaction amount to use nonnegative decimal values but new raw files include quoted text strings. What is the MOST direct residual risk?
- Invalid amounts may silently convert to null and skew revenue aggregates.
- Null business keys can enter Delta targets and make updates or joins nondeterministic.
- Silently coercing a breaking type can erase measurable values while a job appears successful.
- Numerically valid but impossible values can contaminate customer metrics and machine-learning features.
Explanation
Correct Answer
A:The direct residual exposure is: Invalid amounts may silently convert to null and skew revenue aggregates. The triggering condition is: A product data contract requires every transaction amount to use nonnegative decimal values but new raw files include quoted text strings. The omitted safeguard is: Apply type, nullability and range expectations before promoting bronze rows into the silver target. This matters because: Layered quality gates prevent malformed source values from contaminating trusted downstream tables. A corrective test would show: A controlled malformed amount is detected and excluded from published silver while legitimate zero values remain.
Incorrect Answers
B:This failure mode belongs mainly to another situation: An event pipeline must reject rows with a null transaction ID, but the data engineer checks only the row count after load. Its preventative action would be: Enforce a NOT NULL constraint or pipeline expectation on the transaction ID with a defined failure action. The most direct exposure in the current scenario is: Invalid amounts may silently convert to null and skew revenue aggregates.
C:This failure mode belongs mainly to another situation: A source API changes a quantity from integer to nested JSON and the pipeline silently produces nulls downstream. Its preventative action would be: Apply schema enforcement with explicit drift handling and alert on incompatible structural changes. The most direct exposure in the current scenario is: Invalid amounts may silently convert to null and skew revenue aggregates.
D:This failure mode belongs mainly to another situation: A bronze stream sometimes includes ages below zero or above 120 yet the transformation casts every value to integer. Its preventative action would be: Define a range expectation with metrics and the approved drop, fail or quarantine disposition. The most direct exposure in the current scenario is: Invalid amounts may silently convert to null and skew revenue aggregates.
Question 191
During a lakehouse implementation review, engineers observe: A pipeline uses a single notebook to read four sources sequentially even though two can be processed independently. What is the BEST next action?
- Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages.
- Design a separate failure-handling or completion branch with the supported run-if logic for alerts and cleanup.
- Design a task DAG with parallel independent ingestion and an aggregation dependency after both complete.
- Specify the stream watermark and an explicit closeout readiness condition before running downstream aggregation.
Explanation
Correct Answer
C:The most appropriate action is: Design a task DAG with parallel independent ingestion and an aggregation dependency after both complete. It addresses the stated condition: A pipeline uses a single notebook to read four sources sequentially even though two can be processed independently. The technical basis is: Parallelism is safe when tasks have no mutual data dependency and the join point is explicit. Without this control, unnecessary serial execution can extend the critical path beyond the freshness target. A direct acceptance check is: Timing evidence shows independent branches run concurrently and the merge waits for both results.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A scheduled notebook contains both transformation SQL and a long-running external API loop that occasionally hangs. The corresponding measure there would be: Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages. The stated situation instead calls for: Design a task DAG with parallel independent ingestion and an aggregation dependency after both complete.
B:This alternative primarily addresses a different Azure Databricks problem: A source task fails yet the monitoring task never executes because all downstream tasks require upstream success. The corresponding measure there would be: Design a separate failure-handling or completion branch with the supported run-if logic for alerts and cleanup. The stated situation instead calls for: Design a task DAG with parallel independent ingestion and an aggregation dependency after both complete.
D:This alternative primarily addresses a different Azure Databricks problem: A streaming process depends on a downstream batch closeout but no one defines whether it should wait for the latest watermark. The corresponding measure there would be: Specify the stream watermark and an explicit closeout readiness condition before running downstream aggregation. The stated situation instead calls for: Design a task DAG with parallel independent ingestion and an aggregation dependency after both complete.
Question 192
During a lakehouse implementation review, engineers observe: A pipeline experiences escalating costs after a team increases node counts but execution duration barely changes. What is the BEST next action?
- Define meaningful latency, error and backlog thresholds with alert routing based on the business SLO.
- Use run-level compute utilization, cost and stage metrics to identify unused capacity.
- Review deletion-vector behavior, table layout and suitable optimization for the supported runtime.
- Compare incoming event rate, processed records, watermark progress and state-store metrics before scaling.
Explanation
Correct Answer
B:The most appropriate action is: Use run-level compute utilization, cost and stage metrics to identify unused capacity. It addresses the stated condition: A pipeline experiences escalating costs after a team increases node counts but execution duration barely changes. The technical basis is: Cost optimization requires throughput per resource rather than simply maximizing available workers. Without this control, overprovisioning creates higher spend when stages are not parallelizable. A direct acceptance check is: A smaller tested configuration completes within SLA at a lower measured cost.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: An alert triggers on every harmless microbatch delay while real hour-long ingestion stalls go unnoticed. The corresponding measure there would be: Define meaningful latency, error and backlog thresholds with alert routing based on the business SLO. The stated situation instead calls for: Use run-level compute utilization, cost and stage metrics to identify unused capacity.
C:This alternative primarily addresses a different Azure Databricks problem: A query on a Delta table slows after a small percentage of rows are deleted but data files are not optimized. The corresponding measure there would be: Review deletion-vector behavior, table layout and suitable optimization for the supported runtime. The stated situation instead calls for: Use run-level compute utilization, cost and stage metrics to identify unused capacity.
D:This alternative primarily addresses a different Azure Databricks problem: A streaming pipeline's throughput falls although event arrival increases and its input backlog grows over time. The corresponding measure there would be: Compare incoming event rate, processed records, watermark progress and state-store metrics before scaling. The stated situation instead calls for: Use run-level compute utilization, cost and stage metrics to identify unused capacity.
Question 193
A production data platform experiences this condition: A new customer feed is missing optional contact information for 8% of rows and the team treats every null as pipeline failure. Which targeted measure should be selected?
- Collect expectation metrics and reconcile input, accepted, quarantined and rejected record counts.
- Enforce a NOT NULL constraint or pipeline expectation on the transaction ID with a defined failure action.
- Maintain a governed schema contract and integration tests across both notebook and declarative pipeline paths.
- Profile source missingness and define field-specific required versus optional expectations.
Explanation
Correct Answer
D:The most appropriate action is: Profile source missingness and define field-specific required versus optional expectations. It addresses the stated condition: A new customer feed is missing optional contact information for 8% of rows and the team treats every null as pipeline failure. The technical basis is: Completeness thresholds depend on business contracts and cannot be uniform across all columns. Without this control, overly strict validation can block legitimate records while loose validation may permit missing keys. A direct acceptance check is: Quality dashboards separate optional-null rates from prohibited null keys and meet agreed thresholds.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A pipeline reports zero failed rows because bad records are silently dropped without any count of what was removed. The corresponding measure there would be: Collect expectation metrics and reconcile input, accepted, quarantined and rejected record counts. The stated situation instead calls for: Profile source missingness and define field-specific required versus optional expectations.
B:This alternative primarily addresses a different Azure Databricks problem: An event pipeline must reject rows with a null transaction ID, but the data engineer checks only the row count after load. The corresponding measure there would be: Enforce a NOT NULL constraint or pipeline expectation on the transaction ID with a defined failure action. The stated situation instead calls for: Profile source missingness and define field-specific required versus optional expectations.
C:This alternative primarily addresses a different Azure Databricks problem: An engineer changes the expected file schema in one notebook but forgets to update the Lakeflow pipeline definition. The corresponding measure there would be: Maintain a governed schema contract and integration tests across both notebook and declarative pipeline paths. The stated situation instead calls for: Profile source missingness and define field-specific required versus optional expectations.
Question 194
A data engineering team faces this Azure Databricks requirement: A business needs to compare two point-in-time table versions during a financial reconciliation. Which action is MOST appropriate?
- Design a conformed star schema with explicit fact grain and unique dimension keys.
- Choose incremental append or merge based on immutable versus mutable source behavior.
- Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension.
- Use supported Delta table version history and time travel within the documented retention window.
Explanation
Correct Answer
D:The most appropriate action is: Use supported Delta table version history and time travel within the documented retention window. It addresses the stated condition: A business needs to compare two point-in-time table versions during a financial reconciliation. The technical basis is: Delta version history can provide auditable historical snapshots while underlying files are retained. Without this control, assuming indefinite history can fail once physical file retention expires. A direct acceptance check is: A controlled historical version query returns the expected prior balance before maintenance removes eligible old files.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A finance dashboard needs monthly revenue by customer and must join customer dimensions without multiplying facts. The corresponding measure there would be: Design a conformed star schema with explicit fact grain and unique dimension keys. The stated situation instead calls for: Use supported Delta table version history and time travel within the documented retention window.
B:This alternative primarily addresses a different Azure Databricks problem: A raw event lake contains append-only records but a developer chooses daily full overwrite of the curated Delta table. The corresponding measure there would be: Choose incremental append or merge based on immutable versus mutable source behavior. The stated situation instead calls for: Use supported Delta table version history and time travel within the documented retention window.
C:This alternative primarily addresses a different Azure Databricks problem: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. The corresponding measure there would be: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. The stated situation instead calls for: Use supported Delta table version history and time travel within the documented retention window.
Question 195
For this Azure Databricks issue: A warehouse query remains slow even after larger compute is allocated and the profile reports scanning nearly every file. Engineers choose: Inspect query predicates and Delta data skipping or clustering layout before increasing warehouse size again. What acceptance evidence is BEST?
- A selective query reads fewer files and meets latency targets after a controlled layout improvement.
- Spark storage metrics show released cache and unchanged correctness with improved total runtime.
- A peak-load rehearsal stabilizes backlog and keeps end-to-end event age within target.
- A skew-aware plan balances task durations and reduces the critical-stage elapsed time.
Explanation
Correct Answer
A:The strongest acceptance proof is: A selective query reads fewer files and meets latency targets after a controlled layout improvement. It specifically verifies: Inspect query predicates and Delta data skipping or clustering layout before increasing warehouse size again. The original issue was: A warehouse query remains slow even after larger compute is allocated and the profile reports scanning nearly every file. The underlying mechanism is: Full scans caused by poor layout require data organization or predicate changes rather than only more compute. Without it, scaling a query against irrelevant files can increase cost without proportional performance gains.
Incorrect Answers
B:This result would validate a different measure: Use caching only for beneficial reuse and release persisted state when no longer needed. That is relevant to the distinct problem: A team enables Spark cache for a one-time ETL dataset and never unpersists it before processing the next large table. Here the acceptance criterion is: A selective query reads fewer files and meets latency targets after a controlled layout improvement.
C:This result would validate a different measure: Compare incoming event rate, processed records, watermark progress and state-store metrics before scaling. That is relevant to the distinct problem: A streaming pipeline's throughput falls although event arrival increases and its input backlog grows over time. Here the acceptance criterion is: A selective query reads fewer files and meets latency targets after a controlled layout improvement.
D:This result would validate a different measure: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size. That is relevant to the distinct problem: A production Spark query is slow and its stage graph shows one task running far longer than all peers. Here the acceptance criterion is: A selective query reads fewer files and meets latency targets after a controlled layout improvement.
Question 196
During a lakehouse implementation review, engineers observe: A streaming pipeline's throughput falls although event arrival increases and its input backlog grows over time. What is the BEST next action?
- Review deletion-vector behavior, table layout and suitable optimization for the supported runtime.
- Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage.
- Compare incoming event rate, processed records, watermark progress and state-store metrics before scaling.
- Set a measured maintenance cadence or evaluate supported predictive optimization for eligible managed tables.
Explanation
Correct Answer
C:The most appropriate action is: Compare incoming event rate, processed records, watermark progress and state-store metrics before scaling. It addresses the stated condition: A streaming pipeline's throughput falls although event arrival increases and its input backlog grows over time. The technical basis is: Sustained source-to-sink throughput below arrival rate causes accumulating data freshness debt. Without this control, increasing source throughput without repairing processing bottlenecks can worsen queue age. A direct acceptance check is: A peak-load rehearsal stabilizes backlog and keeps end-to-end event age within target.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A query on a Delta table slows after a small percentage of rows are deleted but data files are not optimized. The corresponding measure there would be: Review deletion-vector behavior, table layout and suitable optimization for the supported runtime. The stated situation instead calls for: Compare incoming event rate, processed records, watermark progress and state-store metrics before scaling.
B:This alternative primarily addresses a different Azure Databricks problem: A large aggregation has repeated shuffle spills to disk even though source scans finish quickly. The corresponding measure there would be: Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage. The stated situation instead calls for: Compare incoming event rate, processed records, watermark progress and state-store metrics before scaling.
D:This alternative primarily addresses a different Azure Databricks problem: A new Delta table maintenance job combines repeated OPTIMIZE and VACUUM at every microbatch even though changes are tiny. The corresponding measure there would be: Set a measured maintenance cadence or evaluate supported predictive optimization for eligible managed tables. The stated situation instead calls for: Compare incoming event rate, processed records, watermark progress and state-store metrics before scaling.
Question 197
A platform architect adopts: Apply schema enforcement with explicit drift handling and alert on incompatible structural changes. For this requirement: A source API changes a quantity from integer to nested JSON and the pipeline silently produces nulls downstream. Which rationale MOST directly supports it?
- Measurable quality controls need observable outcomes and accounting for rejected data.
- Primary business-key uniqueness is a semantic data-quality invariant even without enforced physical primary keys.
- Schema evolution must distinguish permissible additive fields from breaking type changes.
- Type checks and temporal semantics must be aligned to prevent incorrect time-series results.
Explanation
Correct Answer
C:The decisive mechanism is: Schema evolution must distinguish permissible additive fields from breaking type changes. That explains the selected action: Apply schema enforcement with explicit drift handling and alert on incompatible structural changes. In the original scenario: A source API changes a quantity from integer to nested JSON and the pipeline silently produces nulls downstream. The specific unmanaged risk is: Silently coercing a breaking type can erase measurable values while a job appears successful. Validate with: A simulated type change is rejected or rescued with an actionable alert and no corrupted curated output.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A pipeline reports zero failed rows because bad records are silently dropped without any count of what was removed. Its targeted response is: Collect expectation metrics and reconcile input, accepted, quarantined and rejected record counts. The current case is governed more directly by: Schema evolution must distinguish permissible additive fields from breaking type changes.
B:This principle concerns a separate data-engineering condition: A curated account table accidentally contains multiple rows per supposedly unique current account ID. Its targeted response is: Run a supported uniqueness or cardinality validation during the transformation and surface violations. The current case is governed more directly by: Schema evolution must distinguish permissible additive fields from breaking type changes.
D:This principle concerns a separate data-engineering condition: A regulated dataset requires all timestamps to parse as UTC but several rows have ambiguous local values. Its targeted response is: Validate timezone-aware timestamp types and quarantine ambiguous or unparseable event values. The current case is governed more directly by: Schema evolution must distinguish permissible additive fields from breaking type changes.
Question 198
A production data team leaves out: Select CTAS for an explicitly materialized snapshot or a scheduled pipeline for ongoing refresh. Its scenario is: A pipeline uses CREATE TABLE AS SELECT for a one-time derived dataset but later assumes the result refreshes automatically. Which failure mode is MOST relevant?
- Inner joining only currently known dimension rows can undercount legitimate transactions.
- Keeping unnecessary historical rows can produce multiple matches and incorrect lookup joins.
- Dashboards can become stale when an initial table snapshot is mistaken for a live derived view.
- Using Type 1 updates may silently rewrite the apparent address on older transaction reports.
Explanation
Correct Answer
C:The direct residual exposure is: Dashboards can become stale when an initial table snapshot is mistaken for a live derived view. The triggering condition is: A pipeline uses CREATE TABLE AS SELECT for a one-time derived dataset but later assumes the result refreshes automatically. The omitted safeguard is: Select CTAS for an explicitly materialized snapshot or a scheduled pipeline for ongoing refresh. This matters because: CTAS populates a table at execution time and does not imply continuous synchronization with source. A corrective test would show: A source update changes the target only through the planned refresh mechanism and produces a recorded watermark.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A late-arriving purchase refers to a customer identifier whose dimension record has not yet been ingested. Its preventative action would be: Define a deliberate unknown-member or late-dimension reconciliation pattern without silently dropping facts. The most direct exposure in the current scenario is: Dashboards can become stale when an initial table snapshot is mistaken for a live derived view.
B:This failure mode belongs mainly to another situation: A reference table stores only the latest airport name and business users explicitly do not need historical values. Its preventative action would be: Implement the required SCD Type 1 overwrite with stable business keys and validated updates. The most direct exposure in the current scenario is: Dashboards can become stale when an initial table snapshot is mistaken for a live derived view.
D:This failure mode belongs mainly to another situation: A retail customer table changes delivery address repeatedly but analysts must reconstruct the address known at any prior invoice date. Its preventative action would be: Model a slowly changing dimension Type 2 with effective dates, current flags and correctly managed surrogate keys. The most direct exposure in the current scenario is: Dashboards can become stale when an initial table snapshot is mistaken for a live derived view.
Question 199
A solution applies: Use a dedicated service principal with scoped Unity Catalog grants and governed credential lifecycle. It addresses: An ETL service principal is granted permissions through a personal employee account that will leave the company. Which operational outcome MOST directly verifies success?
- The pipeline executes under a scoped service identity and the old token is revoked and rejected.
- An approved DDL operation creates the external table at a registered location and an arbitrary path is rejected.
- A new tagged test column receives the required masking behavior without a separate manual table exception.
- The pipeline authenticates using the designated service principal and operates after the employee account is disabled.
Explanation
Correct Answer
D:The strongest acceptance proof is: The pipeline authenticates using the designated service principal and operates after the employee account is disabled. It specifically verifies: Use a dedicated service principal with scoped Unity Catalog grants and governed credential lifecycle. The original issue was: An ETL service principal is granted permissions through a personal employee account that will leave the company. The underlying mechanism is: Workload identities separate automated access from a person's employment and session lifecycle. Without it, an employee departure or password rotation can unexpectedly break critical production ingestion.
Incorrect Answers
A:This result would validate a different measure: Replace committed personal credentials with supported workload authentication and revoke exposed tokens. That is relevant to the distinct problem: A production workload uses a personal access token stored in Git for Databricks REST calls. Here the acceptance criterion is: The pipeline authenticates using the designated service principal and operates after the employee account is disabled.
B:This result would validate a different measure: Configure a supported external location and specific creation privileges rather than bypassing the catalog to write directly. That is relevant to the distinct problem: A user is granted CREATE TABLE but cannot create an external table at an unregistered cloud path. Here the acceptance criterion is: The pipeline authenticates using the designated service principal and operates after the employee account is disabled.
C:This result would validate a different measure: Use centrally governed tags and compatible Unity Catalog ABAC masking policies where supported. That is relevant to the distinct problem: A data compliance team needs consistent redaction of columns tagged as personal data across hundreds of catalog tables. Here the acceptance criterion is: The pipeline authenticates using the designated service principal and operates after the employee account is disabled.
Question 200
An operator introduces: Create a team-specific schema with ownership or delegated creation grants limited to that namespace. The situation is: A newly onboarded product team wants to create its own tables but should not have administration rights over every catalog. Which validation should be required before signoff?
- Discovery tests distinguish raw from certified curated objects and owners confirm their assigned namespaces.
- Catalog lookup and controlled read work from the registered target while unapproved principals cannot bypass it.
- A controlled create-drop rehearsal confirms the expected metadata and physical data lifecycle for the table type.
- The product service principal creates in its own schema and fails the same operation in unrelated schemas.
Explanation
Correct Answer
D:The strongest acceptance proof is: The product service principal creates in its own schema and fails the same operation in unrelated schemas. It specifically verifies: Create a team-specific schema with ownership or delegated creation grants limited to that namespace. The original issue was: A newly onboarded product team wants to create its own tables but should not have administration rights over every catalog. The underlying mechanism is: Delegating within a governed schema supports self-service without tenant-wide administrative access. Without it, global privileges may allow a team to alter or remove other products' data assets.
Incorrect Answers
A:This result would validate a different measure: Define a consistent schema and naming architecture separating layers and business domains by purpose. That is relevant to the distinct problem: Multiple data products put raw, refined and curated assets into a single schema with confusing table names. Here the acceptance criterion is: The product service principal creates in its own schema and fails the same operation in unrelated schemas.
B:This result would validate a different measure: Recreate or update supported external table registration using an approved location and correct access controls. That is relevant to the distinct problem: A managed Delta table is migrated to external storage but the team does not update table registration or grant scopes. Here the acceptance criterion is: The product service principal creates in its own schema and fails the same operation in unrelated schemas.
C:This result would validate a different measure: Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics. That is relevant to the distinct problem: A team expects dropping a managed table to preserve all physical data as though it were an external table. Here the acceptance criterion is: The product service principal creates in its own schema and fails the same operation in unrelated schemas.