Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 71
Question 700
A production data team leaves out: Validate timezone-aware timestamp types and quarantine ambiguous or unparseable event values. Its scenario is: A regulated dataset requires all timestamps to parse as UTC but several rows have ambiguous local values. Which failure mode is MOST relevant?
- Numerically valid but impossible values can contaminate customer metrics and machine-learning features.
- Unclear expectation handling may allow financial errors to reach gold tables or stop all low-risk events unnecessarily.
- Overly strict validation can block legitimate records while loose validation may permit missing keys.
- Ambiguous timestamp parsing can move financial events across reporting windows.
Explanation
Correct Answer
D:The direct residual exposure is: Ambiguous timestamp parsing can move financial events across reporting windows. The triggering condition is: A regulated dataset requires all timestamps to parse as UTC but several rows have ambiguous local values. The omitted safeguard is: Validate timezone-aware timestamp types and quarantine ambiguous or unparseable event values. This matters because: Type checks and temporal semantics must be aligned to prevent incorrect time-series results. A corrective test would show: A sample of valid UTC timestamps passes and known ambiguous strings produce the expected exception record.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A bronze stream sometimes includes ages below zero or above 120 yet the transformation casts every value to integer. Its preventative action would be: Define a range expectation with metrics and the approved drop, fail or quarantine disposition. The most direct exposure in the current scenario is: Ambiguous timestamp parsing can move financial events across reporting windows.
B:This failure mode belongs mainly to another situation: A Lakeflow pipeline has a quality rule but nobody has decided whether invalid transactions should stop publishing. Its preventative action would be: Configure expectations with an explicit fail, drop or retention behavior aligned to business severity. The most direct exposure in the current scenario is: Ambiguous timestamp parsing can move financial events across reporting windows.
C:This failure mode belongs mainly to another situation: A new customer feed is missing optional contact information for 8% of rows and the team treats every null as pipeline failure. Its preventative action would be: Profile source missingness and define field-specific required versus optional expectations. The most direct exposure in the current scenario is: Ambiguous timestamp parsing can move financial events across reporting windows.
Question 701
The following control has NOT been implemented: Maintain a governed schema contract and integration tests across both notebook and declarative pipeline paths. In this case: An engineer changes the expected file schema in one notebook but forgets to update the Lakeflow pipeline definition. Which exposure remains?
- A seemingly successful load can conceal a growing upstream data-loss problem.
- Unclear expectation handling may allow financial errors to reach gold tables or stop all low-risk events unnecessarily.
- Inconsistent schema assumptions create intermittent casts, quarantines or dropped columns.
- Overly strict validation can block legitimate records while loose validation may permit missing keys.
Explanation
Correct Answer
C:The direct residual exposure is: Inconsistent schema assumptions create intermittent casts, quarantines or dropped columns. The triggering condition is: An engineer changes the expected file schema in one notebook but forgets to update the Lakeflow pipeline definition. The omitted safeguard is: Maintain a governed schema contract and integration tests across both notebook and declarative pipeline paths. This matters because: Schema governance must span all ingestion paths that write to a common table. A corrective test would show: Both pipeline paths validate the same additive and incompatible change samples consistently.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A pipeline reports zero failed rows because bad records are silently dropped without any count of what was removed. Its preventative action would be: Collect expectation metrics and reconcile input, accepted, quarantined and rejected record counts. The most direct exposure in the current scenario is: Inconsistent schema assumptions create intermittent casts, quarantines or dropped columns.
B:This failure mode belongs mainly to another situation: A Lakeflow pipeline has a quality rule but nobody has decided whether invalid transactions should stop publishing. Its preventative action would be: Configure expectations with an explicit fail, drop or retention behavior aligned to business severity. The most direct exposure in the current scenario is: Inconsistent schema assumptions create intermittent casts, quarantines or dropped columns.
D:This failure mode belongs mainly to another situation: A new customer feed is missing optional contact information for 8% of rows and the team treats every null as pipeline failure. Its preventative action would be: Profile source missingness and define field-specific required versus optional expectations. The most direct exposure in the current scenario is: Inconsistent schema assumptions create intermittent casts, quarantines or dropped columns.
Question 702
The recommended measure is: Use query-profile evidence and a supported skew mitigation such as adaptive execution or key-aware repartitioning. The operational context is: A DataFrame join spills heavily because a single join key occurs in most rows. Which data engineering concept applies?
- Severe key skew causes uneven task work even with many executors.
- Alert quality depends on actionable thresholds and distinguishing transient variation from sustained failure.
- Maintenance operations themselves consume compute and should correspond to data growth and read patterns.
- A single hot partition can dominate a distributed stage despite idle executor capacity.
Explanation
Correct Answer
A:The decisive mechanism is: Severe key skew causes uneven task work even with many executors. That explains the selected action: Use query-profile evidence and a supported skew mitigation such as adaptive execution or key-aware repartitioning. In the original scenario: A DataFrame join spills heavily because a single join key occurs in most rows. The specific unmanaged risk is: A small number of hot tasks can dominate the join while extra workers remain idle. Validate with: Post-mitigation stage durations are balanced and joined row counts remain correct.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: An alert triggers on every harmless microbatch delay while real hour-long ingestion stalls go unnoticed. Its targeted response is: Define meaningful latency, error and backlog thresholds with alert routing based on the business SLO. The current case is governed more directly by: Severe key skew causes uneven task work even with many executors.
C:This principle concerns a separate data-engineering condition: A new Delta table maintenance job combines repeated OPTIMIZE and VACUUM at every microbatch even though changes are tiny. Its targeted response is: Set a measured maintenance cadence or evaluate supported predictive optimization for eligible managed tables. The current case is governed more directly by: Severe key skew causes uneven task work even with many executors.
D:This principle concerns a separate data-engineering condition: A production Spark query is slow and its stage graph shows one task running far longer than all peers. Its targeted response is: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size. The current case is governed more directly by: Severe key skew causes uneven task work even with many executors.
Question 703
A production data team leaves out: Model a slowly changing dimension Type 2 with effective dates, current flags and correctly managed surrogate keys. Its scenario is: A retail customer table changes delivery address repeatedly but analysts must reconstruct the address known at any prior invoice date. Which failure mode is MOST relevant?
- Using Type 1 updates may silently rewrite the apparent address on older transaction reports.
- Inner joining only currently known dimension rows can undercount legitimate transactions.
- Choosing Delta-specific mutation assumptions for ordinary Parquet files can break consumers.
- Duplicate dimension keys can inflate sums when each fact joins more than one dimension row.
Explanation
Correct Answer
A:The direct residual exposure is: Using Type 1 updates may silently rewrite the apparent address on older transaction reports. The triggering condition is: A retail customer table changes delivery address repeatedly but analysts must reconstruct the address known at any prior invoice date. The omitted safeguard is: Model a slowly changing dimension Type 2 with effective dates, current flags and correctly managed surrogate keys. This matters because: SCD Type 2 preserves historical attribute versions rather than overwriting the prior business state. A corrective test would show: A backdated invoice joins to the address valid on its transaction date while the current record shows the latest address.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A late-arriving purchase refers to a customer identifier whose dimension record has not yet been ingested. Its preventative action would be: Define a deliberate unknown-member or late-dimension reconciliation pattern without silently dropping facts. The most direct exposure in the current scenario is: Using Type 1 updates may silently rewrite the apparent address on older transaction reports.
C:This failure mode belongs mainly to another situation: An external sharing integration needs interoperable columnar read files but not native Delta updates. Its preventative action would be: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements. The most direct exposure in the current scenario is: Using Type 1 updates may silently rewrite the apparent address on older transaction reports.
D:This failure mode belongs mainly to another situation: A finance dashboard needs monthly revenue by customer and must join customer dimensions without multiplying facts. Its preventative action would be: Design a conformed star schema with explicit fact grain and unique dimension keys. The most direct exposure in the current scenario is: Using Type 1 updates may silently rewrite the apparent address on older transaction reports.
Question 704
The design team proposes: Align Azure storage role assignments with the registered Unity Catalog storage credential and external-location privileges. The situation is: A managed identity can read a storage container but a newly created external table is denied in Databricks. Which technical principle justifies the decision?
- Cloud IAM and Unity Catalog are complementary authorization layers that must both permit the intended operation.
- Column masking can transform sensitive values at query time according to the querying principal.
- Object discovery or BROWSE is not a grant to read its underlying data.
- Least privilege includes removing access when the business purpose expires.
Explanation
Correct Answer
A:The decisive mechanism is: Cloud IAM and Unity Catalog are complementary authorization layers that must both permit the intended operation. That explains the selected action: Align Azure storage role assignments with the registered Unity Catalog storage credential and external-location privileges. In the original scenario: A managed identity can read a storage container but a newly created external table is denied in Databricks. The specific unmanaged risk is: Fixing only Azure IAM may leave catalog object access denied while broad grants increase risk. Validate with: A storage access test and catalog grant review both pass for the designated service principal.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A customer service dashboard must show the last four digits of account identifiers but never full account values. Its targeted response is: Apply a supported column mask or ABAC masking rule to the sensitive field and test different roles. The current case is governed more directly by: Cloud IAM and Unity Catalog are complementary authorization layers that must both permit the intended operation.
C:This principle concerns a separate data-engineering condition: An analyst can view the names of confidential tables and assumes browsing metadata implies permission to query records. Its targeted response is: Separate discoverability privileges from USE and SELECT privileges in the catalog permissions model. The current case is governed more directly by: Cloud IAM and Unity Catalog are complementary authorization layers that must both permit the intended operation.
D:This principle concerns a separate data-engineering condition: A vendor's service principal was granted global access for a one-month integration and the project ended. Its targeted response is: Review and revoke stale grants and any linked service credentials, preserving required internal ownership. The current case is governed more directly by: Cloud IAM and Unity Catalog are complementary authorization layers that must both permit the intended operation.
Question 705
An operator introduces: Choose node families and driver/worker sizes using CPU, memory, shuffle and workload profile evidence. The situation is: A new job uses a large memory-optimized node merely because its source data is large, although profiling shows compute-bound transforms. Which validation should be required before signoff?
- A quiet-period rehearsal retains the required streaming operation and subsequent events process without missed offsets.
- A controlled instance-family comparison achieves the same results with improved stage time and total execution cost.
- A forced restart of interactive compute leaves the scheduled production task unaffected.
- A load rehearsal meets dashboard p95 response targets while notebook resource spikes do not affect query service.
Explanation
Correct Answer
B:The strongest acceptance proof is: A controlled instance-family comparison achieves the same results with improved stage time and total execution cost. It specifically verifies: Choose node families and driver/worker sizes using CPU, memory, shuffle and workload profile evidence. The original issue was: A new job uses a large memory-optimized node merely because its source data is large, although profiling shows compute-bound transforms. The underlying mechanism is: Workload bottlenecks rather than raw input size should drive Spark cluster shape and resource selection. Without it, overspending on memory does not relieve a CPU-bound stage and may worsen the cost-performance ratio.
Incorrect Answers
A:This result would validate a different measure: Select stream-appropriate compute and lifecycle settings so monitoring and availability targets are met. That is relevant to the distinct problem: A long-running streaming pipeline is configured with aggressive auto-termination during brief periods without arriving events. Here the acceptance criterion is: A controlled instance-family comparison achieves the same results with improved stage time and total execution cost.
C:This result would validate a different measure: Define a job-owned execution identity and eligible task compute separate from ad hoc analyst cluster lifecycle. That is relevant to the distinct problem: A scheduled ETL run uses shared analyst compute and fails whenever the cluster owner restarts a notebook session. Here the acceptance criterion is: A controlled instance-family comparison achieves the same results with improved stage time and total execution cost.
D:This result would validate a different measure: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. That is relevant to the distinct problem: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. Here the acceptance criterion is: A controlled instance-family comparison achieves the same results with improved stage time and total execution cost.
Question 706
The team implements: Add reviewed Genie instructions, table descriptions and example questions consistent with trusted catalog semantics. The original problem is: An AI/BI Genie space is configured with ambiguous metric terminology and produces misleading natural-language interpretations. Which test MOST convincingly demonstrates correctness?
- Authorized principals list and read a governed volume path while unrelated roles receive denied access.
- Discovery tests distinguish raw from certified curated objects and owners confirm their assigned namespaces.
- A source update followed by the documented refresh produces reconciled aggregate totals.
- A set of known business prompts yields reconciled SQL and approved metric definitions.
Explanation
Correct Answer
D:The strongest acceptance proof is: A set of known business prompts yields reconciled SQL and approved metric definitions. It specifically verifies: Add reviewed Genie instructions, table descriptions and example questions consistent with trusted catalog semantics. The original issue was: An AI/BI Genie space is configured with ambiguous metric terminology and produces misleading natural-language interpretations. The underlying mechanism is: Natural-language discovery needs business context and governed metadata to disambiguate metrics. Without it, an unreviewed semantic instruction can return a plausible but incorrect business measure.
Incorrect Answers
A:This result would validate a different measure: Use Unity Catalog volumes with suitable schema and file privileges for supported non-tabular files. That is relevant to the distinct problem: A departmental data owner stores raw file attachments inside ad hoc DBFS directories instead of a governable data space. Here the acceptance criterion is: A set of known business prompts yields reconciled SQL and approved metric definitions.
B:This result would validate a different measure: Define a consistent schema and naming architecture separating layers and business domains by purpose. That is relevant to the distinct problem: Multiple data products put raw, refined and curated assets into a single schema with confusing table names. Here the acceptance criterion is: A set of known business prompts yields reconciled SQL and approved metric definitions.
C:This result would validate a different measure: Create an eligible materialized view or managed derived table with appropriate refresh and access rules. That is relevant to the distinct problem: A BI group needs to query a materialized aggregate while the source transaction Delta table changes hourly. Here the acceptance criterion is: A set of known business prompts yields reconciled SQL and approved metric definitions.
Question 707
For this Azure Databricks issue: A SaaS database updates and deletes customer records but a pipeline appends every nightly full extract to a Delta table. Engineers choose: Use a supported change-data-capture feed and apply insert, update and delete semantics to the target. What acceptance evidence is BEST?
- Insert-update-delete test events produce one correct current target record per key.
- The workload completes without driver collection and Spark metrics show distributed processing.
- A seed file loads the exact expected columns and invalid rows are reported rather than silently accepted.
- ADF completion evidence and Databricks job dependencies show an end-to-end successful load.
Explanation
Correct Answer
A:The strongest acceptance proof is: Insert-update-delete test events produce one correct current target record per key. It specifically verifies: Use a supported change-data-capture feed and apply insert, update and delete semantics to the target. The original issue was: A SaaS database updates and deletes customer records but a pipeline appends every nightly full extract to a Delta table. The underlying mechanism is: CDC propagates source mutations without duplicating full snapshots. Without it, append-only full extracts can multiply current records and leave deleted source rows active.
Incorrect Answers
B:This result would validate a different measure: Use Spark-native distributed read and transformation APIs or partition-safe processing. That is relevant to the distinct problem: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. Here the acceptance criterion is: Insert-update-delete test events produce one correct current target record per key.
C:This result would validate a different measure: Specify the correct input delimiter, header and schema and validate parsing on a representative file. That is relevant to the distinct problem: An overnight load reads tab-separated files but an engineer assumes comma separators and casts malformed columns to null. Here the acceptance criterion is: Insert-update-delete test events produce one correct current target record per key.
D:This result would validate a different measure: Coordinate ADF orchestration with the appropriate Databricks task and Unity Catalog access model. That is relevant to the distinct problem: A data product needs to copy a supported Azure Data Factory pipeline output into governed Databricks tables. Here the acceptance criterion is: Insert-update-delete test events produce one correct current target record per key.
Question 708
A data product owner adopts: Preserve diagnostic events and Spark UI stage evidence before restart where possible. The reason is: An operator restarts a failed Databricks cluster before collecting the Spark executor and driver logs. What observed result BEST proves effectiveness?
- A selective query reads fewer files and meets latency targets after a controlled layout improvement.
- A simulated failure yields archived logs, failed stage IDs and a documented root-cause path.
- The isolated benchmark meets its percentile duration under simultaneous unrelated workload pressure.
- A peak-concurrency replay meets latency and queue targets with recorded cost.
Explanation
Correct Answer
B:The strongest acceptance proof is: A simulated failure yields archived logs, failed stage IDs and a documented root-cause path. It specifically verifies: Preserve diagnostic events and Spark UI stage evidence before restart where possible. The original issue was: An operator restarts a failed Databricks cluster before collecting the Spark executor and driver logs. The underlying mechanism is: Transient cluster restarts can remove evidence necessary to distinguish dependency, data and resource failures. Without it, A recurring failure may never be diagnosed if its only logs are destroyed repeatedly.
Incorrect Answers
A:This result would validate a different measure: Inspect query predicates and Delta data skipping or clustering layout before increasing warehouse size again. That is relevant to the distinct problem: A warehouse query remains slow even after larger compute is allocated and the profile reports scanning nearly every file. Here the acceptance criterion is: A simulated failure yields archived logs, failed stage IDs and a documented root-cause path.
C:This result would validate a different measure: Compare workload contention and resource reservations, then isolate or resize the critical job compute. That is relevant to the distinct problem: A production notebook intermittently times out only when many other jobs share the same classic cluster. Here the acceptance criterion is: A simulated failure yields archived logs, failed stage IDs and a documented root-cause path.
D:This result would validate a different measure: Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages. That is relevant to the distinct problem: An interactive warehouse has high query queues during the morning but low CPU utilization after lunch. Here the acceptance criterion is: A simulated failure yields archived logs, failed stage IDs and a documented root-cause path.
Question 709
A data product owner adopts: Validate Unity Catalog lineage capture coverage and supplement unsupported external steps with documented provenance. The reason is: A lineage graph shows no relationship for data loaded outside the monitored platform and the team assumes the dataset has no upstream dependency. What observed result BEST proves effectiveness?
- A provenance review identifies both captured Databricks edges and documented external system dependencies.
- An access matrix tests current schema objects and verifies intended principals after migration.
- The designated active service or steward owns the object and prior inappropriate rights are removed.
- A hold rehearsal retrieves the original protected records and documents preservation scope and access.
Explanation
Correct Answer
A:The strongest acceptance proof is: A provenance review identifies both captured Databricks edges and documented external system dependencies. It specifically verifies: Validate Unity Catalog lineage capture coverage and supplement unsupported external steps with documented provenance. The original issue was: A lineage graph shows no relationship for data loaded outside the monitored platform and the team assumes the dataset has no upstream dependency. The underlying mechanism is: Automatic lineage visibility has collection boundaries and does not imply every real-world dependency is recorded. Without it, uncaptured external ingestion may lead to incomplete impact analysis and unsafe upstream changes.
Incorrect Answers
B:This result would validate a different measure: Reconcile catalog object identity, lineage and current grants after schema or owner changes. That is relevant to the distinct problem: A governance team grants dataset access solely by a table's displayed name while records are moved to a new schema. Here the acceptance criterion is: A provenance review identifies both captured Databricks edges and documented external system dependencies.
C:This result would validate a different measure: Transfer ownership through approved governance workflow and review grants after reassignment. That is relevant to the distinct problem: An audit asks who owns a catalog table but object ownership still points to an inactive employee. Here the acceptance criterion is: A provenance review identifies both captured Databricks edges and documented external system dependencies.
D:This result would validate a different measure: Design record-level retention and external legal-hold workflows appropriate to the applicable policy. That is relevant to the distinct problem: A product owner assumes a nightly Delta table overwrite meets a legal hold requirement for old records. Here the acceptance criterion is: A provenance review identifies both captured Databricks edges and documented external system dependencies.
Question 710
A solution applies: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements. It addresses: An external sharing integration needs interoperable columnar read files but not native Delta updates. Which operational outcome MOST directly verifies success?
- A representative account-filter query reads fewer irrelevant files after optimized clustering.
- The top event-time queries show improved data skipping and reconciled row counts.
- The external reader loads approved columns and a documented compatibility test passes.
- Representative business queries return consistent region totals with an acceptable plan.
Explanation
Correct Answer
C:The strongest acceptance proof is: The external reader loads approved columns and a documented compatibility test passes. It specifically verifies: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements. The original issue was: An external sharing integration needs interoperable columnar read files but not native Delta updates. The underlying mechanism is: File and table format choices depend on transactional features and consumer interoperability. Without it, choosing Delta-specific mutation assumptions for ordinary Parquet files can break consumers.
Incorrect Answers
A:This result would validate a different measure: Enable supported liquid clustering with appropriate keys and run compatible optimization. That is relevant to the distinct problem: A large Delta fact table is filtered primarily on high-cardinality account IDs with changing query patterns. Here the acceptance criterion is: The external reader loads approved columns and a documented compatibility test passes.
B:This result would validate a different measure: Reevaluate data layout using representative event-time filters and a suitable clustering key. That is relevant to the distinct problem: An event stream is partitioned by ingestion day but analysts constantly filter by event timestamp ranges crossing midnight. Here the acceptance criterion is: The external reader loads approved columns and a documented compatibility test passes.
D:This result would validate a different measure: Evaluate dimensional normalization against query complexity, maintenance and expected cardinality. That is relevant to the distinct problem: A customer table includes both region hierarchy and customers; teams disagree whether to snowflake descriptive attributes. Here the acceptance criterion is: The external reader loads approved columns and a documented compatibility test passes.