Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 49
Question 480
A production data team leaves out: Respect documented safe retention windows and coordinate VACUUM with readers, streams and time-travel needs. Its scenario is: An engineer runs VACUUM with a zero-hour retention on a shared Delta table used for historical audit queries. Which failure mode is MOST relevant?
- Premature deletion may invalidate time travel or a lagging reader's file references.
- Overprovisioning creates higher spend when stages are not parallelizable.
- Ignoring heavy spills can cause cascading I/O latency and missed pipeline deadlines.
- Adding workers may not speed the one overloaded partition and increases total spend.
Explanation
Correct Answer
A:The direct residual exposure is: Premature deletion may invalidate time travel or a lagging reader's file references. The triggering condition is: An engineer runs VACUUM with a zero-hour retention on a shared Delta table used for historical audit queries. The omitted safeguard is: Respect documented safe retention windows and coordinate VACUUM with readers, streams and time-travel needs. This matters because: VACUUM removes old physical files, so aggressive cleanup can break valid historical references. A corrective test would show: A permitted historical query succeeds before and after approved housekeeping within its retention boundary.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A pipeline experiences escalating costs after a team increases node counts but execution duration barely changes. Its preventative action would be: Use run-level compute utilization, cost and stage metrics to identify unused capacity. The most direct exposure in the current scenario is: Premature deletion may invalidate time travel or a lagging reader's file references.
C:This failure mode belongs mainly to another situation: A large aggregation has repeated shuffle spills to disk even though source scans finish quickly. Its preventative action would be: Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage. The most direct exposure in the current scenario is: Premature deletion may invalidate time travel or a lagging reader's file references.
D:This failure mode belongs mainly to another situation: A production Spark query is slow and its stage graph shows one task running far longer than all peers. Its preventative action would be: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size. The most direct exposure in the current scenario is: Premature deletion may invalidate time travel or a lagging reader's file references.
Question 481
The design team proposes: Implement a catalog stewardship process to review metadata and governed tags at dataset publication. The situation is: A central repository contains schemas with missing column owners and no sensitivity classifications. Which technical principle justifies the decision?
- Data sharing scope should minimize disclosed fields and rows and specify a refresh contract.
- Column mask policies can enforce context-dependent presentation without duplicating raw PII.
- Governance metadata is useful only when ownership and classification are kept current.
- Permission reviews should follow actual securable objects and their dependencies rather than name assumptions.
Explanation
Correct Answer
C:The decisive mechanism is: Governance metadata is useful only when ownership and classification are kept current. That explains the selected action: Implement a catalog stewardship process to review metadata and governed tags at dataset publication. In the original scenario: A central repository contains schemas with missing column owners and no sensitivity classifications. The specific unmanaged risk is: Consumers can use incorrectly labeled confidential fields without recognizing their sensitivity. Validate with: A publish-time check rejects missing required descriptions, owners and classification tags.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A customer wants to share only a daily refreshed aggregate, not raw personally identifiable transactions. Its targeted response is: Publish a restricted derived dataset through Delta Sharing after privacy and freshness validation. The current case is governed more directly by: Governance metadata is useful only when ownership and classification are kept current.
B:This principle concerns a separate data-engineering condition: A data product stores PII in a shared Delta table but several reports require different masking for each audience. Its targeted response is: Develop an approved role-aware column mask and test its outputs across supported query clients. The current case is governed more directly by: Governance metadata is useful only when ownership and classification are kept current.
D:This principle concerns a separate data-engineering condition: A governance team grants dataset access solely by a table's displayed name while records are moved to a new schema. Its targeted response is: Reconcile catalog object identity, lineage and current grants after schema or owner changes. The current case is governed more directly by: Governance metadata is useful only when ownership and classification are kept current.
Question 482
A team omits the recommended measure: Add bounded retries with exponential backoff and observable failure after the retry budget. Its environment shows: A batch enrichment task downloads a remote file and sometimes receives transient HTTP 503 responses. What is the MOST direct residual risk?
- Immediate permanent failure on a brief service outage can miss a delivery SLA.
- Readers can combine incompatible versions when related tables update independently.
- Consumers can query invalid data during the interval before quality checks finish.
- Hand-built ordering can drift from the actual dependency graph as tables are added.
Explanation
Correct Answer
A:The direct residual exposure is: Immediate permanent failure on a brief service outage can miss a delivery SLA. The triggering condition is: A batch enrichment task downloads a remote file and sometimes receives transient HTTP 503 responses. The omitted safeguard is: Add bounded retries with exponential backoff and observable failure after the retry budget. This matters because: Retryable transient failures should be distinguished from permanent data or permission errors. A corrective test would show: An injected transient 503 recovers within the retry budget while repeated failures produce actionable alerts.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A regulated pipeline needs consistent transaction boundaries across multiple table writes but updates are published one at a time. Its preventative action would be: Design a compatible atomic publication or versioned readiness contract so consumers see consistent results. The most direct exposure in the current scenario is: Immediate permanent failure on a brief service outage can miss a delivery SLA.
C:This failure mode belongs mainly to another situation: A workflow writes to production tables before a validation task examines null keys and duplicates. Its preventative action would be: Place validations before publishing and use an atomic cutover or controlled target write. The most direct exposure in the current scenario is: Immediate permanent failure on a brief service outage can miss a delivery SLA.
D:This failure mode belongs mainly to another situation: A continuous feed creates and refreshes tables with consistent declarative dependencies but developers manually chain notebooks. Its preventative action would be: Evaluate Lakeflow Spark Declarative Pipelines for the compatible streaming and transformation graph. The most direct exposure in the current scenario is: Immediate permanent failure on a brief service outage can miss a delivery SLA.
Question 483
A platform architect adopts: Implement a Unity Catalog row filter with deterministic identity-to-tenant mapping and test negative cases. For this requirement: A multi-tenant analytics table relies on application code to append customer_id filtering. Which rationale MOST directly supports it?
- Creating an external table requires governed location and storage authorization beyond a generic schema creation grant.
- Central data-plane access rules reduce reliance on every application path applying tenant filtering.
- Separation of duties requires independent decisions for object administration, data modification and read access.
- Column masking can transform sensitive values at query time according to the querying principal.
Explanation
Correct Answer
B:The decisive mechanism is: Central data-plane access rules reduce reliance on every application path applying tenant filtering. That explains the selected action: Implement a Unity Catalog row filter with deterministic identity-to-tenant mapping and test negative cases. In the original scenario: A multi-tenant analytics table relies on application code to append customer_id filtering. The specific unmanaged risk is: One missing tenant predicate can expose another customer’s records. Validate with: A test account from tenant A cannot retrieve tenant B records through SQL, notebooks or the approved BI route.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A user is granted CREATE TABLE but cannot create an external table at an unregistered cloud path. Its targeted response is: Configure a supported external location and specific creation privileges rather than bypassing the catalog to write directly. The current case is governed more directly by: Central data-plane access rules reduce reliance on every application path applying tenant filtering.
C:This principle concerns a separate data-engineering condition: A development team grants ALL PRIVILEGES on a catalog so that one job can create a temporary table. Its targeted response is: Replace broad grants with documented minimum creation and use privileges in a dedicated schema. The current case is governed more directly by: Central data-plane access rules reduce reliance on every application path applying tenant filtering.
D:This principle concerns a separate data-engineering condition: A customer service dashboard must show the last four digits of account identifiers but never full account values. Its targeted response is: Apply a supported column mask or ABAC masking rule to the sensitive field and test different roles. The current case is governed more directly by: Central data-plane access rules reduce reliance on every application path applying tenant filtering.
Question 484
A solution applies: Define a deliberate unknown-member or late-dimension reconciliation pattern without silently dropping facts. It addresses: A late-arriving purchase refers to a customer identifier whose dimension record has not yet been ingested. Which operational outcome MOST directly verifies success?
- A replay from a preserved source batch reproduces the expected silver and gold data after a code fix.
- Sample invoice lines reconcile exactly to the fact table and roll up to the approved daily total.
- The purchase remains traceable and later dimension arrival resolves the customer association without duplication.
- Representative business queries return consistent region totals with an acceptable plan.
Explanation
Correct Answer
C:The strongest acceptance proof is: The purchase remains traceable and later dimension arrival resolves the customer association without duplication. It specifically verifies: Define a deliberate unknown-member or late-dimension reconciliation pattern without silently dropping facts. The original issue was: A late-arriving purchase refers to a customer identifier whose dimension record has not yet been ingested. The underlying mechanism is: Late facts and missing dimension keys require explicit referential handling. Without it, inner joining only currently known dimension rows can undercount legitimate transactions.
Incorrect Answers
A:This result would validate a different measure: Keep a governed raw or bronze representation before applying irreversible cleansing into silver and gold. That is relevant to the distinct problem: A bronze ingestion layer stores only final transformed values and loses raw source fields needed for incident replay. Here the acceptance criterion is: The purchase remains traceable and later dimension arrival resolves the customer association without duplication.
B:This result would validate a different measure: Define fact-table grain at the required transaction or line-item level before aggregation. That is relevant to the distinct problem: A facts table groups product sales at daily level although auditors must inspect individual transaction adjustments. Here the acceptance criterion is: The purchase remains traceable and later dimension arrival resolves the customer association without duplication.
D:This result would validate a different measure: Evaluate dimensional normalization against query complexity, maintenance and expected cardinality. That is relevant to the distinct problem: A customer table includes both region hierarchy and customers; teams disagree whether to snowflake descriptive attributes. Here the acceptance criterion is: The purchase remains traceable and later dimension arrival resolves the customer association without duplication.
Question 485
For this Azure Databricks issue: A bronze ingestion layer stores only final transformed values and loses raw source fields needed for incident replay. Engineers choose: Keep a governed raw or bronze representation before applying irreversible cleansing into silver and gold. What acceptance evidence is BEST?
- File-count and query-profile comparisons show fewer fragments and faster targeted scans.
- A backdated invoice joins to the address valid on its transaction date while the current record shows the latest address.
- A MERGE transaction updates the expected records atomically and a historical version remains queryable within retention.
- A replay from a preserved source batch reproduces the expected silver and gold data after a code fix.
Explanation
Correct Answer
D:The strongest acceptance proof is: A replay from a preserved source batch reproduces the expected silver and gold data after a code fix. It specifically verifies: Keep a governed raw or bronze representation before applying irreversible cleansing into silver and gold. The original issue was: A bronze ingestion layer stores only final transformed values and loses raw source fields needed for incident replay. The underlying mechanism is: Layered lakehouse processing separates source fidelity from approved business transformations. Without it, without recoverable raw inputs the team cannot replay a transformation after discovering a rule defect.
Incorrect Answers
A:This result would validate a different measure: Use coarse suitable partitioning or liquid clustering selected from filter patterns and cardinality. That is relevant to the distinct problem: An event table is partitioned by the unique transaction identifier and creates millions of tiny partitions. Here the acceptance criterion is: A replay from a preserved source batch reproduces the expected silver and gold data after a code fix.
B:This result would validate a different measure: Model a slowly changing dimension Type 2 with effective dates, current flags and correctly managed surrogate keys. That is relevant to the distinct problem: A retail customer table changes delivery address repeatedly but analysts must reconstruct the address known at any prior invoice date. Here the acceptance criterion is: A replay from a preserved source batch reproduces the expected silver and gold data after a code fix.
C:This result would validate a different measure: Use a managed Delta table for transactional updates and controlled schema evolution. That is relevant to the distinct problem: A reporting team stores frequent updates in raw CSV files and expects atomic MERGE and historical time travel. Here the acceptance criterion is: A replay from a preserved source batch reproduces the expected silver and gold data after a code fix.
Question 486
During a lakehouse implementation review, engineers observe: A daily file load uses append-only mode for periodic snapshots of the same full inventory. What is the BEST next action?
- Use Spark-native distributed read and transformation APIs or partition-safe processing.
- Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records.
- Configure event-time windowing and an appropriate watermark based on observed lateness.
- Persist necessary source metadata and checkpoint lineage for troubleshooting and replay.
Explanation
Correct Answer
B:The most appropriate action is: Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records. It addresses the stated condition: A daily file load uses append-only mode for periodic snapshots of the same full inventory. The technical basis is: Input extraction semantics determine whether repeated rows are new events or refreshed state. Without this control, appending full inventories multiplies items and distorts on-hand quantities. A direct acceptance check is: Two full snapshots produce one current inventory row per item and expected updates.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. The corresponding measure there would be: Use Spark-native distributed read and transformation APIs or partition-safe processing. The stated situation instead calls for: Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records.
C:This alternative primarily addresses a different Azure Databricks problem: A stream must group clicks into five-minute event-time windows while some mobile devices transmit data several minutes late. The corresponding measure there would be: Configure event-time windowing and an appropriate watermark based on observed lateness. The stated situation instead calls for: Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records.
D:This alternative primarily addresses a different Azure Databricks problem: A pipeline ingests Event Hubs messages into a bronze table but never captures the partition, offset or event timestamp. The corresponding measure there would be: Persist necessary source metadata and checkpoint lineage for troubleshooting and replay. The stated situation instead calls for: Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records.
Question 487
A platform architect adopts: Configure event-time windowing and an appropriate watermark based on observed lateness. For this requirement: A stream must group clicks into five-minute event-time windows while some mobile devices transmit data several minutes late. Which rationale MOST directly supports it?
- Watermarks define the tolerated late-event horizon for stateful stream processing.
- Pulling distributed data to a single driver defeats parallel execution and memory limits.
- Data pipeline completeness requires tracking arrivals rather than assuming an exact wall-clock boundary.
- Structured Streaming may retry microbatches, so external effects need idempotency.
Explanation
Correct Answer
A:The decisive mechanism is: Watermarks define the tolerated late-event horizon for stateful stream processing. That explains the selected action: Configure event-time windowing and an appropriate watermark based on observed lateness. In the original scenario: A stream must group clicks into five-minute event-time windows while some mobile devices transmit data several minutes late. The specific unmanaged risk is: Too-short watermark drops legitimate delayed events; too-long state retention may exhaust memory. Validate with: Events arriving within the allowed lateness update the intended window and older events follow documented handling.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. Its targeted response is: Use Spark-native distributed read and transformation APIs or partition-safe processing. The current case is governed more directly by: Watermarks define the tolerated late-event horizon for stateful stream processing.
C:This principle concerns a separate data-engineering condition: A Databricks job receives daily structured extracts but an upstream schedule occasionally delivers files late. Its targeted response is: Use incremental file ingestion with explicit late-arrival cutoffs and a measurable source-to-target watermark. The current case is governed more directly by: Watermarks define the tolerated late-event horizon for stateful stream processing.
D:This principle concerns a separate data-engineering condition: A stream writes into an external sink without stable batch identities and causes duplicate side effects on retry. Its targeted response is: Implement idempotent foreachBatch or compatible transactional sink logic keyed by epoch or business identity. The current case is governed more directly by: Watermarks define the tolerated late-event horizon for stateful stream processing.
Question 488
A platform architect adopts: Limit retries to appropriate transient errors and set accountable failure handling for permanent logic problems. For this requirement: A job's retry policy restarts a deterministic bad SQL query six times even though the table is missing. Which rationale MOST directly supports it?
- Retries cannot repair missing tables or invalid schema references without corrective action.
- Event or file-arrival triggers can reduce idle polling while maintaining responsive execution.
- Production job run identities must be stable and authorized independent of a human notebook editor.
- Concurrency management prevents competing executions from corrupting shared mutable state.
Explanation
Correct Answer
A:The decisive mechanism is: Retries cannot repair missing tables or invalid schema references without corrective action. That explains the selected action: Limit retries to appropriate transient errors and set accountable failure handling for permanent logic problems. In the original scenario: A job's retry policy restarts a deterministic bad SQL query six times even though the table is missing. The specific unmanaged risk is: Repeated invalid retries consume compute and obscure the original fixable error. Validate with: A missing-table test stops with the expected error while a transient failure recovers according to policy.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A cloud file feed needs a job triggered by arriving files instead of polling every minute. Its targeted response is: Use a supported file-arrival trigger where its prerequisites and source location are compatible. The current case is governed more directly by: Retries cannot repair missing tables or invalid schema references without corrective action.
C:This principle concerns a separate data-engineering condition: A job uses a service principal that loses access to the destination schema when an employee leaves. Its targeted response is: Assign a dedicated run-as identity with durable scoped grants and an ownership review. The current case is governed more directly by: Retries cannot repair missing tables or invalid schema references without corrective action.
D:This principle concerns a separate data-engineering condition: A daily data job runs twice during an incident and both executions write to the same target. Its targeted response is: Configure appropriate maximum concurrent runs and idempotent target writes to control overlap. The current case is governed more directly by: Retries cannot repair missing tables or invalid schema references without corrective action.
Question 489
A Databricks engineer recommends: Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change. It responds to: A Spark workload repeatedly fails with executor out-of-memory errors on a high-cardinality groupBy. What underlying mechanism BEST explains why?
- Alert quality depends on actionable thresholds and distinguishing transient variation from sustained failure.
- Driver memory is not an unlimited sink for results of distributed computation.
- Logical deletes can leave physical layout and scan behavior needing maintenance.
- Wide aggregations redistribute data and can create memory-intensive hot partitions.
Explanation
Correct Answer
D:The decisive mechanism is: Wide aggregations redistribute data and can create memory-intensive hot partitions. That explains the selected action: Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change. In the original scenario: A Spark workload repeatedly fails with executor out-of-memory errors on a high-cardinality groupBy. The specific unmanaged risk is: Blindly restarting the cluster may reproduce the same failure at the same stage. Validate with: A controlled representative load finishes without OOM and output group counts reconcile.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: An alert triggers on every harmless microbatch delay while real hour-long ingestion stalls go unnoticed. Its targeted response is: Define meaningful latency, error and backlog thresholds with alert routing based on the business SLO. The current case is governed more directly by: Wide aggregations redistribute data and can create memory-intensive hot partitions.
B:This principle concerns a separate data-engineering condition: A Spark join reads large inputs but the driver reports failures after an analyst calls collect() on the result. Its targeted response is: Keep large results distributed and replace driver collection with bounded sampling or distributed writes. The current case is governed more directly by: Wide aggregations redistribute data and can create memory-intensive hot partitions.
C:This principle concerns a separate data-engineering condition: A query on a Delta table slows after a small percentage of rows are deleted but data files are not optimized. Its targeted response is: Review deletion-vector behavior, table layout and suitable optimization for the supported runtime. The current case is governed more directly by: Wide aggregations redistribute data and can create memory-intensive hot partitions.
Question 490
For this Microsoft DP-750 case: A user creates a column mask on a single table and assumes all tables with the same field name are protected. Which design or operational response BEST fits?
- Use Delta Sharing with an authorized recipient, narrowly scoped shares and lifecycle management.
- Revoke or disable the relevant Delta Sharing recipient/share permissions and verify subsequent access.
- Align table retention properties, scheduled maintenance and legal ownership approvals with the current classification.
- Compare table-level masks against centrally governed ABAC coverage and identify missing assignments.
Explanation
Correct Answer
D:The most appropriate action is: Compare table-level masks against centrally governed ABAC coverage and identify missing assignments. It addresses the stated condition: A user creates a column mask on a single table and assumes all tables with the same field name are protected. The technical basis is: Per-table fine-grained policies do not automatically propagate to unrelated objects. Without this control, an identically named confidential field in a new table could be exposed. A direct acceptance check is: A governance test finds masked behavior across all intended tagged objects and detects any unprotected table.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A data-sharing project emails storage keys to an external partner for reading curated Delta results. The corresponding measure there would be: Use Delta Sharing with an authorized recipient, narrowly scoped shares and lifecycle management. The stated situation instead calls for: Compare table-level masks against centrally governed ABAC coverage and identify missing assignments.
B:This alternative primarily addresses a different Azure Databricks problem: A Delta Sharing recipient must stop receiving future updates after an agreement expires. The corresponding measure there would be: Revoke or disable the relevant Delta Sharing recipient/share permissions and verify subsequent access. The stated situation instead calls for: Compare table-level masks against centrally governed ABAC coverage and identify missing assignments.
C:This alternative primarily addresses a different Azure Databricks problem: A regulated source dataset receives an updated retention period but storage cleanup still follows the obsolete policy. The corresponding measure there would be: Align table retention properties, scheduled maintenance and legal ownership approvals with the current classification. The stated situation instead calls for: Compare table-level masks against centrally governed ABAC coverage and identify missing assignments.