Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 29
Question 280
A team omits the recommended measure: Configure appropriate maximum concurrent runs and idempotent target writes to control overlap. Its environment shows: A daily data job runs twice during an incident and both executions write to the same target. What is the MOST direct residual risk?
- Overlapping runs may race to overwrite outputs or duplicate appended records.
- A clock-only trigger can start before the dependency has produced the expected dataset.
- Temporarily editing the main schedule may skip normal deliveries or replay wrong dates.
- User-linked credentials can break scheduled jobs after account deactivation or team reassignment.
Explanation
Correct Answer
A:The direct residual exposure is: Overlapping runs may race to overwrite outputs or duplicate appended records. The triggering condition is: A daily data job runs twice during an incident and both executions write to the same target. The omitted safeguard is: Configure appropriate maximum concurrent runs and idempotent target writes to control overlap. This matters because: Concurrency management prevents competing executions from corrupting shared mutable state. A corrective test would show: A controlled dual-trigger test respects the run limit and produces one correct final target.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A pipeline must run when an upstream Databricks table is updated but analysts have hardcoded a daily time estimate. Its preventative action would be: Evaluate supported table update triggers or a clear upstream job dependency based on actual availability. The most direct exposure in the current scenario is: Overlapping runs may race to overwrite outputs or duplicate appended records.
C:This failure mode belongs mainly to another situation: An ingestion job requires a one-time backfill without changing the future daily production schedule. Its preventative action would be: Run a controlled parameterized manual invocation for the historical interval while retaining the regular trigger. The most direct exposure in the current scenario is: Overlapping runs may race to overwrite outputs or duplicate appended records.
D:This failure mode belongs mainly to another situation: A job uses a service principal that loses access to the destination schema when an employee leaves. Its preventative action would be: Assign a dedicated run-as identity with durable scoped grants and an ownership review. The most direct exposure in the current scenario is: Overlapping runs may race to overwrite outputs or duplicate appended records.
Question 281
The following control has NOT been implemented: Use row-filter policies on the governed table and test permitted region entitlements. In this case: A shared customer table needs dynamic per-region row restrictions but the business refuses duplicating physical datasets. Which exposure remains?
- A migrated table can retain unintended access or lose required controls after namespace changes.
- Unauthorized grants can remain undetected or be impossible to trace.
- Premature removal of obsolete files can prevent required time travel or restore operations.
- Duplicated datasets can diverge or escape synchronized retention and correction policies.
Explanation
Correct Answer
D:The direct residual exposure is: Duplicated datasets can diverge or escape synchronized retention and correction policies. The triggering condition is: A shared customer table needs dynamic per-region row restrictions but the business refuses duplicating physical datasets. The omitted safeguard is: Use row-filter policies on the governed table and test permitted region entitlements. This matters because: Row filters can restrict query results without maintaining separate physical copies by region. A corrective test would show: Queries from two test regions return only their eligible rows from the same table.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A governance team grants dataset access solely by a table's displayed name while records are moved to a new schema. Its preventative action would be: Reconcile catalog object identity, lineage and current grants after schema or owner changes. The most direct exposure in the current scenario is: Duplicated datasets can diverge or escape synchronized retention and correction policies.
B:This failure mode belongs mainly to another situation: A compliance auditor requests a record of who changed Unity Catalog grants but audit logs are not collected. Its preventative action would be: Configure supported Databricks audit logging and route relevant events to a secured retention destination. The most direct exposure in the current scenario is: Duplicated datasets can diverge or escape synchronized retention and correction policies.
C:This failure mode belongs mainly to another situation: A regulation requires keeping event records for 90 days but a job performs VACUUM after only a few hours. Its preventative action would be: Coordinate Delta retention configuration, VACUUM safety settings and formal recovery requirements. The most direct exposure in the current scenario is: Duplicated datasets can diverge or escape synchronized retention and correction policies.
Question 282
A data product owner adopts: Collect job timing, task duration and relevant Spark stage metrics in a monitored baseline. The reason is: A job's duration doubles but an administrator sees only its final success status with no phase-level metrics. What observed result BEST proves effectiveness?
- A permitted historical query succeeds before and after approved housekeeping within its retention boundary.
- A dashboard highlights task and stage regressions before the total job breaches its SLA.
- A short benign delay does not page staff while a sustained backlog triggers the documented severity.
- Spark storage metrics show released cache and unchanged correctness with improved total runtime.
Explanation
Correct Answer
B:The strongest acceptance proof is: A dashboard highlights task and stage regressions before the total job breaches its SLA. It specifically verifies: Collect job timing, task duration and relevant Spark stage metrics in a monitored baseline. The original issue was: A job's duration doubles but an administrator sees only its final success status with no phase-level metrics. The underlying mechanism is: Success alone cannot show degradation against latency or cost service objectives. Without it, unobserved performance drift can cause the pipeline to miss future delivery windows.
Incorrect Answers
A:This result would validate a different measure: Respect documented safe retention windows and coordinate VACUUM with readers, streams and time-travel needs. That is relevant to the distinct problem: An engineer runs VACUUM with a zero-hour retention on a shared Delta table used for historical audit queries. Here the acceptance criterion is: A dashboard highlights task and stage regressions before the total job breaches its SLA.
C:This result would validate a different measure: Define meaningful latency, error and backlog thresholds with alert routing based on the business SLO. That is relevant to the distinct problem: An alert triggers on every harmless microbatch delay while real hour-long ingestion stalls go unnoticed. Here the acceptance criterion is: A dashboard highlights task and stage regressions before the total job breaches its SLA.
D:This result would validate a different measure: Use caching only for beneficial reuse and release persisted state when no longer needed. That is relevant to the distinct problem: A team enables Spark cache for a one-time ETL dataset and never unpersists it before processing the next large table. Here the acceptance criterion is: A dashboard highlights task and stage regressions before the total job breaches its SLA.
Question 283
For this Azure Databricks issue: An application uses an external Azure Key Vault secret scope but a notebook prints the secret into task logs. Engineers choose: Use supported Databricks secret access with least-privilege scope access and remove secret values from logs. What acceptance evidence is BEST?
- The job authenticates successfully and audited logs contain no raw secret or token.
- A new tagged test column receives the required masking behavior without a separate manual table exception.
- A permitted principal in the allowed workspace succeeds while an unauthorized user or workspace fails.
- The analyst can SELECT the authorized table but cannot UPDATE it or read an unrelated protected table.
Explanation
Correct Answer
A:The strongest acceptance proof is: The job authenticates successfully and audited logs contain no raw secret or token. It specifically verifies: Use supported Databricks secret access with least-privilege scope access and remove secret values from logs. The original issue was: An application uses an external Azure Key Vault secret scope but a notebook prints the secret into task logs. The underlying mechanism is: Secret retrieval must be paired with permissions and safe handling to avoid disclosure after retrieval. Without it, A protected secret can still leak through debug output or unrestricted notebook results.
Incorrect Answers
B:This result would validate a different measure: Use centrally governed tags and compatible Unity Catalog ABAC masking policies where supported. That is relevant to the distinct problem: A data compliance team needs consistent redaction of columns tagged as personal data across hundreds of catalog tables. Here the acceptance criterion is: The job authenticates successfully and audited logs contain no raw secret or token.
C:This result would validate a different measure: Apply both the workspace binding and appropriate Unity Catalog object grants, then test each dimension. That is relevant to the distinct problem: A team restricts access through a catalog-to-workspace binding but treats it as a replacement for data privileges. Here the acceptance criterion is: The job authenticates successfully and audited logs contain no raw secret or token.
D:This result would validate a different measure: Grant only the required USE and SELECT privileges to the analyst group and inspect inherited grants. That is relevant to the distinct problem: A finance analyst must read an approved table but was granted broad MODIFY rights at the schema level. Here the acceptance criterion is: The job authenticates successfully and audited logs contain no raw secret or token.
Question 284
During a lakehouse implementation review, engineers observe: A multi-environment setup uses the same catalog for test and production because bundle target variables were omitted. What is the BEST next action?
- Package declarative automation bundle configurations with environment-specific targets and deploy through the supported CLI.
- Include an integration test using actual Delta table semantics and representative source changes.
- Use idempotent deployment methods or reconcile existing resource IDs before creating replacements.
- Define and validate environment-specific catalog, workspace and identity targets in deployment configuration.
Explanation
Correct Answer
D:The most appropriate action is: Define and validate environment-specific catalog, workspace and identity targets in deployment configuration. It addresses the stated condition: A multi-environment setup uses the same catalog for test and production because bundle target variables were omitted. The technical basis is: Deployment isolation depends on correct values for each environment's object references. Without this control, A staging test might overwrite production assets when default variables point to shared locations. A direct acceptance check is: A rendered staging plan references only staging catalogs and refuses production resource paths.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A team copies Databricks notebooks manually between three workspaces and each uses different hardcoded identifiers. The corresponding measure there would be: Package declarative automation bundle configurations with environment-specific targets and deploy through the supported CLI. The stated situation instead calls for: Define and validate environment-specific catalog, workspace and identity targets in deployment configuration.
B:This alternative primarily addresses a different Azure Databricks problem: A test harness mocks out Delta writes but the real job uses MERGE and unexpectedly changes historical rows. The corresponding measure there would be: Include an integration test using actual Delta table semantics and representative source changes. The stated situation instead calls for: Define and validate environment-specific catalog, workspace and identity targets in deployment configuration.
C:This alternative primarily addresses a different Azure Databricks problem: A REST deployment script creates duplicate jobs every time CI reruns after a timeout. The corresponding measure there would be: Use idempotent deployment methods or reconcile existing resource IDs before creating replacements. The stated situation instead calls for: Define and validate environment-specific catalog, workspace and identity targets in deployment configuration.
Question 285
The team implements: Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans. The original problem is: A pipeline joins a huge fact table with a tiny dimension and a normal shuffle join creates excessive network traffic. Which test MOST convincingly demonstrates correctness?
- The merged dataset preserves typed values and column names from both input feeds.
- Known events around midnight map to the intended store-local date and correct UTC instant.
- Spark UI shows reduced shuffle bytes and matching results without executor out-of-memory.
- A controlled source sample produces exactly the permitted group keys and reconciled sums.
Explanation
Correct Answer
C:The strongest acceptance proof is: Spark UI shows reduced shuffle bytes and matching results without executor out-of-memory. It specifically verifies: Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans. The original issue was: A pipeline joins a huge fact table with a tiny dimension and a normal shuffle join creates excessive network traffic. The underlying mechanism is: Spark can reduce shuffle costs by broadcasting a sufficiently small dimension where appropriate. Without it, blindly broadcasting a large table can exhaust executor memory while unnecessary shuffles slow joins.
Incorrect Answers
A:This result would validate a different measure: Align compatible schemas and use name-based union semantics with explicit type checks. That is relevant to the distinct problem: Two quarterly sales feeds have the same logical fields but different column ordering and a union shifts amounts into dates. Here the acceptance criterion is: Spark UI shows reduced shuffle bytes and matching results without executor out-of-memory.
B:This result would validate a different measure: Normalize time zone semantics before extracting business-local reporting dates. That is relevant to the distinct problem: A date field mixes UTC timestamps and local store times but a pipeline groups all values by the raw date string. Here the acceptance criterion is: Spark UI shows reduced shuffle bytes and matching results without executor out-of-memory.
D:This result would validate a different measure: Apply the required filters and join conditions before aggregation and validate authorized output groups. That is relevant to the distinct problem: An analyst calculates total revenue grouped by country but a join includes countries not approved for reporting. Here the acceptance criterion is: Spark UI shows reduced shuffle bytes and matching results without executor out-of-memory.
Question 286
An implementation skips: Evaluate Lakeflow Spark Declarative Pipelines for the compatible streaming and transformation graph. Engineers are observing: A continuous feed creates and refreshes tables with consistent declarative dependencies but developers manually chain notebooks. What consequence is MOST likely?
- An early downstream task can publish partial business totals without raising an obvious failure.
- Hand-built ordering can drift from the actual dependency graph as tables are added.
- Hard-coded development resources can cause a production job to read or overwrite the wrong data.
- Immediate permanent failure on a brief service outage can miss a delivery SLA.
Explanation
Correct Answer
B:The direct residual exposure is: Hand-built ordering can drift from the actual dependency graph as tables are added. The triggering condition is: A continuous feed creates and refreshes tables with consistent declarative dependencies but developers manually chain notebooks. The omitted safeguard is: Evaluate Lakeflow Spark Declarative Pipelines for the compatible streaming and transformation graph. This matters because: Declarative pipelines express table dependencies and data quality rules without bespoke notebook orchestration. A corrective test would show: A pipeline run updates tables in valid dependency order and records expectation outcomes.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. Its preventative action would be: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. The most direct exposure in the current scenario is: Hand-built ordering can drift from the actual dependency graph as tables are added.
C:This failure mode belongs mainly to another situation: A multi-team notebook uses hard-coded workspace names and credentials, making the same pipeline fail in staging. Its preventative action would be: Parameterize environment-specific paths, catalogs and identities through governed deployment configuration. The most direct exposure in the current scenario is: Hand-built ordering can drift from the actual dependency graph as tables are added.
D:This failure mode belongs mainly to another situation: A batch enrichment task downloads a remote file and sometimes receives transient HTTP 503 responses. Its preventative action would be: Add bounded retries with exponential backoff and observable failure after the retry budget. The most direct exposure in the current scenario is: Hand-built ordering can drift from the actual dependency graph as tables are added.
Question 287
A platform architect adopts: Enable supported liquid clustering with appropriate keys and run compatible optimization. For this requirement: A large Delta fact table is filtered primarily on high-cardinality account IDs with changing query patterns. Which rationale MOST directly supports it?
- Delta version history can provide auditable historical snapshots while underlying files are retained.
- Efficient layout keys are driven by actual access patterns and file statistics.
- Type 1 dimensions replace nonhistorical descriptive attributes without retaining obsolete versions.
- Liquid clustering can organize files for selective filters without static partition boundaries.
Explanation
Correct Answer
D:The decisive mechanism is: Liquid clustering can organize files for selective filters without static partition boundaries. That explains the selected action: Enable supported liquid clustering with appropriate keys and run compatible optimization. In the original scenario: A large Delta fact table is filtered primarily on high-cardinality account IDs with changing query patterns. The specific unmanaged risk is: Rigid directory partitioning can cause imbalance and require costly redesign as filters change. Validate with: A representative account-filter query reads fewer irrelevant files after optimized clustering.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A business needs to compare two point-in-time table versions during a financial reconciliation. Its targeted response is: Use supported Delta table version history and time travel within the documented retention window. The current case is governed more directly by: Liquid clustering can organize files for selective filters without static partition boundaries.
B:This principle concerns a separate data-engineering condition: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. Its targeted response is: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys. The current case is governed more directly by: Liquid clustering can organize files for selective filters without static partition boundaries.
C:This principle concerns a separate data-engineering condition: A reference table stores only the latest airport name and business users explicitly do not need historical values. Its targeted response is: Implement the required SCD Type 1 overwrite with stable business keys and validated updates. The current case is governed more directly by: Liquid clustering can organize files for selective filters without static partition boundaries.
Question 288
A data engineering team faces this Azure Databricks requirement: A job fails during a temporary Azure storage outage but operators rerun every successful upstream task. Which action is MOST appropriate?
- Collect job timing, task duration and relevant Spark stage metrics in a monitored baseline.
- Set a measured maintenance cadence or evaluate supported predictive optimization for eligible managed tables.
- Respect documented safe retention windows and coordinate VACUUM with readers, streams and time-travel needs.
- Use Lakeflow Jobs repair at the failed task when dependency outputs remain valid.
Explanation
Correct Answer
D:The most appropriate action is: Use Lakeflow Jobs repair at the failed task when dependency outputs remain valid. It addresses the stated condition: A job fails during a temporary Azure storage outage but operators rerun every successful upstream task. The technical basis is: Task-level repair can reduce replay cost while preserving successful work. Without this control, unnecessary replay may duplicate writes and extend incident recovery time. A direct acceptance check is: A controlled transient outage is repaired without rerunning completed expensive ingestion.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A job's duration doubles but an administrator sees only its final success status with no phase-level metrics. The corresponding measure there would be: Collect job timing, task duration and relevant Spark stage metrics in a monitored baseline. The stated situation instead calls for: Use Lakeflow Jobs repair at the failed task when dependency outputs remain valid.
B:This alternative primarily addresses a different Azure Databricks problem: A new Delta table maintenance job combines repeated OPTIMIZE and VACUUM at every microbatch even though changes are tiny. The corresponding measure there would be: Set a measured maintenance cadence or evaluate supported predictive optimization for eligible managed tables. The stated situation instead calls for: Use Lakeflow Jobs repair at the failed task when dependency outputs remain valid.
C:This alternative primarily addresses a different Azure Databricks problem: An engineer runs VACUUM with a zero-hour retention on a shared Delta table used for historical audit queries. The corresponding measure there would be: Respect documented safe retention windows and coordinate VACUUM with readers, streams and time-travel needs. The stated situation instead calls for: Use Lakeflow Jobs repair at the failed task when dependency outputs remain valid.
Question 289
A solution applies: Use a compatible unpivot or stack transformation with explicit month and amount columns. It addresses: A large orders dataset has one column per month but a dashboard expects month-value rows for filtering and grouping. Which operational outcome MOST directly verifies success?
- A timed append comparison meets the freshness and duplicate controls with lower compute consumption.
- Test histories with retries and real re-enrollments produce the approved per-student count.
- The unpivoted output contains exactly one row per original item-month and preserves totals.
- A seeded dataset retains zero-quantity items while rows with null quantities follow the documented rule.
Explanation
Correct Answer
C:The strongest acceptance proof is: The unpivoted output contains exactly one row per original item-month and preserves totals. It specifically verifies: Use a compatible unpivot or stack transformation with explicit month and amount columns. The original issue was: A large orders dataset has one column per month but a dashboard expects month-value rows for filtering and grouping. The underlying mechanism is: Unpivoting converts a wide repeated-measure structure into a normalized row representation. Without it, directly summing mismatched month columns can produce fragile and incomplete queries.
Incorrect Answers
A:This result would validate a different measure: Use an incremental append pattern with suitable deduplication when source events cannot change. That is relevant to the distinct problem: An append-only telemetry feed includes newly observed events but engineers use MERGE with expensive full-table scans on every insert. Here the acceptance criterion is: The unpivoted output contains exactly one row per original item-month and preserves totals.
B:This result would validate a different measure: Validate business keys and choose a deterministic latest-enrollment record before aggregation. That is relevant to the distinct problem: A student dataset uses SQL GROUP BY with unhandled duplicate enrollment rows before reporting course completion. Here the acceptance criterion is: The unpivoted output contains exactly one row per original item-month and preserves totals.
D:This result would validate a different measure: Implement explicit null predicates rather than treating zero and null as equivalent. That is relevant to the distinct problem: A team filters out null quantities but needs to preserve rows with zero as valid inventory states. Here the acceptance criterion is: The unpivoted output contains exactly one row per original item-month and preserves totals.
Question 290
A data engineering team faces this Azure Databricks requirement: A regulated pipeline needs consistent transaction boundaries across multiple table writes but updates are published one at a time. Which action is MOST appropriate?
- Design a compatible atomic publication or versioned readiness contract so consumers see consistent results.
- Evaluate Lakeflow Spark Declarative Pipelines for the compatible streaming and transformation graph.
- Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed.
- Separate tasks and use the supported repair-run workflow at the failed stage when dependencies remain valid.
Explanation
Correct Answer
A:The most appropriate action is: Design a compatible atomic publication or versioned readiness contract so consumers see consistent results. It addresses the stated condition: A regulated pipeline needs consistent transaction boundaries across multiple table writes but updates are published one at a time. The technical basis is: Multi-table orchestration requires a defined consistency point even if each individual Delta write is atomic. Without this control, readers can combine incompatible versions when related tables update independently. A direct acceptance check is: A controlled consumer test sees only one approved publication version across dependent datasets.
Incorrect Answers
B:This alternative primarily addresses a different Azure Databricks problem: A continuous feed creates and refreshes tables with consistent declarative dependencies but developers manually chain notebooks. The corresponding measure there would be: Evaluate Lakeflow Spark Declarative Pipelines for the compatible streaming and transformation graph. The stated situation instead calls for: Design a compatible atomic publication or versioned readiness contract so consumers see consistent results.
C:This alternative primarily addresses a different Azure Databricks problem: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. The corresponding measure there would be: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. The stated situation instead calls for: Design a compatible atomic publication or versioned readiness contract so consumers see consistent results.
D:This alternative primarily addresses a different Azure Databricks problem: A bulk repair executes all pipeline tasks again even when only the final reporting notebook failed. The corresponding measure there would be: Separate tasks and use the supported repair-run workflow at the failed stage when dependencies remain valid. The stated situation instead calls for: Design a compatible atomic publication or versioned readiness contract so consumers see consistent results.