Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 90
Question 890
A data engineering team faces this Azure Databricks requirement: A Spark workload repeatedly fails with executor out-of-memory errors on a high-cardinality groupBy. Which action is MOST appropriate?
- Inspect the Spark UI DAG and skewed-key distribution before changing cluster size.
- Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages.
- Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change.
- Use run-level compute utilization, cost and stage metrics to identify unused capacity.
Explanation
Correct Answer
C:The most appropriate action is: Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change. It addresses the stated condition: A Spark workload repeatedly fails with executor out-of-memory errors on a high-cardinality groupBy. The technical basis is: Wide aggregations redistribute data and can create memory-intensive hot partitions. Without this control, blindly restarting the cluster may reproduce the same failure at the same stage. A direct acceptance check is: A controlled representative load finishes without OOM and output group counts reconcile.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A production Spark query is slow and its stage graph shows one task running far longer than all peers. The corresponding measure there would be: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size. The stated situation instead calls for: Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change.
B:This alternative primarily addresses a different Azure Databricks problem: An interactive warehouse has high query queues during the morning but low CPU utilization after lunch. The corresponding measure there would be: Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages. The stated situation instead calls for: Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change.
D:This alternative primarily addresses a different Azure Databricks problem: A pipeline experiences escalating costs after a team increases node counts but execution duration barely changes. The corresponding measure there would be: Use run-level compute utilization, cost and stage metrics to identify unused capacity. The stated situation instead calls for: Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change.
Question 891
The following control has NOT been implemented: Use Delta Sharing with an authorized recipient, narrowly scoped shares and lifecycle management. In this case: A data-sharing project emails storage keys to an external partner for reading curated Delta results. Which exposure remains?
- Unauthorized grants can remain undetected or be impossible to trace.
- Without lineage the team may repair the wrong pipeline after a metric regression.
- Accumulated historical grants may give former project teams access to unrelated data products.
- Long-lived shared storage credentials may expose unrelated files or survive contract termination.
Explanation
Correct Answer
D:The direct residual exposure is: Long-lived shared storage credentials may expose unrelated files or survive contract termination. The triggering condition is: A data-sharing project emails storage keys to an external partner for reading curated Delta results. The omitted safeguard is: Use Delta Sharing with an authorized recipient, narrowly scoped shares and lifecycle management. This matters because: Delta Sharing can provide governed data access without distributing underlying storage account keys. A corrective test would show: The partner reads only the approved shared table and fails to access a nonshared table.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A compliance auditor requests a record of who changed Unity Catalog grants but audit logs are not collected. Its preventative action would be: Configure supported Databricks audit logging and route relevant events to a secured retention destination. The most direct exposure in the current scenario is: Long-lived shared storage credentials may expose unrelated files or survive contract termination.
B:This failure mode belongs mainly to another situation: An analyst wants to know which upstream pipeline produced a metric displayed in a board dashboard. Its preventative action would be: Inspect Unity Catalog data lineage and table history in Catalog Explorer, checking actual supported capture coverage. The most direct exposure in the current scenario is: Long-lived shared storage credentials may expose unrelated files or survive contract termination.
C:This failure mode belongs mainly to another situation: A company needs to review table-owner changes and permissions after organizational restructuring. Its preventative action would be: Run a scheduled ownership and grant recertification process using catalog metadata and audit events. The most direct exposure in the current scenario is: Long-lived shared storage credentials may expose unrelated files or survive contract termination.
Question 892
An implementation skips: Enable supported liquid clustering with appropriate keys and run compatible optimization. Engineers are observing: A large Delta fact table is filtered primarily on high-cardinality account IDs with changing query patterns. What consequence is MOST likely?
- Assuming indefinite history can fail once physical file retention expires.
- Mixing incompatible layout mechanisms can fail operations or leave the team relying on nonexistent optimizations.
- Rigid directory partitioning can cause imbalance and require costly redesign as filters change.
- Excessive partitions create small files and poor query planning without useful pruning.
Explanation
Correct Answer
C:The direct residual exposure is: Rigid directory partitioning can cause imbalance and require costly redesign as filters change. The triggering condition is: A large Delta fact table is filtered primarily on high-cardinality account IDs with changing query patterns. The omitted safeguard is: Enable supported liquid clustering with appropriate keys and run compatible optimization. This matters because: Liquid clustering can organize files for selective filters without static partition boundaries. A corrective test would show: A representative account-filter query reads fewer irrelevant files after optimized clustering.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A business needs to compare two point-in-time table versions during a financial reconciliation. Its preventative action would be: Use supported Delta table version history and time travel within the documented retention window. The most direct exposure in the current scenario is: Rigid directory partitioning can cause imbalance and require costly redesign as filters change.
B:This failure mode belongs mainly to another situation: A team attempts to apply both ZORDER and liquid clustering to the same Delta table. Its preventative action would be: Choose one compatible data layout strategy based on table format, platform support and query evidence. The most direct exposure in the current scenario is: Rigid directory partitioning can cause imbalance and require costly redesign as filters change.
D:This failure mode belongs mainly to another situation: An event table is partitioned by the unique transaction identifier and creates millions of tiny partitions. Its preventative action would be: Use coarse suitable partitioning or liquid clustering selected from filter patterns and cardinality. The most direct exposure in the current scenario is: Rigid directory partitioning can cause imbalance and require costly redesign as filters change.
Question 893
A Databricks engineer recommends: Design a compatible atomic publication or versioned readiness contract so consumers see consistent results. It responds to: A regulated pipeline needs consistent transaction boundaries across multiple table writes but updates are published one at a time. What underlying mechanism BEST explains why?
- Job status must accurately reflect failed required operations for alerting and dependency logic.
- Separating pipeline stages creates observable failure boundaries and supports targeted recovery.
- Multi-table orchestration requires a defined consistency point even if each individual Delta write is atomic.
- Quality gates must precede trusted publication, not merely log errors afterwards.
Explanation
Correct Answer
C:The decisive mechanism is: Multi-table orchestration requires a defined consistency point even if each individual Delta write is atomic. That explains the selected action: Design a compatible atomic publication or versioned readiness contract so consumers see consistent results. In the original scenario: A regulated pipeline needs consistent transaction boundaries across multiple table writes but updates are published one at a time. The specific unmanaged risk is: Readers can combine incompatible versions when related tables update independently. Validate with: A controlled consumer test sees only one approved publication version across dependent datasets.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A workflow uses a notebook task that catches exceptions and returns success even when a write step fails. Its targeted response is: Propagate nonrecoverable exceptions to Lakeflow Jobs and attach actionable error metadata. The current case is governed more directly by: Multi-table orchestration requires a defined consistency point even if each individual Delta write is atomic.
B:This principle concerns a separate data-engineering condition: A multi-step notebook workflow handles source extraction and final reporting in one cell sequence with no recoverable checkpoints. Its targeted response is: Split ingestion, validation and publishing into explicit orchestrated tasks with durable intermediate outputs. The current case is governed more directly by: Multi-table orchestration requires a defined consistency point even if each individual Delta write is atomic.
D:This principle concerns a separate data-engineering condition: A workflow writes to production tables before a validation task examines null keys and duplicates. Its targeted response is: Place validations before publishing and use an atomic cutover or controlled target write. The current case is governed more directly by: Multi-table orchestration requires a defined consistency point even if each individual Delta write is atomic.
Question 894
The team chooses: Use drift detection or a release reconciliation that compares deployed objects against the versioned bundle. Its observed issue is: A Databricks bundle deployment differs from what appears in the repository because someone edits the job in the UI afterward. What is the strongest technical basis?
- Schema and object naming changes can break consumers even when new ETL succeeds.
- Correctness under small inputs does not establish workload performance at production cardinality.
- Infrastructure automation must tolerate retries without multiplying the intended target resources.
- Source control only provides operational truth when actual resource settings match the approved definition.
Explanation
Correct Answer
D:The decisive mechanism is: Source control only provides operational truth when actual resource settings match the approved definition. That explains the selected action: Use drift detection or a release reconciliation that compares deployed objects against the versioned bundle. In the original scenario: A Databricks bundle deployment differs from what appears in the repository because someone edits the job in the UI afterward. The specific unmanaged risk is: Manual post-deployment edits can bypass review and silently change retry or security settings. Validate with: A test UI change is flagged and the approved deployment restores the intended configuration.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A sensitive SQL view is renamed in the release but the team does not test downstream dashboards before promotion. Its targeted response is: Run end-to-end contract tests for dependent readers and obtain owner approval for breaking changes. The current case is governed more directly by: Source control only provides operational truth when actual resource settings match the approved definition.
B:This principle concerns a separate data-engineering condition: An engineering test suite passes on miniature in-memory data yet fails in production on skewed billion-row joins. Its targeted response is: Include performance and end-to-end tests using representative scale and data distributions. The current case is governed more directly by: Source control only provides operational truth when actual resource settings match the approved definition.
C:This principle concerns a separate data-engineering condition: A REST deployment script creates duplicate jobs every time CI reruns after a timeout. Its targeted response is: Use idempotent deployment methods or reconcile existing resource IDs before creating replacements. The current case is governed more directly by: Source control only provides operational truth when actual resource settings match the approved definition.
Question 895
A solution applies: Route deterministic validation failures to quarantine rather than retrying them as transient infrastructure errors. It addresses: A pipeline automatically retries malformed source records indefinitely and consumes its entire compute budget. Which operational outcome MOST directly verifies success?
- A malformed input is rejected once with an error record while later valid records continue.
- A reviewed graph lists every expected parent and a source change propagates to the correct targets.
- A staged failure triggers the documented alert and cleanup while skipping dependent publication.
- A pipeline run updates tables in valid dependency order and records expectation outcomes.
Explanation
Correct Answer
A:The strongest acceptance proof is: A malformed input is rejected once with an error record while later valid records continue. It specifically verifies: Route deterministic validation failures to quarantine rather than retrying them as transient infrastructure errors. The original issue was: A pipeline automatically retries malformed source records indefinitely and consumes its entire compute budget. The underlying mechanism is: Idempotent retries are suitable for transient conditions, not unchanging invalid data. Without it, infinite retries on bad records can block valid work and multiply costs.
Incorrect Answers
B:This result would validate a different measure: Declare consistent table names and dependencies and validate the generated pipeline graph. That is relevant to the distinct problem: A declarative pipeline transforms several datasets but its source names are ambiguous and a data team cannot reconstruct dependencies. Here the acceptance criterion is: A malformed input is rejected once with an error record while later valid records continue.
C:This result would validate a different measure: Design a separate failure-handling or completion branch with the supported run-if logic for alerts and cleanup. That is relevant to the distinct problem: A source task fails yet the monitoring task never executes because all downstream tasks require upstream success. Here the acceptance criterion is: A malformed input is rejected once with an error record while later valid records continue.
D:This result would validate a different measure: Evaluate Lakeflow Spark Declarative Pipelines for the compatible streaming and transformation graph. That is relevant to the distinct problem: A continuous feed creates and refreshes tables with consistent declarative dependencies but developers manually chain notebooks. Here the acceptance criterion is: A malformed input is rejected once with an error record while later valid records continue.
Question 896
The design team proposes: Configure bounded autoscaling using measured workload demand and verify maximum cost and startup behavior. The situation is: A data engineering job sometimes requires twice its usual executor capacity during month-end loads while weekdays are much smaller. Which technical principle justifies the decision?
- Pool capacity can shorten classic cluster startup but reserved idle capacity must still be budgeted.
- SQL warehouses are optimized for SQL analytics and isolate query serving from other compute activities.
- Concurrency queuing is separate from a single-query execution performance problem.
- Autoscaling can adapt worker resources to changing parallel workload demand within a defined limit.
Explanation
Correct Answer
D:The decisive mechanism is: Autoscaling can adapt worker resources to changing parallel workload demand within a defined limit. That explains the selected action: Configure bounded autoscaling using measured workload demand and verify maximum cost and startup behavior. In the original scenario: A data engineering job sometimes requires twice its usual executor capacity during month-end loads while weekdays are much smaller. The specific unmanaged risk is: A fixed undersized cluster misses month-end deadlines while an oversized fixed cluster wastes daily spend. Validate with: Load tests for normal and month-end volumes satisfy runtime targets without exceeding the approved worker ceiling.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A scheduled workflow launches dozens of short-lived classic clusters and loses several minutes each time waiting for virtual machines. Its targeted response is: Evaluate compatible instance pools or supported serverless startup behavior rather than keeping unused all-purpose clusters awake. The current case is governed more directly by: Autoscaling can adapt worker resources to changing parallel workload demand within a defined limit.
B:This principle concerns a separate data-engineering condition: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. Its targeted response is: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. The current case is governed more directly by: Autoscaling can adapt worker resources to changing parallel workload demand within a defined limit.
C:This principle concerns a separate data-engineering condition: A BI warehouse repeatedly slows during morning sign-in peaks even though a single analyst query runs quickly. Its targeted response is: Tune supported SQL warehouse capacity and concurrency scaling based on simultaneous production query measurements. The current case is governed more directly by: Autoscaling can adapt worker resources to changing parallel workload demand within a defined limit.
Question 897
The team chooses: Use supported streaming or sufficiently frequent incremental CDC with measured end-to-end latency. Its observed issue is: A team selects batch processing for customer account changes though downstream applications require minute-scale freshness. What is the strongest technical basis?
- Text ingestion options are part of the data contract and affect parsing correctness.
- CDC propagates source mutations without duplicating full snapshots.
- Freshness requirements should determine batch versus streaming design.
- Input extraction semantics determine whether repeated rows are new events or refreshed state.
Explanation
Correct Answer
C:The decisive mechanism is: Freshness requirements should determine batch versus streaming design. That explains the selected action: Use supported streaming or sufficiently frequent incremental CDC with measured end-to-end latency. In the original scenario: A team selects batch processing for customer account changes though downstream applications require minute-scale freshness. The specific unmanaged risk is: Daily batches cannot satisfy short update deadlines even when their processing is correct. Validate with: A changed customer field reaches the approved target within the measured freshness SLO.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: An overnight load reads tab-separated files but an engineer assumes comma separators and casts malformed columns to null. Its targeted response is: Specify the correct input delimiter, header and schema and validate parsing on a representative file. The current case is governed more directly by: Freshness requirements should determine batch versus streaming design.
B:This principle concerns a separate data-engineering condition: A SaaS database updates and deletes customer records but a pipeline appends every nightly full extract to a Delta table. Its targeted response is: Use a supported change-data-capture feed and apply insert, update and delete semantics to the target. The current case is governed more directly by: Freshness requirements should determine batch versus streaming design.
D:This principle concerns a separate data-engineering condition: A daily file load uses append-only mode for periodic snapshots of the same full inventory. Its targeted response is: Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records. The current case is governed more directly by: Freshness requirements should determine batch versus streaming design.
Question 898
For this Azure Databricks issue: A reporting team stores frequent updates in raw CSV files and expects atomic MERGE and historical time travel. Engineers choose: Use a managed Delta table for transactional updates and controlled schema evolution. What acceptance evidence is BEST?
- A representative account-filter query reads fewer irrelevant files after optimized clustering.
- A controlled historical version query returns the expected prior balance before maintenance removes eligible old files.
- A MERGE transaction updates the expected records atomically and a historical version remains queryable within retention.
- The platform team retains approved storage lifecycle authority while Databricks queries the registered data.
Explanation
Correct Answer
C:The strongest acceptance proof is: A MERGE transaction updates the expected records atomically and a historical version remains queryable within retention. It specifically verifies: Use a managed Delta table for transactional updates and controlled schema evolution. The original issue was: A reporting team stores frequent updates in raw CSV files and expects atomic MERGE and historical time travel. The underlying mechanism is: Delta tables support transactional mutations and versioned metadata beyond raw delimiter files. Without it, directly rewriting CSV in place can expose partial updates and inconsistent reader results.
Incorrect Answers
A:This result would validate a different measure: Enable supported liquid clustering with appropriate keys and run compatible optimization. That is relevant to the distinct problem: A large Delta fact table is filtered primarily on high-cardinality account IDs with changing query patterns. Here the acceptance criterion is: A MERGE transaction updates the expected records atomically and a historical version remains queryable within retention.
B:This result would validate a different measure: Use supported Delta table version history and time travel within the documented retention window. That is relevant to the distinct problem: A business needs to compare two point-in-time table versions during a financial reconciliation. Here the acceptance criterion is: A MERGE transaction updates the expected records atomically and a historical version remains queryable within retention.
D:This result would validate a different measure: Use an external Unity Catalog table with an approved registered location instead of assuming managed storage ownership. That is relevant to the distinct problem: A dataset has a strict requirement that Azure storage ownership and lifecycle remain under a separate platform team. Here the acceptance criterion is: A MERGE transaction updates the expected records atomically and a historical version remains queryable within retention.
Question 899
A production data platform experiences this condition: A batch enrichment task downloads a remote file and sometimes receives transient HTTP 503 responses. Which targeted measure should be selected?
- Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed.
- Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages.
- Add bounded retries with exponential backoff and observable failure after the retry budget.
- Design a compatible atomic publication or versioned readiness contract so consumers see consistent results.
Explanation
Correct Answer
C:The most appropriate action is: Add bounded retries with exponential backoff and observable failure after the retry budget. It addresses the stated condition: A batch enrichment task downloads a remote file and sometimes receives transient HTTP 503 responses. The technical basis is: Retryable transient failures should be distinguished from permanent data or permission errors. Without this control, immediate permanent failure on a brief service outage can miss a delivery SLA. A direct acceptance check is: An injected transient 503 recovers within the retry budget while repeated failures produce actionable alerts.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. The corresponding measure there would be: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. The stated situation instead calls for: Add bounded retries with exponential backoff and observable failure after the retry budget.
B:This alternative primarily addresses a different Azure Databricks problem: A scheduled notebook contains both transformation SQL and a long-running external API loop that occasionally hangs. The corresponding measure there would be: Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages. The stated situation instead calls for: Add bounded retries with exponential backoff and observable failure after the retry budget.
D:This alternative primarily addresses a different Azure Databricks problem: A regulated pipeline needs consistent transaction boundaries across multiple table writes but updates are published one at a time. The corresponding measure there would be: Design a compatible atomic publication or versioned readiness contract so consumers see consistent results. The stated situation instead calls for: Add bounded retries with exponential backoff and observable failure after the retry budget.
Question 900
A solution applies: Review current serverless limitations and select compatible classic jobs compute when necessary. It addresses: A long-running job using an unsupported custom system dependency is moved to serverless without checking feature compatibility. Which operational outcome MOST directly verifies success?
- A compatibility matrix and staging execution demonstrate every necessary dependency on the chosen compute.
- A load rehearsal meets dashboard p95 response targets while notebook resource spikes do not affect query service.
- A controlled instance-family comparison achieves the same results with improved stage time and total execution cost.
- Spark UI and driver metrics show bounded driver memory after correcting the collection pattern.
Explanation
Correct Answer
A:The strongest acceptance proof is: A compatibility matrix and staging execution demonstrate every necessary dependency on the chosen compute. It specifically verifies: Review current serverless limitations and select compatible classic jobs compute when necessary. The original issue was: A long-running job using an unsupported custom system dependency is moved to serverless without checking feature compatibility. The underlying mechanism is: Serverless and classic compute have different configuration surfaces and supported dependency patterns. Without it, A serverless migration may fail when a required library or environment customization cannot be installed.
Incorrect Answers
B:This result would validate a different measure: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. That is relevant to the distinct problem: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. Here the acceptance criterion is: A compatibility matrix and staging execution demonstrate every necessary dependency on the chosen compute.
C:This result would validate a different measure: Choose node families and driver/worker sizes using CPU, memory, shuffle and workload profile evidence. That is relevant to the distinct problem: A new job uses a large memory-optimized node merely because its source data is large, although profiling shows compute-bound transforms. Here the acceptance criterion is: A compatibility matrix and staging execution demonstrate every necessary dependency on the chosen compute.
D:This result would validate a different measure: Inspect driver collection and query planning behavior before simply enlarging every worker. That is relevant to the distinct problem: A SQL transformation consumes large amounts of memory on the driver despite executor tasks processing most source partitions. Here the acceptance criterion is: A compatibility matrix and staging execution demonstrate every necessary dependency on the chosen compute.