Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 64
Question 630
An Azure Databricks architect reviews the following problem: An engineering test suite passes on miniature in-memory data yet fails in production on skewed billion-row joins. What should the team implement FIRST?
- Use drift detection or a release reconciliation that compares deployed objects against the versioned bundle.
- Store deployment source in Git with reviewed branches and pull requests before promoting changes.
- Run end-to-end contract tests for dependent readers and obtain owner approval for breaking changes.
- Include performance and end-to-end tests using representative scale and data distributions.
Explanation
Correct Answer
D:The most appropriate action is: Include performance and end-to-end tests using representative scale and data distributions. It addresses the stated condition: An engineering test suite passes on miniature in-memory data yet fails in production on skewed billion-row joins. The technical basis is: Correctness under small inputs does not establish workload performance at production cardinality. Without this control, skew and memory limits may remain invisible until a full-scale run misses its SLA. A direct acceptance check is: A load test includes known hot keys and verifies latency, spills and output totals within targets.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A Databricks bundle deployment differs from what appears in the repository because someone edits the job in the UI afterward. The corresponding measure there would be: Use drift detection or a release reconciliation that compares deployed objects against the versioned bundle. The stated situation instead calls for: Include performance and end-to-end tests using representative scale and data distributions.
B:This alternative primarily addresses a different Azure Databricks problem: Engineers edit production notebooks directly and lose the ability to identify which code change broke a pipeline. The corresponding measure there would be: Store deployment source in Git with reviewed branches and pull requests before promoting changes. The stated situation instead calls for: Include performance and end-to-end tests using representative scale and data distributions.
C:This alternative primarily addresses a different Azure Databricks problem: A sensitive SQL view is renamed in the release but the team does not test downstream dashboards before promotion. The corresponding measure there would be: Run end-to-end contract tests for dependent readers and obtain owner approval for breaking changes. The stated situation instead calls for: Include performance and end-to-end tests using representative scale and data distributions.
Question 631
A production data team leaves out: Collect Spark stage profiles and test a targeted compute or Photon setting against comparable data. Its scenario is: A team turns on every performance setting for a workload without benchmarking whether disk spills or network shuffle dominate runtime. Which failure mode is MOST relevant?
- Unmeasured tuning changes can hide the original bottleneck or increase compute costs.
- Assuming every transformation accelerates equally may raise costs without reducing job duration.
- Leaving an interactive cluster running around the clock accumulates charges unrelated to completed pipeline work.
- Overspending on memory does not relieve a CPU-bound stage and may worsen the cost-performance ratio.
Explanation
Correct Answer
A:The direct residual exposure is: Unmeasured tuning changes can hide the original bottleneck or increase compute costs. The triggering condition is: A team turns on every performance setting for a workload without benchmarking whether disk spills or network shuffle dominate runtime. The omitted safeguard is: Collect Spark stage profiles and test a targeted compute or Photon setting against comparable data. This matters because: Performance settings should follow measured bottlenecks rather than an assumption that each toggle is beneficial. A corrective test would show: Before-after measurements report comparable inputs, stage timings, spills and total cost.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A Spark SQL pipeline spends most of its time executing wide joins and aggregations but the team has not evaluated a compatible vectorized execution engine. Its preventative action would be: Benchmark Photon on a supported runtime with representative queries and compare cost, latency and correctness. The most direct exposure in the current scenario is: Unmeasured tuning changes can hide the original bottleneck or increase compute costs.
C:This failure mode belongs mainly to another situation: A nightly batch transformation uses a continuously running all-purpose cluster even though no analyst interacts with the workload between runs. Its preventative action would be: Move the scheduled transformation to appropriately configured jobs compute or supported serverless jobs and release resources after execution. The most direct exposure in the current scenario is: Unmeasured tuning changes can hide the original bottleneck or increase compute costs.
D:This failure mode belongs mainly to another situation: A new job uses a large memory-optimized node merely because its source data is large, although profiling shows compute-bound transforms. Its preventative action would be: Choose node families and driver/worker sizes using CPU, memory, shuffle and workload profile evidence. The most direct exposure in the current scenario is: Unmeasured tuning changes can hide the original bottleneck or increase compute costs.
Question 632
An operator introduces: Set appropriate auto-termination or idle controls for interactive compute while excluding workloads requiring continuous availability. The situation is: A newly created cluster remains available for hours after manual exploration even though the organization has a strict idle-cost policy. Which validation should be required before signoff?
- Resource telemetry shows accelerator use where needed and CPU jobs satisfy their SLA at lower cost.
- A load rehearsal meets dashboard p95 response targets while notebook resource spikes do not affect query service.
- Spark UI and driver metrics show bounded driver memory after correcting the collection pattern.
- An inactive test cluster terminates after the configured interval and users can restart it when authorized.
Explanation
Correct Answer
D:The strongest acceptance proof is: An inactive test cluster terminates after the configured interval and users can restart it when authorized. It specifically verifies: Set appropriate auto-termination or idle controls for interactive compute while excluding workloads requiring continuous availability. The original issue was: A newly created cluster remains available for hours after manual exploration even though the organization has a strict idle-cost policy. The underlying mechanism is: Interactive compute should have a governed idle lifecycle distinct from persistent services. Without it, forgotten development clusters can consume cloud resources long after the last useful notebook cell.
Incorrect Answers
A:This result would validate a different measure: Assign GPU-capable compute only to supported GPU workloads and use CPU compute for ordinary transformations. That is relevant to the distinct problem: A feature-engineering notebook requires GPU libraries but the data cleaning jobs never use accelerators. Here the acceptance criterion is: An inactive test cluster terminates after the configured interval and users can restart it when authorized.
B:This result would validate a different measure: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. That is relevant to the distinct problem: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. Here the acceptance criterion is: An inactive test cluster terminates after the configured interval and users can restart it when authorized.
C:This result would validate a different measure: Inspect driver collection and query planning behavior before simply enlarging every worker. That is relevant to the distinct problem: A SQL transformation consumes large amounts of memory on the driver despite executor tasks processing most source partitions. Here the acceptance criterion is: An inactive test cluster terminates after the configured interval and users can restart it when authorized.
Question 633
The team implements: Define a consistent schema and naming architecture separating layers and business domains by purpose. The original problem is: Multiple data products put raw, refined and curated assets into a single schema with confusing table names. Which test MOST convincingly demonstrates correctness?
- Automated DDL completes in both staging and production under the same naming rules.
- A set of known business prompts yields reconciled SQL and approved metric definitions.
- A controlled MERGE and SQL query succeed against the governed Delta table with expected records.
- Discovery tests distinguish raw from certified curated objects and owners confirm their assigned namespaces.
Explanation
Correct Answer
D:The strongest acceptance proof is: Discovery tests distinguish raw from certified curated objects and owners confirm their assigned namespaces. It specifically verifies: Define a consistent schema and naming architecture separating layers and business domains by purpose. The original issue was: Multiple data products put raw, refined and curated assets into a single schema with confusing table names. The underlying mechanism is: Catalog naming and grouping should communicate lifecycle, owner and trust level for consumers. Without it, users may query raw unvalidated records as though they were approved curated metrics.
Incorrect Answers
A:This result would validate a different measure: Adopt a portable naming and quoting convention compatible with Unity Catalog SQL identifiers. That is relevant to the distinct problem: An ingestion team creates a table with a reserved or inconsistent name and automation fails in one environment. Here the acceptance criterion is: Discovery tests distinguish raw from certified curated objects and owners confirm their assigned namespaces.
B:This result would validate a different measure: Add reviewed Genie instructions, table descriptions and example questions consistent with trusted catalog semantics. That is relevant to the distinct problem: An AI/BI Genie space is configured with ambiguous metric terminology and produces misleading natural-language interpretations. Here the acceptance criterion is: Discovery tests distinguish raw from certified curated objects and owners confirm their assigned namespaces.
C:This result would validate a different measure: Register the structured dataset as an appropriate managed Delta table rather than treating the volume itself as a SQL table. That is relevant to the distinct problem: A team exposes structured transaction data as files in a volume but wants SQL analytics and ACID MERGE behavior. Here the acceptance criterion is: Discovery tests distinguish raw from certified curated objects and owners confirm their assigned namespaces.
Question 634
An implementation skips: Propagate nonrecoverable exceptions to Lakeflow Jobs and attach actionable error metadata. Engineers are observing: A workflow uses a notebook task that catches exceptions and returns success even when a write step fails. What consequence is MOST likely?
- A swallowed exception can allow downstream tasks to publish stale or missing outputs.
- Forcing arbitrary procedural logic into a declarative table pipeline may complicate testing and maintenance.
- A downstream report can close before late but permitted events are incorporated.
- Hand-built ordering can drift from the actual dependency graph as tables are added.
Explanation
Correct Answer
A:The direct residual exposure is: A swallowed exception can allow downstream tasks to publish stale or missing outputs. The triggering condition is: A workflow uses a notebook task that catches exceptions and returns success even when a write step fails. The omitted safeguard is: Propagate nonrecoverable exceptions to Lakeflow Jobs and attach actionable error metadata. This matters because: Job status must accurately reflect failed required operations for alerting and dependency logic. A corrective test would show: A deliberately failed write marks the job unsuccessful and prevents dependent publish tasks.
Incorrect Answers
B:This failure mode belongs mainly to another situation: An ad hoc machine-learning preparation task has complex custom Python orchestration and no declarative table graph. Its preventative action would be: Select a notebook task within Lakeflow Jobs when procedural control and custom libraries fit the requirements. The most direct exposure in the current scenario is: A swallowed exception can allow downstream tasks to publish stale or missing outputs.
C:This failure mode belongs mainly to another situation: A streaming process depends on a downstream batch closeout but no one defines whether it should wait for the latest watermark. Its preventative action would be: Specify the stream watermark and an explicit closeout readiness condition before running downstream aggregation. The most direct exposure in the current scenario is: A swallowed exception can allow downstream tasks to publish stale or missing outputs.
D:This failure mode belongs mainly to another situation: A continuous feed creates and refreshes tables with consistent declarative dependencies but developers manually chain notebooks. Its preventative action would be: Evaluate Lakeflow Spark Declarative Pipelines for the compatible streaming and transformation graph. The most direct exposure in the current scenario is: A swallowed exception can allow downstream tasks to publish stale or missing outputs.
Question 635
The following control has NOT been implemented: Align table retention properties, scheduled maintenance and legal ownership approvals with the current classification. In this case: A regulated source dataset receives an updated retention period but storage cleanup still follows the obsolete policy. Which exposure remains?
- Deleting a field without documented change history can leave unanswerable regulatory questions.
- Uncaptured external ingestion may lead to incomplete impact analysis and unsafe upstream changes.
- Consumers can use incorrectly labeled confidential fields without recognizing their sensitivity.
- Old cleanup tasks can destroy files that the revised retention requirement still protects.
Explanation
Correct Answer
D:The direct residual exposure is: Old cleanup tasks can destroy files that the revised retention requirement still protects. The triggering condition is: A regulated source dataset receives an updated retention period but storage cleanup still follows the obsolete policy. The omitted safeguard is: Align table retention properties, scheduled maintenance and legal ownership approvals with the current classification. This matters because: Technical retention settings and organizational policy need an auditable, consistent lifecycle. A corrective test would show: A simulated cleanup retains protected history and an audit maps the new duration to approved settings.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A shared table includes an old regulated field and the business needs reproducible evidence of when it was removed. Its preventative action would be: Use appropriate table history, schema change records and approved retention/audit evidence rather than relying on memory. The most direct exposure in the current scenario is: Old cleanup tasks can destroy files that the revised retention requirement still protects.
B:This failure mode belongs mainly to another situation: A lineage graph shows no relationship for data loaded outside the monitored platform and the team assumes the dataset has no upstream dependency. Its preventative action would be: Validate Unity Catalog lineage capture coverage and supplement unsupported external steps with documented provenance. The most direct exposure in the current scenario is: Old cleanup tasks can destroy files that the revised retention requirement still protects.
C:This failure mode belongs mainly to another situation: A central repository contains schemas with missing column owners and no sensitivity classifications. Its preventative action would be: Implement a catalog stewardship process to review metadata and governed tags at dataset publication. The most direct exposure in the current scenario is: Old cleanup tasks can destroy files that the revised retention requirement still protects.
Question 636
For this Azure Databricks issue: A notebook pipeline uses a shared cluster with unstable third-party dependencies while a critical finance job needs controlled isolation. Engineers choose: Run the finance job on dedicated supported job compute with its own pinned libraries and permissions. What acceptance evidence is BEST?
- A quiet-period rehearsal retains the required streaming operation and subsequent events process without missed offsets.
- Load tests for normal and month-end volumes satisfy runtime targets without exceeding the approved worker ceiling.
- A load rehearsal meets dashboard p95 response targets while notebook resource spikes do not affect query service.
- A test upgrade on the interactive cluster does not change the isolated finance job's dependency versions or results.
Explanation
Correct Answer
D:The strongest acceptance proof is: A test upgrade on the interactive cluster does not change the isolated finance job's dependency versions or results. It specifically verifies: Run the finance job on dedicated supported job compute with its own pinned libraries and permissions. The original issue was: A notebook pipeline uses a shared cluster with unstable third-party dependencies while a critical finance job needs controlled isolation. The underlying mechanism is: Compute isolation reduces interference between unrelated application environments and execution libraries. Without it, an upgrade or runaway task on the shared cluster can compromise finance pipeline availability.
Incorrect Answers
A:This result would validate a different measure: Select stream-appropriate compute and lifecycle settings so monitoring and availability targets are met. That is relevant to the distinct problem: A long-running streaming pipeline is configured with aggressive auto-termination during brief periods without arriving events. Here the acceptance criterion is: A test upgrade on the interactive cluster does not change the isolated finance job's dependency versions or results.
B:This result would validate a different measure: Configure bounded autoscaling using measured workload demand and verify maximum cost and startup behavior. That is relevant to the distinct problem: A data engineering job sometimes requires twice its usual executor capacity during month-end loads while weekdays are much smaller. Here the acceptance criterion is: A test upgrade on the interactive cluster does not change the isolated finance job's dependency versions or results.
C:This result would validate a different measure: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. That is relevant to the distinct problem: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. Here the acceptance criterion is: A test upgrade on the interactive cluster does not change the isolated finance job's dependency versions or results.
Question 637
A platform architect adopts: Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics. For this requirement: A team expects dropping a managed table to preserve all physical data as though it were an external table. Which rationale MOST directly supports it?
- Catalog and schema hierarchy can isolate environments while making object ownership and purpose explicit.
- Lakehouse Federation lets governed queries access supported external systems through a connection abstraction.
- Managed versus external tables differ in who manages data paths and deletion behavior.
- Reading a governed object does not imply permission to create or administer peer objects.
Explanation
Correct Answer
C:The decisive mechanism is: Managed versus external tables differ in who manages data paths and deletion behavior. That explains the selected action: Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics. In the original scenario: A team expects dropping a managed table to preserve all physical data as though it were an external table. The specific unmanaged risk is: A drop operation may remove managed table data that administrators expected to retain. Validate with: A controlled create-drop rehearsal confirms the expected metadata and physical data lifecycle for the table type.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: Production and development teams create identically named data objects in one catalog and accidentally overwrite test tables. Its targeted response is: Create environment-specific Unity Catalog catalogs or schemas and enforce clear ownership and naming conventions. The current case is governed more directly by: Managed versus external tables differ in who manages data paths and deletion behavior.
B:This principle concerns a separate data-engineering condition: A workspace needs to query a foreign transactional source without first loading every row into a Delta table. Its targeted response is: Create an authorized connection and supported foreign catalog for the external source. The current case is governed more directly by: Managed versus external tables differ in who manages data paths and deletion behavior.
D:This principle concerns a separate data-engineering condition: A team expects every analyst to CREATE SCHEMA simply because they can SELECT tables in an existing catalog. Its targeted response is: Grant precise container-use and object-creation privileges based on actual job roles. The current case is governed more directly by: Managed versus external tables differ in who manages data paths and deletion behavior.
Question 638
A Databricks engineer recommends: Deduplicate by stable event identity and business time using documented state and watermark rules. It responds to: An ingestion pipeline reads cloud events with at-least-once delivery and assumes every event ID appears only once. What underlying mechanism BEST explains why?
- Atomic table replacement can avoid exposing a partially loaded intermediate table.
- Managed ingestion connectors can simplify source synchronization when their coverage fits the source.
- CDC correctness depends on source ordering and idempotent update rules.
- Delivery guarantees alone do not remove duplicate producer or consumer events.
Explanation
Correct Answer
D:The decisive mechanism is: Delivery guarantees alone do not remove duplicate producer or consumer events. That explains the selected action: Deduplicate by stable event identity and business time using documented state and watermark rules. In the original scenario: An ingestion pipeline reads cloud events with at-least-once delivery and assumes every event ID appears only once. The specific unmanaged risk is: Replayed events can double-count orders in downstream aggregates. Validate with: A duplicate-event test yields one current business event while preserving valid distinct events.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: An existing table must be rebuilt from corrected historical files while its readers require a consistent atomic replacement. Its targeted response is: Choose a controlled CREATE OR REPLACE TABLE or compatible atomic table-replacement strategy and validate schema/permissions. The current case is governed more directly by: Delivery guarantees alone do not remove duplicate producer or consumer events.
B:This principle concerns a separate data-engineering condition: A batch integration needs a connector-managed path from a supported operational source into the lakehouse. Its targeted response is: Evaluate supported Lakeflow Connect ingestion rather than hardcoding ad hoc database extraction. The current case is governed more directly by: Delivery guarantees alone do not remove duplicate producer or consumer events.
C:This principle concerns a separate data-engineering condition: A transactional feed produces out-of-order CDC events and a naive MERGE overwrites a newer status with an older update. Its targeted response is: Apply ordered CDC sequencing using source change version or timestamp and consistent delete handling. The current case is governed more directly by: Delivery guarantees alone do not remove duplicate producer or consumer events.
Question 639
A production data team leaves out: Tune supported SQL warehouse capacity and concurrency scaling based on simultaneous production query measurements. Its scenario is: A BI warehouse repeatedly slows during morning sign-in peaks even though a single analyst query runs quickly. Which failure mode is MOST relevant?
- An upgrade or runaway task on the shared cluster can compromise finance pipeline availability.
- An unverified runtime switch may break a previously working production ingestion notebook.
- A default GPU cluster may charge premium rates while most tasks leave accelerators unused.
- Sizing from isolated benchmark queries can leave hundreds of dashboard viewers waiting in a query queue.
Explanation
Correct Answer
D:The direct residual exposure is: Sizing from isolated benchmark queries can leave hundreds of dashboard viewers waiting in a query queue. The triggering condition is: A BI warehouse repeatedly slows during morning sign-in peaks even though a single analyst query runs quickly. The omitted safeguard is: Tune supported SQL warehouse capacity and concurrency scaling based on simultaneous production query measurements. This matters because: Concurrency queuing is separate from a single-query execution performance problem. A corrective test would show: A concurrent-user benchmark meets percentile latency and queue-depth targets with acceptable cost.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A notebook pipeline uses a shared cluster with unstable third-party dependencies while a critical finance job needs controlled isolation. Its preventative action would be: Run the finance job on dedicated supported job compute with its own pinned libraries and permissions. The most direct exposure in the current scenario is: Sizing from isolated benchmark queries can leave hundreds of dashboard viewers waiting in a query queue.
B:This failure mode belongs mainly to another situation: A developer selects a new Databricks Runtime version for production despite an untested JDBC driver dependency. Its preventative action would be: Pin a supported Databricks Runtime version and validate application libraries against it before deployment. The most direct exposure in the current scenario is: Sizing from isolated benchmark queries can leave hundreds of dashboard viewers waiting in a query queue.
C:This failure mode belongs mainly to another situation: A feature-engineering notebook requires GPU libraries but the data cleaning jobs never use accelerators. Its preventative action would be: Assign GPU-capable compute only to supported GPU workloads and use CPU compute for ordinary transformations. The most direct exposure in the current scenario is: Sizing from isolated benchmark queries can leave hundreds of dashboard viewers waiting in a query queue.
Question 640
A solution applies: Design a separate failure-handling or completion branch with the supported run-if logic for alerts and cleanup. It addresses: A source task fails yet the monitoring task never executes because all downstream tasks require upstream success. Which operational outcome MOST directly verifies success?
- Invalid input blocks publication while the previous approved dataset remains available.
- A staged failure triggers the documented alert and cleanup while skipping dependent publication.
- A simulated hung request times out, alerts the owner and avoids publishing incomplete data.
- A delayed-ingestion test prevents gold refresh until bronze and silver are completed successfully.
Explanation
Correct Answer
B:The strongest acceptance proof is: A staged failure triggers the documented alert and cleanup while skipping dependent publication. It specifically verifies: Design a separate failure-handling or completion branch with the supported run-if logic for alerts and cleanup. The original issue was: A source task fails yet the monitoring task never executes because all downstream tasks require upstream success. The underlying mechanism is: Orchestration may need error paths that execute regardless of normal success dependencies. Without it, an upstream failure can remain invisible and leave temporary resources without cleanup.
Incorrect Answers
A:This result would validate a different measure: Place validations before publishing and use an atomic cutover or controlled target write. That is relevant to the distinct problem: A workflow writes to production tables before a validation task examines null keys and duplicates. Here the acceptance criterion is: A staged failure triggers the documented alert and cleanup while skipping dependent publication.
C:This result would validate a different measure: Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages. That is relevant to the distinct problem: A scheduled notebook contains both transformation SQL and a long-running external API loop that occasionally hangs. Here the acceptance criterion is: A staged failure triggers the documented alert and cleanup while skipping dependent publication.
D:This result would validate a different measure: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. That is relevant to the distinct problem: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. Here the acceptance criterion is: A staged failure triggers the documented alert and cleanup while skipping dependent publication.