Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 39
Question 380
The recommended measure is: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. The operational context is: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. Which data engineering concept applies?
- SQL warehouses are optimized for SQL analytics and isolate query serving from other compute activities.
- Workspace compute permissions and access modes help maintain workload and security boundaries.
- Ephemeral task-oriented compute can avoid idle capacity costs while retaining reproducible batch execution.
- Spark and runtime upgrades can change supported APIs, dependency versions and execution behavior.
Explanation
Correct Answer
A:The decisive mechanism is: SQL warehouses are optimized for SQL analytics and isolate query serving from other compute activities. That explains the selected action: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. In the original scenario: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. The specific unmanaged risk is: A shared notebook cluster can cause dashboards to stall when unrelated Spark notebooks consume executors. Validate with: A load rehearsal meets dashboard p95 response targets while notebook resource spikes do not affect query service.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A multi-team interactive cluster permits users to attach arbitrary unapproved libraries and run notebooks under broad shared privileges. Its targeted response is: Apply suitable compute access modes, policies and entitlements that restrict who can attach or modify shared compute. The current case is governed more directly by: SQL warehouses are optimized for SQL analytics and isolate query serving from other compute activities.
C:This principle concerns a separate data-engineering condition: A nightly batch transformation uses a continuously running all-purpose cluster even though no analyst interacts with the workload between runs. Its targeted response is: Move the scheduled transformation to appropriately configured jobs compute or supported serverless jobs and release resources after execution. The current case is governed more directly by: SQL warehouses are optimized for SQL analytics and isolate query serving from other compute activities.
D:This principle concerns a separate data-engineering condition: A developer selects a new Databricks Runtime version for production despite an untested JDBC driver dependency. Its targeted response is: Pin a supported Databricks Runtime version and validate application libraries against it before deployment. The current case is governed more directly by: SQL warehouses are optimized for SQL analytics and isolate query serving from other compute activities.
Question 381
The design team proposes: Validate per-target bundle variable resolution and compare rendered job definitions before deployment. The situation is: A data engineer updates a bundled job trigger but another environment's deployment silently uses the wrong schedule. Which technical principle justifies the decision?
- Reproducible deployments require versioned execution dependencies as well as application code.
- Automated releases should not depend on personal accounts or broadly exposed long-lived credentials.
- Configuration-as-code requires environment-specific parameter checks as well as common source control.
- Version control and code review establish provenance and repeatable release history.
Explanation
Correct Answer
C:The decisive mechanism is: Configuration-as-code requires environment-specific parameter checks as well as common source control. That explains the selected action: Validate per-target bundle variable resolution and compare rendered job definitions before deployment. In the original scenario: A data engineer updates a bundled job trigger but another environment's deployment silently uses the wrong schedule. The specific unmanaged risk is: A wrong schedule can duplicate loads or miss critical reporting windows. Validate with: Deployment plans show the intended trigger for each target and a test job fires at the correct interval.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A team cannot reproduce a prior release because package versions are resolved dynamically during deployment. Its targeted response is: Pin package and runtime versions in the governed release artifact and record its dependency manifest. The current case is governed more directly by: Configuration-as-code requires environment-specific parameter checks as well as common source control.
B:This principle concerns a separate data-engineering condition: A CI pipeline deploys Databricks assets under an engineer's personal token stored in repository variables. Its targeted response is: Use a secure service identity and protected credential handling for bundle or REST API deployment. The current case is governed more directly by: Configuration-as-code requires environment-specific parameter checks as well as common source control.
D:This principle concerns a separate data-engineering condition: Engineers edit production notebooks directly and lose the ability to identify which code change broke a pipeline. Its targeted response is: Store deployment source in Git with reviewed branches and pull requests before promoting changes. The current case is governed more directly by: Configuration-as-code requires environment-specific parameter checks as well as common source control.
Question 382
For this Azure Databricks issue: A new customer feed is missing optional contact information for 8% of rows and the team treats every null as pipeline failure. Engineers choose: Profile source missingness and define field-specific required versus optional expectations. What acceptance evidence is BEST?
- Quality dashboards separate optional-null rates from prohibited null keys and meet agreed thresholds.
- A seeded repeated account ID is detected before the curated dataset is marked ready.
- A simulated type change is rejected or rescued with an actionable alert and no corrupted curated output.
- A sample of valid UTC timestamps passes and known ambiguous strings produce the expected exception record.
Explanation
Correct Answer
A:The strongest acceptance proof is: Quality dashboards separate optional-null rates from prohibited null keys and meet agreed thresholds. It specifically verifies: Profile source missingness and define field-specific required versus optional expectations. The original issue was: A new customer feed is missing optional contact information for 8% of rows and the team treats every null as pipeline failure. The underlying mechanism is: Completeness thresholds depend on business contracts and cannot be uniform across all columns. Without it, overly strict validation can block legitimate records while loose validation may permit missing keys.
Incorrect Answers
B:This result would validate a different measure: Run a supported uniqueness or cardinality validation during the transformation and surface violations. That is relevant to the distinct problem: A curated account table accidentally contains multiple rows per supposedly unique current account ID. Here the acceptance criterion is: Quality dashboards separate optional-null rates from prohibited null keys and meet agreed thresholds.
C:This result would validate a different measure: Apply schema enforcement with explicit drift handling and alert on incompatible structural changes. That is relevant to the distinct problem: A source API changes a quantity from integer to nested JSON and the pipeline silently produces nulls downstream. Here the acceptance criterion is: Quality dashboards separate optional-null rates from prohibited null keys and meet agreed thresholds.
D:This result would validate a different measure: Validate timezone-aware timestamp types and quarantine ambiguous or unparseable event values. That is relevant to the distinct problem: A regulated dataset requires all timestamps to parse as UTC but several rows have ambiguous local values. Here the acceptance criterion is: Quality dashboards separate optional-null rates from prohibited null keys and meet agreed thresholds.
Question 383
During a lakehouse implementation review, engineers observe: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. What is the BEST next action?
- Define a supported typed schema for nested structures and expected optional fields before writing to curated Delta.
- Enable supported liquid clustering with appropriate keys and run compatible optimization.
- Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys.
- Use supported Delta table version history and time travel within the documented retention window.
Explanation
Correct Answer
C:The most appropriate action is: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys. It addresses the stated condition: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. The technical basis is: Efficient layout keys are driven by actual access patterns and file statistics. Without this control, poorly chosen keys increase full scans and maintenance overhead as traffic grows. A direct acceptance check is: A measured workload benchmark shows scan reduction across the highest-value query templates.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A stream produces nested JSON events with optional fields but an analyst hardcodes every field as STRING. The corresponding measure there would be: Define a supported typed schema for nested structures and expected optional fields before writing to curated Delta. The stated situation instead calls for: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys.
B:This alternative primarily addresses a different Azure Databricks problem: A large Delta fact table is filtered primarily on high-cardinality account IDs with changing query patterns. The corresponding measure there would be: Enable supported liquid clustering with appropriate keys and run compatible optimization. The stated situation instead calls for: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys.
D:This alternative primarily addresses a different Azure Databricks problem: A business needs to compare two point-in-time table versions during a financial reconciliation. The corresponding measure there would be: Use supported Delta table version history and time travel within the documented retention window. The stated situation instead calls for: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys.
Question 384
For this Azure Databricks issue: A raw event lake contains append-only records but a developer chooses daily full overwrite of the curated Delta table. Engineers choose: Choose incremental append or merge based on immutable versus mutable source behavior. What acceptance evidence is BEST?
- The external reader loads approved columns and a documented compatibility test passes.
- The platform team retains approved storage lifecycle authority while Databricks queries the registered data.
- A measured workload benchmark shows scan reduction across the highest-value query templates.
- Two successive batches load only new records while preserved history and counts reconcile.
Explanation
Correct Answer
D:The strongest acceptance proof is: Two successive batches load only new records while preserved history and counts reconcile. It specifically verifies: Choose incremental append or merge based on immutable versus mutable source behavior. The original issue was: A raw event lake contains append-only records but a developer chooses daily full overwrite of the curated Delta table. The underlying mechanism is: Ingestion strategy should match whether source records can be corrected or deleted. Without it, unnecessary overwrites increase compute cost and can cause data loss during partial reruns.
Incorrect Answers
A:This result would validate a different measure: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements. That is relevant to the distinct problem: An external sharing integration needs interoperable columnar read files but not native Delta updates. Here the acceptance criterion is: Two successive batches load only new records while preserved history and counts reconcile.
B:This result would validate a different measure: Use an external Unity Catalog table with an approved registered location instead of assuming managed storage ownership. That is relevant to the distinct problem: A dataset has a strict requirement that Azure storage ownership and lifecycle remain under a separate platform team. Here the acceptance criterion is: Two successive batches load only new records while preserved history and counts reconcile.
C:This result would validate a different measure: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys. That is relevant to the distinct problem: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. Here the acceptance criterion is: Two successive batches load only new records while preserved history and counts reconcile.
Question 385
An Azure Databricks architect reviews the following problem: A continuous feed creates and refreshes tables with consistent declarative dependencies but developers manually chain notebooks. What should the team implement FIRST?
- Evaluate Lakeflow Spark Declarative Pipelines for the compatible streaming and transformation graph.
- Select a notebook task within Lakeflow Jobs when procedural control and custom libraries fit the requirements.
- Design a separate failure-handling or completion branch with the supported run-if logic for alerts and cleanup.
- Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed.
Explanation
Correct Answer
A:The most appropriate action is: Evaluate Lakeflow Spark Declarative Pipelines for the compatible streaming and transformation graph. It addresses the stated condition: A continuous feed creates and refreshes tables with consistent declarative dependencies but developers manually chain notebooks. The technical basis is: Declarative pipelines express table dependencies and data quality rules without bespoke notebook orchestration. Without this control, hand-built ordering can drift from the actual dependency graph as tables are added. A direct acceptance check is: A pipeline run updates tables in valid dependency order and records expectation outcomes.
Incorrect Answers
B:This alternative primarily addresses a different Azure Databricks problem: An ad hoc machine-learning preparation task has complex custom Python orchestration and no declarative table graph. The corresponding measure there would be: Select a notebook task within Lakeflow Jobs when procedural control and custom libraries fit the requirements. The stated situation instead calls for: Evaluate Lakeflow Spark Declarative Pipelines for the compatible streaming and transformation graph.
C:This alternative primarily addresses a different Azure Databricks problem: A source task fails yet the monitoring task never executes because all downstream tasks require upstream success. The corresponding measure there would be: Design a separate failure-handling or completion branch with the supported run-if logic for alerts and cleanup. The stated situation instead calls for: Evaluate Lakeflow Spark Declarative Pipelines for the compatible streaming and transformation graph.
D:This alternative primarily addresses a different Azure Databricks problem: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. The corresponding measure there would be: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. The stated situation instead calls for: Evaluate Lakeflow Spark Declarative Pipelines for the compatible streaming and transformation graph.
Question 386
For this Microsoft DP-750 case: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. Which design or operational response BEST fits?
- Select stream-appropriate compute and lifecycle settings so monitoring and availability targets are met.
- Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries.
- Create compute policies with approved node types, worker limits, tags and idle settings for the permitted users.
- Benchmark Photon on a supported runtime with representative queries and compare cost, latency and correctness.
Explanation
Correct Answer
B:The most appropriate action is: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. It addresses the stated condition: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. The technical basis is: SQL warehouses are optimized for SQL analytics and isolate query serving from other compute activities. Without this control, A shared notebook cluster can cause dashboards to stall when unrelated Spark notebooks consume executors. A direct acceptance check is: A load rehearsal meets dashboard p95 response targets while notebook resource spikes do not affect query service.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A long-running streaming pipeline is configured with aggressive auto-termination during brief periods without arriving events. The corresponding measure there would be: Select stream-appropriate compute and lifecycle settings so monitoring and availability targets are met. The stated situation instead calls for: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries.
C:This alternative primarily addresses a different Azure Databricks problem: A department wants a self-service sandbox but must prevent users from creating huge unrestricted clusters. The corresponding measure there would be: Create compute policies with approved node types, worker limits, tags and idle settings for the permitted users. The stated situation instead calls for: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries.
D:This alternative primarily addresses a different Azure Databricks problem: A Spark SQL pipeline spends most of its time executing wide joins and aggregations but the team has not evaluated a compatible vectorized execution engine. The corresponding measure there would be: Benchmark Photon on a supported runtime with representative queries and compare cost, latency and correctness. The stated situation instead calls for: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries.
Question 387
For this Azure Databricks issue: A multi-team notebook uses hard-coded workspace names and credentials, making the same pipeline fail in staging. Engineers choose: Parameterize environment-specific paths, catalogs and identities through governed deployment configuration. What acceptance evidence is BEST?
- A job executes the procedural notebook under controlled dependencies and produces the validated output.
- After a final-task failure, a repair completes the target without reprocessing validated upstream data.
- A staging and production run use the same code with distinct approved configuration and no credential leakage.
- Two executions of the same input batch yield one approved current record per order.
Explanation
Correct Answer
C:The strongest acceptance proof is: A staging and production run use the same code with distinct approved configuration and no credential leakage. It specifically verifies: Parameterize environment-specific paths, catalogs and identities through governed deployment configuration. The original issue was: A multi-team notebook uses hard-coded workspace names and credentials, making the same pipeline fail in staging. The underlying mechanism is: Portable pipelines should separate code logic from environment-specific configuration. Without it, hard-coded development resources can cause a production job to read or overwrite the wrong data.
Incorrect Answers
A:This result would validate a different measure: Select a notebook task within Lakeflow Jobs when procedural control and custom libraries fit the requirements. That is relevant to the distinct problem: An ad hoc machine-learning preparation task has complex custom Python orchestration and no declarative table graph. Here the acceptance criterion is: A staging and production run use the same code with distinct approved configuration and no credential leakage.
B:This result would validate a different measure: Separate tasks and use the supported repair-run workflow at the failed stage when dependencies remain valid. That is relevant to the distinct problem: A bulk repair executes all pipeline tasks again even when only the final reporting notebook failed. Here the acceptance criterion is: A staging and production run use the same code with distinct approved configuration and no credential leakage.
D:This result would validate a different measure: Use idempotent writes or MERGE keyed by stable order identity for repeatable task execution. That is relevant to the distinct problem: A nightly pipeline rerun appends the same orders to the target and doubles totals after an operator presses Retry. Here the acceptance criterion is: A staging and production run use the same code with distinct approved configuration and no credential leakage.
Question 388
The team chooses: Use a managed Delta table for transactional updates and controlled schema evolution. Its observed issue is: A reporting team stores frequent updates in raw CSV files and expects atomic MERGE and historical time travel. What is the strongest technical basis?
- External tables separate catalog management from independently controlled underlying file lifecycle.
- Delta tables support transactional mutations and versioned metadata beyond raw delimiter files.
- Liquid clustering can organize files for selective filters without static partition boundaries.
- Temporal tables describe the validity of state over time rather than only its latest value.
Explanation
Correct Answer
B:The decisive mechanism is: Delta tables support transactional mutations and versioned metadata beyond raw delimiter files. That explains the selected action: Use a managed Delta table for transactional updates and controlled schema evolution. In the original scenario: A reporting team stores frequent updates in raw CSV files and expects atomic MERGE and historical time travel. The specific unmanaged risk is: Directly rewriting CSV in place can expose partial updates and inconsistent reader results. Validate with: A MERGE transaction updates the expected records atomically and a historical version remains queryable within retention.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A dataset has a strict requirement that Azure storage ownership and lifecycle remain under a separate platform team. Its targeted response is: Use an external Unity Catalog table with an approved registered location instead of assuming managed storage ownership. The current case is governed more directly by: Delta tables support transactional mutations and versioned metadata beyond raw delimiter files.
C:This principle concerns a separate data-engineering condition: A large Delta fact table is filtered primarily on high-cardinality account IDs with changing query patterns. Its targeted response is: Enable supported liquid clustering with appropriate keys and run compatible optimization. The current case is governed more directly by: Delta tables support transactional mutations and versioned metadata beyond raw delimiter files.
D:This principle concerns a separate data-engineering condition: A historical compliance table must show how account status changed at precise effective intervals. Its targeted response is: Use a temporal history model with nonoverlapping effective-from and effective-to boundaries. The current case is governed more directly by: Delta tables support transactional mutations and versioned metadata beyond raw delimiter files.
Question 389
An operator introduces: Apply ordered CDC sequencing using source change version or timestamp and consistent delete handling. The situation is: A transactional feed produces out-of-order CDC events and a naive MERGE overwrites a newer status with an older update. Which validation should be required before signoff?
- Events arriving within the allowed lateness update the intended window and older events follow documented handling.
- A shuffled CDC test results in the correct highest-version value after replay.
- Injected duplicate and malformed source samples follow the intended dedupe or quarantine rules.
- The workload completes without driver collection and Spark metrics show distributed processing.
Explanation
Correct Answer
B:The strongest acceptance proof is: A shuffled CDC test results in the correct highest-version value after replay. It specifically verifies: Apply ordered CDC sequencing using source change version or timestamp and consistent delete handling. The original issue was: A transactional feed produces out-of-order CDC events and a naive MERGE overwrites a newer status with an older update. The underlying mechanism is: CDC correctness depends on source ordering and idempotent update rules. Without it, out-of-order events can regress a target from its latest state to a stale state.
Incorrect Answers
A:This result would validate a different measure: Configure event-time windowing and an appropriate watermark based on observed lateness. That is relevant to the distinct problem: A stream must group clicks into five-minute event-time windows while some mobile devices transmit data several minutes late. Here the acceptance criterion is: A shuffled CDC test results in the correct highest-version value after replay.
C:This result would validate a different measure: Define Lakeflow Spark Declarative Pipelines source options, incremental behavior and expectations with a durable target. That is relevant to the distinct problem: A declarative pipeline loads cloud object files continuously but its configuration omits how duplicate files and bad records are handled. Here the acceptance criterion is: A shuffled CDC test results in the correct highest-version value after replay.
D:This result would validate a different measure: Use Spark-native distributed read and transformation APIs or partition-safe processing. That is relevant to the distinct problem: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. Here the acceptance criterion is: A shuffled CDC test results in the correct highest-version value after replay.
Question 390
A data engineering team faces this Azure Databricks requirement: A large aggregation has repeated shuffle spills to disk even though source scans finish quickly. Which action is MOST appropriate?
- Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage.
- Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages.
- Use query-profile evidence and a supported skew mitigation such as adaptive execution or key-aware repartitioning.
- Compare incoming event rate, processed records, watermark progress and state-store metrics before scaling.
Explanation
Correct Answer
A:The most appropriate action is: Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage. It addresses the stated condition: A large aggregation has repeated shuffle spills to disk even though source scans finish quickly. The technical basis is: Shuffle spills indicate data exceeded available in-memory execution resources for a stage. Without this control, ignoring heavy spills can cause cascading I/O latency and missed pipeline deadlines. A direct acceptance check is: A representative run shows reduced spill bytes and matching aggregate results with improved runtime.
Incorrect Answers
B:This alternative primarily addresses a different Azure Databricks problem: An interactive warehouse has high query queues during the morning but low CPU utilization after lunch. The corresponding measure there would be: Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages. The stated situation instead calls for: Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage.
C:This alternative primarily addresses a different Azure Databricks problem: A DataFrame join spills heavily because a single join key occurs in most rows. The corresponding measure there would be: Use query-profile evidence and a supported skew mitigation such as adaptive execution or key-aware repartitioning. The stated situation instead calls for: Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage.
D:This alternative primarily addresses a different Azure Databricks problem: A streaming pipeline's throughput falls although event arrival increases and its input backlog grows over time. The corresponding measure there would be: Compare incoming event rate, processed records, watermark progress and state-store metrics before scaling. The stated situation instead calls for: Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage.