Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 77
Question 760
A data product owner adopts: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. The reason is: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. What observed result BEST proves effectiveness?
- A backdated invoice joins to the address valid on its transaction date while the current record shows the latest address.
- File-count and query-profile comparisons show fewer fragments and faster targeted scans.
- Storage metrics show a manageable file distribution and point lookups meet query SLAs.
- A controlled late event appears in the correct historical period according to the approved policy.
Explanation
Correct Answer
C:The strongest acceptance proof is: Storage metrics show a manageable file distribution and point lookups meet query SLAs. It specifically verifies: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. The original issue was: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. The underlying mechanism is: Partition pruning benefits can be outweighed by extremely fragmented high-cardinality directory layouts. Without it, one-file-per-customer patterns can overwhelm metadata and worsen query startup.
Incorrect Answers
A:This result would validate a different measure: Model a slowly changing dimension Type 2 with effective dates, current flags and correctly managed surrogate keys. That is relevant to the distinct problem: A retail customer table changes delivery address repeatedly but analysts must reconstruct the address known at any prior invoice date. Here the acceptance criterion is: Storage metrics show a manageable file distribution and point lookups meet query SLAs.
B:This result would validate a different measure: Use coarse suitable partitioning or liquid clustering selected from filter patterns and cardinality. That is relevant to the distinct problem: An event table is partitioned by the unique transaction identifier and creates millions of tiny partitions. Here the acceptance criterion is: Storage metrics show a manageable file distribution and point lookups meet query SLAs.
D:This result would validate a different measure: Design event-time fields, ingestion timestamps and late-data behavior suited to replay and temporal queries. That is relevant to the distinct problem: A lakehouse receives sensor observations with event timestamps that arrive out of order. Here the acceptance criterion is: Storage metrics show a manageable file distribution and point lookups meet query SLAs.
Question 761
During a lakehouse implementation review, engineers observe: A central repository contains schemas with missing column owners and no sensitivity classifications. What is the BEST next action?
- Implement a catalog stewardship process to review metadata and governed tags at dataset publication.
- Use Delta Sharing with an authorized recipient, narrowly scoped shares and lifecycle management.
- Publish a restricted derived dataset through Delta Sharing after privacy and freshness validation.
- Develop an approved role-aware column mask and test its outputs across supported query clients.
Explanation
Correct Answer
A:The most appropriate action is: Implement a catalog stewardship process to review metadata and governed tags at dataset publication. It addresses the stated condition: A central repository contains schemas with missing column owners and no sensitivity classifications. The technical basis is: Governance metadata is useful only when ownership and classification are kept current. Without this control, consumers can use incorrectly labeled confidential fields without recognizing their sensitivity. A direct acceptance check is: A publish-time check rejects missing required descriptions, owners and classification tags.
Incorrect Answers
B:This alternative primarily addresses a different Azure Databricks problem: A data-sharing project emails storage keys to an external partner for reading curated Delta results. The corresponding measure there would be: Use Delta Sharing with an authorized recipient, narrowly scoped shares and lifecycle management. The stated situation instead calls for: Implement a catalog stewardship process to review metadata and governed tags at dataset publication.
C:This alternative primarily addresses a different Azure Databricks problem: A customer wants to share only a daily refreshed aggregate, not raw personally identifiable transactions. The corresponding measure there would be: Publish a restricted derived dataset through Delta Sharing after privacy and freshness validation. The stated situation instead calls for: Implement a catalog stewardship process to review metadata and governed tags at dataset publication.
D:This alternative primarily addresses a different Azure Databricks problem: A data product stores PII in a shared Delta table but several reports require different masking for each audience. The corresponding measure there would be: Develop an approved role-aware column mask and test its outputs across supported query clients. The stated situation instead calls for: Implement a catalog stewardship process to review metadata and governed tags at dataset publication.
Question 762
An implementation skips: Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages. Engineers are observing: A scheduled notebook contains both transformation SQL and a long-running external API loop that occasionally hangs. What consequence is MOST likely?
- A stalled API call can hold an entire pipeline indefinitely and consume compute.
- Unnecessary serial execution can extend the critical path beyond the freshness target.
- Full reruns can repeat expensive historical ingestion and duplicate side effects.
- Hard-coded development resources can cause a production job to read or overwrite the wrong data.
Explanation
Correct Answer
A:The direct residual exposure is: A stalled API call can hold an entire pipeline indefinitely and consume compute. The triggering condition is: A scheduled notebook contains both transformation SQL and a long-running external API loop that occasionally hangs. The omitted safeguard is: Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages. This matters because: Task boundaries make duration limits and operational ownership explicit. A corrective test would show: A simulated hung request times out, alerts the owner and avoids publishing incomplete data.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A pipeline uses a single notebook to read four sources sequentially even though two can be processed independently. Its preventative action would be: Design a task DAG with parallel independent ingestion and an aggregation dependency after both complete. The most direct exposure in the current scenario is: A stalled API call can hold an entire pipeline indefinitely and consume compute.
C:This failure mode belongs mainly to another situation: A bulk repair executes all pipeline tasks again even when only the final reporting notebook failed. Its preventative action would be: Separate tasks and use the supported repair-run workflow at the failed stage when dependencies remain valid. The most direct exposure in the current scenario is: A stalled API call can hold an entire pipeline indefinitely and consume compute.
D:This failure mode belongs mainly to another situation: A multi-team notebook uses hard-coded workspace names and credentials, making the same pipeline fail in staging. Its preventative action would be: Parameterize environment-specific paths, catalogs and identities through governed deployment configuration. The most direct exposure in the current scenario is: A stalled API call can hold an entire pipeline indefinitely and consume compute.
Question 763
A team omits the recommended measure: Use supported streaming or sufficiently frequent incremental CDC with measured end-to-end latency. Its environment shows: A team selects batch processing for customer account changes though downstream applications require minute-scale freshness. What is the MOST direct residual risk?
- Deleting stream state can cause full reprocessing or skipped incremental data.
- Repeated listing and manual offsets can miss files or become too expensive as volume grows.
- A fixed midnight file list can omit valid delayed uploads.
- Daily batches cannot satisfy short update deadlines even when their processing is correct.
Explanation
Correct Answer
D:The direct residual exposure is: Daily batches cannot satisfy short update deadlines even when their processing is correct. The triggering condition is: A team selects batch processing for customer account changes though downstream applications require minute-scale freshness. The omitted safeguard is: Use supported streaming or sufficiently frequent incremental CDC with measured end-to-end latency. This matters because: Freshness requirements should determine batch versus streaming design. A corrective test would show: A changed customer field reaches the approved target within the measured freshness SLO.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A long-running ingestion job persists its checkpoint in the same temporary folder that daily cleanup deletes. Its preventative action would be: Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle. The most direct exposure in the current scenario is: Daily batches cannot satisfy short update deadlines even when their processing is correct.
B:This failure mode belongs mainly to another situation: An Azure storage account receives tens of thousands of small parquet files per hour and a notebook lists the whole container on every run. Its preventative action would be: Use Auto Loader with durable checkpointing and an appropriate discovery mode to ingest new files incrementally. The most direct exposure in the current scenario is: Daily batches cannot satisfy short update deadlines even when their processing is correct.
C:This failure mode belongs mainly to another situation: A Databricks job receives daily structured extracts but an upstream schedule occasionally delivers files late. Its preventative action would be: Use incremental file ingestion with explicit late-arrival cutoffs and a measurable source-to-target watermark. The most direct exposure in the current scenario is: Daily batches cannot satisfy short update deadlines even when their processing is correct.
Question 764
During a lakehouse implementation review, engineers observe: An overnight load reads tab-separated files but an engineer assumes comma separators and casts malformed columns to null. What is the BEST next action?
- Choose a controlled CREATE OR REPLACE TABLE or compatible atomic table-replacement strategy and validate schema/permissions.
- Persist necessary source metadata and checkpoint lineage for troubleshooting and replay.
- Specify the correct input delimiter, header and schema and validate parsing on a representative file.
- Deduplicate by stable event identity and business time using documented state and watermark rules.
Explanation
Correct Answer
C:The most appropriate action is: Specify the correct input delimiter, header and schema and validate parsing on a representative file. It addresses the stated condition: An overnight load reads tab-separated files but an engineer assumes comma separators and casts malformed columns to null. The technical basis is: Text ingestion options are part of the data contract and affect parsing correctness. Without this control, misparsed columns can populate nulls while the batch appears to complete successfully. A direct acceptance check is: A seed file loads the exact expected columns and invalid rows are reported rather than silently accepted.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: An existing table must be rebuilt from corrected historical files while its readers require a consistent atomic replacement. The corresponding measure there would be: Choose a controlled CREATE OR REPLACE TABLE or compatible atomic table-replacement strategy and validate schema/permissions. The stated situation instead calls for: Specify the correct input delimiter, header and schema and validate parsing on a representative file.
B:This alternative primarily addresses a different Azure Databricks problem: A pipeline ingests Event Hubs messages into a bronze table but never captures the partition, offset or event timestamp. The corresponding measure there would be: Persist necessary source metadata and checkpoint lineage for troubleshooting and replay. The stated situation instead calls for: Specify the correct input delimiter, header and schema and validate parsing on a representative file.
D:This alternative primarily addresses a different Azure Databricks problem: An ingestion pipeline reads cloud events with at-least-once delivery and assumes every event ID appears only once. The corresponding measure there would be: Deduplicate by stable event identity and business time using documented state and watermark rules. The stated situation instead calls for: Specify the correct input delimiter, header and schema and validate parsing on a representative file.
Question 765
The design team proposes: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. The situation is: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. Which technical principle justifies the decision?
- A recoverable pipeline should produce the same logical state after an equivalent retry.
- Retryable transient failures should be distinguished from permanent data or permission errors.
- Task-level repair reduces unnecessary recomputation while preserving dependency correctness.
- DAG-based orchestration coordinates data readiness and avoids race conditions across dependent tasks.
Explanation
Correct Answer
D:The decisive mechanism is: DAG-based orchestration coordinates data readiness and avoids race conditions across dependent tasks. That explains the selected action: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. In the original scenario: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. The specific unmanaged risk is: An early downstream task can publish partial business totals without raising an obvious failure. Validate with: A delayed-ingestion test prevents gold refresh until bronze and silver are completed successfully.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A nightly pipeline rerun appends the same orders to the target and doubles totals after an operator presses Retry. Its targeted response is: Use idempotent writes or MERGE keyed by stable order identity for repeatable task execution. The current case is governed more directly by: DAG-based orchestration coordinates data readiness and avoids race conditions across dependent tasks.
B:This principle concerns a separate data-engineering condition: A batch enrichment task downloads a remote file and sometimes receives transient HTTP 503 responses. Its targeted response is: Add bounded retries with exponential backoff and observable failure after the retry budget. The current case is governed more directly by: DAG-based orchestration coordinates data readiness and avoids race conditions across dependent tasks.
C:This principle concerns a separate data-engineering condition: A bulk repair executes all pipeline tasks again even when only the final reporting notebook failed. Its targeted response is: Separate tasks and use the supported repair-run workflow at the failed stage when dependencies remain valid. The current case is governed more directly by: DAG-based orchestration coordinates data readiness and avoids race conditions across dependent tasks.
Question 766
A team omits the recommended measure: Use a supported file-arrival trigger where its prerequisites and source location are compatible. Its environment shows: A cloud file feed needs a job triggered by arriving files instead of polling every minute. What is the MOST direct residual risk?
- Mixed processing dates across tasks can create unreconciled reports and repeated records.
- User-linked credentials can break scheduled jobs after account deactivation or team reassignment.
- Constant scheduling may incur empty-run costs and still miss arrivals between checks.
- Replaying an entire DAG without need can duplicate side effects and increase cost.
Explanation
Correct Answer
C:The direct residual exposure is: Constant scheduling may incur empty-run costs and still miss arrivals between checks. The triggering condition is: A cloud file feed needs a job triggered by arriving files instead of polling every minute. The omitted safeguard is: Use a supported file-arrival trigger where its prerequisites and source location are compatible. This matters because: Event or file-arrival triggers can reduce idle polling while maintaining responsive execution. A corrective test would show: A test file arrives and starts one valid job run without repeated empty polling.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A workflow needs to pass a date parameter from a scheduled trigger into all its downstream notebook tasks. Its preventative action would be: Define validated job parameters and propagate them through supported task parameter bindings. The most direct exposure in the current scenario is: Constant scheduling may incur empty-run costs and still miss arrivals between checks.
B:This failure mode belongs mainly to another situation: A job uses a service principal that loses access to the destination schema when an employee leaves. Its preventative action would be: Assign a dedicated run-as identity with durable scoped grants and an ownership review. The most direct exposure in the current scenario is: Constant scheduling may incur empty-run costs and still miss arrivals between checks.
D:This failure mode belongs mainly to another situation: A team assumes that automatic job retries are the same as recovering a failed multi-step run at its last unsuccessful task. Its preventative action would be: Choose the appropriate retry and repair-run functionality based on failed-task state and valid dependencies. The most direct exposure in the current scenario is: Constant scheduling may incur empty-run costs and still miss arrivals between checks.
Question 767
A Databricks engineer recommends: Apply ordered CDC sequencing using source change version or timestamp and consistent delete handling. It responds to: A transactional feed produces out-of-order CDC events and a naive MERGE overwrites a newer status with an older update. What underlying mechanism BEST explains why?
- Data pipeline completeness requires tracking arrivals rather than assuming an exact wall-clock boundary.
- Freshness requirements should determine batch versus streaming design.
- Pulling distributed data to a single driver defeats parallel execution and memory limits.
- CDC correctness depends on source ordering and idempotent update rules.
Explanation
Correct Answer
D:The decisive mechanism is: CDC correctness depends on source ordering and idempotent update rules. That explains the selected action: Apply ordered CDC sequencing using source change version or timestamp and consistent delete handling. In the original scenario: A transactional feed produces out-of-order CDC events and a naive MERGE overwrites a newer status with an older update. The specific unmanaged risk is: Out-of-order events can regress a target from its latest state to a stale state. Validate with: A shuffled CDC test results in the correct highest-version value after replay.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A Databricks job receives daily structured extracts but an upstream schedule occasionally delivers files late. Its targeted response is: Use incremental file ingestion with explicit late-arrival cutoffs and a measurable source-to-target watermark. The current case is governed more directly by: CDC correctness depends on source ordering and idempotent update rules.
B:This principle concerns a separate data-engineering condition: A team selects batch processing for customer account changes though downstream applications require minute-scale freshness. Its targeted response is: Use supported streaming or sufficiently frequent incremental CDC with measured end-to-end latency. The current case is governed more directly by: CDC correctness depends on source ordering and idempotent update rules.
C:This principle concerns a separate data-engineering condition: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. Its targeted response is: Use Spark-native distributed read and transformation APIs or partition-safe processing. The current case is governed more directly by: CDC correctness depends on source ordering and idempotent update rules.
Question 768
The team chooses: Implement the required SCD Type 1 overwrite with stable business keys and validated updates. Its observed issue is: A reference table stores only the latest airport name and business users explicitly do not need historical values. What is the strongest technical basis?
- Liquid clustering and traditional ZORDER/partitioning have compatibility constraints and different maintenance paths.
- Event time and processing time represent different facts and affect historical interpretation.
- Type 1 dimensions replace nonhistorical descriptive attributes without retaining obsolete versions.
- Physical layout should match selective consumer predicates rather than convenience of ingestion.
Explanation
Correct Answer
C:The decisive mechanism is: Type 1 dimensions replace nonhistorical descriptive attributes without retaining obsolete versions. That explains the selected action: Implement the required SCD Type 1 overwrite with stable business keys and validated updates. In the original scenario: A reference table stores only the latest airport name and business users explicitly do not need historical values. The specific unmanaged risk is: Keeping unnecessary historical rows can produce multiple matches and incorrect lookup joins. Validate with: The business key resolves to one current row after an update and no unintended duplicate joins occur.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A team attempts to apply both ZORDER and liquid clustering to the same Delta table. Its targeted response is: Choose one compatible data layout strategy based on table format, platform support and query evidence. The current case is governed more directly by: Type 1 dimensions replace nonhistorical descriptive attributes without retaining obsolete versions.
B:This principle concerns a separate data-engineering condition: A lakehouse receives sensor observations with event timestamps that arrive out of order. Its targeted response is: Design event-time fields, ingestion timestamps and late-data behavior suited to replay and temporal queries. The current case is governed more directly by: Type 1 dimensions replace nonhistorical descriptive attributes without retaining obsolete versions.
D:This principle concerns a separate data-engineering condition: An event stream is partitioned by ingestion day but analysts constantly filter by event timestamp ranges crossing midnight. Its targeted response is: Reevaluate data layout using representative event-time filters and a suitable clustering key. The current case is governed more directly by: Type 1 dimensions replace nonhistorical descriptive attributes without retaining obsolete versions.
Question 769
The team chooses: Create compute policies with approved node types, worker limits, tags and idle settings for the permitted users. Its observed issue is: A department wants a self-service sandbox but must prevent users from creating huge unrestricted clusters. What is the strongest technical basis?
- GPU resources are specialized and do not automatically accelerate every Spark or SQL transformation.
- Compute policies provide governance boundaries for permissible resource configurations.
- Concurrency queuing is separate from a single-query execution performance problem.
- Serverless and classic compute have different configuration surfaces and supported dependency patterns.
Explanation
Correct Answer
B:The decisive mechanism is: Compute policies provide governance boundaries for permissible resource configurations. That explains the selected action: Create compute policies with approved node types, worker limits, tags and idle settings for the permitted users. In the original scenario: A department wants a self-service sandbox but must prevent users from creating huge unrestricted clusters. The specific unmanaged risk is: Unrestricted cluster creation can create unpredictable spending and inconsistent operational standards. Validate with: An ordinary user can start an approved small cluster but cannot exceed policy size or runtime limits.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A feature-engineering notebook requires GPU libraries but the data cleaning jobs never use accelerators. Its targeted response is: Assign GPU-capable compute only to supported GPU workloads and use CPU compute for ordinary transformations. The current case is governed more directly by: Compute policies provide governance boundaries for permissible resource configurations.
C:This principle concerns a separate data-engineering condition: A BI warehouse repeatedly slows during morning sign-in peaks even though a single analyst query runs quickly. Its targeted response is: Tune supported SQL warehouse capacity and concurrency scaling based on simultaneous production query measurements. The current case is governed more directly by: Compute policies provide governance boundaries for permissible resource configurations.
D:This principle concerns a separate data-engineering condition: A long-running job using an unsupported custom system dependency is moved to serverless without checking feature compatibility. Its targeted response is: Review current serverless limitations and select compatible classic jobs compute when necessary. The current case is governed more directly by: Compute policies provide governance boundaries for permissible resource configurations.
Question 770
The team implements: Validate timezone-aware timestamp types and quarantine ambiguous or unparseable event values. The original problem is: A regulated dataset requires all timestamps to parse as UTC but several rows have ambiguous local values. Which test MOST convincingly demonstrates correctness?
- A sample of valid UTC timestamps passes and known ambiguous strings produce the expected exception record.
- Each run reports complete input-to-output reconciliation and alerts when rejected fraction exceeds its SLO.
- Injected critical and noncritical bad rows follow their respective approved actions and emit metrics.
- A controlled negative-age record triggers the configured expectation event without being silently published.
Explanation
Correct Answer
A:The strongest acceptance proof is: A sample of valid UTC timestamps passes and known ambiguous strings produce the expected exception record. It specifically verifies: Validate timezone-aware timestamp types and quarantine ambiguous or unparseable event values. The original issue was: A regulated dataset requires all timestamps to parse as UTC but several rows have ambiguous local values. The underlying mechanism is: Type checks and temporal semantics must be aligned to prevent incorrect time-series results. Without it, ambiguous timestamp parsing can move financial events across reporting windows.
Incorrect Answers
B:This result would validate a different measure: Collect expectation metrics and reconcile input, accepted, quarantined and rejected record counts. That is relevant to the distinct problem: A pipeline reports zero failed rows because bad records are silently dropped without any count of what was removed. Here the acceptance criterion is: A sample of valid UTC timestamps passes and known ambiguous strings produce the expected exception record.
C:This result would validate a different measure: Configure expectations with an explicit fail, drop or retention behavior aligned to business severity. That is relevant to the distinct problem: A Lakeflow pipeline has a quality rule but nobody has decided whether invalid transactions should stop publishing. Here the acceptance criterion is: A sample of valid UTC timestamps passes and known ambiguous strings produce the expected exception record.
D:This result would validate a different measure: Define a range expectation with metrics and the approved drop, fail or quarantine disposition. That is relevant to the distinct problem: A bronze stream sometimes includes ages below zero or above 120 yet the transformation casts every value to integer. Here the acceptance criterion is: A sample of valid UTC timestamps passes and known ambiguous strings produce the expected exception record.