Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 47
Question 460
An implementation skips: Inspect Unity Catalog data lineage and table history in Catalog Explorer, checking actual supported capture coverage. Engineers are observing: An analyst wants to know which upstream pipeline produced a metric displayed in a board dashboard. What consequence is MOST likely?
- Manual protection drift can expose fields on newly created tables before a review.
- Without lineage the team may repair the wrong pipeline after a metric regression.
- Long-lived shared storage credentials may expose unrelated files or survive contract termination.
- Sharing raw transaction tables can disclose protected customer attributes unnecessarily.
Explanation
Correct Answer
B:The direct residual exposure is: Without lineage the team may repair the wrong pipeline after a metric regression. The triggering condition is: An analyst wants to know which upstream pipeline produced a metric displayed in a board dashboard. The omitted safeguard is: Inspect Unity Catalog data lineage and table history in Catalog Explorer, checking actual supported capture coverage. This matters because: Data lineage links producers, transformations and consumers where observed activity is supported. A corrective test would show: A seeded transformation appears with its expected upstream source and downstream target in lineage.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A central security team manually adds row masking rules to every department table and new datasets repeatedly lack protection. Its preventative action would be: Create compatible central ABAC policies based on governed classifications and verify inherited coverage. The most direct exposure in the current scenario is: Without lineage the team may repair the wrong pipeline after a metric regression.
C:This failure mode belongs mainly to another situation: A data-sharing project emails storage keys to an external partner for reading curated Delta results. Its preventative action would be: Use Delta Sharing with an authorized recipient, narrowly scoped shares and lifecycle management. The most direct exposure in the current scenario is: Without lineage the team may repair the wrong pipeline after a metric regression.
D:This failure mode belongs mainly to another situation: A customer wants to share only a daily refreshed aggregate, not raw personally identifiable transactions. Its preventative action would be: Publish a restricted derived dataset through Delta Sharing after privacy and freshness validation. The most direct exposure in the current scenario is: Without lineage the team may repair the wrong pipeline after a metric regression.
Question 461
An Azure Databricks architect reviews the following problem: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. What should the team implement FIRST?
- Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension.
- Implement the required SCD Type 1 overwrite with stable business keys and validated updates.
- Design event-time fields, ingestion timestamps and late-data behavior suited to replay and temporal queries.
- Select CTAS for an explicitly materialized snapshot or a scheduled pipeline for ongoing refresh.
Explanation
Correct Answer
A:The most appropriate action is: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. It addresses the stated condition: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. The technical basis is: Partition pruning benefits can be outweighed by extremely fragmented high-cardinality directory layouts. Without this control, one-file-per-customer patterns can overwhelm metadata and worsen query startup. A direct acceptance check is: Storage metrics show a manageable file distribution and point lookups meet query SLAs.
Incorrect Answers
B:This alternative primarily addresses a different Azure Databricks problem: A reference table stores only the latest airport name and business users explicitly do not need historical values. The corresponding measure there would be: Implement the required SCD Type 1 overwrite with stable business keys and validated updates. The stated situation instead calls for: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension.
C:This alternative primarily addresses a different Azure Databricks problem: A lakehouse receives sensor observations with event timestamps that arrive out of order. The corresponding measure there would be: Design event-time fields, ingestion timestamps and late-data behavior suited to replay and temporal queries. The stated situation instead calls for: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension.
D:This alternative primarily addresses a different Azure Databricks problem: A pipeline uses CREATE TABLE AS SELECT for a one-time derived dataset but later assumes the result refreshes automatically. The corresponding measure there would be: Select CTAS for an explicitly materialized snapshot or a scheduled pipeline for ongoing refresh. The stated situation instead calls for: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension.
Question 462
For this Microsoft DP-750 case: An append-only telemetry feed includes newly observed events but engineers use MERGE with expensive full-table scans on every insert. Which design or operational response BEST fits?
- Use an incremental append pattern with suitable deduplication when source events cannot change.
- Apply a deterministic deduplication rule based on stable order identity and source version before aggregation.
- Use a compatible unpivot or stack transformation with explicit month and amount columns.
- Implement explicit null predicates rather than treating zero and null as equivalent.
Explanation
Correct Answer
A:The most appropriate action is: Use an incremental append pattern with suitable deduplication when source events cannot change. It addresses the stated condition: An append-only telemetry feed includes newly observed events but engineers use MERGE with expensive full-table scans on every insert. The technical basis is: Append is generally simpler for immutable events than key-by-key mutation logic. Without this control, unnecessary MERGE operations can add substantial cost and contention without correctness benefit. A direct acceptance check is: A timed append comparison meets the freshness and duplicate controls with lower compute consumption.
Incorrect Answers
B:This alternative primarily addresses a different Azure Databricks problem: An incoming orders table contains repeated order IDs because a vendor retransmits the same daily extract. The corresponding measure there would be: Apply a deterministic deduplication rule based on stable order identity and source version before aggregation. The stated situation instead calls for: Use an incremental append pattern with suitable deduplication when source events cannot change.
C:This alternative primarily addresses a different Azure Databricks problem: A large orders dataset has one column per month but a dashboard expects month-value rows for filtering and grouping. The corresponding measure there would be: Use a compatible unpivot or stack transformation with explicit month and amount columns. The stated situation instead calls for: Use an incremental append pattern with suitable deduplication when source events cannot change.
D:This alternative primarily addresses a different Azure Databricks problem: A team filters out null quantities but needs to preserve rows with zero as valid inventory states. The corresponding measure there would be: Implement explicit null predicates rather than treating zero and null as equivalent. The stated situation instead calls for: Use an incremental append pattern with suitable deduplication when source events cannot change.
Question 463
A data product owner adopts: Propagate nonrecoverable exceptions to Lakeflow Jobs and attach actionable error metadata. The reason is: A workflow uses a notebook task that catches exceptions and returns success even when a write step fails. What observed result BEST proves effectiveness?
- A delayed-ingestion test prevents gold refresh until bronze and silver are completed successfully.
- A seeded mismatch prevents cutover and the prior certified table remains queryable.
- A deliberately failed write marks the job unsuccessful and prevents dependent publish tasks.
- A simulated hung request times out, alerts the owner and avoids publishing incomplete data.
Explanation
Correct Answer
C:The strongest acceptance proof is: A deliberately failed write marks the job unsuccessful and prevents dependent publish tasks. It specifically verifies: Propagate nonrecoverable exceptions to Lakeflow Jobs and attach actionable error metadata. The original issue was: A workflow uses a notebook task that catches exceptions and returns success even when a write step fails. The underlying mechanism is: Job status must accurately reflect failed required operations for alerting and dependency logic. Without it, A swallowed exception can allow downstream tasks to publish stale or missing outputs.
Incorrect Answers
A:This result would validate a different measure: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. That is relevant to the distinct problem: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. Here the acceptance criterion is: A deliberately failed write marks the job unsuccessful and prevents dependent publish tasks.
B:This result would validate a different measure: Write to a staging target, validate row totals and invariants, then perform the approved publication step. That is relevant to the distinct problem: A finance transformation writes its output directly into a published table before reconciling totals against the source. Here the acceptance criterion is: A deliberately failed write marks the job unsuccessful and prevents dependent publish tasks.
D:This result would validate a different measure: Use controlled task-level timeouts and isolate unpredictable external calls from bounded transform stages. That is relevant to the distinct problem: A scheduled notebook contains both transformation SQL and a long-running external API loop that occasionally hangs. Here the acceptance criterion is: A deliberately failed write marks the job unsuccessful and prevents dependent publish tasks.
Question 464
The recommended measure is: Apply a supported column mask or ABAC masking rule to the sensitive field and test different roles. The operational context is: A customer service dashboard must show the last four digits of account identifiers but never full account values. Which data engineering concept applies?
- Column masking can transform sensitive values at query time according to the querying principal.
- Managed identity authentication avoids exposing reusable static secrets and ties authorization to a governed identity.
- Workspace access scope and principal privileges enforce different aspects of authorization.
- Data access delegation need not transfer administrative ownership of data objects.
Explanation
Correct Answer
A:The decisive mechanism is: Column masking can transform sensitive values at query time according to the querying principal. That explains the selected action: Apply a supported column mask or ABAC masking rule to the sensitive field and test different roles. In the original scenario: A customer service dashboard must show the last four digits of account identifiers but never full account values. The specific unmanaged risk is: Granting SELECT without masking may expose full account identifiers to unauthorized readers. Validate with: Role-based sample queries show masked values for support and exact values only for approved security operators.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A scheduled cloud-data ingestion needs an Azure storage identity but engineers embed a reusable account key inside notebooks. Its targeted response is: Use a supported Azure managed identity and Unity Catalog storage credential or equivalent approved access path. The current case is governed more directly by: Column masking can transform sensitive values at query time according to the querying principal.
C:This principle concerns a separate data-engineering condition: A team restricts access through a catalog-to-workspace binding but treats it as a replacement for data privileges. Its targeted response is: Apply both the workspace binding and appropriate Unity Catalog object grants, then test each dimension. The current case is governed more directly by: Column masking can transform sensitive values at query time according to the querying principal.
D:This principle concerns a separate data-engineering condition: A data steward wants to grant temporary read access without handing over Unity Catalog ownership of a schema. Its targeted response is: Issue time-bounded group membership or an approved scoped SELECT grant with an access review. The current case is governed more directly by: Column masking can transform sensitive values at query time according to the querying principal.
Question 465
The following control has NOT been implemented: Define a range expectation with metrics and the approved drop, fail or quarantine disposition. In this case: A bronze stream sometimes includes ages below zero or above 120 yet the transformation casts every value to integer. Which exposure remains?
- Null business keys can enter Delta targets and make updates or joins nondeterministic.
- Unclear expectation handling may allow financial errors to reach gold tables or stop all low-risk events unnecessarily.
- Silently coercing a breaking type can erase measurable values while a job appears successful.
- Numerically valid but impossible values can contaminate customer metrics and machine-learning features.
Explanation
Correct Answer
D:The direct residual exposure is: Numerically valid but impossible values can contaminate customer metrics and machine-learning features. The triggering condition is: A bronze stream sometimes includes ages below zero or above 120 yet the transformation casts every value to integer. The omitted safeguard is: Define a range expectation with metrics and the approved drop, fail or quarantine disposition. This matters because: Valid data type does not guarantee semantic range correctness. A corrective test would show: A controlled negative-age record triggers the configured expectation event without being silently published.
Incorrect Answers
A:This failure mode belongs mainly to another situation: An event pipeline must reject rows with a null transaction ID, but the data engineer checks only the row count after load. Its preventative action would be: Enforce a NOT NULL constraint or pipeline expectation on the transaction ID with a defined failure action. The most direct exposure in the current scenario is: Numerically valid but impossible values can contaminate customer metrics and machine-learning features.
B:This failure mode belongs mainly to another situation: A Lakeflow pipeline has a quality rule but nobody has decided whether invalid transactions should stop publishing. Its preventative action would be: Configure expectations with an explicit fail, drop or retention behavior aligned to business severity. The most direct exposure in the current scenario is: Numerically valid but impossible values can contaminate customer metrics and machine-learning features.
C:This failure mode belongs mainly to another situation: A source API changes a quantity from integer to nested JSON and the pipeline silently produces nulls downstream. Its preventative action would be: Apply schema enforcement with explicit drift handling and alert on incompatible structural changes. The most direct exposure in the current scenario is: Numerically valid but impossible values can contaminate customer metrics and machine-learning features.
Question 466
An implementation skips: Pin package and runtime versions in the governed release artifact and record its dependency manifest. Engineers are observing: A team cannot reproduce a prior release because package versions are resolved dynamically during deployment. What consequence is MOST likely?
- Skew and memory limits may remain invisible until a full-scale run misses its SLA.
- The same Git commit may run differently after a dependency's incompatible update.
- A deployment may succeed while the first production job cannot access its storage or schema.
- A logically incorrect aggregation can pass compilation while corrupting metrics.
Explanation
Correct Answer
B:The direct residual exposure is: The same Git commit may run differently after a dependency's incompatible update. The triggering condition is: A team cannot reproduce a prior release because package versions are resolved dynamically during deployment. The omitted safeguard is: Pin package and runtime versions in the governed release artifact and record its dependency manifest. This matters because: Reproducible deployments require versioned execution dependencies as well as application code. A corrective test would show: Rebuilding the prior release resolves identical pinned versions and passes the saved regression suite.
Incorrect Answers
A:This failure mode belongs mainly to another situation: An engineering test suite passes on miniature in-memory data yet fails in production on skewed billion-row joins. Its preventative action would be: Include performance and end-to-end tests using representative scale and data distributions. The most direct exposure in the current scenario is: The same Git commit may run differently after a dependency's incompatible update.
C:This failure mode belongs mainly to another situation: A new pipeline passes unit tests but production fails because the service principal lacks a catalog grant. Its preventative action would be: Include integration tests of identity, Unity Catalog permissions and destination connectivity in staging. The most direct exposure in the current scenario is: The same Git commit may run differently after a dependency's incompatible update.
D:This failure mode belongs mainly to another situation: A release modifies transformation logic but tests only whether Python files compile. Its preventative action would be: Add representative unit and integration tests asserting transformation results and schema contracts. The most direct exposure in the current scenario is: The same Git commit may run differently after a dependency's incompatible update.
Question 467
The team implements: Model a slowly changing dimension Type 2 with effective dates, current flags and correctly managed surrogate keys. The original problem is: A retail customer table changes delivery address repeatedly but analysts must reconstruct the address known at any prior invoice date. Which test MOST convincingly demonstrates correctness?
- A backdated invoice joins to the address valid on its transaction date while the current record shows the latest address.
- All required engines pass a controlled read and supported mutation matrix before production.
- Storage metrics show a manageable file distribution and point lookups meet query SLAs.
- A supported optimization command completes and query plans show the chosen layout works.
Explanation
Correct Answer
A:The strongest acceptance proof is: A backdated invoice joins to the address valid on its transaction date while the current record shows the latest address. It specifically verifies: Model a slowly changing dimension Type 2 with effective dates, current flags and correctly managed surrogate keys. The original issue was: A retail customer table changes delivery address repeatedly but analysts must reconstruct the address known at any prior invoice date. The underlying mechanism is: SCD Type 2 preserves historical attribute versions rather than overwriting the prior business state. Without it, using Type 1 updates may silently rewrite the apparent address on older transaction reports.
Incorrect Answers
B:This result would validate a different measure: Choose and document a supported format and catalog integration with explicit read/write compatibility tests. That is relevant to the distinct problem: A cross-engine consumer expects a Delta table but source is plain Iceberg and engineers assume identical mutation semantics. Here the acceptance criterion is: A backdated invoice joins to the address valid on its transaction date while the current record shows the latest address.
C:This result would validate a different measure: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. That is relevant to the distinct problem: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. Here the acceptance criterion is: A backdated invoice joins to the address valid on its transaction date while the current record shows the latest address.
D:This result would validate a different measure: Choose one compatible data layout strategy based on table format, platform support and query evidence. That is relevant to the distinct problem: A team attempts to apply both ZORDER and liquid clustering to the same Delta table. Here the acceptance criterion is: A backdated invoice joins to the address valid on its transaction date while the current record shows the latest address.
Question 468
The following control has NOT been implemented: Normalize time zone semantics before extracting business-local reporting dates. In this case: A date field mixes UTC timestamps and local store times but a pipeline groups all values by the raw date string. Which exposure remains?
- An unrestricted join can multiply keys and exaggerate customer counts.
- Directly summing mismatched month columns can produce fragile and incomplete queries.
- Invented defaults can distort regional regulatory or financial reporting.
- Orders near midnight may be reported under the wrong business day.
Explanation
Correct Answer
D:The direct residual exposure is: Orders near midnight may be reported under the wrong business day. The triggering condition is: A date field mixes UTC timestamps and local store times but a pipeline groups all values by the raw date string. The omitted safeguard is: Normalize time zone semantics before extracting business-local reporting dates. This matters because: Time zone conversion and calendar boundary choice affect the grouping of events. A corrective test would show: Known events around midnight map to the intended store-local date and correct UTC instant.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A team wants only customer IDs present in both systems but uses an inner join that multiplies repeated rows. Its preventative action would be: Use an appropriate intersect or semi-join operation based on set and duplicate requirements. The most direct exposure in the current scenario is: Orders near midnight may be reported under the wrong business day.
B:This failure mode belongs mainly to another situation: A large orders dataset has one column per month but a dashboard expects month-value rows for filtering and grouping. Its preventative action would be: Use a compatible unpivot or stack transformation with explicit month and amount columns. The most direct exposure in the current scenario is: Orders near midnight may be reported under the wrong business day.
C:This failure mode belongs mainly to another situation: A customer reference dataset has missing country codes but a pipeline replaces every null with an arbitrary default country. Its preventative action would be: Define field-specific missing-value rules and quarantine or enrich unknown values before assigning business meaning. The most direct exposure in the current scenario is: Orders near midnight may be reported under the wrong business day.
Question 469
For this Azure Databricks issue: A sensitive SQL view is renamed in the release but the team does not test downstream dashboards before promotion. Engineers choose: Run end-to-end contract tests for dependent readers and obtain owner approval for breaking changes. What acceptance evidence is BEST?
- Two identical deployment executions leave exactly one intended job with approved settings.
- A rehearsed rollback restores a prior tested bundle and recovers the expected job definitions.
- A staging dashboard loads its documented fields and a breaking-change simulation is rejected.
- A production run references an approved immutable commit and a prior version can be restored.
Explanation
Correct Answer
C:The strongest acceptance proof is: A staging dashboard loads its documented fields and a breaking-change simulation is rejected. It specifically verifies: Run end-to-end contract tests for dependent readers and obtain owner approval for breaking changes. The original issue was: A sensitive SQL view is renamed in the release but the team does not test downstream dashboards before promotion. The underlying mechanism is: Schema and object naming changes can break consumers even when new ETL succeeds. Without it, A release may finish successfully while critical reports cannot resolve their expected view.
Incorrect Answers
A:This result would validate a different measure: Use idempotent deployment methods or reconcile existing resource IDs before creating replacements. That is relevant to the distinct problem: A REST deployment script creates duplicate jobs every time CI reruns after a timeout. Here the acceptance criterion is: A staging dashboard loads its documented fields and a breaking-change simulation is rejected.
B:This result would validate a different measure: Tag and retain approved bundle releases and document a verified redeploy or rollback procedure. That is relevant to the distinct problem: An operations group wants to roll back a broken Lakeflow workflow but keeps no previous package version. Here the acceptance criterion is: A staging dashboard loads its documented fields and a breaking-change simulation is rejected.
D:This result would validate a different measure: Store deployment source in Git with reviewed branches and pull requests before promoting changes. That is relevant to the distinct problem: Engineers edit production notebooks directly and lose the ability to identify which code change broke a pipeline. Here the acceptance criterion is: A staging dashboard loads its documented fields and a breaking-change simulation is rejected.
Question 470
A Databricks engineer recommends: Apply a controlled pivot with a defined set of expected categories and aggregation rule. It responds to: A quality report needs one column for each product status category but stores status as separate rows. What underlying mechanism BEST explains why?
- Deduplication should use meaningful business keys, not arbitrary full-row equality alone.
- Pivoting transposes category values into columns but requires stable categories and aggregation semantics.
- Spark can reduce shuffle costs by broadcasting a sufficiently small dimension where appropriate.
- SQL NULL indicates absent or unknown values while numeric zero is a valid value.
Explanation
Correct Answer
B:The decisive mechanism is: Pivoting transposes category values into columns but requires stable categories and aggregation semantics. That explains the selected action: Apply a controlled pivot with a defined set of expected categories and aggregation rule. In the original scenario: A quality report needs one column for each product status category but stores status as separate rows. The specific unmanaged risk is: Unknown statuses may be dropped or duplicated if pivot categories are assumed without validation. Validate with: A sample includes all approved categories and status totals reconcile to source rows.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: An incoming orders table contains repeated order IDs because a vendor retransmits the same daily extract. Its targeted response is: Apply a deterministic deduplication rule based on stable order identity and source version before aggregation. The current case is governed more directly by: Pivoting transposes category values into columns but requires stable categories and aggregation semantics.
C:This principle concerns a separate data-engineering condition: A pipeline joins a huge fact table with a tiny dimension and a normal shuffle join creates excessive network traffic. Its targeted response is: Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans. The current case is governed more directly by: Pivoting transposes category values into columns but requires stable categories and aggregation semantics.
D:This principle concerns a separate data-engineering condition: A team filters out null quantities but needs to preserve rows with zero as valid inventory states. Its targeted response is: Implement explicit null predicates rather than treating zero and null as equivalent. The current case is governed more directly by: Pivoting transposes category values into columns but requires stable categories and aggregation semantics.