Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 56
Question 550
A solution applies: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys. It addresses: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. Which operational outcome MOST directly verifies success?
- File-count and query-profile comparisons show fewer fragments and faster targeted scans.
- The external reader loads approved columns and a documented compatibility test passes.
- A measured workload benchmark shows scan reduction across the highest-value query templates.
- The top event-time queries show improved data skipping and reconciled row counts.
Explanation
Correct Answer
C:The strongest acceptance proof is: A measured workload benchmark shows scan reduction across the highest-value query templates. It specifically verifies: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys. The original issue was: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. The underlying mechanism is: Efficient layout keys are driven by actual access patterns and file statistics. Without it, poorly chosen keys increase full scans and maintenance overhead as traffic grows.
Incorrect Answers
A:This result would validate a different measure: Use coarse suitable partitioning or liquid clustering selected from filter patterns and cardinality. That is relevant to the distinct problem: An event table is partitioned by the unique transaction identifier and creates millions of tiny partitions. Here the acceptance criterion is: A measured workload benchmark shows scan reduction across the highest-value query templates.
B:This result would validate a different measure: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements. That is relevant to the distinct problem: An external sharing integration needs interoperable columnar read files but not native Delta updates. Here the acceptance criterion is: A measured workload benchmark shows scan reduction across the highest-value query templates.
D:This result would validate a different measure: Reevaluate data layout using representative event-time filters and a suitable clustering key. That is relevant to the distinct problem: An event stream is partitioned by ingestion day but analysts constantly filter by event timestamp ranges crossing midnight. Here the acceptance criterion is: A measured workload benchmark shows scan reduction across the highest-value query templates.
Question 551
The design team proposes: Use a dedicated service principal with scoped Unity Catalog grants and governed credential lifecycle. The situation is: An ETL service principal is granted permissions through a personal employee account that will leave the company. Which technical principle justifies the decision?
- Cloud IAM and Unity Catalog are complementary authorization layers that must both permit the intended operation.
- Automation authentication should be auditable, nonpersonal and resistant to repository exposure.
- Separation of duties requires independent decisions for object administration, data modification and read access.
- Workload identities separate automated access from a person's employment and session lifecycle.
Explanation
Correct Answer
D:The decisive mechanism is: Workload identities separate automated access from a person's employment and session lifecycle. That explains the selected action: Use a dedicated service principal with scoped Unity Catalog grants and governed credential lifecycle. In the original scenario: An ETL service principal is granted permissions through a personal employee account that will leave the company. The specific unmanaged risk is: An employee departure or password rotation can unexpectedly break critical production ingestion. Validate with: The pipeline authenticates using the designated service principal and operates after the employee account is disabled.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A managed identity can read a storage container but a newly created external table is denied in Databricks. Its targeted response is: Align Azure storage role assignments with the registered Unity Catalog storage credential and external-location privileges. The current case is governed more directly by: Workload identities separate automated access from a person's employment and session lifecycle.
B:This principle concerns a separate data-engineering condition: A production workload uses a personal access token stored in Git for Databricks REST calls. Its targeted response is: Replace committed personal credentials with supported workload authentication and revoke exposed tokens. The current case is governed more directly by: Workload identities separate automated access from a person's employment and session lifecycle.
C:This principle concerns a separate data-engineering condition: A development team grants ALL PRIVILEGES on a catalog so that one job can create a temporary table. Its targeted response is: Replace broad grants with documented minimum creation and use privileges in a dedicated schema. The current case is governed more directly by: Workload identities separate automated access from a person's employment and session lifecycle.
Question 552
The team chooses: Apply type, nullability and range expectations before promoting bronze rows into the silver target. Its observed issue is: A product data contract requires every transaction amount to use nonnegative decimal values but new raw files include quoted text strings. What is the strongest technical basis?
- Explicit quality constraints protect critical key semantics rather than relying on aggregate row totals.
- Layered quality gates prevent malformed source values from contaminating trusted downstream tables.
- Type checks and temporal semantics must be aligned to prevent incorrect time-series results.
- Schema evolution must distinguish permissible additive fields from breaking type changes.
Explanation
Correct Answer
B:The decisive mechanism is: Layered quality gates prevent malformed source values from contaminating trusted downstream tables. That explains the selected action: Apply type, nullability and range expectations before promoting bronze rows into the silver target. In the original scenario: A product data contract requires every transaction amount to use nonnegative decimal values but new raw files include quoted text strings. The specific unmanaged risk is: Invalid amounts may silently convert to null and skew revenue aggregates. Validate with: A controlled malformed amount is detected and excluded from published silver while legitimate zero values remain.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: An event pipeline must reject rows with a null transaction ID, but the data engineer checks only the row count after load. Its targeted response is: Enforce a NOT NULL constraint or pipeline expectation on the transaction ID with a defined failure action. The current case is governed more directly by: Layered quality gates prevent malformed source values from contaminating trusted downstream tables.
C:This principle concerns a separate data-engineering condition: A regulated dataset requires all timestamps to parse as UTC but several rows have ambiguous local values. Its targeted response is: Validate timezone-aware timestamp types and quarantine ambiguous or unparseable event values. The current case is governed more directly by: Layered quality gates prevent malformed source values from contaminating trusted downstream tables.
D:This principle concerns a separate data-engineering condition: A source API changes a quantity from integer to nested JSON and the pipeline silently produces nulls downstream. Its targeted response is: Apply schema enforcement with explicit drift handling and alert on incompatible structural changes. The current case is governed more directly by: Layered quality gates prevent malformed source values from contaminating trusted downstream tables.
Question 553
The design team proposes: Define and test consistent trimming, Unicode normalization and case comparison for the business key. The situation is: A customer-name cleansing script uses an arbitrary lowercase conversion without preserving Unicode and leading/trailing whitespace rules. Which technical principle justifies the decision?
- Null remediation must preserve the difference between genuinely unknown data and a legitimate default.
- Text normalization must be deterministic and tailored to identifier semantics.
- Set difference helps identify migration gaps without retrieving matched records.
- Aggregations require correctly modeled financial direction and data types.
Explanation
Correct Answer
B:The decisive mechanism is: Text normalization must be deterministic and tailored to identifier semantics. That explains the selected action: Define and test consistent trimming, Unicode normalization and case comparison for the business key. In the original scenario: A customer-name cleansing script uses an arbitrary lowercase conversion without preserving Unicode and leading/trailing whitespace rules. The specific unmanaged risk is: Inconsistent name normalization can create duplicate customers or merge distinct identities incorrectly. Validate with: Mixed-case, whitespace and non-ASCII test records resolve according to the documented mapping policy.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A customer reference dataset has missing country codes but a pipeline replaces every null with an arbitrary default country. Its targeted response is: Define field-specific missing-value rules and quarantine or enrich unknown values before assigning business meaning. The current case is governed more directly by: Text normalization must be deterministic and tailored to identifier semantics.
C:This principle concerns a separate data-engineering condition: An analyst must find records present in the source system but absent in a target migration. Its targeted response is: Use a left anti-join or EXCEPT operation with aligned schemas and explicit null semantics. The current case is governed more directly by: Text normalization must be deterministic and tailored to identifier semantics.
D:This principle concerns a separate data-engineering condition: A daily revenue report double-counts refunds because both debit and credit rows are summed as positive numbers. Its targeted response is: Normalize transaction sign semantics before grouping by business date and customer. The current case is governed more directly by: Text normalization must be deterministic and tailored to identifier semantics.
Question 554
A Databricks engineer recommends: Define a range expectation with metrics and the approved drop, fail or quarantine disposition. It responds to: A bronze stream sometimes includes ages below zero or above 120 yet the transformation casts every value to integer. What underlying mechanism BEST explains why?
- Layered quality gates prevent malformed source values from contaminating trusted downstream tables.
- Explicit quality constraints protect critical key semantics rather than relying on aggregate row totals.
- Valid data type does not guarantee semantic range correctness.
- Schema evolution must distinguish permissible additive fields from breaking type changes.
Explanation
Correct Answer
C:The decisive mechanism is: Valid data type does not guarantee semantic range correctness. That explains the selected action: Define a range expectation with metrics and the approved drop, fail or quarantine disposition. In the original scenario: A bronze stream sometimes includes ages below zero or above 120 yet the transformation casts every value to integer. The specific unmanaged risk is: Numerically valid but impossible values can contaminate customer metrics and machine-learning features. Validate with: A controlled negative-age record triggers the configured expectation event without being silently published.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A product data contract requires every transaction amount to use nonnegative decimal values but new raw files include quoted text strings. Its targeted response is: Apply type, nullability and range expectations before promoting bronze rows into the silver target. The current case is governed more directly by: Valid data type does not guarantee semantic range correctness.
B:This principle concerns a separate data-engineering condition: An event pipeline must reject rows with a null transaction ID, but the data engineer checks only the row count after load. Its targeted response is: Enforce a NOT NULL constraint or pipeline expectation on the transaction ID with a defined failure action. The current case is governed more directly by: Valid data type does not guarantee semantic range correctness.
D:This principle concerns a separate data-engineering condition: A source API changes a quantity from integer to nested JSON and the pipeline silently produces nulls downstream. Its targeted response is: Apply schema enforcement with explicit drift handling and alert on incompatible structural changes. The current case is governed more directly by: Valid data type does not guarantee semantic range correctness.
Question 555
A pipeline owner declines the action: Review and revoke stale grants and any linked service credentials, preserving required internal ownership. The business situation is: A vendor's service principal was granted global access for a one-month integration and the project ended. What specific failure can follow?
- An object may be available to overprivileged identities in unintended contexts if either boundary is ignored.
- A user may bypass the masked presentation by reading the unmasked source table.
- A retired vendor workload can continue reading business data through unchanged credentials.
- Granting SELECT alone may not make a catalog object usable and lead to excessive troubleshooting grants.
Explanation
Correct Answer
C:The direct residual exposure is: A retired vendor workload can continue reading business data through unchanged credentials. The triggering condition is: A vendor's service principal was granted global access for a one-month integration and the project ended. The omitted safeguard is: Review and revoke stale grants and any linked service credentials, preserving required internal ownership. This matters because: Least privilege includes removing access when the business purpose expires. A corrective test would show: After revocation the vendor principal cannot list or query protected objects while internal jobs remain healthy.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A team restricts access through a catalog-to-workspace binding but treats it as a replacement for data privileges. Its preventative action would be: Apply both the workspace binding and appropriate Unity Catalog object grants, then test each dimension. The most direct exposure in the current scenario is: A retired vendor workload can continue reading business data through unchanged credentials.
B:This failure mode belongs mainly to another situation: A data owner builds a masked view but analysts can still query the underlying sensitive table directly. Its preventative action would be: Revoke inappropriate base-table access and publish only the authorized view or policy-governed object. The most direct exposure in the current scenario is: A retired vendor workload can continue reading business data through unchanged credentials.
D:This failure mode belongs mainly to another situation: A contractor receives SELECT on a table yet gets an access-denied error when querying with a three-part name. Its preventative action would be: Check required parent catalog and schema usage privileges alongside the table SELECT grant. The most direct exposure in the current scenario is: A retired vendor workload can continue reading business data through unchanged credentials.
Question 556
The recommended measure is: Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans. The operational context is: A pipeline joins a huge fact table with a tiny dimension and a normal shuffle join creates excessive network traffic. Which data engineering concept applies?
- Spark can reduce shuffle costs by broadcasting a sufficiently small dimension where appropriate.
- Union operations concatenate rows and require aligned column meanings.
- Filter placement and join predicates determine which rows contribute to a grouped measure.
- Append is generally simpler for immutable events than key-by-key mutation logic.
Explanation
Correct Answer
A:The decisive mechanism is: Spark can reduce shuffle costs by broadcasting a sufficiently small dimension where appropriate. That explains the selected action: Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans. In the original scenario: A pipeline joins a huge fact table with a tiny dimension and a normal shuffle join creates excessive network traffic. The specific unmanaged risk is: Blindly broadcasting a large table can exhaust executor memory while unnecessary shuffles slow joins. Validate with: Spark UI shows reduced shuffle bytes and matching results without executor out-of-memory.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: Two quarterly sales feeds have the same logical fields but different column ordering and a union shifts amounts into dates. Its targeted response is: Align compatible schemas and use name-based union semantics with explicit type checks. The current case is governed more directly by: Spark can reduce shuffle costs by broadcasting a sufficiently small dimension where appropriate.
C:This principle concerns a separate data-engineering condition: An analyst calculates total revenue grouped by country but a join includes countries not approved for reporting. Its targeted response is: Apply the required filters and join conditions before aggregation and validate authorized output groups. The current case is governed more directly by: Spark can reduce shuffle costs by broadcasting a sufficiently small dimension where appropriate.
D:This principle concerns a separate data-engineering condition: An append-only telemetry feed includes newly observed events but engineers use MERGE with expensive full-table scans on every insert. Its targeted response is: Use an incremental append pattern with suitable deduplication when source events cannot change. The current case is governed more directly by: Spark can reduce shuffle costs by broadcasting a sufficiently small dimension where appropriate.
Question 557
An implementation skips: Limit retries to appropriate transient errors and set accountable failure handling for permanent logic problems. Engineers are observing: A job's retry policy restarts a deterministic bad SQL query six times even though the table is missing. What consequence is MOST likely?
- A missed overnight job can remain unnoticed until business users complain.
- Repeated invalid retries consume compute and obscure the original fixable error.
- Misunderstood skip conditions can leave expected cleanup or alerts unexecuted.
- Constant scheduling may incur empty-run costs and still miss arrivals between checks.
Explanation
Correct Answer
B:The direct residual exposure is: Repeated invalid retries consume compute and obscure the original fixable error. The triggering condition is: A job's retry policy restarts a deterministic bad SQL query six times even though the table is missing. The omitted safeguard is: Limit retries to appropriate transient errors and set accountable failure handling for permanent logic problems. This matters because: Retries cannot repair missing tables or invalid schema references without corrective action. A corrective test would show: A missing-table test stops with the expected error while a transient failure recovers according to policy.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A scheduled Lakeflow Job has no owner alert when its final task fails overnight. Its preventative action would be: Configure job failure notifications to approved recipients and validate incident routing. The most direct exposure in the current scenario is: Repeated invalid retries consume compute and obscure the original fixable error.
C:This failure mode belongs mainly to another situation: A job owner cannot explain why a task skipped execution after a predecessor failed. Its preventative action would be: Review explicit dependency and run-if settings and validate skip behavior through a staged failure. The most direct exposure in the current scenario is: Repeated invalid retries consume compute and obscure the original fixable error.
D:This failure mode belongs mainly to another situation: A cloud file feed needs a job triggered by arriving files instead of polling every minute. Its preventative action would be: Use a supported file-arrival trigger where its prerequisites and source location are compatible. The most direct exposure in the current scenario is: Repeated invalid retries consume compute and obscure the original fixable error.
Question 558
An implementation skips: Apply a deterministic deduplication rule based on stable order identity and source version before aggregation. Engineers are observing: An incoming orders table contains repeated order IDs because a vendor retransmits the same daily extract. What consequence is MOST likely?
- Completion metrics may be inflated or undercounted when duplicate business events are not resolved.
- Duplicate orders inflate sales totals and can trigger incorrect downstream payouts.
- A simple key join duplicates click events and misattributes prior users.
- Refund amounts treated as sales inflate net revenue even when record counts match.
Explanation
Correct Answer
B:The direct residual exposure is: Duplicate orders inflate sales totals and can trigger incorrect downstream payouts. The triggering condition is: An incoming orders table contains repeated order IDs because a vendor retransmits the same daily extract. The omitted safeguard is: Apply a deterministic deduplication rule based on stable order identity and source version before aggregation. This matters because: Deduplication should use meaningful business keys, not arbitrary full-row equality alone. A corrective test would show: A seeded repeated-ID sample yields one chosen current order version and reconciled totals.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A student dataset uses SQL GROUP BY with unhandled duplicate enrollment rows before reporting course completion. Its preventative action would be: Validate business keys and choose a deterministic latest-enrollment record before aggregation. The most direct exposure in the current scenario is: Duplicate orders inflate sales totals and can trigger incorrect downstream payouts.
C:This failure mode belongs mainly to another situation: A clickstream fact table is joined to a customer table with multiple status versions but no effective-date condition. Its preventative action would be: Join historical dimension rows using business key and validity interval at event time. The most direct exposure in the current scenario is: Duplicate orders inflate sales totals and can trigger incorrect downstream payouts.
D:This failure mode belongs mainly to another situation: A daily revenue report double-counts refunds because both debit and credit rows are summed as positive numbers. Its preventative action would be: Normalize transaction sign semantics before grouping by business date and customer. The most direct exposure in the current scenario is: Duplicate orders inflate sales totals and can trigger incorrect downstream payouts.
Question 559
A production data platform experiences this condition: An interactive warehouse has high query queues during the morning but low CPU utilization after lunch. Which targeted measure should be selected?
- Keep large results distributed and replace driver collection with bounded sampling or distributed writes.
- Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages.
- Export relevant logs and metrics through a supported Log Analytics or Azure Monitor integration with controlled retention.
- Define meaningful latency, error and backlog thresholds with alert routing based on the business SLO.
Explanation
Correct Answer
B:The most appropriate action is: Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages. It addresses the stated condition: An interactive warehouse has high query queues during the morning but low CPU utilization after lunch. The technical basis is: Concurrent demand peaks can drive queue delays independently of average compute utilization. Without this control, A warehouse sized to average load may breach dashboard p95 latency during business opening. A direct acceptance check is: A peak-concurrency replay meets latency and queue targets with recorded cost.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A Spark join reads large inputs but the driver reports failures after an analyst calls collect() on the result. The corresponding measure there would be: Keep large results distributed and replace driver collection with bounded sampling or distributed writes. The stated situation instead calls for: Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages.
C:This alternative primarily addresses a different Azure Databricks problem: An Azure Databricks pipeline has recurring errors but its logs remain only in a short-lived cluster session. The corresponding measure there would be: Export relevant logs and metrics through a supported Log Analytics or Azure Monitor integration with controlled retention. The stated situation instead calls for: Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages.
D:This alternative primarily addresses a different Azure Databricks problem: An alert triggers on every harmless microbatch delay while real hour-long ingestion stalls go unnoticed. The corresponding measure there would be: Define meaningful latency, error and backlog thresholds with alert routing based on the business SLO. The stated situation instead calls for: Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages.
Question 560
A data engineering team faces this Azure Databricks requirement: A newly onboarded product team wants to create its own tables but should not have administration rights over every catalog. Which action is MOST appropriate?
- Create environment-specific Unity Catalog catalogs or schemas and enforce clear ownership and naming conventions.
- Use fully qualified names and a controlled default namespace in deployment SQL and notebooks.
- Create a team-specific schema with ownership or delegated creation grants limited to that namespace.
- Adopt a portable naming and quoting convention compatible with Unity Catalog SQL identifiers.
Explanation
Correct Answer
C:The most appropriate action is: Create a team-specific schema with ownership or delegated creation grants limited to that namespace. It addresses the stated condition: A newly onboarded product team wants to create its own tables but should not have administration rights over every catalog. The technical basis is: Delegating within a governed schema supports self-service without tenant-wide administrative access. Without this control, global privileges may allow a team to alter or remove other products' data assets. A direct acceptance check is: The product service principal creates in its own schema and fails the same operation in unrelated schemas.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: Production and development teams create identically named data objects in one catalog and accidentally overwrite test tables. The corresponding measure there would be: Create environment-specific Unity Catalog catalogs or schemas and enforce clear ownership and naming conventions. The stated situation instead calls for: Create a team-specific schema with ownership or delegated creation grants limited to that namespace.
B:This alternative primarily addresses a different Azure Databricks problem: An automation pipeline issues CREATE TABLE without qualifying the target catalog and schema. The corresponding measure there would be: Use fully qualified names and a controlled default namespace in deployment SQL and notebooks. The stated situation instead calls for: Create a team-specific schema with ownership or delegated creation grants limited to that namespace.
D:This alternative primarily addresses a different Azure Databricks problem: An ingestion team creates a table with a reserved or inconsistent name and automation fails in one environment. The corresponding measure there would be: Adopt a portable naming and quoting convention compatible with Unity Catalog SQL identifiers. The stated situation instead calls for: Create a team-specific schema with ownership or delegated creation grants limited to that namespace.