Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 51
Question 500
An implementation skips: Use Spark-native distributed read and transformation APIs or partition-safe processing. Engineers are observing: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. What consequence is MOST likely?
- A stream can write duplicate or malformed records into curated output without policy.
- Uncoordinated retries can run transformations before ingestion finishes or under the wrong identity.
- Driver out-of-memory can fail the load despite adequate executor resources.
- Ignoring changed source structures can halt ingestion or silently lose fields.
Explanation
Correct Answer
C:The direct residual exposure is: Driver out-of-memory can fail the load despite adequate executor resources. The triggering condition is: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. The omitted safeguard is: Use Spark-native distributed read and transformation APIs or partition-safe processing. This matters because: Pulling distributed data to a single driver defeats parallel execution and memory limits. A corrective test would show: The workload completes without driver collection and Spark metrics show distributed processing.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A declarative pipeline loads cloud object files continuously but its configuration omits how duplicate files and bad records are handled. Its preventative action would be: Define Lakeflow Spark Declarative Pipelines source options, incremental behavior and expectations with a durable target. The most direct exposure in the current scenario is: Driver out-of-memory can fail the load despite adequate executor resources.
B:This failure mode belongs mainly to another situation: A data product needs to copy a supported Azure Data Factory pipeline output into governed Databricks tables. Its preventative action would be: Coordinate ADF orchestration with the appropriate Databricks task and Unity Catalog access model. The most direct exposure in the current scenario is: Driver out-of-memory can fail the load despite adequate executor resources.
D:This failure mode belongs mainly to another situation: A file feed introduces an unexpected optional column and the stream fails after the first changed file. Its preventative action would be: Configure supported schema evolution or rescued-data handling with explicit compatibility review. The most direct exposure in the current scenario is: Driver out-of-memory can fail the load despite adequate executor resources.
Question 501
A team omits the recommended measure: Keep a governed raw or bronze representation before applying irreversible cleansing into silver and gold. Its environment shows: A bronze ingestion layer stores only final transformed values and loses raw source fields needed for incident replay. What is the MOST direct residual risk?
- Without recoverable raw inputs the team cannot replay a transformation after discovering a rule defect.
- Migrating data into a managed table can violate external storage governance agreements.
- Unneeded normalization can add joins while denormalization without governance can produce inconsistent descriptions.
- Duplicating physical dimensions can leave teams on inconsistent updates or retention periods.
Explanation
Correct Answer
A:The direct residual exposure is: Without recoverable raw inputs the team cannot replay a transformation after discovering a rule defect. The triggering condition is: A bronze ingestion layer stores only final transformed values and loses raw source fields needed for incident replay. The omitted safeguard is: Keep a governed raw or bronze representation before applying irreversible cleansing into silver and gold. This matters because: Layered lakehouse processing separates source fidelity from approved business transformations. A corrective test would show: A replay from a preserved source batch reproduces the expected silver and gold data after a code fix.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A dataset has a strict requirement that Azure storage ownership and lifecycle remain under a separate platform team. Its preventative action would be: Use an external Unity Catalog table with an approved registered location instead of assuming managed storage ownership. The most direct exposure in the current scenario is: Without recoverable raw inputs the team cannot replay a transformation after discovering a rule defect.
C:This failure mode belongs mainly to another situation: A customer table includes both region hierarchy and customers; teams disagree whether to snowflake descriptive attributes. Its preventative action would be: Evaluate dimensional normalization against query complexity, maintenance and expected cardinality. The most direct exposure in the current scenario is: Without recoverable raw inputs the team cannot replay a transformation after discovering a rule defect.
D:This failure mode belongs mainly to another situation: A data product creates multiple physical copies of a curated dimension because teams need different column names. Its preventative action would be: Expose stable SQL views over a governed base table when only presentation aliases differ. The most direct exposure in the current scenario is: Without recoverable raw inputs the team cannot replay a transformation after discovering a rule defect.
Question 502
The recommended measure is: Configure the Lakeflow Job schedule with the intended time zone and verify daylight-saving behavior where relevant. The operational context is: A business report must refresh daily at 06:00 in a defined local time zone but a developer schedules it in UTC without conversion. Which data engineering concept applies?
- Event-driven triggers align pipeline execution to upstream readiness where supported.
- Automatic recovery should be limited by failure budgets and should not hide persistent faults.
- Event or file-arrival triggers can reduce idle polling while maintaining responsive execution.
- Scheduler time zone semantics affect when recurring business windows run.
Explanation
Correct Answer
D:The decisive mechanism is: Scheduler time zone semantics affect when recurring business windows run. That explains the selected action: Configure the Lakeflow Job schedule with the intended time zone and verify daylight-saving behavior where relevant. In the original scenario: A business report must refresh daily at 06:00 in a defined local time zone but a developer schedules it in UTC without conversion. The specific unmanaged risk is: An implicit UTC schedule can cause delivery an hour or more away from the required business cutover. Validate with: Run history shows the job starts at the agreed local times across representative date transitions.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A pipeline must run when an upstream Databricks table is updated but analysts have hardcoded a daily time estimate. Its targeted response is: Evaluate supported table update triggers or a clear upstream job dependency based on actual availability. The current case is governed more directly by: Scheduler time zone semantics affect when recurring business windows run.
B:This principle concerns a separate data-engineering condition: A continuously running pipeline is scheduled to restart automatically but restart storms consume a growing compute budget. Its targeted response is: Configure bounded restarts and alerting with root-cause investigation for repeated failures. The current case is governed more directly by: Scheduler time zone semantics affect when recurring business windows run.
C:This principle concerns a separate data-engineering condition: A cloud file feed needs a job triggered by arriving files instead of polling every minute. Its targeted response is: Use a supported file-arrival trigger where its prerequisites and source location are compatible. The current case is governed more directly by: Scheduler time zone semantics affect when recurring business windows run.
Question 503
A production data platform experiences this condition: A customer reference dataset has missing country codes but a pipeline replaces every null with an arbitrary default country. Which targeted measure should be selected?
- Join historical dimension rows using business key and validity interval at event time.
- Define field-specific missing-value rules and quarantine or enrich unknown values before assigning business meaning.
- Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans.
- Normalize transaction sign semantics before grouping by business date and customer.
Explanation
Correct Answer
B:The most appropriate action is: Define field-specific missing-value rules and quarantine or enrich unknown values before assigning business meaning. It addresses the stated condition: A customer reference dataset has missing country codes but a pipeline replaces every null with an arbitrary default country. The technical basis is: Null remediation must preserve the difference between genuinely unknown data and a legitimate default. Without this control, invented defaults can distort regional regulatory or financial reporting. A direct acceptance check is: Rows with absent country are flagged for resolution and approved true defaults are applied consistently.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A clickstream fact table is joined to a customer table with multiple status versions but no effective-date condition. The corresponding measure there would be: Join historical dimension rows using business key and validity interval at event time. The stated situation instead calls for: Define field-specific missing-value rules and quarantine or enrich unknown values before assigning business meaning.
C:This alternative primarily addresses a different Azure Databricks problem: A pipeline joins a huge fact table with a tiny dimension and a normal shuffle join creates excessive network traffic. The corresponding measure there would be: Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans. The stated situation instead calls for: Define field-specific missing-value rules and quarantine or enrich unknown values before assigning business meaning.
D:This alternative primarily addresses a different Azure Databricks problem: A daily revenue report double-counts refunds because both debit and credit rows are summed as positive numbers. The corresponding measure there would be: Normalize transaction sign semantics before grouping by business date and customer. The stated situation instead calls for: Define field-specific missing-value rules and quarantine or enrich unknown values before assigning business meaning.
Question 504
The design team proposes: Use a secure service identity and protected credential handling for bundle or REST API deployment. The situation is: A CI pipeline deploys Databricks assets under an engineer's personal token stored in repository variables. Which technical principle justifies the decision?
- Conflict resolution must preserve intended behavior from both change sets.
- Automated releases should not depend on personal accounts or broadly exposed long-lived credentials.
- Correctness under small inputs does not establish workload performance at production cardinality.
- Version control and code review establish provenance and repeatable release history.
Explanation
Correct Answer
B:The decisive mechanism is: Automated releases should not depend on personal accounts or broadly exposed long-lived credentials. That explains the selected action: Use a secure service identity and protected credential handling for bundle or REST API deployment. In the original scenario: A CI pipeline deploys Databricks assets under an engineer's personal token stored in repository variables. The specific unmanaged risk is: An exposed or disabled personal token can compromise or halt all production deployments. Validate with: CI deploys under the named service principal and the personal token is rejected and revoked.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: Two engineers change the same ingestion function on separate branches and one silently overwrites the other's fix. Its targeted response is: Resolve Git merge conflicts with tests and reviewed integration before merging the release branch. The current case is governed more directly by: Automated releases should not depend on personal accounts or broadly exposed long-lived credentials.
C:This principle concerns a separate data-engineering condition: An engineering test suite passes on miniature in-memory data yet fails in production on skewed billion-row joins. Its targeted response is: Include performance and end-to-end tests using representative scale and data distributions. The current case is governed more directly by: Automated releases should not depend on personal accounts or broadly exposed long-lived credentials.
D:This principle concerns a separate data-engineering condition: Engineers edit production notebooks directly and lose the ability to identify which code change broke a pipeline. Its targeted response is: Store deployment source in Git with reviewed branches and pull requests before promoting changes. The current case is governed more directly by: Automated releases should not depend on personal accounts or broadly exposed long-lived credentials.
Question 505
A data engineering team faces this Azure Databricks requirement: A team exposes structured transaction data as files in a volume but wants SQL analytics and ACID MERGE behavior. Which action is MOST appropriate?
- Check federation connector capabilities and distinguish read-only remote access from supported writable local tables.
- Register the structured dataset as an appropriate managed Delta table rather than treating the volume itself as a SQL table.
- Use Unity Catalog volumes with suitable schema and file privileges for supported non-tabular files.
- Assign view ownership and narrowly scoped permissions under the appropriate Unity Catalog schema.
Explanation
Correct Answer
B:The most appropriate action is: Register the structured dataset as an appropriate managed Delta table rather than treating the volume itself as a SQL table. It addresses the stated condition: A team exposes structured transaction data as files in a volume but wants SQL analytics and ACID MERGE behavior. The technical basis is: Volumes organize files, whereas Delta tables offer tabular metadata and transactional query operations. Without this control, file-only storage may lack table-level transactional update and discoverability semantics. A direct acceptance check is: A controlled MERGE and SQL query succeed against the governed Delta table with expected records.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A foreign catalog is connected to a remote database but an analyst expects standard managed Delta writes against it. The corresponding measure there would be: Check federation connector capabilities and distinguish read-only remote access from supported writable local tables. The stated situation instead calls for: Register the structured dataset as an appropriate managed Delta table rather than treating the volume itself as a SQL table.
C:This alternative primarily addresses a different Azure Databricks problem: A departmental data owner stores raw file attachments inside ad hoc DBFS directories instead of a governable data space. The corresponding measure there would be: Use Unity Catalog volumes with suitable schema and file privileges for supported non-tabular files. The stated situation instead calls for: Register the structured dataset as an appropriate managed Delta table rather than treating the volume itself as a SQL table.
D:This alternative primarily addresses a different Azure Databricks problem: A central team creates global views that hide source objects but forgets to document ownership and access grants. The corresponding measure there would be: Assign view ownership and narrowly scoped permissions under the appropriate Unity Catalog schema. The stated situation instead calls for: Register the structured dataset as an appropriate managed Delta table rather than treating the volume itself as a SQL table.
Question 506
An Azure Databricks architect reviews the following problem: A team cannot reproduce a prior release because package versions are resolved dynamically during deployment. What should the team implement FIRST?
- Package declarative automation bundle configurations with environment-specific targets and deploy through the supported CLI.
- Pin package and runtime versions in the governed release artifact and record its dependency manifest.
- Include integration tests of identity, Unity Catalog permissions and destination connectivity in staging.
- Add representative unit and integration tests asserting transformation results and schema contracts.
Explanation
Correct Answer
B:The most appropriate action is: Pin package and runtime versions in the governed release artifact and record its dependency manifest. It addresses the stated condition: A team cannot reproduce a prior release because package versions are resolved dynamically during deployment. The technical basis is: Reproducible deployments require versioned execution dependencies as well as application code. Without this control, the same Git commit may run differently after a dependency's incompatible update. A direct acceptance check is: Rebuilding the prior release resolves identical pinned versions and passes the saved regression suite.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A team copies Databricks notebooks manually between three workspaces and each uses different hardcoded identifiers. The corresponding measure there would be: Package declarative automation bundle configurations with environment-specific targets and deploy through the supported CLI. The stated situation instead calls for: Pin package and runtime versions in the governed release artifact and record its dependency manifest.
C:This alternative primarily addresses a different Azure Databricks problem: A new pipeline passes unit tests but production fails because the service principal lacks a catalog grant. The corresponding measure there would be: Include integration tests of identity, Unity Catalog permissions and destination connectivity in staging. The stated situation instead calls for: Pin package and runtime versions in the governed release artifact and record its dependency manifest.
D:This alternative primarily addresses a different Azure Databricks problem: A release modifies transformation logic but tests only whether Python files compile. The corresponding measure there would be: Add representative unit and integration tests asserting transformation results and schema contracts. The stated situation instead calls for: Pin package and runtime versions in the governed release artifact and record its dependency manifest.
Question 507
A production data team leaves out: Develop an approved role-aware column mask and test its outputs across supported query clients. Its scenario is: A data product stores PII in a shared Delta table but several reports require different masking for each audience. Which failure mode is MOST relevant?
- An orphaned ownership chain can delay access review or emergency remediation.
- Sharing raw transaction tables can disclose protected customer attributes unnecessarily.
- Old cleanup tasks can destroy files that the revised retention requirement still protects.
- Independent report-level masking may be bypassed by direct SQL access to the source.
Explanation
Correct Answer
D:The direct residual exposure is: Independent report-level masking may be bypassed by direct SQL access to the source. The triggering condition is: A data product stores PII in a shared Delta table but several reports require different masking for each audience. The omitted safeguard is: Develop an approved role-aware column mask and test its outputs across supported query clients. This matters because: Column mask policies can enforce context-dependent presentation without duplicating raw PII. A corrective test would show: Authorized privacy reviewers see full test values while ordinary report roles see only permitted redactions.
Incorrect Answers
A:This failure mode belongs mainly to another situation: An audit asks who owns a catalog table but object ownership still points to an inactive employee. Its preventative action would be: Transfer ownership through approved governance workflow and review grants after reassignment. The most direct exposure in the current scenario is: Independent report-level masking may be bypassed by direct SQL access to the source.
B:This failure mode belongs mainly to another situation: A customer wants to share only a daily refreshed aggregate, not raw personally identifiable transactions. Its preventative action would be: Publish a restricted derived dataset through Delta Sharing after privacy and freshness validation. The most direct exposure in the current scenario is: Independent report-level masking may be bypassed by direct SQL access to the source.
C:This failure mode belongs mainly to another situation: A regulated source dataset receives an updated retention period but storage cleanup still follows the obsolete policy. Its preventative action would be: Align table retention properties, scheduled maintenance and legal ownership approvals with the current classification. The most direct exposure in the current scenario is: Independent report-level masking may be bypassed by direct SQL access to the source.
Question 508
A team omits the recommended measure: Assign GPU-capable compute only to supported GPU workloads and use CPU compute for ordinary transformations. Its environment shows: A feature-engineering notebook requires GPU libraries but the data cleaning jobs never use accelerators. What is the MOST direct residual risk?
- Overspending on memory does not relieve a CPU-bound stage and may worsen the cost-performance ratio.
- A serverless migration may fail when a required library or environment customization cannot be installed.
- A default GPU cluster may charge premium rates while most tasks leave accelerators unused.
- Unrestricted cluster creation can create unpredictable spending and inconsistent operational standards.
Explanation
Correct Answer
C:The direct residual exposure is: A default GPU cluster may charge premium rates while most tasks leave accelerators unused. The triggering condition is: A feature-engineering notebook requires GPU libraries but the data cleaning jobs never use accelerators. The omitted safeguard is: Assign GPU-capable compute only to supported GPU workloads and use CPU compute for ordinary transformations. This matters because: GPU resources are specialized and do not automatically accelerate every Spark or SQL transformation. A corrective test would show: Resource telemetry shows accelerator use where needed and CPU jobs satisfy their SLA at lower cost.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A new job uses a large memory-optimized node merely because its source data is large, although profiling shows compute-bound transforms. Its preventative action would be: Choose node families and driver/worker sizes using CPU, memory, shuffle and workload profile evidence. The most direct exposure in the current scenario is: A default GPU cluster may charge premium rates while most tasks leave accelerators unused.
B:This failure mode belongs mainly to another situation: A long-running job using an unsupported custom system dependency is moved to serverless without checking feature compatibility. Its preventative action would be: Review current serverless limitations and select compatible classic jobs compute when necessary. The most direct exposure in the current scenario is: A default GPU cluster may charge premium rates while most tasks leave accelerators unused.
D:This failure mode belongs mainly to another situation: A department wants a self-service sandbox but must prevent users from creating huge unrestricted clusters. Its preventative action would be: Create compute policies with approved node types, worker limits, tags and idle settings for the permitted users. The most direct exposure in the current scenario is: A default GPU cluster may charge premium rates while most tasks leave accelerators unused.
Question 509
A team omits the recommended measure: Assign view ownership and narrowly scoped permissions under the appropriate Unity Catalog schema. Its environment shows: A central team creates global views that hide source objects but forgets to document ownership and access grants. What is the MOST direct residual risk?
- An unqualified table reference may redirect a production job to a developer's test dataset.
- An ownerless shared view may expose unreviewed columns or become impossible to maintain safely.
- An environment-default schema can route production data definitions into an incorrect catalog.
- Credentials embedded in notebooks can be copied and used outside the intended catalog authorization.
Explanation
Correct Answer
B:The direct residual exposure is: An ownerless shared view may expose unreviewed columns or become impossible to maintain safely. The triggering condition is: A central team creates global views that hide source objects but forgets to document ownership and access grants. The omitted safeguard is: Assign view ownership and narrowly scoped permissions under the appropriate Unity Catalog schema. This matters because: Governance of views includes ownership, discovery and authorization just as for tables. A corrective test would show: An access review identifies the view steward and confirms a prohibited reader cannot query it.
Incorrect Answers
A:This failure mode belongs mainly to another situation: Production and development teams create identically named data objects in one catalog and accidentally overwrite test tables. Its preventative action would be: Create environment-specific Unity Catalog catalogs or schemas and enforce clear ownership and naming conventions. The most direct exposure in the current scenario is: An ownerless shared view may expose unreviewed columns or become impossible to maintain safely.
C:This failure mode belongs mainly to another situation: An automation pipeline issues CREATE TABLE without qualifying the target catalog and schema. Its preventative action would be: Use fully qualified names and a controlled default namespace in deployment SQL and notebooks. The most direct exposure in the current scenario is: An ownerless shared view may expose unreviewed columns or become impossible to maintain safely.
D:This failure mode belongs mainly to another situation: A legacy external Delta table points at a cloud location with an unmanaged storage credential stored in notebook text. Its preventative action would be: Register supported external locations and storage credentials, then define governed external tables. The most direct exposure in the current scenario is: An ownerless shared view may expose unreviewed columns or become impossible to maintain safely.
Question 510
An implementation skips: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. Engineers are observing: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. What consequence is MOST likely?
- Excessive partitions create small files and poor query planning without useful pruning.
- Duplicating physical dimensions can leave teams on inconsistent updates or retention periods.
- One-file-per-customer patterns can overwhelm metadata and worsen query startup.
- Migrating data into a managed table can violate external storage governance agreements.
Explanation
Correct Answer
C:The direct residual exposure is: One-file-per-customer patterns can overwhelm metadata and worsen query startup. The triggering condition is: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. The omitted safeguard is: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. This matters because: Partition pruning benefits can be outweighed by extremely fragmented high-cardinality directory layouts. A corrective test would show: Storage metrics show a manageable file distribution and point lookups meet query SLAs.
Incorrect Answers
A:This failure mode belongs mainly to another situation: An event table is partitioned by the unique transaction identifier and creates millions of tiny partitions. Its preventative action would be: Use coarse suitable partitioning or liquid clustering selected from filter patterns and cardinality. The most direct exposure in the current scenario is: One-file-per-customer patterns can overwhelm metadata and worsen query startup.
B:This failure mode belongs mainly to another situation: A data product creates multiple physical copies of a curated dimension because teams need different column names. Its preventative action would be: Expose stable SQL views over a governed base table when only presentation aliases differ. The most direct exposure in the current scenario is: One-file-per-customer patterns can overwhelm metadata and worsen query startup.
D:This failure mode belongs mainly to another situation: A dataset has a strict requirement that Azure storage ownership and lifecycle remain under a separate platform team. Its preventative action would be: Use an external Unity Catalog table with an approved registered location instead of assuming managed storage ownership. The most direct exposure in the current scenario is: One-file-per-customer patterns can overwhelm metadata and worsen query startup.