Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 48
Question 470
A Databricks engineer recommends: Apply a controlled pivot with a defined set of expected categories and aggregation rule. It responds to: A quality report needs one column for each product status category but stores status as separate rows. What underlying mechanism BEST explains why?
- Deduplication should use meaningful business keys, not arbitrary full-row equality alone.
- Pivoting transposes category values into columns but requires stable categories and aggregation semantics.
- Spark can reduce shuffle costs by broadcasting a sufficiently small dimension where appropriate.
- SQL NULL indicates absent or unknown values while numeric zero is a valid value.
Explanation
Correct Answer
B:The decisive mechanism is: Pivoting transposes category values into columns but requires stable categories and aggregation semantics. That explains the selected action: Apply a controlled pivot with a defined set of expected categories and aggregation rule. In the original scenario: A quality report needs one column for each product status category but stores status as separate rows. The specific unmanaged risk is: Unknown statuses may be dropped or duplicated if pivot categories are assumed without validation. Validate with: A sample includes all approved categories and status totals reconcile to source rows.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: An incoming orders table contains repeated order IDs because a vendor retransmits the same daily extract. Its targeted response is: Apply a deterministic deduplication rule based on stable order identity and source version before aggregation. The current case is governed more directly by: Pivoting transposes category values into columns but requires stable categories and aggregation semantics.
C:This principle concerns a separate data-engineering condition: A pipeline joins a huge fact table with a tiny dimension and a normal shuffle join creates excessive network traffic. Its targeted response is: Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans. The current case is governed more directly by: Pivoting transposes category values into columns but requires stable categories and aggregation semantics.
D:This principle concerns a separate data-engineering condition: A team filters out null quantities but needs to preserve rows with zero as valid inventory states. Its targeted response is: Implement explicit null predicates rather than treating zero and null as equivalent. The current case is governed more directly by: Pivoting transposes category values into columns but requires stable categories and aggregation semantics.
Question 471
The following control has NOT been implemented: Add user acceptance tests for the relevant reporting contract before production promotion. In this case: A developer changes a transformation with a user-visible business effect but omits acceptance testing by the data product owner. Which exposure remains?
- Unreviewed overwrites can remove validation or idempotency logic from the deployed pipeline.
- A release may finish successfully while critical reports cannot resolve their expected view.
- An exposed or disabled personal token can compromise or halt all production deployments.
- A numerically correct but semantically changed field can mislead business users.
Explanation
Correct Answer
D:The direct residual exposure is: A numerically correct but semantically changed field can mislead business users. The triggering condition is: A developer changes a transformation with a user-visible business effect but omits acceptance testing by the data product owner. The omitted safeguard is: Add user acceptance tests for the relevant reporting contract before production promotion. This matters because: Technical tests cannot replace business approval of the reported meaning of data. A corrective test would show: A product owner signs off a sample report and its reconciled acceptance criteria before rollout.
Incorrect Answers
A:This failure mode belongs mainly to another situation: Two engineers change the same ingestion function on separate branches and one silently overwrites the other's fix. Its preventative action would be: Resolve Git merge conflicts with tests and reviewed integration before merging the release branch. The most direct exposure in the current scenario is: A numerically correct but semantically changed field can mislead business users.
B:This failure mode belongs mainly to another situation: A sensitive SQL view is renamed in the release but the team does not test downstream dashboards before promotion. Its preventative action would be: Run end-to-end contract tests for dependent readers and obtain owner approval for breaking changes. The most direct exposure in the current scenario is: A numerically correct but semantically changed field can mislead business users.
C:This failure mode belongs mainly to another situation: A CI pipeline deploys Databricks assets under an engineer's personal token stored in repository variables. Its preventative action would be: Use a secure service identity and protected credential handling for bundle or REST API deployment. The most direct exposure in the current scenario is: A numerically correct but semantically changed field can mislead business users.
Question 472
An operator introduces: Configure job failure notifications to approved recipients and validate incident routing. The situation is: A scheduled Lakeflow Job has no owner alert when its final task fails overnight. Which validation should be required before signoff?
- A staged task failure produces an actionable notification with job ID and run link.
- A missing-table test stops with the expected error while a transient failure recovers according to policy.
- Repeated injected failures hit the configured stop threshold and create a human-actionable incident.
- A scheduled run succeeds under the named service principal after the former owner account is disabled.
Explanation
Correct Answer
A:The strongest acceptance proof is: A staged task failure produces an actionable notification with job ID and run link. It specifically verifies: Configure job failure notifications to approved recipients and validate incident routing. The original issue was: A scheduled Lakeflow Job has no owner alert when its final task fails overnight. The underlying mechanism is: Operational scheduling without actionable failure reporting is not sufficient for production reliability. Without it, A missed overnight job can remain unnoticed until business users complain.
Incorrect Answers
B:This result would validate a different measure: Limit retries to appropriate transient errors and set accountable failure handling for permanent logic problems. That is relevant to the distinct problem: A job's retry policy restarts a deterministic bad SQL query six times even though the table is missing. Here the acceptance criterion is: A staged task failure produces an actionable notification with job ID and run link.
C:This result would validate a different measure: Configure bounded restarts and alerting with root-cause investigation for repeated failures. That is relevant to the distinct problem: A continuously running pipeline is scheduled to restart automatically but restart storms consume a growing compute budget. Here the acceptance criterion is: A staged task failure produces an actionable notification with job ID and run link.
D:This result would validate a different measure: Assign a dedicated run-as identity with durable scoped grants and an ownership review. That is relevant to the distinct problem: A job uses a service principal that loses access to the destination schema when an employee leaves. Here the acceptance criterion is: A staged task failure produces an actionable notification with job ID and run link.
Question 473
A production data team leaves out: Compare table-level masks against centrally governed ABAC coverage and identify missing assignments. Its scenario is: A user creates a column mask on a single table and assumes all tables with the same field name are protected. Which failure mode is MOST relevant?
- Uncaptured external ingestion may lead to incomplete impact analysis and unsafe upstream changes.
- An identically named confidential field in a new table could be exposed.
- Expired partners may retain ongoing data access despite a terminated agreement.
- Old cleanup tasks can destroy files that the revised retention requirement still protects.
Explanation
Correct Answer
B:The direct residual exposure is: An identically named confidential field in a new table could be exposed. The triggering condition is: A user creates a column mask on a single table and assumes all tables with the same field name are protected. The omitted safeguard is: Compare table-level masks against centrally governed ABAC coverage and identify missing assignments. This matters because: Per-table fine-grained policies do not automatically propagate to unrelated objects. A corrective test would show: A governance test finds masked behavior across all intended tagged objects and detects any unprotected table.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A lineage graph shows no relationship for data loaded outside the monitored platform and the team assumes the dataset has no upstream dependency. Its preventative action would be: Validate Unity Catalog lineage capture coverage and supplement unsupported external steps with documented provenance. The most direct exposure in the current scenario is: An identically named confidential field in a new table could be exposed.
C:This failure mode belongs mainly to another situation: A Delta Sharing recipient must stop receiving future updates after an agreement expires. Its preventative action would be: Revoke or disable the relevant Delta Sharing recipient/share permissions and verify subsequent access. The most direct exposure in the current scenario is: An identically named confidential field in a new table could be exposed.
D:This failure mode belongs mainly to another situation: A regulated source dataset receives an updated retention period but storage cleanup still follows the obsolete policy. Its preventative action would be: Align table retention properties, scheduled maintenance and legal ownership approvals with the current classification. The most direct exposure in the current scenario is: An identically named confidential field in a new table could be exposed.
Question 474
The recommended measure is: Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options. The operational context is: An integration performs a one-off initial import of an existing CSV folder into a Delta table. Which data engineering concept applies?
- Schema drift must be classified as an acceptable extension or a breaking contract change.
- One-time or incremental file loading can be expressed with SQL ingestion commands without building a persistent streaming service.
- Pulling distributed data to a single driver defeats parallel execution and memory limits.
- CDC propagates source mutations without duplicating full snapshots.
Explanation
Correct Answer
B:The decisive mechanism is: One-time or incremental file loading can be expressed with SQL ingestion commands without building a persistent streaming service. That explains the selected action: Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options. In the original scenario: An integration performs a one-off initial import of an existing CSV folder into a Delta table. The specific unmanaged risk is: A bespoke long-running stream for a one-time load adds complexity and idle cost. Validate with: The import reads the intended input files once and reconciles the expected row count and schema.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A file feed introduces an unexpected optional column and the stream fails after the first changed file. Its targeted response is: Configure supported schema evolution or rescued-data handling with explicit compatibility review. The current case is governed more directly by: One-time or incremental file loading can be expressed with SQL ingestion commands without building a persistent streaming service.
C:This principle concerns a separate data-engineering condition: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. Its targeted response is: Use Spark-native distributed read and transformation APIs or partition-safe processing. The current case is governed more directly by: One-time or incremental file loading can be expressed with SQL ingestion commands without building a persistent streaming service.
D:This principle concerns a separate data-engineering condition: A SaaS database updates and deletes customer records but a pipeline appends every nightly full extract to a Delta table. Its targeted response is: Use a supported change-data-capture feed and apply insert, update and delete semantics to the target. The current case is governed more directly by: One-time or incremental file loading can be expressed with SQL ingestion commands without building a persistent streaming service.
Question 475
The recommended measure is: Protect the main branch with peer review, required tests and controlled promotion windows. The operational context is: A release process merges unreviewed notebook changes directly into the main branch minutes before a production run. Which data engineering concept applies?
- Configuration-as-code requires environment-specific parameter checks as well as common source control.
- Reviewed changes with required status checks reduce uncontrolled production configuration drift.
- Conflict resolution must preserve intended behavior from both change sets.
- Version control and code review establish provenance and repeatable release history.
Explanation
Correct Answer
B:The decisive mechanism is: Reviewed changes with required status checks reduce uncontrolled production configuration drift. That explains the selected action: Protect the main branch with peer review, required tests and controlled promotion windows. In the original scenario: A release process merges unreviewed notebook changes directly into the main branch minutes before a production run. The specific unmanaged risk is: A faulty last-minute edit can create an untested release with no accountable reviewer. Validate with: An unauthorized direct push is rejected and a reviewed pull request completes all required gates.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A data engineer updates a bundled job trigger but another environment's deployment silently uses the wrong schedule. Its targeted response is: Validate per-target bundle variable resolution and compare rendered job definitions before deployment. The current case is governed more directly by: Reviewed changes with required status checks reduce uncontrolled production configuration drift.
C:This principle concerns a separate data-engineering condition: Two engineers change the same ingestion function on separate branches and one silently overwrites the other's fix. Its targeted response is: Resolve Git merge conflicts with tests and reviewed integration before merging the release branch. The current case is governed more directly by: Reviewed changes with required status checks reduce uncontrolled production configuration drift.
D:This principle concerns a separate data-engineering condition: Engineers edit production notebooks directly and lose the ability to identify which code change broke a pipeline. Its targeted response is: Store deployment source in Git with reviewed branches and pull requests before promoting changes. The current case is governed more directly by: Reviewed changes with required status checks reduce uncontrolled production configuration drift.
Question 476
The design team proposes: Specify the correct input delimiter, header and schema and validate parsing on a representative file. The situation is: An overnight load reads tab-separated files but an engineer assumes comma separators and casts malformed columns to null. Which technical principle justifies the decision?
- Streaming source metadata supports provenance and exact-range incident investigation.
- CDC correctness depends on source ordering and idempotent update rules.
- Text ingestion options are part of the data contract and affect parsing correctness.
- Managed ingestion connectors can simplify source synchronization when their coverage fits the source.
Explanation
Correct Answer
C:The decisive mechanism is: Text ingestion options are part of the data contract and affect parsing correctness. That explains the selected action: Specify the correct input delimiter, header and schema and validate parsing on a representative file. In the original scenario: An overnight load reads tab-separated files but an engineer assumes comma separators and casts malformed columns to null. The specific unmanaged risk is: Misparsed columns can populate nulls while the batch appears to complete successfully. Validate with: A seed file loads the exact expected columns and invalid rows are reported rather than silently accepted.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A pipeline ingests Event Hubs messages into a bronze table but never captures the partition, offset or event timestamp. Its targeted response is: Persist necessary source metadata and checkpoint lineage for troubleshooting and replay. The current case is governed more directly by: Text ingestion options are part of the data contract and affect parsing correctness.
B:This principle concerns a separate data-engineering condition: A transactional feed produces out-of-order CDC events and a naive MERGE overwrites a newer status with an older update. Its targeted response is: Apply ordered CDC sequencing using source change version or timestamp and consistent delete handling. The current case is governed more directly by: Text ingestion options are part of the data contract and affect parsing correctness.
D:This principle concerns a separate data-engineering condition: A batch integration needs a connector-managed path from a supported operational source into the lakehouse. Its targeted response is: Evaluate supported Lakeflow Connect ingestion rather than hardcoding ad hoc database extraction. The current case is governed more directly by: Text ingestion options are part of the data contract and affect parsing correctness.
Question 477
An implementation skips: Revoke or disable the relevant Delta Sharing recipient/share permissions and verify subsequent access. Engineers are observing: A Delta Sharing recipient must stop receiving future updates after an agreement expires. What consequence is MOST likely?
- Expired partners may retain ongoing data access despite a terminated agreement.
- Historical evidence may be lost even when the latest dataset remains complete.
- Accumulated historical grants may give former project teams access to unrelated data products.
- An orphaned ownership chain can delay access review or emergency remediation.
Explanation
Correct Answer
A:The direct residual exposure is: Expired partners may retain ongoing data access despite a terminated agreement. The triggering condition is: A Delta Sharing recipient must stop receiving future updates after an agreement expires. The omitted safeguard is: Revoke or disable the relevant Delta Sharing recipient/share permissions and verify subsequent access. This matters because: External sharing must include explicit revocation and monitoring, not only initial provisioning. A corrective test would show: A partner query fails after revocation while internal consumers remain able to read the dataset.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A product owner assumes a nightly Delta table overwrite meets a legal hold requirement for old records. Its preventative action would be: Design record-level retention and external legal-hold workflows appropriate to the applicable policy. The most direct exposure in the current scenario is: Expired partners may retain ongoing data access despite a terminated agreement.
C:This failure mode belongs mainly to another situation: A company needs to review table-owner changes and permissions after organizational restructuring. Its preventative action would be: Run a scheduled ownership and grant recertification process using catalog metadata and audit events. The most direct exposure in the current scenario is: Expired partners may retain ongoing data access despite a terminated agreement.
D:This failure mode belongs mainly to another situation: An audit asks who owns a catalog table but object ownership still points to an inactive employee. Its preventative action would be: Transfer ownership through approved governance workflow and review grants after reassignment. The most direct exposure in the current scenario is: Expired partners may retain ongoing data access despite a terminated agreement.
Question 478
An operator introduces: Use row-filter policies on the governed table and test permitted region entitlements. The situation is: A shared customer table needs dynamic per-region row restrictions but the business refuses duplicating physical datasets. Which validation should be required before signoff?
- A data steward can find the certified revenue definition and verify the correct field in Catalog Explorer.
- A newly classified test table receives the prescribed mask and an authorized exception is documented.
- Queries from two test regions return only their eligible rows from the same table.
- Authorized privacy reviewers see full test values while ordinary report roles see only permitted redactions.
Explanation
Correct Answer
C:The strongest acceptance proof is: Queries from two test regions return only their eligible rows from the same table. It specifically verifies: Use row-filter policies on the governed table and test permitted region entitlements. The original issue was: A shared customer table needs dynamic per-region row restrictions but the business refuses duplicating physical datasets. The underlying mechanism is: Row filters can restrict query results without maintaining separate physical copies by region. Without it, duplicated datasets can diverge or escape synchronized retention and correction policies.
Incorrect Answers
A:This result would validate a different measure: Add reviewed Unity Catalog table and column comments, owners and business definitions for consumer discovery. That is relevant to the distinct problem: A business glossary has table names but no indication of which field is net revenue versus gross sales. Here the acceptance criterion is: Queries from two test regions return only their eligible rows from the same table.
B:This result would validate a different measure: Create compatible central ABAC policies based on governed classifications and verify inherited coverage. That is relevant to the distinct problem: A central security team manually adds row masking rules to every department table and new datasets repeatedly lack protection. Here the acceptance criterion is: Queries from two test regions return only their eligible rows from the same table.
D:This result would validate a different measure: Develop an approved role-aware column mask and test its outputs across supported query clients. That is relevant to the distinct problem: A data product stores PII in a shared Delta table but several reports require different masking for each audience. Here the acceptance criterion is: Queries from two test regions return only their eligible rows from the same table.
Question 479
A team omits the recommended measure: Inspect query predicates and Delta data skipping or clustering layout before increasing warehouse size again. Its environment shows: A warehouse query remains slow even after larger compute is allocated and the profile reports scanning nearly every file. What is the MOST direct residual risk?
- Unused cached data can cause eviction, spilling and unexpected failures for later stages.
- Premature deletion may invalidate time travel or a lagging reader's file references.
- Scaling a query against irrelevant files can increase cost without proportional performance gains.
- Overprovisioning creates higher spend when stages are not parallelizable.
Explanation
Correct Answer
C:The direct residual exposure is: Scaling a query against irrelevant files can increase cost without proportional performance gains. The triggering condition is: A warehouse query remains slow even after larger compute is allocated and the profile reports scanning nearly every file. The omitted safeguard is: Inspect query predicates and Delta data skipping or clustering layout before increasing warehouse size again. This matters because: Full scans caused by poor layout require data organization or predicate changes rather than only more compute. A corrective test would show: A selective query reads fewer files and meets latency targets after a controlled layout improvement.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A team enables Spark cache for a one-time ETL dataset and never unpersists it before processing the next large table. Its preventative action would be: Use caching only for beneficial reuse and release persisted state when no longer needed. The most direct exposure in the current scenario is: Scaling a query against irrelevant files can increase cost without proportional performance gains.
B:This failure mode belongs mainly to another situation: An engineer runs VACUUM with a zero-hour retention on a shared Delta table used for historical audit queries. Its preventative action would be: Respect documented safe retention windows and coordinate VACUUM with readers, streams and time-travel needs. The most direct exposure in the current scenario is: Scaling a query against irrelevant files can increase cost without proportional performance gains.
D:This failure mode belongs mainly to another situation: A pipeline experiences escalating costs after a team increases node counts but execution duration barely changes. Its preventative action would be: Use run-level compute utilization, cost and stage metrics to identify unused capacity. The most direct exposure in the current scenario is: Scaling a query against irrelevant files can increase cost without proportional performance gains.
Question 480
A production data team leaves out: Respect documented safe retention windows and coordinate VACUUM with readers, streams and time-travel needs. Its scenario is: An engineer runs VACUUM with a zero-hour retention on a shared Delta table used for historical audit queries. Which failure mode is MOST relevant?
- Premature deletion may invalidate time travel or a lagging reader's file references.
- Overprovisioning creates higher spend when stages are not parallelizable.
- Ignoring heavy spills can cause cascading I/O latency and missed pipeline deadlines.
- Adding workers may not speed the one overloaded partition and increases total spend.
Explanation
Correct Answer
A:The direct residual exposure is: Premature deletion may invalidate time travel or a lagging reader's file references. The triggering condition is: An engineer runs VACUUM with a zero-hour retention on a shared Delta table used for historical audit queries. The omitted safeguard is: Respect documented safe retention windows and coordinate VACUUM with readers, streams and time-travel needs. This matters because: VACUUM removes old physical files, so aggressive cleanup can break valid historical references. A corrective test would show: A permitted historical query succeeds before and after approved housekeeping within its retention boundary.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A pipeline experiences escalating costs after a team increases node counts but execution duration barely changes. Its preventative action would be: Use run-level compute utilization, cost and stage metrics to identify unused capacity. The most direct exposure in the current scenario is: Premature deletion may invalidate time travel or a lagging reader's file references.
C:This failure mode belongs mainly to another situation: A large aggregation has repeated shuffle spills to disk even though source scans finish quickly. Its preventative action would be: Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage. The most direct exposure in the current scenario is: Premature deletion may invalidate time travel or a lagging reader's file references.
D:This failure mode belongs mainly to another situation: A production Spark query is slow and its stage graph shows one task running far longer than all peers. Its preventative action would be: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size. The most direct exposure in the current scenario is: Premature deletion may invalidate time travel or a lagging reader's file references.