Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 15
Question 140
The following control has NOT been implemented: Use appropriate table history, schema change records and approved retention/audit evidence rather than relying on memory. In this case: A shared table includes an old regulated field and the business needs reproducible evidence of when it was removed. Which exposure remains?
- Expired partners may retain ongoing data access despite a terminated agreement.
- Manual protection drift can expose fields on newly created tables before a review.
- Deleting a field without documented change history can leave unanswerable regulatory questions.
- Duplicated datasets can diverge or escape synchronized retention and correction policies.
Explanation
Correct Answer
C:The direct residual exposure is: Deleting a field without documented change history can leave unanswerable regulatory questions. The triggering condition is: A shared table includes an old regulated field and the business needs reproducible evidence of when it was removed. The omitted safeguard is: Use appropriate table history, schema change records and approved retention/audit evidence rather than relying on memory. This matters because: Metadata history and audit trails can support investigation of structure and access changes. A corrective test would show: The change record ties the removed column to an authorized deployment and preserved evidence.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A Delta Sharing recipient must stop receiving future updates after an agreement expires. Its preventative action would be: Revoke or disable the relevant Delta Sharing recipient/share permissions and verify subsequent access. The most direct exposure in the current scenario is: Deleting a field without documented change history can leave unanswerable regulatory questions.
B:This failure mode belongs mainly to another situation: A central security team manually adds row masking rules to every department table and new datasets repeatedly lack protection. Its preventative action would be: Create compatible central ABAC policies based on governed classifications and verify inherited coverage. The most direct exposure in the current scenario is: Deleting a field without documented change history can leave unanswerable regulatory questions.
D:This failure mode belongs mainly to another situation: A shared customer table needs dynamic per-region row restrictions but the business refuses duplicating physical datasets. Its preventative action would be: Use row-filter policies on the governed table and test permitted region entitlements. The most direct exposure in the current scenario is: Deleting a field without documented change history can leave unanswerable regulatory questions.
Question 141
A platform architect adopts: Validate Unity Catalog lineage capture coverage and supplement unsupported external steps with documented provenance. For this requirement: A lineage graph shows no relationship for data loaded outside the monitored platform and the team assumes the dataset has no upstream dependency. Which rationale MOST directly supports it?
- Security governance requires attributable administrative activity and durable evidence.
- Automatic lineage visibility has collection boundaries and does not imply every real-world dependency is recorded.
- Tag-based access control relies on trusted management of the tags themselves.
- Permission reviews should follow actual securable objects and their dependencies rather than name assumptions.
Explanation
Correct Answer
B:The decisive mechanism is: Automatic lineage visibility has collection boundaries and does not imply every real-world dependency is recorded. That explains the selected action: Validate Unity Catalog lineage capture coverage and supplement unsupported external steps with documented provenance. In the original scenario: A lineage graph shows no relationship for data loaded outside the monitored platform and the team assumes the dataset has no upstream dependency. The specific unmanaged risk is: Uncaptured external ingestion may lead to incomplete impact analysis and unsafe upstream changes. Validate with: A provenance review identifies both captured Databricks edges and documented external system dependencies.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A compliance auditor requests a record of who changed Unity Catalog grants but audit logs are not collected. Its targeted response is: Configure supported Databricks audit logging and route relevant events to a secured retention destination. The current case is governed more directly by: Automatic lineage visibility has collection boundaries and does not imply every real-world dependency is recorded.
C:This principle concerns a separate data-engineering condition: A governance team tags columns as restricted but anyone can remove the tag to bypass attached policies. Its targeted response is: Restrict governed-tag administration and audit changes to protected metadata. The current case is governed more directly by: Automatic lineage visibility has collection boundaries and does not imply every real-world dependency is recorded.
D:This principle concerns a separate data-engineering condition: A governance team grants dataset access solely by a table's displayed name while records are moved to a new schema. Its targeted response is: Reconcile catalog object identity, lineage and current grants after schema or owner changes. The current case is governed more directly by: Automatic lineage visibility has collection boundaries and does not imply every real-world dependency is recorded.
Question 142
A production data team leaves out: Evaluate supported Lakeflow Connect ingestion rather than hardcoding ad hoc database extraction. Its scenario is: A batch integration needs a connector-managed path from a supported operational source into the lakehouse. Which failure mode is MOST relevant?
- Misparsed columns can populate nulls while the batch appears to complete successfully.
- Driver out-of-memory can fail the load despite adequate executor resources.
- Deleting stream state can cause full reprocessing or skipped incremental data.
- Custom polling without checkpoint or schema handling may miss updates and require fragile maintenance.
Explanation
Correct Answer
D:The direct residual exposure is: Custom polling without checkpoint or schema handling may miss updates and require fragile maintenance. The triggering condition is: A batch integration needs a connector-managed path from a supported operational source into the lakehouse. The omitted safeguard is: Evaluate supported Lakeflow Connect ingestion rather than hardcoding ad hoc database extraction. This matters because: Managed ingestion connectors can simplify source synchronization when their coverage fits the source. A corrective test would show: A test source update appears in the target with correct keys and measurable ingestion lag.
Incorrect Answers
A:This failure mode belongs mainly to another situation: An overnight load reads tab-separated files but an engineer assumes comma separators and casts malformed columns to null. Its preventative action would be: Specify the correct input delimiter, header and schema and validate parsing on a representative file. The most direct exposure in the current scenario is: Custom polling without checkpoint or schema handling may miss updates and require fragile maintenance.
B:This failure mode belongs mainly to another situation: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. Its preventative action would be: Use Spark-native distributed read and transformation APIs or partition-safe processing. The most direct exposure in the current scenario is: Custom polling without checkpoint or schema handling may miss updates and require fragile maintenance.
C:This failure mode belongs mainly to another situation: A long-running ingestion job persists its checkpoint in the same temporary folder that daily cleanup deletes. Its preventative action would be: Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle. The most direct exposure in the current scenario is: Custom polling without checkpoint or schema handling may miss updates and require fragile maintenance.
Question 143
During a lakehouse implementation review, engineers observe: An ETL service principal is granted permissions through a personal employee account that will leave the company. What is the BEST next action?
- Configure a supported external location and specific creation privileges rather than bypassing the catalog to write directly.
- Grant only the required USE and SELECT privileges to the analyst group and inspect inherited grants.
- Check required parent catalog and schema usage privileges alongside the table SELECT grant.
- Use a dedicated service principal with scoped Unity Catalog grants and governed credential lifecycle.
Explanation
Correct Answer
D:The most appropriate action is: Use a dedicated service principal with scoped Unity Catalog grants and governed credential lifecycle. It addresses the stated condition: An ETL service principal is granted permissions through a personal employee account that will leave the company. The technical basis is: Workload identities separate automated access from a person's employment and session lifecycle. Without this control, an employee departure or password rotation can unexpectedly break critical production ingestion. A direct acceptance check is: The pipeline authenticates using the designated service principal and operates after the employee account is disabled.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A user is granted CREATE TABLE but cannot create an external table at an unregistered cloud path. The corresponding measure there would be: Configure a supported external location and specific creation privileges rather than bypassing the catalog to write directly. The stated situation instead calls for: Use a dedicated service principal with scoped Unity Catalog grants and governed credential lifecycle.
B:This alternative primarily addresses a different Azure Databricks problem: A finance analyst must read an approved table but was granted broad MODIFY rights at the schema level. The corresponding measure there would be: Grant only the required USE and SELECT privileges to the analyst group and inspect inherited grants. The stated situation instead calls for: Use a dedicated service principal with scoped Unity Catalog grants and governed credential lifecycle.
C:This alternative primarily addresses a different Azure Databricks problem: A contractor receives SELECT on a table yet gets an access-denied error when querying with a three-part name. The corresponding measure there would be: Check required parent catalog and schema usage privileges alongside the table SELECT grant. The stated situation instead calls for: Use a dedicated service principal with scoped Unity Catalog grants and governed credential lifecycle.
Question 144
A production data team leaves out: Use a left anti-join or EXCEPT operation with aligned schemas and explicit null semantics. Its scenario is: An analyst must find records present in the source system but absent in a target migration. Which failure mode is MOST relevant?
- Unknown statuses may be dropped or duplicated if pivot categories are assumed without validation.
- Position-based union may quietly corrupt data when column positions differ.
- An inner join hides precisely the missing records the reconciliation needs to surface.
- Blindly broadcasting a large table can exhaust executor memory while unnecessary shuffles slow joins.
Explanation
Correct Answer
C:The direct residual exposure is: An inner join hides precisely the missing records the reconciliation needs to surface. The triggering condition is: An analyst must find records present in the source system but absent in a target migration. The omitted safeguard is: Use a left anti-join or EXCEPT operation with aligned schemas and explicit null semantics. This matters because: Set difference helps identify migration gaps without retrieving matched records. A corrective test would show: The gap report returns seeded source-only records and excludes matched records.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A quality report needs one column for each product status category but stores status as separate rows. Its preventative action would be: Apply a controlled pivot with a defined set of expected categories and aggregation rule. The most direct exposure in the current scenario is: An inner join hides precisely the missing records the reconciliation needs to surface.
B:This failure mode belongs mainly to another situation: Two quarterly sales feeds have the same logical fields but different column ordering and a union shifts amounts into dates. Its preventative action would be: Align compatible schemas and use name-based union semantics with explicit type checks. The most direct exposure in the current scenario is: An inner join hides precisely the missing records the reconciliation needs to surface.
D:This failure mode belongs mainly to another situation: A pipeline joins a huge fact table with a tiny dimension and a normal shuffle join creates excessive network traffic. Its preventative action would be: Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans. The most direct exposure in the current scenario is: An inner join hides precisely the missing records the reconciliation needs to surface.
Question 145
A production data platform experiences this condition: A Spark SQL pipeline spends most of its time executing wide joins and aggregations but the team has not evaluated a compatible vectorized execution engine. Which targeted measure should be selected?
- Configure bounded autoscaling using measured workload demand and verify maximum cost and startup behavior.
- Apply suitable compute access modes, policies and entitlements that restrict who can attach or modify shared compute.
- Benchmark Photon on a supported runtime with representative queries and compare cost, latency and correctness.
- Define a job-owned execution identity and eligible task compute separate from ad hoc analyst cluster lifecycle.
Explanation
Correct Answer
C:The most appropriate action is: Benchmark Photon on a supported runtime with representative queries and compare cost, latency and correctness. It addresses the stated condition: A Spark SQL pipeline spends most of its time executing wide joins and aggregations but the team has not evaluated a compatible vectorized execution engine. The technical basis is: Photon can accelerate many supported SQL and DataFrame operations but requires workload-specific measurements. Without this control, assuming every transformation accelerates equally may raise costs without reducing job duration. A direct acceptance check is: Identical data and query workloads yield reconciled outputs and a recorded improvement in elapsed time per unit cost.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A data engineering job sometimes requires twice its usual executor capacity during month-end loads while weekdays are much smaller. The corresponding measure there would be: Configure bounded autoscaling using measured workload demand and verify maximum cost and startup behavior. The stated situation instead calls for: Benchmark Photon on a supported runtime with representative queries and compare cost, latency and correctness.
B:This alternative primarily addresses a different Azure Databricks problem: A multi-team interactive cluster permits users to attach arbitrary unapproved libraries and run notebooks under broad shared privileges. The corresponding measure there would be: Apply suitable compute access modes, policies and entitlements that restrict who can attach or modify shared compute. The stated situation instead calls for: Benchmark Photon on a supported runtime with representative queries and compare cost, latency and correctness.
D:This alternative primarily addresses a different Azure Databricks problem: A scheduled ETL run uses shared analyst compute and fails whenever the cluster owner restarts a notebook session. The corresponding measure there would be: Define a job-owned execution identity and eligible task compute separate from ad hoc analyst cluster lifecycle. The stated situation instead calls for: Benchmark Photon on a supported runtime with representative queries and compare cost, latency and correctness.
Question 146
For this Azure Databricks issue: A new Delta table maintenance job combines repeated OPTIMIZE and VACUUM at every microbatch even though changes are tiny. Engineers choose: Set a measured maintenance cadence or evaluate supported predictive optimization for eligible managed tables. What acceptance evidence is BEST?
- A dashboard highlights task and stage regressions before the total job breaches its SLA.
- Before-and-after file statistics show fewer small files with identical row counts and faster queries.
- A benchmark shows lower total maintenance-plus-query cost without compromising retention.
- The job writes results through distributed output and driver memory stays below the approved threshold.
Explanation
Correct Answer
C:The strongest acceptance proof is: A benchmark shows lower total maintenance-plus-query cost without compromising retention. It specifically verifies: Set a measured maintenance cadence or evaluate supported predictive optimization for eligible managed tables. The original issue was: A new Delta table maintenance job combines repeated OPTIMIZE and VACUUM at every microbatch even though changes are tiny. The underlying mechanism is: Maintenance operations themselves consume compute and should correspond to data growth and read patterns. Without it, over-maintenance can spend more resources rewriting files than it saves in queries.
Incorrect Answers
A:This result would validate a different measure: Collect job timing, task duration and relevant Spark stage metrics in a monitored baseline. That is relevant to the distinct problem: A job's duration doubles but an administrator sees only its final success status with no phase-level metrics. Here the acceptance criterion is: A benchmark shows lower total maintenance-plus-query cost without compromising retention.
B:This result would validate a different measure: Run supported OPTIMIZE or predictive optimization and validate file-size distribution and query cost. That is relevant to the distinct problem: A Delta table contains millions of tiny files after frequent microbatch updates and SQL scans spend time opening files. Here the acceptance criterion is: A benchmark shows lower total maintenance-plus-query cost without compromising retention.
D:This result would validate a different measure: Keep large results distributed and replace driver collection with bounded sampling or distributed writes. That is relevant to the distinct problem: A Spark join reads large inputs but the driver reports failures after an analyst calls collect() on the result. Here the acceptance criterion is: A benchmark shows lower total maintenance-plus-query cost without compromising retention.
Question 147
The team chooses: Profile source missingness and define field-specific required versus optional expectations. Its observed issue is: A new customer feed is missing optional contact information for 8% of rows and the team treats every null as pipeline failure. What is the strongest technical basis?
- Schema evolution must distinguish permissible additive fields from breaking type changes.
- Primary business-key uniqueness is a semantic data-quality invariant even without enforced physical primary keys.
- Layered quality gates prevent malformed source values from contaminating trusted downstream tables.
- Completeness thresholds depend on business contracts and cannot be uniform across all columns.
Explanation
Correct Answer
D:The decisive mechanism is: Completeness thresholds depend on business contracts and cannot be uniform across all columns. That explains the selected action: Profile source missingness and define field-specific required versus optional expectations. In the original scenario: A new customer feed is missing optional contact information for 8% of rows and the team treats every null as pipeline failure. The specific unmanaged risk is: Overly strict validation can block legitimate records while loose validation may permit missing keys. Validate with: Quality dashboards separate optional-null rates from prohibited null keys and meet agreed thresholds.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A source API changes a quantity from integer to nested JSON and the pipeline silently produces nulls downstream. Its targeted response is: Apply schema enforcement with explicit drift handling and alert on incompatible structural changes. The current case is governed more directly by: Completeness thresholds depend on business contracts and cannot be uniform across all columns.
B:This principle concerns a separate data-engineering condition: A curated account table accidentally contains multiple rows per supposedly unique current account ID. Its targeted response is: Run a supported uniqueness or cardinality validation during the transformation and surface violations. The current case is governed more directly by: Completeness thresholds depend on business contracts and cannot be uniform across all columns.
C:This principle concerns a separate data-engineering condition: A product data contract requires every transaction amount to use nonnegative decimal values but new raw files include quoted text strings. Its targeted response is: Apply type, nullability and range expectations before promoting bronze rows into the silver target. The current case is governed more directly by: Completeness thresholds depend on business contracts and cannot be uniform across all columns.
Question 148
During a lakehouse implementation review, engineers observe: A production Spark query is slow and its stage graph shows one task running far longer than all peers. What is the BEST next action?
- Inspect the Spark UI DAG and skewed-key distribution before changing cluster size.
- Preserve diagnostic events and Spark UI stage evidence before restart where possible.
- Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage.
- Inspect query predicates and Delta data skipping or clustering layout before increasing warehouse size again.
Explanation
Correct Answer
A:The most appropriate action is: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size. It addresses the stated condition: A production Spark query is slow and its stage graph shows one task running far longer than all peers. The technical basis is: A single hot partition can dominate a distributed stage despite idle executor capacity. Without this control, adding workers may not speed the one overloaded partition and increases total spend. A direct acceptance check is: A skew-aware plan balances task durations and reduces the critical-stage elapsed time.
Incorrect Answers
B:This alternative primarily addresses a different Azure Databricks problem: An operator restarts a failed Databricks cluster before collecting the Spark executor and driver logs. The corresponding measure there would be: Preserve diagnostic events and Spark UI stage evidence before restart where possible. The stated situation instead calls for: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size.
C:This alternative primarily addresses a different Azure Databricks problem: A large aggregation has repeated shuffle spills to disk even though source scans finish quickly. The corresponding measure there would be: Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage. The stated situation instead calls for: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size.
D:This alternative primarily addresses a different Azure Databricks problem: A warehouse query remains slow even after larger compute is allocated and the profile reports scanning nearly every file. The corresponding measure there would be: Inspect query predicates and Delta data skipping or clustering layout before increasing warehouse size again. The stated situation instead calls for: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size.
Question 149
A data product owner adopts: Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics. The reason is: A team expects dropping a managed table to preserve all physical data as though it were an external table. What observed result BEST proves effectiveness?
- Approved regional workspaces access their catalogs while a disallowed workspace fails its access test.
- Permitted users query only expected columns and rows through the view without a separately exported file.
- A controlled create-drop rehearsal confirms the expected metadata and physical data lifecycle for the table type.
- A set of known business prompts yields reconciled SQL and approved metric definitions.
Explanation
Correct Answer
C:The strongest acceptance proof is: A controlled create-drop rehearsal confirms the expected metadata and physical data lifecycle for the table type. It specifically verifies: Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics. The original issue was: A team expects dropping a managed table to preserve all physical data as though it were an external table. The underlying mechanism is: Managed versus external tables differ in who manages data paths and deletion behavior. Without it, A drop operation may remove managed table data that administrators expected to retain.
Incorrect Answers
A:This result would validate a different measure: Split catalogs or governance boundaries along residency and ownership requirements and validate workspace bindings. That is relevant to the distinct problem: A platform architect creates one catalog for subsidiaries with incompatible regional data-residency permissions. Here the acceptance criterion is: A controlled create-drop rehearsal confirms the expected metadata and physical data lifecycle for the table type.
B:This result would validate a different measure: Create a Unity Catalog view with approved permissions and verify underlying access semantics. That is relevant to the distinct problem: An analyst wants to expose a filtered projection of a governed table without copying sensitive source rows. Here the acceptance criterion is: A controlled create-drop rehearsal confirms the expected metadata and physical data lifecycle for the table type.
D:This result would validate a different measure: Add reviewed Genie instructions, table descriptions and example questions consistent with trusted catalog semantics. That is relevant to the distinct problem: An AI/BI Genie space is configured with ambiguous metric terminology and produces misleading natural-language interpretations. Here the acceptance criterion is: A controlled create-drop rehearsal confirms the expected metadata and physical data lifecycle for the table type.
Question 150
The team implements: Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options. The original problem is: An integration performs a one-off initial import of an existing CSV folder into a Delta table. Which test MOST convincingly demonstrates correctness?
- The import reads the intended input files once and reconciles the expected row count and schema.
- A test source update appears in the target with correct keys and measurable ingestion lag.
- A shuffled CDC test results in the correct highest-version value after replay.
- Injected duplicate and malformed source samples follow the intended dedupe or quarantine rules.
Explanation
Correct Answer
A:The strongest acceptance proof is: The import reads the intended input files once and reconciles the expected row count and schema. It specifically verifies: Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options. The original issue was: An integration performs a one-off initial import of an existing CSV folder into a Delta table. The underlying mechanism is: One-time or incremental file loading can be expressed with SQL ingestion commands without building a persistent streaming service. Without it, A bespoke long-running stream for a one-time load adds complexity and idle cost.
Incorrect Answers
B:This result would validate a different measure: Evaluate supported Lakeflow Connect ingestion rather than hardcoding ad hoc database extraction. That is relevant to the distinct problem: A batch integration needs a connector-managed path from a supported operational source into the lakehouse. Here the acceptance criterion is: The import reads the intended input files once and reconciles the expected row count and schema.
C:This result would validate a different measure: Apply ordered CDC sequencing using source change version or timestamp and consistent delete handling. That is relevant to the distinct problem: A transactional feed produces out-of-order CDC events and a naive MERGE overwrites a newer status with an older update. Here the acceptance criterion is: The import reads the intended input files once and reconciles the expected row count and schema.
D:This result would validate a different measure: Define Lakeflow Spark Declarative Pipelines source options, incremental behavior and expectations with a durable target. That is relevant to the distinct problem: A declarative pipeline loads cloud object files continuously but its configuration omits how duplicate files and bad records are handled. Here the acceptance criterion is: The import reads the intended input files once and reconciles the expected row count and schema.