Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 52
Question 510
An implementation skips: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. Engineers are observing: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. What consequence is MOST likely?
- Excessive partitions create small files and poor query planning without useful pruning.
- Duplicating physical dimensions can leave teams on inconsistent updates or retention periods.
- One-file-per-customer patterns can overwhelm metadata and worsen query startup.
- Migrating data into a managed table can violate external storage governance agreements.
Explanation
Correct Answer
C:The direct residual exposure is: One-file-per-customer patterns can overwhelm metadata and worsen query startup. The triggering condition is: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. The omitted safeguard is: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. This matters because: Partition pruning benefits can be outweighed by extremely fragmented high-cardinality directory layouts. A corrective test would show: Storage metrics show a manageable file distribution and point lookups meet query SLAs.
Incorrect Answers
A:This failure mode belongs mainly to another situation: An event table is partitioned by the unique transaction identifier and creates millions of tiny partitions. Its preventative action would be: Use coarse suitable partitioning or liquid clustering selected from filter patterns and cardinality. The most direct exposure in the current scenario is: One-file-per-customer patterns can overwhelm metadata and worsen query startup.
B:This failure mode belongs mainly to another situation: A data product creates multiple physical copies of a curated dimension because teams need different column names. Its preventative action would be: Expose stable SQL views over a governed base table when only presentation aliases differ. The most direct exposure in the current scenario is: One-file-per-customer patterns can overwhelm metadata and worsen query startup.
D:This failure mode belongs mainly to another situation: A dataset has a strict requirement that Azure storage ownership and lifecycle remain under a separate platform team. Its preventative action would be: Use an external Unity Catalog table with an approved registered location instead of assuming managed storage ownership. The most direct exposure in the current scenario is: One-file-per-customer patterns can overwhelm metadata and worsen query startup.
Question 511
The following control has NOT been implemented: Wait for or explicitly verify required external or asynchronous work before reporting task success. In this case: A job is successful in the orchestrator but the business target is not updated because the notebook returns before async work completes. Which exposure remains?
- Premature success can start downstream tasks against incomplete writes.
- Replaying an entire DAG without need can duplicate side effects and increase cost.
- A clock-only trigger can start before the dependency has produced the expected dataset.
- Temporarily editing the main schedule may skip normal deliveries or replay wrong dates.
Explanation
Correct Answer
A:The direct residual exposure is: Premature success can start downstream tasks against incomplete writes. The triggering condition is: A job is successful in the orchestrator but the business target is not updated because the notebook returns before async work completes. The omitted safeguard is: Wait for or explicitly verify required external or asynchronous work before reporting task success. This matters because: Task success must correspond to the completion of required data side effects, not just function invocation. A corrective test would show: A test run finishes only after the target watermark and expected row counts are confirmed.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A team assumes that automatic job retries are the same as recovering a failed multi-step run at its last unsuccessful task. Its preventative action would be: Choose the appropriate retry and repair-run functionality based on failed-task state and valid dependencies. The most direct exposure in the current scenario is: Premature success can start downstream tasks against incomplete writes.
C:This failure mode belongs mainly to another situation: A pipeline must run when an upstream Databricks table is updated but analysts have hardcoded a daily time estimate. Its preventative action would be: Evaluate supported table update triggers or a clear upstream job dependency based on actual availability. The most direct exposure in the current scenario is: Premature success can start downstream tasks against incomplete writes.
D:This failure mode belongs mainly to another situation: An ingestion job requires a one-time backfill without changing the future daily production schedule. Its preventative action would be: Run a controlled parameterized manual invocation for the historical interval while retaining the regular trigger. The most direct exposure in the current scenario is: Premature success can start downstream tasks against incomplete writes.
Question 512
For this Azure Databricks issue: A Databricks job receives daily structured extracts but an upstream schedule occasionally delivers files late. Engineers choose: Use incremental file ingestion with explicit late-arrival cutoffs and a measurable source-to-target watermark. What acceptance evidence is BEST?
- A sampled bronze row links to the expected stream partition and source event time.
- Injected duplicate and malformed source samples follow the intended dedupe or quarantine rules.
- A deliberately delayed file is ingested exactly once within the allowed arrival window.
- ADF completion evidence and Databricks job dependencies show an end-to-end successful load.
Explanation
Correct Answer
C:The strongest acceptance proof is: A deliberately delayed file is ingested exactly once within the allowed arrival window. It specifically verifies: Use incremental file ingestion with explicit late-arrival cutoffs and a measurable source-to-target watermark. The original issue was: A Databricks job receives daily structured extracts but an upstream schedule occasionally delivers files late. The underlying mechanism is: Data pipeline completeness requires tracking arrivals rather than assuming an exact wall-clock boundary. Without it, A fixed midnight file list can omit valid delayed uploads.
Incorrect Answers
A:This result would validate a different measure: Persist necessary source metadata and checkpoint lineage for troubleshooting and replay. That is relevant to the distinct problem: A pipeline ingests Event Hubs messages into a bronze table but never captures the partition, offset or event timestamp. Here the acceptance criterion is: A deliberately delayed file is ingested exactly once within the allowed arrival window.
B:This result would validate a different measure: Define Lakeflow Spark Declarative Pipelines source options, incremental behavior and expectations with a durable target. That is relevant to the distinct problem: A declarative pipeline loads cloud object files continuously but its configuration omits how duplicate files and bad records are handled. Here the acceptance criterion is: A deliberately delayed file is ingested exactly once within the allowed arrival window.
D:This result would validate a different measure: Coordinate ADF orchestration with the appropriate Databricks task and Unity Catalog access model. That is relevant to the distinct problem: A data product needs to copy a supported Azure Data Factory pipeline output into governed Databricks tables. Here the acceptance criterion is: A deliberately delayed file is ingested exactly once within the allowed arrival window.
Question 513
The design team proposes: Enforce a NOT NULL constraint or pipeline expectation on the transaction ID with a defined failure action. The situation is: An event pipeline must reject rows with a null transaction ID, but the data engineer checks only the row count after load. Which technical principle justifies the decision?
- Schema evolution must distinguish permissible additive fields from breaking type changes.
- Measurable quality controls need observable outcomes and accounting for rejected data.
- Explicit quality constraints protect critical key semantics rather than relying on aggregate row totals.
- Completeness thresholds depend on business contracts and cannot be uniform across all columns.
Explanation
Correct Answer
C:The decisive mechanism is: Explicit quality constraints protect critical key semantics rather than relying on aggregate row totals. That explains the selected action: Enforce a NOT NULL constraint or pipeline expectation on the transaction ID with a defined failure action. In the original scenario: An event pipeline must reject rows with a null transaction ID, but the data engineer checks only the row count after load. The specific unmanaged risk is: Null business keys can enter Delta targets and make updates or joins nondeterministic. Validate with: A seeded null-ID row is blocked or quarantined while valid records complete successfully.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A source API changes a quantity from integer to nested JSON and the pipeline silently produces nulls downstream. Its targeted response is: Apply schema enforcement with explicit drift handling and alert on incompatible structural changes. The current case is governed more directly by: Explicit quality constraints protect critical key semantics rather than relying on aggregate row totals.
B:This principle concerns a separate data-engineering condition: A pipeline reports zero failed rows because bad records are silently dropped without any count of what was removed. Its targeted response is: Collect expectation metrics and reconcile input, accepted, quarantined and rejected record counts. The current case is governed more directly by: Explicit quality constraints protect critical key semantics rather than relying on aggregate row totals.
D:This principle concerns a separate data-engineering condition: A new customer feed is missing optional contact information for 8% of rows and the team treats every null as pipeline failure. Its targeted response is: Profile source missingness and define field-specific required versus optional expectations. The current case is governed more directly by: Explicit quality constraints protect critical key semantics rather than relying on aggregate row totals.
Question 514
The following control has NOT been implemented: Split ingestion, validation and publishing into explicit orchestrated tasks with durable intermediate outputs. In this case: A multi-step notebook workflow handles source extraction and final reporting in one cell sequence with no recoverable checkpoints. Which exposure remains?
- Consumers can query invalid data during the interval before quality checks finish.
- Readers can combine incompatible versions when related tables update independently.
- One late-stage error may force a costly replay of all preceding successful processing.
- Forcing arbitrary procedural logic into a declarative table pipeline may complicate testing and maintenance.
Explanation
Correct Answer
C:The direct residual exposure is: One late-stage error may force a costly replay of all preceding successful processing. The triggering condition is: A multi-step notebook workflow handles source extraction and final reporting in one cell sequence with no recoverable checkpoints. The omitted safeguard is: Split ingestion, validation and publishing into explicit orchestrated tasks with durable intermediate outputs. This matters because: Separating pipeline stages creates observable failure boundaries and supports targeted recovery. A corrective test would show: A simulated final-task failure can be repaired without rerunning already validated upstream ingestion.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A workflow writes to production tables before a validation task examines null keys and duplicates. Its preventative action would be: Place validations before publishing and use an atomic cutover or controlled target write. The most direct exposure in the current scenario is: One late-stage error may force a costly replay of all preceding successful processing.
B:This failure mode belongs mainly to another situation: A regulated pipeline needs consistent transaction boundaries across multiple table writes but updates are published one at a time. Its preventative action would be: Design a compatible atomic publication or versioned readiness contract so consumers see consistent results. The most direct exposure in the current scenario is: One late-stage error may force a costly replay of all preceding successful processing.
D:This failure mode belongs mainly to another situation: An ad hoc machine-learning preparation task has complex custom Python orchestration and no declarative table graph. Its preventative action would be: Select a notebook task within Lakeflow Jobs when procedural control and custom libraries fit the requirements. The most direct exposure in the current scenario is: One late-stage error may force a costly replay of all preceding successful processing.
Question 515
The team implements: Tune supported SQL warehouse capacity and concurrency scaling based on simultaneous production query measurements. The original problem is: A BI warehouse repeatedly slows during morning sign-in peaks even though a single analyst query runs quickly. Which test MOST convincingly demonstrates correctness?
- A load rehearsal meets dashboard p95 response targets while notebook resource spikes do not affect query service.
- A concurrent-user benchmark meets percentile latency and queue-depth targets with acceptable cost.
- An ordinary user can start an approved small cluster but cannot exceed policy size or runtime limits.
- The scheduled job completes within its service objective and compute billing shows no unattended idle cluster hours.
Explanation
Correct Answer
B:The strongest acceptance proof is: A concurrent-user benchmark meets percentile latency and queue-depth targets with acceptable cost. It specifically verifies: Tune supported SQL warehouse capacity and concurrency scaling based on simultaneous production query measurements. The original issue was: A BI warehouse repeatedly slows during morning sign-in peaks even though a single analyst query runs quickly. The underlying mechanism is: Concurrency queuing is separate from a single-query execution performance problem. Without it, sizing from isolated benchmark queries can leave hundreds of dashboard viewers waiting in a query queue.
Incorrect Answers
A:This result would validate a different measure: Use an appropriately sized SQL warehouse with workload isolation and measured concurrency for dashboard queries. That is relevant to the distinct problem: Analysts run short SQL dashboard queries against a shared lakehouse but their notebooks share a general-purpose Spark cluster with training experiments. Here the acceptance criterion is: A concurrent-user benchmark meets percentile latency and queue-depth targets with acceptable cost.
C:This result would validate a different measure: Create compute policies with approved node types, worker limits, tags and idle settings for the permitted users. That is relevant to the distinct problem: A department wants a self-service sandbox but must prevent users from creating huge unrestricted clusters. Here the acceptance criterion is: A concurrent-user benchmark meets percentile latency and queue-depth targets with acceptable cost.
D:This result would validate a different measure: Move the scheduled transformation to appropriately configured jobs compute or supported serverless jobs and release resources after execution. That is relevant to the distinct problem: A nightly batch transformation uses a continuously running all-purpose cluster even though no analyst interacts with the workload between runs. Here the acceptance criterion is: A concurrent-user benchmark meets percentile latency and queue-depth targets with acceptable cost.
Question 516
A pipeline owner declines the action: Use a secure service identity and protected credential handling for bundle or REST API deployment. The business situation is: A CI pipeline deploys Databricks assets under an engineer's personal token stored in repository variables. What specific failure can follow?
- An exposed or disabled personal token can compromise or halt all production deployments.
- A release may finish successfully while critical reports cannot resolve their expected view.
- A faulty last-minute edit can create an untested release with no accountable reviewer.
- A logically incorrect aggregation can pass compilation while corrupting metrics.
Explanation
Correct Answer
A:The direct residual exposure is: An exposed or disabled personal token can compromise or halt all production deployments. The triggering condition is: A CI pipeline deploys Databricks assets under an engineer's personal token stored in repository variables. The omitted safeguard is: Use a secure service identity and protected credential handling for bundle or REST API deployment. This matters because: Automated releases should not depend on personal accounts or broadly exposed long-lived credentials. A corrective test would show: CI deploys under the named service principal and the personal token is rejected and revoked.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A sensitive SQL view is renamed in the release but the team does not test downstream dashboards before promotion. Its preventative action would be: Run end-to-end contract tests for dependent readers and obtain owner approval for breaking changes. The most direct exposure in the current scenario is: An exposed or disabled personal token can compromise or halt all production deployments.
C:This failure mode belongs mainly to another situation: A release process merges unreviewed notebook changes directly into the main branch minutes before a production run. Its preventative action would be: Protect the main branch with peer review, required tests and controlled promotion windows. The most direct exposure in the current scenario is: An exposed or disabled personal token can compromise or halt all production deployments.
D:This failure mode belongs mainly to another situation: A release modifies transformation logic but tests only whether Python files compile. Its preventative action would be: Add representative unit and integration tests asserting transformation results and schema contracts. The most direct exposure in the current scenario is: An exposed or disabled personal token can compromise or halt all production deployments.
Question 517
During a lakehouse implementation review, engineers observe: A CI pipeline deploys Databricks assets under an engineer's personal token stored in repository variables. What is the BEST next action?
- Add user acceptance tests for the relevant reporting contract before production promotion.
- Use a secure service identity and protected credential handling for bundle or REST API deployment.
- Add representative unit and integration tests asserting transformation results and schema contracts.
- Resolve Git merge conflicts with tests and reviewed integration before merging the release branch.
Explanation
Correct Answer
B:The most appropriate action is: Use a secure service identity and protected credential handling for bundle or REST API deployment. It addresses the stated condition: A CI pipeline deploys Databricks assets under an engineer's personal token stored in repository variables. The technical basis is: Automated releases should not depend on personal accounts or broadly exposed long-lived credentials. Without this control, an exposed or disabled personal token can compromise or halt all production deployments. A direct acceptance check is: CI deploys under the named service principal and the personal token is rejected and revoked.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A developer changes a transformation with a user-visible business effect but omits acceptance testing by the data product owner. The corresponding measure there would be: Add user acceptance tests for the relevant reporting contract before production promotion. The stated situation instead calls for: Use a secure service identity and protected credential handling for bundle or REST API deployment.
C:This alternative primarily addresses a different Azure Databricks problem: A release modifies transformation logic but tests only whether Python files compile. The corresponding measure there would be: Add representative unit and integration tests asserting transformation results and schema contracts. The stated situation instead calls for: Use a secure service identity and protected credential handling for bundle or REST API deployment.
D:This alternative primarily addresses a different Azure Databricks problem: Two engineers change the same ingestion function on separate branches and one silently overwrites the other's fix. The corresponding measure there would be: Resolve Git merge conflicts with tests and reviewed integration before merging the release branch. The stated situation instead calls for: Use a secure service identity and protected credential handling for bundle or REST API deployment.
Question 518
During a lakehouse implementation review, engineers observe: An external sharing integration needs interoperable columnar read files but not native Delta updates. What is the BEST next action?
- Choose and document a supported format and catalog integration with explicit read/write compatibility tests.
- Design event-time fields, ingestion timestamps and late-data behavior suited to replay and temporal queries.
- Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys.
- Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements.
Explanation
Correct Answer
D:The most appropriate action is: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements. It addresses the stated condition: An external sharing integration needs interoperable columnar read files but not native Delta updates. The technical basis is: File and table format choices depend on transactional features and consumer interoperability. Without this control, choosing Delta-specific mutation assumptions for ordinary Parquet files can break consumers. A direct acceptance check is: The external reader loads approved columns and a documented compatibility test passes.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A cross-engine consumer expects a Delta table but source is plain Iceberg and engineers assume identical mutation semantics. The corresponding measure there would be: Choose and document a supported format and catalog integration with explicit read/write compatibility tests. The stated situation instead calls for: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements.
B:This alternative primarily addresses a different Azure Databricks problem: A lakehouse receives sensor observations with event timestamps that arrive out of order. The corresponding measure there would be: Design event-time fields, ingestion timestamps and late-data behavior suited to replay and temporal queries. The stated situation instead calls for: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements.
C:This alternative primarily addresses a different Azure Databricks problem: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. The corresponding measure there would be: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys. The stated situation instead calls for: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements.
Question 519
For this Azure Databricks issue: A BI group needs to query a materialized aggregate while the source transaction Delta table changes hourly. Engineers choose: Create an eligible materialized view or managed derived table with appropriate refresh and access rules. What acceptance evidence is BEST?
- Discovery tests distinguish raw from certified curated objects and owners confirm their assigned namespaces.
- A source update followed by the documented refresh produces reconciled aggregate totals.
- An analyst can query approved tables but cannot create a schema without a separate grant.
- A controlled MERGE and SQL query succeed against the governed Delta table with expected records.
Explanation
Correct Answer
B:The strongest acceptance proof is: A source update followed by the documented refresh produces reconciled aggregate totals. It specifically verifies: Create an eligible materialized view or managed derived table with appropriate refresh and access rules. The original issue was: A BI group needs to query a materialized aggregate while the source transaction Delta table changes hourly. The underlying mechanism is: Materialized views can persist derived query results, but refresh behavior must match freshness expectations. Without it, users can unknowingly consume stale aggregates if refresh ownership and schedule are unspecified.
Incorrect Answers
A:This result would validate a different measure: Define a consistent schema and naming architecture separating layers and business domains by purpose. That is relevant to the distinct problem: Multiple data products put raw, refined and curated assets into a single schema with confusing table names. Here the acceptance criterion is: A source update followed by the documented refresh produces reconciled aggregate totals.
C:This result would validate a different measure: Grant precise container-use and object-creation privileges based on actual job roles. That is relevant to the distinct problem: A team expects every analyst to CREATE SCHEMA simply because they can SELECT tables in an existing catalog. Here the acceptance criterion is: A source update followed by the documented refresh produces reconciled aggregate totals.
D:This result would validate a different measure: Register the structured dataset as an appropriate managed Delta table rather than treating the volume itself as a SQL table. That is relevant to the distinct problem: A team exposes structured transaction data as files in a volume but wants SQL analytics and ACID MERGE behavior. Here the acceptance criterion is: A source update followed by the documented refresh produces reconciled aggregate totals.
Question 520
A production data platform experiences this condition: An application uses an external Azure Key Vault secret scope but a notebook prints the secret into task logs. Which targeted measure should be selected?
- Check required parent catalog and schema usage privileges alongside the table SELECT grant.
- Grant only the required USE and SELECT privileges to the analyst group and inspect inherited grants.
- Test the identity-aware filter function, its permissions and supported execution context on representative queries.
- Use supported Databricks secret access with least-privilege scope access and remove secret values from logs.
Explanation
Correct Answer
D:The most appropriate action is: Use supported Databricks secret access with least-privilege scope access and remove secret values from logs. It addresses the stated condition: An application uses an external Azure Key Vault secret scope but a notebook prints the secret into task logs. The technical basis is: Secret retrieval must be paired with permissions and safe handling to avoid disclosure after retrieval. Without this control, A protected secret can still leak through debug output or unrestricted notebook results. A direct acceptance check is: The job authenticates successfully and audited logs contain no raw secret or token.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A contractor receives SELECT on a table yet gets an access-denied error when querying with a three-part name. The corresponding measure there would be: Check required parent catalog and schema usage privileges alongside the table SELECT grant. The stated situation instead calls for: Use supported Databricks secret access with least-privilege scope access and remove secret values from logs.
B:This alternative primarily addresses a different Azure Databricks problem: A finance analyst must read an approved table but was granted broad MODIFY rights at the schema level. The corresponding measure there would be: Grant only the required USE and SELECT privileges to the analyst group and inspect inherited grants. The stated situation instead calls for: Use supported Databricks secret access with least-privilege scope access and remove secret values from logs.
C:This alternative primarily addresses a different Azure Databricks problem: An administrator attaches a row-filter function that returns errors for unauthorized query contexts and treats the issue as data corruption. The corresponding measure there would be: Test the identity-aware filter function, its permissions and supported execution context on representative queries. The stated situation instead calls for: Use supported Databricks secret access with least-privilege scope access and remove secret values from logs.