Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 87
Question 860
An implementation skips: Check federation connector capabilities and distinguish read-only remote access from supported writable local tables. Engineers are observing: A foreign catalog is connected to a remote database but an analyst expects standard managed Delta writes against it. What consequence is MOST likely?
- An environment-default schema can route production data definitions into an incorrect catalog.
- An unreviewed semantic instruction can return a plausible but incorrect business measure.
- Assuming full DDL or mutation support may cause failed deployments or inadvertent source changes.
- Users may query raw unvalidated records as though they were approved curated metrics.
Explanation
Correct Answer
C:The direct residual exposure is: Assuming full DDL or mutation support may cause failed deployments or inadvertent source changes. The triggering condition is: A foreign catalog is connected to a remote database but an analyst expects standard managed Delta writes against it. The omitted safeguard is: Check federation connector capabilities and distinguish read-only remote access from supported writable local tables. This matters because: Foreign catalog semantics and connector capabilities differ from native managed Delta table operations. A corrective test would show: A capability test documents supported query and write operations and prevents unsupported mutations.
Incorrect Answers
A:This failure mode belongs mainly to another situation: An automation pipeline issues CREATE TABLE without qualifying the target catalog and schema. Its preventative action would be: Use fully qualified names and a controlled default namespace in deployment SQL and notebooks. The most direct exposure in the current scenario is: Assuming full DDL or mutation support may cause failed deployments or inadvertent source changes.
B:This failure mode belongs mainly to another situation: An AI/BI Genie space is configured with ambiguous metric terminology and produces misleading natural-language interpretations. Its preventative action would be: Add reviewed Genie instructions, table descriptions and example questions consistent with trusted catalog semantics. The most direct exposure in the current scenario is: Assuming full DDL or mutation support may cause failed deployments or inadvertent source changes.
D:This failure mode belongs mainly to another situation: Multiple data products put raw, refined and curated assets into a single schema with confusing table names. Its preventative action would be: Define a consistent schema and naming architecture separating layers and business domains by purpose. The most direct exposure in the current scenario is: Assuming full DDL or mutation support may cause failed deployments or inadvertent source changes.
Question 861
An Azure Databricks architect reviews the following problem: An incoming orders table contains repeated order IDs because a vendor retransmits the same daily extract. What should the team implement FIRST?
- Apply a deterministic deduplication rule based on stable order identity and source version before aggregation.
- Define and test consistent trimming, Unicode normalization and case comparison for the business key.
- Use an appropriate intersect or semi-join operation based on set and duplicate requirements.
- Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans.
Explanation
Correct Answer
A:The most appropriate action is: Apply a deterministic deduplication rule based on stable order identity and source version before aggregation. It addresses the stated condition: An incoming orders table contains repeated order IDs because a vendor retransmits the same daily extract. The technical basis is: Deduplication should use meaningful business keys, not arbitrary full-row equality alone. Without this control, duplicate orders inflate sales totals and can trigger incorrect downstream payouts. A direct acceptance check is: A seeded repeated-ID sample yields one chosen current order version and reconciled totals.
Incorrect Answers
B:This alternative primarily addresses a different Azure Databricks problem: A customer-name cleansing script uses an arbitrary lowercase conversion without preserving Unicode and leading/trailing whitespace rules. The corresponding measure there would be: Define and test consistent trimming, Unicode normalization and case comparison for the business key. The stated situation instead calls for: Apply a deterministic deduplication rule based on stable order identity and source version before aggregation.
C:This alternative primarily addresses a different Azure Databricks problem: A team wants only customer IDs present in both systems but uses an inner join that multiplies repeated rows. The corresponding measure there would be: Use an appropriate intersect or semi-join operation based on set and duplicate requirements. The stated situation instead calls for: Apply a deterministic deduplication rule based on stable order identity and source version before aggregation.
D:This alternative primarily addresses a different Azure Databricks problem: A pipeline joins a huge fact table with a tiny dimension and a normal shuffle join creates excessive network traffic. The corresponding measure there would be: Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans. The stated situation instead calls for: Apply a deterministic deduplication rule based on stable order identity and source version before aggregation.
Question 862
A solution applies: Review and revoke stale grants and any linked service credentials, preserving required internal ownership. It addresses: A vendor's service principal was granted global access for a one-month integration and the project ended. Which operational outcome MOST directly verifies success?
- An approved DDL operation creates the external table at a registered location and an arbitrary path is rejected.
- After revocation the vendor principal cannot list or query protected objects while internal jobs remain healthy.
- The job authenticates successfully and audited logs contain no raw secret or token.
- Approved and disallowed identities receive the expected records without filter execution errors.
Explanation
Correct Answer
B:The strongest acceptance proof is: After revocation the vendor principal cannot list or query protected objects while internal jobs remain healthy. It specifically verifies: Review and revoke stale grants and any linked service credentials, preserving required internal ownership. The original issue was: A vendor's service principal was granted global access for a one-month integration and the project ended. The underlying mechanism is: Least privilege includes removing access when the business purpose expires. Without it, A retired vendor workload can continue reading business data through unchanged credentials.
Incorrect Answers
A:This result would validate a different measure: Configure a supported external location and specific creation privileges rather than bypassing the catalog to write directly. That is relevant to the distinct problem: A user is granted CREATE TABLE but cannot create an external table at an unregistered cloud path. Here the acceptance criterion is: After revocation the vendor principal cannot list or query protected objects while internal jobs remain healthy.
C:This result would validate a different measure: Use supported Databricks secret access with least-privilege scope access and remove secret values from logs. That is relevant to the distinct problem: An application uses an external Azure Key Vault secret scope but a notebook prints the secret into task logs. Here the acceptance criterion is: After revocation the vendor principal cannot list or query protected objects while internal jobs remain healthy.
D:This result would validate a different measure: Test the identity-aware filter function, its permissions and supported execution context on representative queries. That is relevant to the distinct problem: An administrator attaches a row-filter function that returns errors for unauthorized query contexts and treats the issue as data corruption. Here the acceptance criterion is: After revocation the vendor principal cannot list or query protected objects while internal jobs remain healthy.
Question 863
The following control has NOT been implemented: Reevaluate data layout using representative event-time filters and a suitable clustering key. In this case: An event stream is partitioned by ingestion day but analysts constantly filter by event timestamp ranges crossing midnight. Which exposure remains?
- A mismatched partition scheme forces broad scans across unrelated arrival-day files.
- Poorly chosen keys increase full scans and maintenance overhead as traffic grows.
- Without recoverable raw inputs the team cannot replay a transformation after discovering a rule defect.
- Unnecessary overwrites increase compute cost and can cause data loss during partial reruns.
Explanation
Correct Answer
A:The direct residual exposure is: A mismatched partition scheme forces broad scans across unrelated arrival-day files. The triggering condition is: An event stream is partitioned by ingestion day but analysts constantly filter by event timestamp ranges crossing midnight. The omitted safeguard is: Reevaluate data layout using representative event-time filters and a suitable clustering key. This matters because: Physical layout should match selective consumer predicates rather than convenience of ingestion. A corrective test would show: The top event-time queries show improved data skipping and reconciled row counts.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. Its preventative action would be: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys. The most direct exposure in the current scenario is: A mismatched partition scheme forces broad scans across unrelated arrival-day files.
C:This failure mode belongs mainly to another situation: A bronze ingestion layer stores only final transformed values and loses raw source fields needed for incident replay. Its preventative action would be: Keep a governed raw or bronze representation before applying irreversible cleansing into silver and gold. The most direct exposure in the current scenario is: A mismatched partition scheme forces broad scans across unrelated arrival-day files.
D:This failure mode belongs mainly to another situation: A raw event lake contains append-only records but a developer chooses daily full overwrite of the curated Delta table. Its preventative action would be: Choose incremental append or merge based on immutable versus mutable source behavior. The most direct exposure in the current scenario is: A mismatched partition scheme forces broad scans across unrelated arrival-day files.
Question 864
A Databricks engineer recommends: Wait for or explicitly verify required external or asynchronous work before reporting task success. It responds to: A job is successful in the orchestrator but the business target is not updated because the notebook returns before async work completes. What underlying mechanism BEST explains why?
- Task success must correspond to the completion of required data side effects, not just function invocation.
- Job task state transitions follow dependency and conditional execution rules.
- Retries cannot repair missing tables or invalid schema references without corrective action.
- Automatic recovery should be limited by failure budgets and should not hide persistent faults.
Explanation
Correct Answer
A:The decisive mechanism is: Task success must correspond to the completion of required data side effects, not just function invocation. That explains the selected action: Wait for or explicitly verify required external or asynchronous work before reporting task success. In the original scenario: A job is successful in the orchestrator but the business target is not updated because the notebook returns before async work completes. The specific unmanaged risk is: Premature success can start downstream tasks against incomplete writes. Validate with: A test run finishes only after the target watermark and expected row counts are confirmed.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A job owner cannot explain why a task skipped execution after a predecessor failed. Its targeted response is: Review explicit dependency and run-if settings and validate skip behavior through a staged failure. The current case is governed more directly by: Task success must correspond to the completion of required data side effects, not just function invocation.
C:This principle concerns a separate data-engineering condition: A job's retry policy restarts a deterministic bad SQL query six times even though the table is missing. Its targeted response is: Limit retries to appropriate transient errors and set accountable failure handling for permanent logic problems. The current case is governed more directly by: Task success must correspond to the completion of required data side effects, not just function invocation.
D:This principle concerns a separate data-engineering condition: A continuously running pipeline is scheduled to restart automatically but restart storms consume a growing compute budget. Its targeted response is: Configure bounded restarts and alerting with root-cause investigation for repeated failures. The current case is governed more directly by: Task success must correspond to the completion of required data side effects, not just function invocation.
Question 865
An operator introduces: Separate the small validation into an eligible lightweight or serverless task and measure total end-to-end runtime. The situation is: A data team uses a high-power job cluster for a low-volume data-quality check that spends most runtime starting the cluster. Which validation should be required before signoff?
- A representative run measures total cost and elapsed time, including initialization, and meets the freshness budget.
- A concurrent-user benchmark meets percentile latency and queue-depth targets with acceptable cost.
- Before-after measurements report comparable inputs, stage timings, spills and total cost.
- Spark UI and driver metrics show bounded driver memory after correcting the collection pattern.
Explanation
Correct Answer
A:The strongest acceptance proof is: A representative run measures total cost and elapsed time, including initialization, and meets the freshness budget. It specifically verifies: Separate the small validation into an eligible lightweight or serverless task and measure total end-to-end runtime. The original issue was: A data team uses a high-power job cluster for a low-volume data-quality check that spends most runtime starting the cluster. The underlying mechanism is: End-to-end task time includes compute startup and orchestration overhead, not just transformation CPU time. Without it, overprovisioned validation can cost more in startup overhead than the data being checked warrants.
Incorrect Answers
B:This result would validate a different measure: Tune supported SQL warehouse capacity and concurrency scaling based on simultaneous production query measurements. That is relevant to the distinct problem: A BI warehouse repeatedly slows during morning sign-in peaks even though a single analyst query runs quickly. Here the acceptance criterion is: A representative run measures total cost and elapsed time, including initialization, and meets the freshness budget.
C:This result would validate a different measure: Collect Spark stage profiles and test a targeted compute or Photon setting against comparable data. That is relevant to the distinct problem: A team turns on every performance setting for a workload without benchmarking whether disk spills or network shuffle dominate runtime. Here the acceptance criterion is: A representative run measures total cost and elapsed time, including initialization, and meets the freshness budget.
D:This result would validate a different measure: Inspect driver collection and query planning behavior before simply enlarging every worker. That is relevant to the distinct problem: A SQL transformation consumes large amounts of memory on the driver despite executor tasks processing most source partitions. Here the acceptance criterion is: A representative run measures total cost and elapsed time, including initialization, and meets the freshness budget.
Question 866
A pipeline owner declines the action: Validate per-target bundle variable resolution and compare rendered job definitions before deployment. The business situation is: A data engineer updates a bundled job trigger but another environment's deployment silently uses the wrong schedule. What specific failure can follow?
- A faulty last-minute edit can create an untested release with no accountable reviewer.
- Skew and memory limits may remain invisible until a full-scale run misses its SLA.
- A wrong schedule can duplicate loads or miss critical reporting windows.
- A deployment may succeed while the first production job cannot access its storage or schema.
Explanation
Correct Answer
C:The direct residual exposure is: A wrong schedule can duplicate loads or miss critical reporting windows. The triggering condition is: A data engineer updates a bundled job trigger but another environment's deployment silently uses the wrong schedule. The omitted safeguard is: Validate per-target bundle variable resolution and compare rendered job definitions before deployment. This matters because: Configuration-as-code requires environment-specific parameter checks as well as common source control. A corrective test would show: Deployment plans show the intended trigger for each target and a test job fires at the correct interval.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A release process merges unreviewed notebook changes directly into the main branch minutes before a production run. Its preventative action would be: Protect the main branch with peer review, required tests and controlled promotion windows. The most direct exposure in the current scenario is: A wrong schedule can duplicate loads or miss critical reporting windows.
B:This failure mode belongs mainly to another situation: An engineering test suite passes on miniature in-memory data yet fails in production on skewed billion-row joins. Its preventative action would be: Include performance and end-to-end tests using representative scale and data distributions. The most direct exposure in the current scenario is: A wrong schedule can duplicate loads or miss critical reporting windows.
D:This failure mode belongs mainly to another situation: A new pipeline passes unit tests but production fails because the service principal lacks a catalog grant. Its preventative action would be: Include integration tests of identity, Unity Catalog permissions and destination connectivity in staging. The most direct exposure in the current scenario is: A wrong schedule can duplicate loads or miss critical reporting windows.
Question 867
The team chooses: Configure supported Databricks audit logging and route relevant events to a secured retention destination. Its observed issue is: A compliance auditor requests a record of who changed Unity Catalog grants but audit logs are not collected. What is the strongest technical basis?
- Security governance requires attributable administrative activity and durable evidence.
- Delta Sharing can provide governed data access without distributing underlying storage account keys.
- Retention policy must account for physical file cleanup as well as logical table content.
- ABAC can scale consistent data protection across tables matching policy attributes.
Explanation
Correct Answer
A:The decisive mechanism is: Security governance requires attributable administrative activity and durable evidence. That explains the selected action: Configure supported Databricks audit logging and route relevant events to a secured retention destination. In the original scenario: A compliance auditor requests a record of who changed Unity Catalog grants but audit logs are not collected. The specific unmanaged risk is: Unauthorized grants can remain undetected or be impossible to trace. Validate with: A test GRANT or REVOKE event appears with actor, timestamp and target in the audit sink.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A data-sharing project emails storage keys to an external partner for reading curated Delta results. Its targeted response is: Use Delta Sharing with an authorized recipient, narrowly scoped shares and lifecycle management. The current case is governed more directly by: Security governance requires attributable administrative activity and durable evidence.
C:This principle concerns a separate data-engineering condition: A regulation requires keeping event records for 90 days but a job performs VACUUM after only a few hours. Its targeted response is: Coordinate Delta retention configuration, VACUUM safety settings and formal recovery requirements. The current case is governed more directly by: Security governance requires attributable administrative activity and durable evidence.
D:This principle concerns a separate data-engineering condition: A central security team manually adds row masking rules to every department table and new datasets repeatedly lack protection. Its targeted response is: Create compatible central ABAC policies based on governed classifications and verify inherited coverage. The current case is governed more directly by: Security governance requires attributable administrative activity and durable evidence.
Question 868
An operator introduces: Apply a controlled pivot with a defined set of expected categories and aggregation rule. The situation is: A quality report needs one column for each product status category but stores status as separate rows. Which validation should be required before signoff?
- A sample includes all approved categories and status totals reconcile to source rows.
- A controlled source sample produces exactly the permitted group keys and reconciled sums.
- The unpivoted output contains exactly one row per original item-month and preserves totals.
- Spark UI shows reduced shuffle bytes and matching results without executor out-of-memory.
Explanation
Correct Answer
A:The strongest acceptance proof is: A sample includes all approved categories and status totals reconcile to source rows. It specifically verifies: Apply a controlled pivot with a defined set of expected categories and aggregation rule. The original issue was: A quality report needs one column for each product status category but stores status as separate rows. The underlying mechanism is: Pivoting transposes category values into columns but requires stable categories and aggregation semantics. Without it, unknown statuses may be dropped or duplicated if pivot categories are assumed without validation.
Incorrect Answers
B:This result would validate a different measure: Apply the required filters and join conditions before aggregation and validate authorized output groups. That is relevant to the distinct problem: An analyst calculates total revenue grouped by country but a join includes countries not approved for reporting. Here the acceptance criterion is: A sample includes all approved categories and status totals reconcile to source rows.
C:This result would validate a different measure: Use a compatible unpivot or stack transformation with explicit month and amount columns. That is relevant to the distinct problem: A large orders dataset has one column per month but a dashboard expects month-value rows for filtering and grouping. Here the acceptance criterion is: A sample includes all approved categories and status totals reconcile to source rows.
D:This result would validate a different measure: Evaluate a suitable broadcast join based on measured dimension size and Spark execution plans. That is relevant to the distinct problem: A pipeline joins a huge fact table with a tiny dimension and a normal shuffle join creates excessive network traffic. Here the acceptance criterion is: A sample includes all approved categories and status totals reconcile to source rows.
Question 869
The team chooses: Use a supported change-data-capture feed and apply insert, update and delete semantics to the target. Its observed issue is: A SaaS database updates and deletes customer records but a pipeline appends every nightly full extract to a Delta table. What is the strongest technical basis?
- Data pipeline completeness requires tracking arrivals rather than assuming an exact wall-clock boundary.
- Checkpoint loss removes the stream's reliable progress and state record.
- Structured Streaming may retry microbatches, so external effects need idempotency.
- CDC propagates source mutations without duplicating full snapshots.
Explanation
Correct Answer
D:The decisive mechanism is: CDC propagates source mutations without duplicating full snapshots. That explains the selected action: Use a supported change-data-capture feed and apply insert, update and delete semantics to the target. In the original scenario: A SaaS database updates and deletes customer records but a pipeline appends every nightly full extract to a Delta table. The specific unmanaged risk is: Append-only full extracts can multiply current records and leave deleted source rows active. Validate with: Insert-update-delete test events produce one correct current target record per key.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A Databricks job receives daily structured extracts but an upstream schedule occasionally delivers files late. Its targeted response is: Use incremental file ingestion with explicit late-arrival cutoffs and a measurable source-to-target watermark. The current case is governed more directly by: CDC propagates source mutations without duplicating full snapshots.
B:This principle concerns a separate data-engineering condition: A long-running ingestion job persists its checkpoint in the same temporary folder that daily cleanup deletes. Its targeted response is: Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle. The current case is governed more directly by: CDC propagates source mutations without duplicating full snapshots.
C:This principle concerns a separate data-engineering condition: A stream writes into an external sink without stable batch identities and causes duplicate side effects on retry. Its targeted response is: Implement idempotent foreachBatch or compatible transactional sink logic keyed by epoch or business identity. The current case is governed more directly by: CDC propagates source mutations without duplicating full snapshots.
Question 870
A platform architect adopts: Check federation connector capabilities and distinguish read-only remote access from supported writable local tables. For this requirement: A foreign catalog is connected to a remote database but an analyst expects standard managed Delta writes against it. Which rationale MOST directly supports it?
- Foreign catalog semantics and connector capabilities differ from native managed Delta table operations.
- The object type should reflect whether data is computed on read or persisted and maintained.
- Managed versus external tables differ in who manages data paths and deletion behavior.
- Governance of views includes ownership, discovery and authorization just as for tables.
Explanation
Correct Answer
A:The decisive mechanism is: Foreign catalog semantics and connector capabilities differ from native managed Delta table operations. That explains the selected action: Check federation connector capabilities and distinguish read-only remote access from supported writable local tables. In the original scenario: A foreign catalog is connected to a remote database but an analyst expects standard managed Delta writes against it. The specific unmanaged risk is: Assuming full DDL or mutation support may cause failed deployments or inadvertent source changes. Validate with: A capability test documents supported query and write operations and prevents unsupported mutations.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A data pipeline creates a physical table for a reporting transformation that would be better maintained as a query definition. Its targeted response is: Choose a view, materialized view or stored table based on freshness, cost and refresh responsibilities. The current case is governed more directly by: Foreign catalog semantics and connector capabilities differ from native managed Delta table operations.
C:This principle concerns a separate data-engineering condition: A team expects dropping a managed table to preserve all physical data as though it were an external table. Its targeted response is: Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics. The current case is governed more directly by: Foreign catalog semantics and connector capabilities differ from native managed Delta table operations.
D:This principle concerns a separate data-engineering condition: A central team creates global views that hide source objects but forgets to document ownership and access grants. Its targeted response is: Assign view ownership and narrowly scoped permissions under the appropriate Unity Catalog schema. The current case is governed more directly by: Foreign catalog semantics and connector capabilities differ from native managed Delta table operations.