Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 79
Question 780
An operator introduces: Deduplicate by stable event identity and business time using documented state and watermark rules. The situation is: An ingestion pipeline reads cloud events with at-least-once delivery and assumes every event ID appears only once. Which validation should be required before signoff?
- A shuffled CDC test results in the correct highest-version value after replay.
- A duplicate-event test yields one current business event while preserving valid distinct events.
- A deliberately delayed file is ingested exactly once within the allowed arrival window.
- An event batch survives a controlled consumer restart and downstream totals match the expected sample.
Explanation
Correct Answer
B:The strongest acceptance proof is: A duplicate-event test yields one current business event while preserving valid distinct events. It specifically verifies: Deduplicate by stable event identity and business time using documented state and watermark rules. The original issue was: An ingestion pipeline reads cloud events with at-least-once delivery and assumes every event ID appears only once. The underlying mechanism is: Delivery guarantees alone do not remove duplicate producer or consumer events. Without it, replayed events can double-count orders in downstream aggregates.
Incorrect Answers
A:This result would validate a different measure: Apply ordered CDC sequencing using source change version or timestamp and consistent delete handling. That is relevant to the distinct problem: A transactional feed produces out-of-order CDC events and a naive MERGE overwrites a newer status with an older update. Here the acceptance criterion is: A duplicate-event test yields one current business event while preserving valid distinct events.
C:This result would validate a different measure: Use incremental file ingestion with explicit late-arrival cutoffs and a measurable source-to-target watermark. That is relevant to the distinct problem: A Databricks job receives daily structured extracts but an upstream schedule occasionally delivers files late. Here the acceptance criterion is: A duplicate-event test yields one current business event while preserving valid distinct events.
D:This result would validate a different measure: Use a supported Spark Structured Streaming or Lakeflow streaming source with an Event Hubs connection and checkpoint. That is relevant to the distinct problem: A product emits append-only telemetry to Azure Event Hubs and dashboards need near-real-time processing. Here the acceptance criterion is: A duplicate-event test yields one current business event while preserving valid distinct events.
Question 781
A team omits the recommended measure: Choose a view, materialized view or stored table based on freshness, cost and refresh responsibilities. Its environment shows: A data pipeline creates a physical table for a reporting transformation that would be better maintained as a query definition. What is the MOST direct residual risk?
- Assuming full DDL or mutation support may cause failed deployments or inadvertent source changes.
- Credentials embedded in notebooks can be copied and used outside the intended catalog authorization.
- Replicated extracts may fall outside the original source's governance and retention controls.
- Persisting every intermediate query can create stale and duplicate datasets.
Explanation
Correct Answer
D:The direct residual exposure is: Persisting every intermediate query can create stale and duplicate datasets. The triggering condition is: A data pipeline creates a physical table for a reporting transformation that would be better maintained as a query definition. The omitted safeguard is: Choose a view, materialized view or stored table based on freshness, cost and refresh responsibilities. This matters because: The object type should reflect whether data is computed on read or persisted and maintained. A corrective test would show: An architecture test proves the selected object's refresh and query cost meet defined needs.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A foreign catalog is connected to a remote database but an analyst expects standard managed Delta writes against it. Its preventative action would be: Check federation connector capabilities and distinguish read-only remote access from supported writable local tables. The most direct exposure in the current scenario is: Persisting every intermediate query can create stale and duplicate datasets.
B:This failure mode belongs mainly to another situation: A legacy external Delta table points at a cloud location with an unmanaged storage credential stored in notebook text. Its preventative action would be: Register supported external locations and storage credentials, then define governed external tables. The most direct exposure in the current scenario is: Persisting every intermediate query can create stale and duplicate datasets.
C:This failure mode belongs mainly to another situation: An analyst wants to expose a filtered projection of a governed table without copying sensitive source rows. Its preventative action would be: Create a Unity Catalog view with approved permissions and verify underlying access semantics. The most direct exposure in the current scenario is: Persisting every intermediate query can create stale and duplicate datasets.
Question 782
A solution applies: Evaluate supported table update triggers or a clear upstream job dependency based on actual availability. It addresses: A pipeline must run when an upstream Databricks table is updated but analysts have hardcoded a daily time estimate. Which operational outcome MOST directly verifies success?
- A simulated stall exceeds the threshold, stops correctly and alerts the support team.
- A delayed upstream refresh starts the dependent job only after the designated table event.
- A test file arrives and starts one valid job run without repeated empty polling.
- A missing-table test stops with the expected error while a transient failure recovers according to policy.
Explanation
Correct Answer
B:The strongest acceptance proof is: A delayed upstream refresh starts the dependent job only after the designated table event. It specifically verifies: Evaluate supported table update triggers or a clear upstream job dependency based on actual availability. The original issue was: A pipeline must run when an upstream Databricks table is updated but analysts have hardcoded a daily time estimate. The underlying mechanism is: Event-driven triggers align pipeline execution to upstream readiness where supported. Without it, A clock-only trigger can start before the dependency has produced the expected dataset.
Incorrect Answers
A:This result would validate a different measure: Set task and job timeout limits with notifications and a documented retry and recovery route. That is relevant to the distinct problem: A long-running job occasionally hangs but there is no configured runtime ceiling or escalation. Here the acceptance criterion is: A delayed upstream refresh starts the dependent job only after the designated table event.
C:This result would validate a different measure: Use a supported file-arrival trigger where its prerequisites and source location are compatible. That is relevant to the distinct problem: A cloud file feed needs a job triggered by arriving files instead of polling every minute. Here the acceptance criterion is: A delayed upstream refresh starts the dependent job only after the designated table event.
D:This result would validate a different measure: Limit retries to appropriate transient errors and set accountable failure handling for permanent logic problems. That is relevant to the distinct problem: A job's retry policy restarts a deterministic bad SQL query six times even though the table is missing. Here the acceptance criterion is: A delayed upstream refresh starts the dependent job only after the designated table event.
Question 783
A pipeline owner declines the action: Implement a catalog stewardship process to review metadata and governed tags at dataset publication. The business situation is: A central repository contains schemas with missing column owners and no sensitivity classifications. What specific failure can follow?
- Consumers can use incorrectly labeled confidential fields without recognizing their sensitivity.
- Old cleanup tasks can destroy files that the revised retention requirement still protects.
- Historical evidence may be lost even when the latest dataset remains complete.
- A migrated table can retain unintended access or lose required controls after namespace changes.
Explanation
Correct Answer
A:The direct residual exposure is: Consumers can use incorrectly labeled confidential fields without recognizing their sensitivity. The triggering condition is: A central repository contains schemas with missing column owners and no sensitivity classifications. The omitted safeguard is: Implement a catalog stewardship process to review metadata and governed tags at dataset publication. This matters because: Governance metadata is useful only when ownership and classification are kept current. A corrective test would show: A publish-time check rejects missing required descriptions, owners and classification tags.
Incorrect Answers
B:This failure mode belongs mainly to another situation: A regulated source dataset receives an updated retention period but storage cleanup still follows the obsolete policy. Its preventative action would be: Align table retention properties, scheduled maintenance and legal ownership approvals with the current classification. The most direct exposure in the current scenario is: Consumers can use incorrectly labeled confidential fields without recognizing their sensitivity.
C:This failure mode belongs mainly to another situation: A product owner assumes a nightly Delta table overwrite meets a legal hold requirement for old records. Its preventative action would be: Design record-level retention and external legal-hold workflows appropriate to the applicable policy. The most direct exposure in the current scenario is: Consumers can use incorrectly labeled confidential fields without recognizing their sensitivity.
D:This failure mode belongs mainly to another situation: A governance team grants dataset access solely by a table's displayed name while records are moved to a new schema. Its preventative action would be: Reconcile catalog object identity, lineage and current grants after schema or owner changes. The most direct exposure in the current scenario is: Consumers can use incorrectly labeled confidential fields without recognizing their sensitivity.
Question 784
A platform architect adopts: Configure a supported external location and specific creation privileges rather than bypassing the catalog to write directly. For this requirement: A user is granted CREATE TABLE but cannot create an external table at an unregistered cloud path. Which rationale MOST directly supports it?
- Row-level security evaluates access rules at query time rather than relying on users to remember a WHERE clause.
- Data access delegation need not transfer administrative ownership of data objects.
- Cloud IAM and Unity Catalog are complementary authorization layers that must both permit the intended operation.
- Creating an external table requires governed location and storage authorization beyond a generic schema creation grant.
Explanation
Correct Answer
D:The decisive mechanism is: Creating an external table requires governed location and storage authorization beyond a generic schema creation grant. That explains the selected action: Configure a supported external location and specific creation privileges rather than bypassing the catalog to write directly. In the original scenario: A user is granted CREATE TABLE but cannot create an external table at an unregistered cloud path. The specific unmanaged risk is: Direct-path writes can bypass registration and leave unowned storage artifacts. Validate with: An approved DDL operation creates the external table at a registered location and an arbitrary path is rejected.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A user must see only the East region rows of a common sales table while managers see all territories. Its targeted response is: Configure a tested Unity Catalog row filter or suitable ABAC policy based on the authorized identity. The current case is governed more directly by: Creating an external table requires governed location and storage authorization beyond a generic schema creation grant.
B:This principle concerns a separate data-engineering condition: A data steward wants to grant temporary read access without handing over Unity Catalog ownership of a schema. Its targeted response is: Issue time-bounded group membership or an approved scoped SELECT grant with an access review. The current case is governed more directly by: Creating an external table requires governed location and storage authorization beyond a generic schema creation grant.
C:This principle concerns a separate data-engineering condition: A managed identity can read a storage container but a newly created external table is denied in Databricks. Its targeted response is: Align Azure storage role assignments with the registered Unity Catalog storage credential and external-location privileges. The current case is governed more directly by: Creating an external table requires governed location and storage authorization beyond a generic schema creation grant.
Question 785
A solution applies: Choose a controlled CREATE OR REPLACE TABLE or compatible atomic table-replacement strategy and validate schema/permissions. It addresses: An existing table must be rebuilt from corrected historical files while its readers require a consistent atomic replacement. Which operational outcome MOST directly verifies success?
- Readers observe a complete validated target before or after cutover, not partial data.
- A changed customer field reaches the approved target within the measured freshness SLO.
- A sampled bronze row links to the expected stream partition and source event time.
- After a restart, already committed files are not duplicated and newly arrived files are ingested.
Explanation
Correct Answer
A:The strongest acceptance proof is: Readers observe a complete validated target before or after cutover, not partial data. It specifically verifies: Choose a controlled CREATE OR REPLACE TABLE or compatible atomic table-replacement strategy and validate schema/permissions. The original issue was: An existing table must be rebuilt from corrected historical files while its readers require a consistent atomic replacement. The underlying mechanism is: Atomic table replacement can avoid exposing a partially loaded intermediate table. Without it, dropping then recreating tables may interrupt readers and lose associated governance settings.
Incorrect Answers
B:This result would validate a different measure: Use supported streaming or sufficiently frequent incremental CDC with measured end-to-end latency. That is relevant to the distinct problem: A team selects batch processing for customer account changes though downstream applications require minute-scale freshness. Here the acceptance criterion is: Readers observe a complete validated target before or after cutover, not partial data.
C:This result would validate a different measure: Persist necessary source metadata and checkpoint lineage for troubleshooting and replay. That is relevant to the distinct problem: A pipeline ingests Event Hubs messages into a bronze table but never captures the partition, offset or event timestamp. Here the acceptance criterion is: Readers observe a complete validated target before or after cutover, not partial data.
D:This result would validate a different measure: Use Auto Loader with durable checkpointing and an appropriate discovery mode to ingest new files incrementally. That is relevant to the distinct problem: An Azure storage account receives tens of thousands of small parquet files per hour and a notebook lists the whole container on every run. Here the acceptance criterion is: Readers observe a complete validated target before or after cutover, not partial data.
Question 786
A data product owner adopts: Enable supported liquid clustering with appropriate keys and run compatible optimization. The reason is: A large Delta fact table is filtered primarily on high-cardinality account IDs with changing query patterns. What observed result BEST proves effectiveness?
- A representative account-filter query reads fewer irrelevant files after optimized clustering.
- A controlled late event appears in the correct historical period according to the approved policy.
- The platform team retains approved storage lifecycle authority while Databricks queries the registered data.
- A replay from a preserved source batch reproduces the expected silver and gold data after a code fix.
Explanation
Correct Answer
A:The strongest acceptance proof is: A representative account-filter query reads fewer irrelevant files after optimized clustering. It specifically verifies: Enable supported liquid clustering with appropriate keys and run compatible optimization. The original issue was: A large Delta fact table is filtered primarily on high-cardinality account IDs with changing query patterns. The underlying mechanism is: Liquid clustering can organize files for selective filters without static partition boundaries. Without it, rigid directory partitioning can cause imbalance and require costly redesign as filters change.
Incorrect Answers
B:This result would validate a different measure: Design event-time fields, ingestion timestamps and late-data behavior suited to replay and temporal queries. That is relevant to the distinct problem: A lakehouse receives sensor observations with event timestamps that arrive out of order. Here the acceptance criterion is: A representative account-filter query reads fewer irrelevant files after optimized clustering.
C:This result would validate a different measure: Use an external Unity Catalog table with an approved registered location instead of assuming managed storage ownership. That is relevant to the distinct problem: A dataset has a strict requirement that Azure storage ownership and lifecycle remain under a separate platform team. Here the acceptance criterion is: A representative account-filter query reads fewer irrelevant files after optimized clustering.
D:This result would validate a different measure: Keep a governed raw or bronze representation before applying irreversible cleansing into silver and gold. That is relevant to the distinct problem: A bronze ingestion layer stores only final transformed values and loses raw source fields needed for incident replay. Here the acceptance criterion is: A representative account-filter query reads fewer irrelevant files after optimized clustering.
Question 787
The recommended measure is: Choose one compatible data layout strategy based on table format, platform support and query evidence. The operational context is: A team attempts to apply both ZORDER and liquid clustering to the same Delta table. Which data engineering concept applies?
- Partition pruning benefits can be outweighed by extremely fragmented high-cardinality directory layouts.
- Liquid clustering and traditional ZORDER/partitioning have compatibility constraints and different maintenance paths.
- File and table format choices depend on transactional features and consumer interoperability.
- A star schema separates measurable events from descriptive dimensions and controls join cardinality.
Explanation
Correct Answer
B:The decisive mechanism is: Liquid clustering and traditional ZORDER/partitioning have compatibility constraints and different maintenance paths. That explains the selected action: Choose one compatible data layout strategy based on table format, platform support and query evidence. In the original scenario: A team attempts to apply both ZORDER and liquid clustering to the same Delta table. The specific unmanaged risk is: Mixing incompatible layout mechanisms can fail operations or leave the team relying on nonexistent optimizations. Validate with: A supported optimization command completes and query plans show the chosen layout works.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. Its targeted response is: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. The current case is governed more directly by: Liquid clustering and traditional ZORDER/partitioning have compatibility constraints and different maintenance paths.
C:This principle concerns a separate data-engineering condition: An external sharing integration needs interoperable columnar read files but not native Delta updates. Its targeted response is: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements. The current case is governed more directly by: Liquid clustering and traditional ZORDER/partitioning have compatibility constraints and different maintenance paths.
D:This principle concerns a separate data-engineering condition: A finance dashboard needs monthly revenue by customer and must join customer dimensions without multiplying facts. Its targeted response is: Design a conformed star schema with explicit fact grain and unique dimension keys. The current case is governed more directly by: Liquid clustering and traditional ZORDER/partitioning have compatibility constraints and different maintenance paths.
Question 788
For this Azure Databricks issue: An alert triggers on every harmless microbatch delay while real hour-long ingestion stalls go unnoticed. Engineers choose: Define meaningful latency, error and backlog thresholds with alert routing based on the business SLO. What acceptance evidence is BEST?
- Load testing shows completion within the target window and queued runs do not overlap unsafely.
- A short benign delay does not page staff while a sustained backlog triggers the documented severity.
- A peak-load rehearsal stabilizes backlog and keeps end-to-end event age within target.
- A controlled transient outage is repaired without rerunning completed expensive ingestion.
Explanation
Correct Answer
B:The strongest acceptance proof is: A short benign delay does not page staff while a sustained backlog triggers the documented severity. It specifically verifies: Define meaningful latency, error and backlog thresholds with alert routing based on the business SLO. The original issue was: An alert triggers on every harmless microbatch delay while real hour-long ingestion stalls go unnoticed. The underlying mechanism is: Alert quality depends on actionable thresholds and distinguishing transient variation from sustained failure. Without it, excessive false alerts can obscure serious freshness incidents.
Incorrect Answers
A:This result would validate a different measure: Investigate stage runtime growth and enforce appropriate concurrency policy before raising frequency. That is relevant to the distinct problem: A job scheduled each hour starts running for 80 minutes and overlaps the next run. Here the acceptance criterion is: A short benign delay does not page staff while a sustained backlog triggers the documented severity.
C:This result would validate a different measure: Compare incoming event rate, processed records, watermark progress and state-store metrics before scaling. That is relevant to the distinct problem: A streaming pipeline's throughput falls although event arrival increases and its input backlog grows over time. Here the acceptance criterion is: A short benign delay does not page staff while a sustained backlog triggers the documented severity.
D:This result would validate a different measure: Use Lakeflow Jobs repair at the failed task when dependency outputs remain valid. That is relevant to the distinct problem: A job fails during a temporary Azure storage outage but operators rerun every successful upstream task. Here the acceptance criterion is: A short benign delay does not page staff while a sustained backlog triggers the documented severity.
Question 789
The design team proposes: Adopt a portable naming and quoting convention compatible with Unity Catalog SQL identifiers. The situation is: An ingestion team creates a table with a reserved or inconsistent name and automation fails in one environment. Which technical principle justifies the decision?
- Consistent identifiers reduce accidental parsing and namespace resolution differences.
- Governance of views includes ownership, discovery and authorization just as for tables.
- The object type should reflect whether data is computed on read or persisted and maintained.
- Managed versus external tables differ in who manages data paths and deletion behavior.
Explanation
Correct Answer
A:The decisive mechanism is: Consistent identifiers reduce accidental parsing and namespace resolution differences. That explains the selected action: Adopt a portable naming and quoting convention compatible with Unity Catalog SQL identifiers. In the original scenario: An ingestion team creates a table with a reserved or inconsistent name and automation fails in one environment. The specific unmanaged risk is: An inconsistent identifier forces manual quoting and breaks repeatable deployment scripts. Validate with: Automated DDL completes in both staging and production under the same naming rules.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A central team creates global views that hide source objects but forgets to document ownership and access grants. Its targeted response is: Assign view ownership and narrowly scoped permissions under the appropriate Unity Catalog schema. The current case is governed more directly by: Consistent identifiers reduce accidental parsing and namespace resolution differences.
C:This principle concerns a separate data-engineering condition: A data pipeline creates a physical table for a reporting transformation that would be better maintained as a query definition. Its targeted response is: Choose a view, materialized view or stored table based on freshness, cost and refresh responsibilities. The current case is governed more directly by: Consistent identifiers reduce accidental parsing and namespace resolution differences.
D:This principle concerns a separate data-engineering condition: A team expects dropping a managed table to preserve all physical data as though it were an external table. Its targeted response is: Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics. The current case is governed more directly by: Consistent identifiers reduce accidental parsing and namespace resolution differences.
Question 790
A team omits the recommended measure: Set a measured maintenance cadence or evaluate supported predictive optimization for eligible managed tables. Its environment shows: A new Delta table maintenance job combines repeated OPTIMIZE and VACUUM at every microbatch even though changes are tiny. What is the MOST direct residual risk?
- Scaling a query against irrelevant files can increase cost without proportional performance gains.
- Over-maintenance can spend more resources rewriting files than it saves in queries.
- Unused cached data can cause eviction, spilling and unexpected failures for later stages.
- Collecting billions of rows can crash the driver despite idle executors.
Explanation
Correct Answer
B:The direct residual exposure is: Over-maintenance can spend more resources rewriting files than it saves in queries. The triggering condition is: A new Delta table maintenance job combines repeated OPTIMIZE and VACUUM at every microbatch even though changes are tiny. The omitted safeguard is: Set a measured maintenance cadence or evaluate supported predictive optimization for eligible managed tables. This matters because: Maintenance operations themselves consume compute and should correspond to data growth and read patterns. A corrective test would show: A benchmark shows lower total maintenance-plus-query cost without compromising retention.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A warehouse query remains slow even after larger compute is allocated and the profile reports scanning nearly every file. Its preventative action would be: Inspect query predicates and Delta data skipping or clustering layout before increasing warehouse size again. The most direct exposure in the current scenario is: Over-maintenance can spend more resources rewriting files than it saves in queries.
C:This failure mode belongs mainly to another situation: A team enables Spark cache for a one-time ETL dataset and never unpersists it before processing the next large table. Its preventative action would be: Use caching only for beneficial reuse and release persisted state when no longer needed. The most direct exposure in the current scenario is: Over-maintenance can spend more resources rewriting files than it saves in queries.
D:This failure mode belongs mainly to another situation: A Spark join reads large inputs but the driver reports failures after an analyst calls collect() on the result. Its preventative action would be: Keep large results distributed and replace driver collection with bounded sampling or distributed writes. The most direct exposure in the current scenario is: Over-maintenance can spend more resources rewriting files than it saves in queries.