Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 55
Question 540
A team omits the recommended measure: Test the identity-aware filter function, its permissions and supported execution context on representative queries. Its environment shows: An administrator attaches a row-filter function that returns errors for unauthorized query contexts and treats the issue as data corruption. What is the MOST direct residual risk?
- Manual per-table rules can miss newly created columns and expose sensitive values.
- A broken policy function may deny intended users or expose inconsistent query results if bypassed.
- An object may be available to overprivileged identities in unintended contexts if either boundary is ignored.
- A retired vendor workload can continue reading business data through unchanged credentials.
Explanation
Correct Answer
B:The direct residual exposure is: A broken policy function may deny intended users or expose inconsistent query results if bypassed. The triggering condition is: An administrator attaches a row-filter function that returns errors for unauthorized query contexts and treats the issue as data corruption. The omitted safeguard is: Test the identity-aware filter function, its permissions and supported execution context on representative queries. This matters because: Fine-grained security functions are part of the query execution path and can affect availability. A corrective test would show: Approved and disallowed identities receive the expected records without filter execution errors.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A data compliance team needs consistent redaction of columns tagged as personal data across hundreds of catalog tables. Its preventative action would be: Use centrally governed tags and compatible Unity Catalog ABAC masking policies where supported. The most direct exposure in the current scenario is: A broken policy function may deny intended users or expose inconsistent query results if bypassed.
C:This failure mode belongs mainly to another situation: A team restricts access through a catalog-to-workspace binding but treats it as a replacement for data privileges. Its preventative action would be: Apply both the workspace binding and appropriate Unity Catalog object grants, then test each dimension. The most direct exposure in the current scenario is: A broken policy function may deny intended users or expose inconsistent query results if bypassed.
D:This failure mode belongs mainly to another situation: A vendor's service principal was granted global access for a one-month integration and the project ended. Its preventative action would be: Review and revoke stale grants and any linked service credentials, preserving required internal ownership. The most direct exposure in the current scenario is: A broken policy function may deny intended users or expose inconsistent query results if bypassed.
Question 541
A production data team leaves out: Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options. Its scenario is: An integration performs a one-off initial import of an existing CSV folder into a Delta table. Which failure mode is MOST relevant?
- Appending full inventories multiplies items and distorts on-hand quantities.
- A bespoke long-running stream for a one-time load adds complexity and idle cost.
- A failed data-quality rule cannot be traced back to the event-bus record that produced it.
- Dropping then recreating tables may interrupt readers and lose associated governance settings.
Explanation
Correct Answer
B:The direct residual exposure is: A bespoke long-running stream for a one-time load adds complexity and idle cost. The triggering condition is: An integration performs a one-off initial import of an existing CSV folder into a Delta table. The omitted safeguard is: Use a suitable COPY INTO or supported batch ingestion statement with validated format and path options. This matters because: One-time or incremental file loading can be expressed with SQL ingestion commands without building a persistent streaming service. A corrective test would show: The import reads the intended input files once and reconciles the expected row count and schema.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A daily file load uses append-only mode for periodic snapshots of the same full inventory. Its preventative action would be: Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records. The most direct exposure in the current scenario is: A bespoke long-running stream for a one-time load adds complexity and idle cost.
C:This failure mode belongs mainly to another situation: A pipeline ingests Event Hubs messages into a bronze table but never captures the partition, offset or event timestamp. Its preventative action would be: Persist necessary source metadata and checkpoint lineage for troubleshooting and replay. The most direct exposure in the current scenario is: A bespoke long-running stream for a one-time load adds complexity and idle cost.
D:This failure mode belongs mainly to another situation: An existing table must be rebuilt from corrected historical files while its readers require a consistent atomic replacement. Its preventative action would be: Choose a controlled CREATE OR REPLACE TABLE or compatible atomic table-replacement strategy and validate schema/permissions. The most direct exposure in the current scenario is: A bespoke long-running stream for a one-time load adds complexity and idle cost.
Question 542
The design team proposes: Use a compatible unpivot or stack transformation with explicit month and amount columns. The situation is: A large orders dataset has one column per month but a dashboard expects month-value rows for filtering and grouping. Which technical principle justifies the decision?
- Deduplication should use meaningful business keys, not arbitrary full-row equality alone.
- MERGE combines inserts and updates atomically against a keyed target.
- Append is generally simpler for immutable events than key-by-key mutation logic.
- Unpivoting converts a wide repeated-measure structure into a normalized row representation.
Explanation
Correct Answer
D:The decisive mechanism is: Unpivoting converts a wide repeated-measure structure into a normalized row representation. That explains the selected action: Use a compatible unpivot or stack transformation with explicit month and amount columns. In the original scenario: A large orders dataset has one column per month but a dashboard expects month-value rows for filtering and grouping. The specific unmanaged risk is: Directly summing mismatched month columns can produce fragile and incomplete queries. Validate with: The unpivoted output contains exactly one row per original item-month and preserves totals.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: An incoming orders table contains repeated order IDs because a vendor retransmits the same daily extract. Its targeted response is: Apply a deterministic deduplication rule based on stable order identity and source version before aggregation. The current case is governed more directly by: Unpivoting converts a wide repeated-measure structure into a normalized row representation.
B:This principle concerns a separate data-engineering condition: A product catalog receives corrections for existing SKUs plus entirely new items on each incremental load. Its targeted response is: Use a Delta MERGE keyed by SKU with appropriate matched-update and not-matched-insert logic. The current case is governed more directly by: Unpivoting converts a wide repeated-measure structure into a normalized row representation.
C:This principle concerns a separate data-engineering condition: An append-only telemetry feed includes newly observed events but engineers use MERGE with expensive full-table scans on every insert. Its targeted response is: Use an incremental append pattern with suitable deduplication when source events cannot change. The current case is governed more directly by: Unpivoting converts a wide repeated-measure structure into a normalized row representation.
Question 543
A data product owner adopts: Run supported OPTIMIZE or predictive optimization and validate file-size distribution and query cost. The reason is: A Delta table contains millions of tiny files after frequent microbatch updates and SQL scans spend time opening files. What observed result BEST proves effectiveness?
- A simulated failure yields archived logs, failed stage IDs and a documented root-cause path.
- Before-and-after file statistics show fewer small files with identical row counts and faster queries.
- A benchmark shows lower total maintenance-plus-query cost without compromising retention.
- A skew-aware plan balances task durations and reduces the critical-stage elapsed time.
Explanation
Correct Answer
B:The strongest acceptance proof is: Before-and-after file statistics show fewer small files with identical row counts and faster queries. It specifically verifies: Run supported OPTIMIZE or predictive optimization and validate file-size distribution and query cost. The original issue was: A Delta table contains millions of tiny files after frequent microbatch updates and SQL scans spend time opening files. The underlying mechanism is: File compaction reduces metadata and small-file overhead without changing logical table records. Without it, A growing small-file count can dominate query planning and scan overhead.
Incorrect Answers
A:This result would validate a different measure: Preserve diagnostic events and Spark UI stage evidence before restart where possible. That is relevant to the distinct problem: An operator restarts a failed Databricks cluster before collecting the Spark executor and driver logs. Here the acceptance criterion is: Before-and-after file statistics show fewer small files with identical row counts and faster queries.
C:This result would validate a different measure: Set a measured maintenance cadence or evaluate supported predictive optimization for eligible managed tables. That is relevant to the distinct problem: A new Delta table maintenance job combines repeated OPTIMIZE and VACUUM at every microbatch even though changes are tiny. Here the acceptance criterion is: Before-and-after file statistics show fewer small files with identical row counts and faster queries.
D:This result would validate a different measure: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size. That is relevant to the distinct problem: A production Spark query is slow and its stage graph shows one task running far longer than all peers. Here the acceptance criterion is: Before-and-after file statistics show fewer small files with identical row counts and faster queries.
Question 544
An operator introduces: Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change. The situation is: A Spark workload repeatedly fails with executor out-of-memory errors on a high-cardinality groupBy. Which validation should be required before signoff?
- A controlled representative load finishes without OOM and output group counts reconcile.
- A skew-aware plan balances task durations and reduces the critical-stage elapsed time.
- A representative run shows reduced spill bytes and matching aggregate results with improved runtime.
- A smaller tested configuration completes within SLA at a lower measured cost.
Explanation
Correct Answer
A:The strongest acceptance proof is: A controlled representative load finishes without OOM and output group counts reconcile. It specifically verifies: Profile aggregation key distribution, shuffle partitions and executor memory before choosing a targeted change. The original issue was: A Spark workload repeatedly fails with executor out-of-memory errors on a high-cardinality groupBy. The underlying mechanism is: Wide aggregations redistribute data and can create memory-intensive hot partitions. Without it, blindly restarting the cluster may reproduce the same failure at the same stage.
Incorrect Answers
B:This result would validate a different measure: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size. That is relevant to the distinct problem: A production Spark query is slow and its stage graph shows one task running far longer than all peers. Here the acceptance criterion is: A controlled representative load finishes without OOM and output group counts reconcile.
C:This result would validate a different measure: Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage. That is relevant to the distinct problem: A large aggregation has repeated shuffle spills to disk even though source scans finish quickly. Here the acceptance criterion is: A controlled representative load finishes without OOM and output group counts reconcile.
D:This result would validate a different measure: Use run-level compute utilization, cost and stage metrics to identify unused capacity. That is relevant to the distinct problem: A pipeline experiences escalating costs after a team increases node counts but execution duration barely changes. Here the acceptance criterion is: A controlled representative load finishes without OOM and output group counts reconcile.
Question 545
The following control has NOT been implemented: Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage. In this case: A large aggregation has repeated shuffle spills to disk even though source scans finish quickly. Which exposure remains?
- Adding workers may not speed the one overloaded partition and increases total spend.
- Unused cached data can cause eviction, spilling and unexpected failures for later stages.
- A warehouse sized to average load may breach dashboard p95 latency during business opening.
- Ignoring heavy spills can cause cascading I/O latency and missed pipeline deadlines.
Explanation
Correct Answer
D:The direct residual exposure is: Ignoring heavy spills can cause cascading I/O latency and missed pipeline deadlines. The triggering condition is: A large aggregation has repeated shuffle spills to disk even though source scans finish quickly. The omitted safeguard is: Examine shuffle metrics, memory pressure and partition sizing to target the spilling stage. This matters because: Shuffle spills indicate data exceeded available in-memory execution resources for a stage. A corrective test would show: A representative run shows reduced spill bytes and matching aggregate results with improved runtime.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A production Spark query is slow and its stage graph shows one task running far longer than all peers. Its preventative action would be: Inspect the Spark UI DAG and skewed-key distribution before changing cluster size. The most direct exposure in the current scenario is: Ignoring heavy spills can cause cascading I/O latency and missed pipeline deadlines.
B:This failure mode belongs mainly to another situation: A team enables Spark cache for a one-time ETL dataset and never unpersists it before processing the next large table. Its preventative action would be: Use caching only for beneficial reuse and release persisted state when no longer needed. The most direct exposure in the current scenario is: Ignoring heavy spills can cause cascading I/O latency and missed pipeline deadlines.
C:This failure mode belongs mainly to another situation: An interactive warehouse has high query queues during the morning but low CPU utilization after lunch. Its preventative action would be: Analyze concurrent query load and scaling policies rather than selecting capacity from daily averages. The most direct exposure in the current scenario is: Ignoring heavy spills can cause cascading I/O latency and missed pipeline deadlines.
Question 546
A team omits the recommended measure: Parameterize environment-specific paths, catalogs and identities through governed deployment configuration. Its environment shows: A multi-team notebook uses hard-coded workspace names and credentials, making the same pipeline fail in staging. What is the MOST direct residual risk?
- Readers can combine incompatible versions when related tables update independently.
- An early downstream task can publish partial business totals without raising an obvious failure.
- Hard-coded development resources can cause a production job to read or overwrite the wrong data.
- An upstream failure can remain invisible and leave temporary resources without cleanup.
Explanation
Correct Answer
C:The direct residual exposure is: Hard-coded development resources can cause a production job to read or overwrite the wrong data. The triggering condition is: A multi-team notebook uses hard-coded workspace names and credentials, making the same pipeline fail in staging. The omitted safeguard is: Parameterize environment-specific paths, catalogs and identities through governed deployment configuration. This matters because: Portable pipelines should separate code logic from environment-specific configuration. A corrective test would show: A staging and production run use the same code with distinct approved configuration and no credential leakage.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A regulated pipeline needs consistent transaction boundaries across multiple table writes but updates are published one at a time. Its preventative action would be: Design a compatible atomic publication or versioned readiness contract so consumers see consistent results. The most direct exposure in the current scenario is: Hard-coded development resources can cause a production job to read or overwrite the wrong data.
B:This failure mode belongs mainly to another situation: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. Its preventative action would be: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. The most direct exposure in the current scenario is: Hard-coded development resources can cause a production job to read or overwrite the wrong data.
D:This failure mode belongs mainly to another situation: A source task fails yet the monitoring task never executes because all downstream tasks require upstream success. Its preventative action would be: Design a separate failure-handling or completion branch with the supported run-if logic for alerts and cleanup. The most direct exposure in the current scenario is: Hard-coded development resources can cause a production job to read or overwrite the wrong data.
Question 547
A production data platform experiences this condition: A customer table includes both region hierarchy and customers; teams disagree whether to snowflake descriptive attributes. Which targeted measure should be selected?
- Choose and document a supported format and catalog integration with explicit read/write compatibility tests.
- Evaluate dimensional normalization against query complexity, maintenance and expected cardinality.
- Keep a governed raw or bronze representation before applying irreversible cleansing into silver and gold.
- Use an external Unity Catalog table with an approved registered location instead of assuming managed storage ownership.
Explanation
Correct Answer
B:The most appropriate action is: Evaluate dimensional normalization against query complexity, maintenance and expected cardinality. It addresses the stated condition: A customer table includes both region hierarchy and customers; teams disagree whether to snowflake descriptive attributes. The technical basis is: Data models balance redundancy and query simplicity according to workload requirements. Without this control, unneeded normalization can add joins while denormalization without governance can produce inconsistent descriptions. A direct acceptance check is: Representative business queries return consistent region totals with an acceptable plan.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A cross-engine consumer expects a Delta table but source is plain Iceberg and engineers assume identical mutation semantics. The corresponding measure there would be: Choose and document a supported format and catalog integration with explicit read/write compatibility tests. The stated situation instead calls for: Evaluate dimensional normalization against query complexity, maintenance and expected cardinality.
C:This alternative primarily addresses a different Azure Databricks problem: A bronze ingestion layer stores only final transformed values and loses raw source fields needed for incident replay. The corresponding measure there would be: Keep a governed raw or bronze representation before applying irreversible cleansing into silver and gold. The stated situation instead calls for: Evaluate dimensional normalization against query complexity, maintenance and expected cardinality.
D:This alternative primarily addresses a different Azure Databricks problem: A dataset has a strict requirement that Azure storage ownership and lifecycle remain under a separate platform team. The corresponding measure there would be: Use an external Unity Catalog table with an approved registered location instead of assuming managed storage ownership. The stated situation instead calls for: Evaluate dimensional normalization against query complexity, maintenance and expected cardinality.
Question 548
A pipeline owner declines the action: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. The business situation is: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. What specific failure can follow?
- A bad transformation may make a broken dataset immediately visible to consumers.
- An early downstream task can publish partial business totals without raising an obvious failure.
- At-least-once task execution can duplicate side effects when outputs are append-only.
- Consumers can query invalid data during the interval before quality checks finish.
Explanation
Correct Answer
B:The direct residual exposure is: An early downstream task can publish partial business totals without raising an obvious failure. The triggering condition is: A sales pipeline aggregates gold metrics before the bronze ingestion task finishes and occasionally reports incomplete totals. The omitted safeguard is: Define explicit Lakeflow Jobs task dependencies so silver and gold steps run only after their required predecessors succeed. This matters because: DAG-based orchestration coordinates data readiness and avoids race conditions across dependent tasks. A corrective test would show: A delayed-ingestion test prevents gold refresh until bronze and silver are completed successfully.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A finance transformation writes its output directly into a published table before reconciling totals against the source. Its preventative action would be: Write to a staging target, validate row totals and invariants, then perform the approved publication step. The most direct exposure in the current scenario is: An early downstream task can publish partial business totals without raising an obvious failure.
C:This failure mode belongs mainly to another situation: A nightly pipeline rerun appends the same orders to the target and doubles totals after an operator presses Retry. Its preventative action would be: Use idempotent writes or MERGE keyed by stable order identity for repeatable task execution. The most direct exposure in the current scenario is: An early downstream task can publish partial business totals without raising an obvious failure.
D:This failure mode belongs mainly to another situation: A workflow writes to production tables before a validation task examines null keys and duplicates. Its preventative action would be: Place validations before publishing and use an atomic cutover or controlled target write. The most direct exposure in the current scenario is: An early downstream task can publish partial business totals without raising an obvious failure.
Question 549
The team implements: Select CTAS for an explicitly materialized snapshot or a scheduled pipeline for ongoing refresh. The original problem is: A pipeline uses CREATE TABLE AS SELECT for a one-time derived dataset but later assumes the result refreshes automatically. Which test MOST convincingly demonstrates correctness?
- The purchase remains traceable and later dimension arrival resolves the customer association without duplication.
- The top event-time queries show improved data skipping and reconciled row counts.
- Storage metrics show a manageable file distribution and point lookups meet query SLAs.
- A source update changes the target only through the planned refresh mechanism and produces a recorded watermark.
Explanation
Correct Answer
D:The strongest acceptance proof is: A source update changes the target only through the planned refresh mechanism and produces a recorded watermark. It specifically verifies: Select CTAS for an explicitly materialized snapshot or a scheduled pipeline for ongoing refresh. The original issue was: A pipeline uses CREATE TABLE AS SELECT for a one-time derived dataset but later assumes the result refreshes automatically. The underlying mechanism is: CTAS populates a table at execution time and does not imply continuous synchronization with source. Without it, dashboards can become stale when an initial table snapshot is mistaken for a live derived view.
Incorrect Answers
A:This result would validate a different measure: Define a deliberate unknown-member or late-dimension reconciliation pattern without silently dropping facts. That is relevant to the distinct problem: A late-arriving purchase refers to a customer identifier whose dimension record has not yet been ingested. Here the acceptance criterion is: A source update changes the target only through the planned refresh mechanism and produces a recorded watermark.
B:This result would validate a different measure: Reevaluate data layout using representative event-time filters and a suitable clustering key. That is relevant to the distinct problem: An event stream is partitioned by ingestion day but analysts constantly filter by event timestamp ranges crossing midnight. Here the acceptance criterion is: A source update changes the target only through the planned refresh mechanism and produces a recorded watermark.
C:This result would validate a different measure: Prefer appropriate clustering or lower-cardinality partitioning and assess file sizes for the dimension. That is relevant to the distinct problem: A data engineer chooses the distinct customer ID as a partition directory for an SCD dimension with millions of customers. Here the acceptance criterion is: A source update changes the target only through the planned refresh mechanism and produces a recorded watermark.
Question 550
A solution applies: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys. It addresses: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. Which operational outcome MOST directly verifies success?
- File-count and query-profile comparisons show fewer fragments and faster targeted scans.
- The external reader loads approved columns and a documented compatibility test passes.
- A measured workload benchmark shows scan reduction across the highest-value query templates.
- The top event-time queries show improved data skipping and reconciled row counts.
Explanation
Correct Answer
C:The strongest acceptance proof is: A measured workload benchmark shows scan reduction across the highest-value query templates. It specifically verifies: Compare selective filter patterns and data-skipping statistics before committing partition and clustering keys. The original issue was: A query repeatedly filters by customer, date and product but the physical data design was selected without query-profile evidence. The underlying mechanism is: Efficient layout keys are driven by actual access patterns and file statistics. Without it, poorly chosen keys increase full scans and maintenance overhead as traffic grows.
Incorrect Answers
A:This result would validate a different measure: Use coarse suitable partitioning or liquid clustering selected from filter patterns and cardinality. That is relevant to the distinct problem: An event table is partitioned by the unique transaction identifier and creates millions of tiny partitions. Here the acceptance criterion is: A measured workload benchmark shows scan reduction across the highest-value query templates.
B:This result would validate a different measure: Evaluate Parquet or supported open-table formats against reader compatibility and mutation requirements. That is relevant to the distinct problem: An external sharing integration needs interoperable columnar read files but not native Delta updates. Here the acceptance criterion is: A measured workload benchmark shows scan reduction across the highest-value query templates.
D:This result would validate a different measure: Reevaluate data layout using representative event-time filters and a suitable clustering key. That is relevant to the distinct problem: An event stream is partitioned by ingestion day but analysts constantly filter by event timestamp ranges crossing midnight. Here the acceptance criterion is: A measured workload benchmark shows scan reduction across the highest-value query templates.