Implementing Data Engineering Solutions Using Azure Databricks DP-750 Practice Test 36
Question 350
A platform architect adopts: Reconcile catalog object identity, lineage and current grants after schema or owner changes. For this requirement: A governance team grants dataset access solely by a table's displayed name while records are moved to a new schema. Which rationale MOST directly supports it?
- Object governance is ongoing and must follow organizational role and service identity changes.
- Automatic lineage visibility has collection boundaries and does not imply every real-world dependency is recorded.
- Governance depends on maintained object ownership as teams and responsibilities change.
- Permission reviews should follow actual securable objects and their dependencies rather than name assumptions.
Explanation
Correct Answer
D:The decisive mechanism is: Permission reviews should follow actual securable objects and their dependencies rather than name assumptions. That explains the selected action: Reconcile catalog object identity, lineage and current grants after schema or owner changes. In the original scenario: A governance team grants dataset access solely by a table's displayed name while records are moved to a new schema. The specific unmanaged risk is: A migrated table can retain unintended access or lose required controls after namespace changes. Validate with: An access matrix tests current schema objects and verifies intended principals after migration.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A company needs to review table-owner changes and permissions after organizational restructuring. Its targeted response is: Run a scheduled ownership and grant recertification process using catalog metadata and audit events. The current case is governed more directly by: Permission reviews should follow actual securable objects and their dependencies rather than name assumptions.
B:This principle concerns a separate data-engineering condition: A lineage graph shows no relationship for data loaded outside the monitored platform and the team assumes the dataset has no upstream dependency. Its targeted response is: Validate Unity Catalog lineage capture coverage and supplement unsupported external steps with documented provenance. The current case is governed more directly by: Permission reviews should follow actual securable objects and their dependencies rather than name assumptions.
C:This principle concerns a separate data-engineering condition: An audit asks who owns a catalog table but object ownership still points to an inactive employee. Its targeted response is: Transfer ownership through approved governance workflow and review grants after reassignment. The current case is governed more directly by: Permission reviews should follow actual securable objects and their dependencies rather than name assumptions.
Question 351
A data product owner adopts: Use idempotent deployment methods or reconcile existing resource IDs before creating replacements. The reason is: A REST deployment script creates duplicate jobs every time CI reruns after a timeout. What observed result BEST proves effectiveness?
- A rehearsed rollback restores a prior tested bundle and recovers the expected job definitions.
- CI deploys under the named service principal and the personal token is rejected and revoked.
- Two identical deployment executions leave exactly one intended job with approved settings.
- An injected API rejection makes CI fail with a traceable error and leaves production unchanged.
Explanation
Correct Answer
C:The strongest acceptance proof is: Two identical deployment executions leave exactly one intended job with approved settings. It specifically verifies: Use idempotent deployment methods or reconcile existing resource IDs before creating replacements. The original issue was: A REST deployment script creates duplicate jobs every time CI reruns after a timeout. The underlying mechanism is: Infrastructure automation must tolerate retries without multiplying the intended target resources. Without it, duplicate jobs may run on the same schedule and load data twice.
Incorrect Answers
A:This result would validate a different measure: Tag and retain approved bundle releases and document a verified redeploy or rollback procedure. That is relevant to the distinct problem: An operations group wants to roll back a broken Lakeflow workflow but keeps no previous package version. Here the acceptance criterion is: Two identical deployment executions leave exactly one intended job with approved settings.
B:This result would validate a different measure: Use a secure service identity and protected credential handling for bundle or REST API deployment. That is relevant to the distinct problem: A CI pipeline deploys Databricks assets under an engineer's personal token stored in repository variables. Here the acceptance criterion is: Two identical deployment executions leave exactly one intended job with approved settings.
D:This result would validate a different measure: Validate API status and returned resource identifiers, then use bounded retries only for transient failures. That is relevant to the distinct problem: An automation job uses Databricks REST APIs but omits response-code checks and assumes every HTTP request succeeded. Here the acceptance criterion is: Two identical deployment executions leave exactly one intended job with approved settings.
Question 352
The design team proposes: Create a Unity Catalog view with approved permissions and verify underlying access semantics. The situation is: An analyst wants to expose a filtered projection of a governed table without copying sensitive source rows. Which technical principle justifies the decision?
- The object type should reflect whether data is computed on read or persisted and maintained.
- Delegating within a governed schema supports self-service without tenant-wide administrative access.
- A view can define a query abstraction without requiring a second physical copy of source data.
- Managed versus external tables differ in who manages data paths and deletion behavior.
Explanation
Correct Answer
C:The decisive mechanism is: A view can define a query abstraction without requiring a second physical copy of source data. That explains the selected action: Create a Unity Catalog view with approved permissions and verify underlying access semantics. In the original scenario: An analyst wants to expose a filtered projection of a governed table without copying sensitive source rows. The specific unmanaged risk is: Replicated extracts may fall outside the original source's governance and retention controls. Validate with: Permitted users query only expected columns and rows through the view without a separately exported file.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A data pipeline creates a physical table for a reporting transformation that would be better maintained as a query definition. Its targeted response is: Choose a view, materialized view or stored table based on freshness, cost and refresh responsibilities. The current case is governed more directly by: A view can define a query abstraction without requiring a second physical copy of source data.
B:This principle concerns a separate data-engineering condition: A newly onboarded product team wants to create its own tables but should not have administration rights over every catalog. Its targeted response is: Create a team-specific schema with ownership or delegated creation grants limited to that namespace. The current case is governed more directly by: A view can define a query abstraction without requiring a second physical copy of source data.
D:This principle concerns a separate data-engineering condition: A team expects dropping a managed table to preserve all physical data as though it were an external table. Its targeted response is: Select managed or external Unity Catalog table ownership intentionally and verify lifecycle semantics. The current case is governed more directly by: A view can define a query abstraction without requiring a second physical copy of source data.
Question 353
A solution applies: Keep large results distributed and replace driver collection with bounded sampling or distributed writes. It addresses: A Spark join reads large inputs but the driver reports failures after an analyst calls collect() on the result. Which operational outcome MOST directly verifies success?
- A short benign delay does not page staff while a sustained backlog triggers the documented severity.
- Post-mitigation stage durations are balanced and joined row counts remain correct.
- The job writes results through distributed output and driver memory stays below the approved threshold.
- A simulated failure leaves accessible run logs and correlation identifiers in the approved monitoring destination.
Explanation
Correct Answer
C:The strongest acceptance proof is: The job writes results through distributed output and driver memory stays below the approved threshold. It specifically verifies: Keep large results distributed and replace driver collection with bounded sampling or distributed writes. The original issue was: A Spark join reads large inputs but the driver reports failures after an analyst calls collect() on the result. The underlying mechanism is: Driver memory is not an unlimited sink for results of distributed computation. Without it, collecting billions of rows can crash the driver despite idle executors.
Incorrect Answers
A:This result would validate a different measure: Define meaningful latency, error and backlog thresholds with alert routing based on the business SLO. That is relevant to the distinct problem: An alert triggers on every harmless microbatch delay while real hour-long ingestion stalls go unnoticed. Here the acceptance criterion is: The job writes results through distributed output and driver memory stays below the approved threshold.
B:This result would validate a different measure: Use query-profile evidence and a supported skew mitigation such as adaptive execution or key-aware repartitioning. That is relevant to the distinct problem: A DataFrame join spills heavily because a single join key occurs in most rows. Here the acceptance criterion is: The job writes results through distributed output and driver memory stays below the approved threshold.
D:This result would validate a different measure: Export relevant logs and metrics through a supported Log Analytics or Azure Monitor integration with controlled retention. That is relevant to the distinct problem: An Azure Databricks pipeline has recurring errors but its logs remain only in a short-lived cluster session. Here the acceptance criterion is: The job writes results through distributed output and driver memory stays below the approved threshold.
Question 354
The design team proposes: Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records. The situation is: A daily file load uses append-only mode for periodic snapshots of the same full inventory. Which technical principle justifies the decision?
- Data pipeline completeness requires tracking arrivals rather than assuming an exact wall-clock boundary.
- Atomic table replacement can avoid exposing a partially loaded intermediate table.
- Input extraction semantics determine whether repeated rows are new events or refreshed state.
- Incremental file discovery avoids repeated full-directory scans and supports recoverable progress.
Explanation
Correct Answer
C:The decisive mechanism is: Input extraction semantics determine whether repeated rows are new events or refreshed state. That explains the selected action: Use a snapshot replacement or keyed MERGE strategy instead of appending every full snapshot as new records. In the original scenario: A daily file load uses append-only mode for periodic snapshots of the same full inventory. The specific unmanaged risk is: Appending full inventories multiplies items and distorts on-hand quantities. Validate with: Two full snapshots produce one current inventory row per item and expected updates.
Incorrect Answers
A:This principle concerns a separate data-engineering condition: A Databricks job receives daily structured extracts but an upstream schedule occasionally delivers files late. Its targeted response is: Use incremental file ingestion with explicit late-arrival cutoffs and a measurable source-to-target watermark. The current case is governed more directly by: Input extraction semantics determine whether repeated rows are new events or refreshed state.
B:This principle concerns a separate data-engineering condition: An existing table must be rebuilt from corrected historical files while its readers require a consistent atomic replacement. Its targeted response is: Choose a controlled CREATE OR REPLACE TABLE or compatible atomic table-replacement strategy and validate schema/permissions. The current case is governed more directly by: Input extraction semantics determine whether repeated rows are new events or refreshed state.
D:This principle concerns a separate data-engineering condition: An Azure storage account receives tens of thousands of small parquet files per hour and a notebook lists the whole container on every run. Its targeted response is: Use Auto Loader with durable checkpointing and an appropriate discovery mode to ingest new files incrementally. The current case is governed more directly by: Input extraction semantics determine whether repeated rows are new events or refreshed state.
Question 355
An Azure Databricks architect reviews the following problem: An engineering pipeline attempts to insert records into a production Delta table using an account with SELECT only. What should the team implement FIRST?
- Configure a tested Unity Catalog row filter or suitable ABAC policy based on the authorized identity.
- Replace committed personal credentials with supported workload authentication and revoke exposed tokens.
- Use a supported Azure managed identity and Unity Catalog storage credential or equivalent approved access path.
- Grant a dedicated writer identity the minimum supported modification permissions and revoke broad user write grants.
Explanation
Correct Answer
D:The most appropriate action is: Grant a dedicated writer identity the minimum supported modification permissions and revoke broad user write grants. It addresses the stated condition: An engineering pipeline attempts to insert records into a production Delta table using an account with SELECT only. The technical basis is: Read and write operations require distinct Unity Catalog privileges and operating identities. Without this control, giving all analysts write rights merely to repair one pipeline expands accidental corruption risk. A direct acceptance check is: The writer inserts a validated test row and reader accounts are denied modification attempts.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A user must see only the East region rows of a common sales table while managers see all territories. The corresponding measure there would be: Configure a tested Unity Catalog row filter or suitable ABAC policy based on the authorized identity. The stated situation instead calls for: Grant a dedicated writer identity the minimum supported modification permissions and revoke broad user write grants.
B:This alternative primarily addresses a different Azure Databricks problem: A production workload uses a personal access token stored in Git for Databricks REST calls. The corresponding measure there would be: Replace committed personal credentials with supported workload authentication and revoke exposed tokens. The stated situation instead calls for: Grant a dedicated writer identity the minimum supported modification permissions and revoke broad user write grants.
C:This alternative primarily addresses a different Azure Databricks problem: A scheduled cloud-data ingestion needs an Azure storage identity but engineers embed a reusable account key inside notebooks. The corresponding measure there would be: Use a supported Azure managed identity and Unity Catalog storage credential or equivalent approved access path. The stated situation instead calls for: Grant a dedicated writer identity the minimum supported modification permissions and revoke broad user write grants.
Question 356
An implementation skips: Use Auto Loader with durable checkpointing and an appropriate discovery mode to ingest new files incrementally. Engineers are observing: An Azure storage account receives tens of thousands of small parquet files per hour and a notebook lists the whole container on every run. What consequence is MOST likely?
- Driver out-of-memory can fail the load despite adequate executor resources.
- Repeated listing and manual offsets can miss files or become too expensive as volume grows.
- Replayed events can double-count orders in downstream aggregates.
- Ignoring changed source structures can halt ingestion or silently lose fields.
Explanation
Correct Answer
B:The direct residual exposure is: Repeated listing and manual offsets can miss files or become too expensive as volume grows. The triggering condition is: An Azure storage account receives tens of thousands of small parquet files per hour and a notebook lists the whole container on every run. The omitted safeguard is: Use Auto Loader with durable checkpointing and an appropriate discovery mode to ingest new files incrementally. This matters because: Incremental file discovery avoids repeated full-directory scans and supports recoverable progress. A corrective test would show: After a restart, already committed files are not duplicated and newly arrived files are ingested.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A Python notebook reads a massive table into pandas on the driver before processing data intended for Spark executors. Its preventative action would be: Use Spark-native distributed read and transformation APIs or partition-safe processing. The most direct exposure in the current scenario is: Repeated listing and manual offsets can miss files or become too expensive as volume grows.
C:This failure mode belongs mainly to another situation: An ingestion pipeline reads cloud events with at-least-once delivery and assumes every event ID appears only once. Its preventative action would be: Deduplicate by stable event identity and business time using documented state and watermark rules. The most direct exposure in the current scenario is: Repeated listing and manual offsets can miss files or become too expensive as volume grows.
D:This failure mode belongs mainly to another situation: A file feed introduces an unexpected optional column and the stream fails after the first changed file. Its preventative action would be: Configure supported schema evolution or rescued-data handling with explicit compatibility review. The most direct exposure in the current scenario is: Repeated listing and manual offsets can miss files or become too expensive as volume grows.
Question 357
A data engineering team faces this Azure Databricks requirement: A developer changes a transformation with a user-visible business effect but omits acceptance testing by the data product owner. Which action is MOST appropriate?
- Define and validate environment-specific catalog, workspace and identity targets in deployment configuration.
- Add representative unit and integration tests asserting transformation results and schema contracts.
- Add user acceptance tests for the relevant reporting contract before production promotion.
- Store deployment source in Git with reviewed branches and pull requests before promoting changes.
Explanation
Correct Answer
C:The most appropriate action is: Add user acceptance tests for the relevant reporting contract before production promotion. It addresses the stated condition: A developer changes a transformation with a user-visible business effect but omits acceptance testing by the data product owner. The technical basis is: Technical tests cannot replace business approval of the reported meaning of data. Without this control, A numerically correct but semantically changed field can mislead business users. A direct acceptance check is: A product owner signs off a sample report and its reconciled acceptance criteria before rollout.
Incorrect Answers
A:This alternative primarily addresses a different Azure Databricks problem: A multi-environment setup uses the same catalog for test and production because bundle target variables were omitted. The corresponding measure there would be: Define and validate environment-specific catalog, workspace and identity targets in deployment configuration. The stated situation instead calls for: Add user acceptance tests for the relevant reporting contract before production promotion.
B:This alternative primarily addresses a different Azure Databricks problem: A release modifies transformation logic but tests only whether Python files compile. The corresponding measure there would be: Add representative unit and integration tests asserting transformation results and schema contracts. The stated situation instead calls for: Add user acceptance tests for the relevant reporting contract before production promotion.
D:This alternative primarily addresses a different Azure Databricks problem: Engineers edit production notebooks directly and lose the ability to identify which code change broke a pipeline. The corresponding measure there would be: Store deployment source in Git with reviewed branches and pull requests before promoting changes. The stated situation instead calls for: Add user acceptance tests for the relevant reporting contract before production promotion.
Question 358
An implementation skips: Define a stable contract through managed views and test compatibility before changing source objects. Engineers are observing: A reporting service needs to keep a stable presentation schema while underlying source columns are renamed by engineering. What consequence is MOST likely?
- An ownerless shared view may expose unreviewed columns or become impossible to maintain safely.
- A broadly shared catalog can expose regulated data to unintended region workspaces.
- Unregistered working files can bypass object discovery and authorization reviews.
- Direct downstream table references can break when upstream columns are renamed.
Explanation
Correct Answer
D:The direct residual exposure is: Direct downstream table references can break when upstream columns are renamed. The triggering condition is: A reporting service needs to keep a stable presentation schema while underlying source columns are renamed by engineering. The omitted safeguard is: Define a stable contract through managed views and test compatibility before changing source objects. This matters because: Abstraction views can separate a consumer-facing contract from internal physical table evolution. A corrective test would show: Contract tests continue to select documented columns through the published view after a staged schema change.
Incorrect Answers
A:This failure mode belongs mainly to another situation: A central team creates global views that hide source objects but forgets to document ownership and access grants. Its preventative action would be: Assign view ownership and narrowly scoped permissions under the appropriate Unity Catalog schema. The most direct exposure in the current scenario is: Direct downstream table references can break when upstream columns are renamed.
B:This failure mode belongs mainly to another situation: A platform architect creates one catalog for subsidiaries with incompatible regional data-residency permissions. Its preventative action would be: Split catalogs or governance boundaries along residency and ownership requirements and validate workspace bindings. The most direct exposure in the current scenario is: Direct downstream table references can break when upstream columns are renamed.
C:This failure mode belongs mainly to another situation: A departmental data owner stores raw file attachments inside ad hoc DBFS directories instead of a governable data space. Its preventative action would be: Use Unity Catalog volumes with suitable schema and file privileges for supported non-tabular files. The most direct exposure in the current scenario is: Direct downstream table references can break when upstream columns are renamed.
Question 359
The team chooses: Run end-to-end contract tests for dependent readers and obtain owner approval for breaking changes. Its observed issue is: A sensitive SQL view is renamed in the release but the team does not test downstream dashboards before promotion. What is the strongest technical basis?
- Schema and object naming changes can break consumers even when new ETL succeeds.
- Release readiness includes environment contracts and authorization beyond pure transformation logic.
- Deployment isolation depends on correct values for each environment's object references.
- Automated releases should not depend on personal accounts or broadly exposed long-lived credentials.
Explanation
Correct Answer
A:The decisive mechanism is: Schema and object naming changes can break consumers even when new ETL succeeds. That explains the selected action: Run end-to-end contract tests for dependent readers and obtain owner approval for breaking changes. In the original scenario: A sensitive SQL view is renamed in the release but the team does not test downstream dashboards before promotion. The specific unmanaged risk is: A release may finish successfully while critical reports cannot resolve their expected view. Validate with: A staging dashboard loads its documented fields and a breaking-change simulation is rejected.
Incorrect Answers
B:This principle concerns a separate data-engineering condition: A new pipeline passes unit tests but production fails because the service principal lacks a catalog grant. Its targeted response is: Include integration tests of identity, Unity Catalog permissions and destination connectivity in staging. The current case is governed more directly by: Schema and object naming changes can break consumers even when new ETL succeeds.
C:This principle concerns a separate data-engineering condition: A multi-environment setup uses the same catalog for test and production because bundle target variables were omitted. Its targeted response is: Define and validate environment-specific catalog, workspace and identity targets in deployment configuration. The current case is governed more directly by: Schema and object naming changes can break consumers even when new ETL succeeds.
D:This principle concerns a separate data-engineering condition: A CI pipeline deploys Databricks assets under an engineer's personal token stored in repository variables. Its targeted response is: Use a secure service identity and protected credential handling for bundle or REST API deployment. The current case is governed more directly by: Schema and object naming changes can break consumers even when new ETL succeeds.
Question 360
A data product owner adopts: Implement idempotent foreachBatch or compatible transactional sink logic keyed by epoch or business identity. The reason is: A stream writes into an external sink without stable batch identities and causes duplicate side effects on retry. What observed result BEST proves effectiveness?
- Replaying the same microbatch produces one target effect per business key.
- A maintenance simulation retains the checkpoint and resumes ingestion from the expected offset.
- A deliberately delayed file is ingested exactly once within the allowed arrival window.
- A changed customer field reaches the approved target within the measured freshness SLO.
Explanation
Correct Answer
A:The strongest acceptance proof is: Replaying the same microbatch produces one target effect per business key. It specifically verifies: Implement idempotent foreachBatch or compatible transactional sink logic keyed by epoch or business identity. The original issue was: A stream writes into an external sink without stable batch identities and causes duplicate side effects on retry. The underlying mechanism is: Structured Streaming may retry microbatches, so external effects need idempotency. Without it, retries can duplicate invoices or notifications even if the source checkpoint is correct.
Incorrect Answers
B:This result would validate a different measure: Store checkpoint and schema tracking files in durable protected locations with retention matching the stream lifecycle. That is relevant to the distinct problem: A long-running ingestion job persists its checkpoint in the same temporary folder that daily cleanup deletes. Here the acceptance criterion is: Replaying the same microbatch produces one target effect per business key.
C:This result would validate a different measure: Use incremental file ingestion with explicit late-arrival cutoffs and a measurable source-to-target watermark. That is relevant to the distinct problem: A Databricks job receives daily structured extracts but an upstream schedule occasionally delivers files late. Here the acceptance criterion is: Replaying the same microbatch produces one target effect per business key.
D:This result would validate a different measure: Use supported streaming or sufficiently frequent incremental CDC with measured end-to-end latency. That is relevant to the distinct problem: A team selects batch processing for customer account changes though downstream applications require minute-scale freshness. Here the acceptance criterion is: Replaying the same microbatch produces one target effect per business key.