Certified Data Engineer Associate Practice Test 58
Question 570
At a city utilities office, one shuffle stage has a single task reading gigabytes more data than its peers and dominates total execution time. The dataset `experiment_samples` is in scope, and the proposed fix needs before-and-after measurements. What verification provides evidence of correct behavior under the original adverse condition?
- Run only a normal, unchanged `experiment_samples` sample and review max versus median shuffle read.
- Review a platform-level availability report for `experiment_samples` without testing whether one heavily skewed key concentrates work in a straggling shuffle partition.
- Apply the selected partitioning change and compare longest task time, spill and aggregate correctness
- Check that the intended configuration exists for `experiment_samples` and that a healthy run succeeded, without inducing the earlier failure.
Explanation
Correct Answer
C:Apply the selected partitioning change and compare longest task time, spill and aggregate correctness. This reproduces the scenario-dependent condition, verifies max versus median shuffle read, stage task times, spill bytes, and heavy key counts, and directly tests whether one heavily skewed key concentrates work in a straggling shuffle partition is resolved. Observed results, not intent, determine approval.
Incorrect Answers
A:Run only a normal, unchanged `experiment_samples` sample and review max versus median shuffle read. A normal sample does not challenge one heavily skewed key concentrates work in a straggling shuffle partition.
B:Review a platform-level availability report for `experiment_samples` without testing whether one heavily skewed key concentrates work in a straggling shuffle partition. Aggregate availability does not establish the behavior of affected `experiment_samples` records. It cannot replace the direct exercise: apply the selected partitioning change and compare longest task time, spill and aggregate correctness.
D:Check that the intended configuration exists for `experiment_samples` and that a healthy run succeeded, without inducing the earlier failure. Installed configuration and a healthy run are insufficient to prove the failure mode is gone. The original issue was one heavily skewed key concentrates work in a straggling shuffle partition.
Question 571
At a scientific imaging provider, a new Delta table is frequently filtered on high-cardinality customer_id while daily partitions create thousands of tiny files. The dataset `account_statements` is in scope, and fault analysis must preserve actual stage-level evidence. Which investigation best tests the stated constraint with direct data?
- Correlate query filter columns, file count and size across affected and unaffected `R034-CASE-115` runs.
- Review completed run status and aggregate cluster health without record-level correlation
- Compare the current configuration and the signed change approval, without source-row evidence
- Examine only a normal successful run of `account_statements` and trend query filter columns without recreating the fault.
Explanation
Correct Answer
A:Correlate query filter columns, file count and size across affected and unaffected `R034-CASE-115` runs.. By joining query filter columns, file count and size, clustering configuration, and query profile across the affected and unaffected conditions, the investigation directly tests the recorded mechanism: rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern. It also sets a measurable baseline for compare selective customer queries before and after clustering and optimization on representative data.
Incorrect Answers
B:Review completed run status and aggregate cluster health without record-level correlation. A successful aggregate platform state can coexist with rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern. This check never observes the affected `account_statements` records or their causal path.
C:Compare the current configuration and the signed change approval, without source-row evidence. A desired configuration and approval ticket are not runtime proof. In this case they omit query filter columns, file count and size, clustering configuration, and query profile, the direct means of distinguishing rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern.
D:Examine only a normal successful run of `account_statements` and trend query filter columns without recreating the fault. A healthy-run trend can be useful, but it does not expose rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern. The failing-path query filter columns, file count and size, clustering configuration, and query profile has to be correlated before attributing cause.
Question 572
At a scientific imaging provider, a new Delta table is frequently filtered on high-cardinality customer_id while daily partitions create thousands of tiny files. The dataset `account_statements` is in scope, and fault analysis must preserve actual stage-level evidence. Which option targets the actual technical dependency rather than an incidental symptom?
- Enable supported liquid clustering on useful query keys and use optimization to organize files based on actual filters
- Partition by the high-cardinality customer identifier into millions of folders; use that approach for `account_statements`.
- Add ZORDER and liquid clustering simultaneously to the same table; use that approach for `account_statements`.
- Disable all compaction because more files increase parallelism; use that approach for `account_statements`.
Explanation
Correct Answer
A:Enable supported liquid clustering on useful query keys and use optimization to organize files based on actual filters. This directly addresses rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern. The design should be accepted only after compare selective customer queries before and after clustering and optimization on representative data; the related operational boundary is to allow data layout to evolve with workloads rather than hardwire high-cardinality partitions.
Incorrect Answers
B:Partition by the high-cardinality customer identifier into millions of folders; use that approach for `account_statements`. Extreme partition cardinality increases metadata and small-file overhead. For `account_statements`, this cannot remove the causal limitation: rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern.
C:Add ZORDER and liquid clustering simultaneously to the same table; use that approach for `account_statements`. Liquid clustering and ZORDER are mutually incompatible table-layout strategies. For `account_statements`, this cannot remove the causal limitation: rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern.
D:Disable all compaction because more files increase parallelism; use that approach for `account_statements`. Excess small files can worsen file discovery and query setup overhead. For `account_statements`, this cannot remove the causal limitation: rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern.
Question 573
At a scientific imaging provider, a new Delta table is frequently filtered on high-cardinality customer_id while daily partitions create thousands of tiny files. The dataset `account_statements` is in scope, and fault analysis must preserve actual stage-level evidence. A teammate proposes: "Partition by the high-cardinality customer identifier into millions of folders." Which technical limitation is the decisive reason not to approve that specific proposal?
- Excess small files can worsen file discovery and query setup overhead. This would apply to an alternative change for `account_statements`.
- Extreme partition cardinality increases metadata and small-file overhead. The proposal does not repair `account_statements`.
- Liquid clustering and ZORDER are mutually incompatible table-layout strategies. This would apply to another option for `account_statements`.
- The task is waiting for its next schedule; Spark stage metrics and task failure records are irrelevant.
Explanation
Correct Answer
B:Extreme partition cardinality increases metadata and small-file overhead. The proposal does not repair `account_statements`. This identifies the known limitation of the exact proposed action. Extreme partition cardinality increases metadata and small-file overhead. The underlying problem remains rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern; Enable supported liquid clustering on useful query keys and use optimization to organize files based on actual filters instead targets that failure.
Incorrect Answers
A:Excess small files can worsen file discovery and query setup overhead. This would apply to an alternative change for `account_statements`. This failure applies to the separate proposal "Disable all compaction because more files increase parallelism".
C:Liquid clustering and ZORDER are mutually incompatible table-layout strategies. This would apply to another option for `account_statements`. This limitation belongs to the different alternative "Add ZORDER and liquid clustering simultaneously to the same table". The stem calls for the specific failure of "Partition by the high-cardinality customer identifier into millions of folders".
D:The task is waiting for its next schedule; Spark stage metrics and task failure records are irrelevant. The case provides a concrete failing data or control path; a mere alternate theory does not address rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern.
Question 574
At a scientific imaging provider, a new Delta table is frequently filtered on high-cardinality customer_id while daily partitions create thousands of tiny files. The dataset `account_statements` is in scope, and fault analysis must preserve actual stage-level evidence. What operational boundary should be documented for approval?
- Disable all compaction because more files increase parallelism
- Add ZORDER and liquid clustering simultaneously to the same table
- Allow data layout to evolve with workloads rather than hardwire high-cardinality partitions
- Partition by the high-cardinality customer identifier into millions of folders; monitor results only after launch of `account_statements`.
Explanation
Correct Answer
C:Allow data layout to evolve with workloads rather than hardwire high-cardinality partitions. This explicitly bounds the compromise while still addressing rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern. Acceptance relies on compare selective customer queries before and after clustering and optimization on representative data rather than simply declaring the change complete.
Incorrect Answers
A:Disable all compaction because more files increase parallelism. Excess small files can worsen file discovery and query setup overhead. Monitoring after deployment does not make this approach a safe bounded tradeoff for `account_statements`.
B:Add ZORDER and liquid clustering simultaneously to the same table. Liquid clustering and ZORDER are mutually incompatible table-layout strategies. Monitoring after deployment does not make this approach a safe bounded tradeoff for `account_statements`.
D:Partition by the high-cardinality customer identifier into millions of folders; monitor results only after launch of `account_statements`. Extreme partition cardinality increases metadata and small-file overhead. Monitoring after deployment does not make this approach a safe bounded tradeoff for `account_statements`.
Question 575
At a scientific imaging provider, a new Delta table is frequently filtered on high-cardinality customer_id while daily partitions create thousands of tiny files. The dataset `account_statements` is in scope, and fault analysis must preserve actual stage-level evidence. Which validation should be required in the release record?
- Check that the intended configuration exists for `account_statements` and that a healthy run succeeded, without inducing the earlier failure.
- Review a platform-level availability report for `account_statements` without testing whether rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern.
- Compare selective customer queries before and after clustering and optimization on representative data
- Run only a normal, unchanged `account_statements` sample and review query filter columns.
Explanation
Correct Answer
C:Compare selective customer queries before and after clustering and optimization on representative data. This reproduces the scenario-dependent condition, verifies query filter columns, file count and size, clustering configuration, and query profile, and directly tests whether rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern is resolved. Observed results, not intent, determine approval.
Incorrect Answers
A:Check that the intended configuration exists for `account_statements` and that a healthy run succeeded, without inducing the earlier failure. Installed configuration and a healthy run are insufficient to prove the failure mode is gone. The original issue was rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern.
B:Review a platform-level availability report for `account_statements` without testing whether rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern. Aggregate availability does not establish the behavior of affected `account_statements` records. It cannot replace the direct exercise: compare selective customer queries before and after clustering and optimization on representative data.
D:Run only a normal, unchanged `account_statements` sample and review query filter columns. A normal sample does not challenge rigid daily partitioning creates file fragmentation and does not match the selective customer lookup pattern.
Question 576
At an agricultural sensor program, a Unity Catalog managed Delta table has rising maintenance overhead, but the operator is manually running daily file compaction regardless of workload. The dataset `fraud_signals` is in scope, and the cost increase must be justified by better completion times. Before choosing a remedy, which investigation actually distinguishes the failure mechanism from a green dashboard?
- Inspect normal-period success metrics for the stated dataset without reproducing the source error
- Correlate table type, predictive optimization eligibility across affected and unaffected `R034-CASE-116` runs.
- Compare the current configuration and the signed change approval, without source-row evidence
- Review completed run status and aggregate cluster health without record-level correlation
Explanation
Correct Answer
B:Correlate table type, predictive optimization eligibility across affected and unaffected `R034-CASE-116` runs.. By joining table type, predictive optimization eligibility, maintenance events, file layout, and query duration across the affected and unaffected conditions, the investigation directly tests the recorded mechanism: fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions. It also sets a measurable baseline for enable the supported setting on a test managed table and compare maintenance actions and query cost.
Incorrect Answers
A:Inspect normal-period success metrics for the stated dataset without reproducing the source error. A healthy-run trend can be useful, but it does not expose fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions. The failing-path table type, predictive optimization eligibility, maintenance events, file layout, and query duration has to be correlated before attributing cause.
C:Compare the current configuration and the signed change approval, without source-row evidence. A desired configuration and approval ticket are not runtime proof. In this case they omit table type, predictive optimization eligibility, maintenance events, file layout, and query duration, the direct means of distinguishing fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions.
D:Review completed run status and aggregate cluster health without record-level correlation. A successful aggregate platform state can coexist with fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions. This check never observes the affected `fraud_signals` records or their causal path.
Question 577
At an agricultural sensor program, a Unity Catalog managed Delta table has rising maintenance overhead, but the operator is manually running daily file compaction regardless of workload. The dataset `fraud_signals` is in scope, and the cost increase must be justified by better completion times. Which implementation best resolves the stated defect without breaking the business constraint?
- Run unbounded OPTIMIZE on every table each hour regardless of need; use that approach for `fraud_signals`.
- Use unsupported layout features on an external table without checking eligibility; use that approach for `fraud_signals`.
- Evaluate eligible predictive optimization to automate supported maintenance and monitor cost and table performance
- Delete the table history manually as a substitute for automated optimization; use that approach for `fraud_signals`.
Explanation
Correct Answer
C:Evaluate eligible predictive optimization to automate supported maintenance and monitor cost and table performance. This directly addresses fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions. The design should be accepted only after enable the supported setting on a test managed table and compare maintenance actions and query cost; the related operational boundary is to accept controlled automated maintenance where supported rather than blanket scheduled rewrites.
Incorrect Answers
A:Run unbounded OPTIMIZE on every table each hour regardless of need; use that approach for `fraud_signals`. Indiscriminate compaction adds significant compute cost for unchanged tables. For `fraud_signals`, this cannot remove the causal limitation: fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions.
B:Use unsupported layout features on an external table without checking eligibility; use that approach for `fraud_signals`. Feature support differs by table management and runtime, so eligibility must be established. For `fraud_signals`, this cannot remove the causal limitation: fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions.
D:Delete the table history manually as a substitute for automated optimization; use that approach for `fraud_signals`. Destructive history removal is not an appropriate performance management policy. For `fraud_signals`, this cannot remove the causal limitation: fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions.
Question 578
At an agricultural sensor program, a Unity Catalog managed Delta table has rising maintenance overhead, but the operator is manually running daily file compaction regardless of workload. The dataset `fraud_signals` is in scope, and the cost increase must be justified by better completion times. A teammate proposes: "Delete the table history manually as a substitute for automated optimization." Which technical limitation is the decisive reason not to approve that specific proposal?
- The task is waiting for its next schedule; Spark stage metrics and task failure records are irrelevant.
- Destructive history removal is not an appropriate performance management policy. The proposal does not repair `fraud_signals`.
- Feature support differs by table management and runtime, so eligibility must be established. This would apply to another option for `fraud_signals`.
- Indiscriminate compaction adds significant compute cost for unchanged tables. This would apply to an alternative change for `fraud_signals`.
Explanation
Correct Answer
B:Destructive history removal is not an appropriate performance management policy. The proposal does not repair `fraud_signals`. This identifies the known limitation of the exact proposed action. Destructive history removal is not an appropriate performance management policy. The underlying problem remains fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions; Evaluate eligible predictive optimization to automate supported maintenance and monitor cost and table performance instead targets that failure.
Incorrect Answers
A:The task is waiting for its next schedule; Spark stage metrics and task failure records are irrelevant. The case provides a concrete failing data or control path; a mere alternate theory does not address fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions.
C:Feature support differs by table management and runtime, so eligibility must be established. This would apply to another option for `fraud_signals`. This limitation belongs to the different alternative "Use unsupported layout features on an external table without checking eligibility". The stem calls for the specific failure of "Delete the table history manually as a substitute for automated optimization".
D:Indiscriminate compaction adds significant compute cost for unchanged tables. This would apply to an alternative change for `fraud_signals`. This failure applies to the separate proposal "Run unbounded OPTIMIZE on every table each hour regardless of need".
Question 579
At an agricultural sensor program, a Unity Catalog managed Delta table has rising maintenance overhead, but the operator is manually running daily file compaction regardless of workload. The dataset `fraud_signals` is in scope, and the cost increase must be justified by better completion times. Which bounded tradeoff would preserve the core outcome and control operational risk?
- Run unbounded OPTIMIZE on every table each hour regardless of need
- Delete the table history manually as a substitute for automated optimization
- Use unsupported layout features on an external table without checking eligibility
- Accept controlled automated maintenance where supported rather than blanket scheduled rewrites
Explanation
Correct Answer
D:Accept controlled automated maintenance where supported rather than blanket scheduled rewrites. This explicitly bounds the compromise while still addressing fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions. Acceptance relies on enable the supported setting on a test managed table and compare maintenance actions and query cost rather than simply declaring the change complete.
Incorrect Answers
A:Run unbounded OPTIMIZE on every table each hour regardless of need. Indiscriminate compaction adds significant compute cost for unchanged tables. Monitoring after deployment does not make this approach a safe bounded tradeoff for `fraud_signals`.
B:Delete the table history manually as a substitute for automated optimization. Destructive history removal is not an appropriate performance management policy. Monitoring after deployment does not make this approach a safe bounded tradeoff for `fraud_signals`.
C:Use unsupported layout features on an external table without checking eligibility. Feature support differs by table management and runtime, so eligibility must be established. Monitoring after deployment does not make this approach a safe bounded tradeoff for `fraud_signals`.
Question 580
At an agricultural sensor program, a Unity Catalog managed Delta table has rising maintenance overhead, but the operator is manually running daily file compaction regardless of workload. The dataset `fraud_signals` is in scope, and the cost increase must be justified by better completion times. Before release, which acceptance test would actually demonstrate that the root problem is controlled?
- Run only a normal, unchanged `fraud_signals` sample and review table type.
- Enable the supported setting on a test managed table and compare maintenance actions and query cost
- Review a platform-level availability report for `fraud_signals` without testing whether fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions.
- Check that the intended configuration exists for `fraud_signals` and that a healthy run succeeded, without inducing the earlier failure.
Explanation
Correct Answer
B:Enable the supported setting on a test managed table and compare maintenance actions and query cost. This reproduces the scenario-dependent condition, verifies table type, predictive optimization eligibility, maintenance events, file layout, and query duration, and directly tests whether fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions is resolved. Observed results, not intent, determine approval.
Incorrect Answers
A:Run only a normal, unchanged `fraud_signals` sample and review table type. A normal sample does not challenge fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions.
C:Review a platform-level availability report for `fraud_signals` without testing whether fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions. Aggregate availability does not establish the behavior of affected `fraud_signals` records. It cannot replace the direct exercise: enable the supported setting on a test managed table and compare maintenance actions and query cost.
D:Check that the intended configuration exists for `fraud_signals` and that a healthy run succeeded, without inducing the earlier failure. Installed configuration and a healthy run are insufficient to prove the failure mode is gone. The original issue was fixed manual maintenance may run unnecessarily or omit the most valuable optimization actions.