Certified Data Engineer Associate Practice Test 23
Question 220
At a regional banking group, a large fact DataFrame is joined to a very small reference dimension but shuffle volume dominates runtime. The dataset `prescription_claims` is in scope, and raw records must remain available for replay. Before release, which acceptance test would actually demonstrate that the root problem is controlled?
- Check that the intended configuration exists for `prescription_claims` and that a healthy run succeeded, without inducing the earlier failure.
- Run only a normal, unchanged `prescription_claims` sample and review reference table size.
- Compare join plans and row counts with and without broadcasting the small dimension
- Review a platform-level availability report for `prescription_claims` without testing whether shuffling both sides is unnecessary when the small reference fits within safe broadcast memory.
Explanation
Correct Answer
C:Compare join plans and row counts with and without broadcasting the small dimension. This reproduces the scenario-dependent condition, verifies reference table size, Spark physical plan, shuffle bytes, and joined row cardinality, and directly tests whether shuffling both sides is unnecessary when the small reference fits within safe broadcast memory is resolved. Observed results, not intent, determine approval.
Incorrect Answers
A:Check that the intended configuration exists for `prescription_claims` and that a healthy run succeeded, without inducing the earlier failure. Installed configuration and a healthy run are insufficient to prove the failure mode is gone. The original issue was shuffling both sides is unnecessary when the small reference fits within safe broadcast memory.
B:Run only a normal, unchanged `prescription_claims` sample and review reference table size. A normal sample does not challenge shuffling both sides is unnecessary when the small reference fits within safe broadcast memory.
D:Review a platform-level availability report for `prescription_claims` without testing whether shuffling both sides is unnecessary when the small reference fits within safe broadcast memory. Aggregate availability does not establish the behavior of affected `prescription_claims` records. It cannot replace the direct exercise: compare join plans and row counts with and without broadcasting the small dimension.
Question 221
At a census statistics agency, a customer event enrichment drops customers without matching segmentation rows even though the report requires all customers. The dataset `daily_receipts` is in scope, and downstream dashboards cannot accept fabricated values. What evidence should the engineer collect to establish the cause rather than a coincidental symptom?
- Examine only a normal successful run of `daily_receipts` and trend left-side customer count without recreating the fault.
- Compare the saved configuration with the signed change ticket for `daily_receipts` while omitting source and output records.
- Correlate left-side customer count, unmatched key sample across affected and unaffected `R034-CASE-045` runs.
- Review aggregate cluster availability and job completion for `daily_receipts` without correlating the observed error path.
Explanation
Correct Answer
C:Correlate left-side customer count, unmatched key sample across affected and unaffected `R034-CASE-045` runs.. By joining left-side customer count, unmatched key sample, join type, and missing segment rate across the affected and unaffected conditions, the investigation directly tests the recorded mechanism: an inner join discards left-side rows without a corresponding dimension match. It also sets a measurable baseline for include a customer with no segment and confirm that customer remains with a null or default segment.
Incorrect Answers
A:Examine only a normal successful run of `daily_receipts` and trend left-side customer count without recreating the fault. A healthy-run trend can be useful, but it does not expose an inner join discards left-side rows without a corresponding dimension match. The failing-path left-side customer count, unmatched key sample, join type, and missing segment rate has to be correlated before attributing cause.
B:Compare the saved configuration with the signed change ticket for `daily_receipts` while omitting source and output records. A desired configuration and approval ticket are not runtime proof. In this case they omit left-side customer count, unmatched key sample, join type, and missing segment rate, the direct means of distinguishing an inner join discards left-side rows without a corresponding dimension match.
D:Review aggregate cluster availability and job completion for `daily_receipts` without correlating the observed error path. A successful aggregate platform state can coexist with an inner join discards left-side rows without a corresponding dimension match. This check never observes the affected `daily_receipts` records or their causal path.
Question 222
At a census statistics agency, a customer event enrichment drops customers without matching segmentation rows even though the report requires all customers. The dataset `daily_receipts` is in scope, and downstream dashboards cannot accept fabricated values. Which platform configuration or data-engineering change is most defensible?
- Append both datasets with unionByName instead of joining them; use that approach for `daily_receipts`.
- Use a left join from the customer DataFrame to segmentation and handle unmatched dimension fields explicitly
- Use a cross join to preserve all customers without join keys; use that approach for `daily_receipts`.
- Keep the inner join and fill null segmentation fields afterward; use that approach for `daily_receipts`.
Explanation
Correct Answer
B:Use a left join from the customer DataFrame to segmentation and handle unmatched dimension fields explicitly. This directly addresses an inner join discards left-side rows without a corresponding dimension match. The design should be accepted only after include a customer with no segment and confirm that customer remains with a null or default segment; the related operational boundary is to retain all required primary records while documenting the dimension-match quality gap.
Incorrect Answers
A:Append both datasets with unionByName instead of joining them; use that approach for `daily_receipts`. Union stacks rows; it does not enrich each customer with its dimension attributes. For `daily_receipts`, this cannot remove the causal limitation: an inner join discards left-side rows without a corresponding dimension match.
C:Use a cross join to preserve all customers without join keys; use that approach for `daily_receipts`. A Cartesian product creates spurious combinations instead of valid matches. For `daily_receipts`, this cannot remove the causal limitation: an inner join discards left-side rows without a corresponding dimension match.
D:Keep the inner join and fill null segmentation fields afterward; use that approach for `daily_receipts`. Rows removed by the inner join cannot be recovered by a later fillna. For `daily_receipts`, this cannot remove the causal limitation: an inner join discards left-side rows without a corresponding dimension match.
Question 223
At a census statistics agency, a customer event enrichment drops customers without matching segmentation rows even though the report requires all customers. The dataset `daily_receipts` is in scope, and downstream dashboards cannot accept fabricated values. A teammate proposes: "Use a cross join to preserve all customers without join keys." Which technical limitation is the decisive reason not to approve that specific proposal?
- Rows removed by the inner join cannot be recovered by a later fillna. This would apply to another option for `daily_receipts`.
- Union stacks rows; it does not enrich each customer with its dimension attributes. This would apply to an alternative change for `daily_receipts`.
- A Cartesian product creates spurious combinations instead of valid matches. The proposal does not repair `daily_receipts`.
- The dashboard cached the prior result; no DataFrame operation or underlying data requires correction.
Explanation
Correct Answer
C:A Cartesian product creates spurious combinations instead of valid matches. The proposal does not repair `daily_receipts`. This identifies the known limitation of the exact proposed action. A Cartesian product creates spurious combinations instead of valid matches. The underlying problem remains an inner join discards left-side rows without a corresponding dimension match; Use a left join from the customer DataFrame to segmentation and handle unmatched dimension fields explicitly instead targets that failure.
Incorrect Answers
A:Rows removed by the inner join cannot be recovered by a later fillna. This would apply to another option for `daily_receipts`. This limitation belongs to the different alternative "Keep the inner join and fill null segmentation fields afterward". The stem calls for the specific failure of "Use a cross join to preserve all customers without join keys".
B:Union stacks rows; it does not enrich each customer with its dimension attributes. This would apply to an alternative change for `daily_receipts`. This failure applies to the separate proposal "Append both datasets with unionByName instead of joining them".
D:The dashboard cached the prior result; no DataFrame operation or underlying data requires correction. The case provides a concrete failing data or control path; a mere alternate theory does not address an inner join discards left-side rows without a corresponding dimension match.
Question 224
At a census statistics agency, a customer event enrichment drops customers without matching segmentation rows even though the report requires all customers. The dataset `daily_receipts` is in scope, and downstream dashboards cannot accept fabricated values. What risk-and-cost compromise should the owner accept instead of the proposed shortcuts?
- Keep the inner join and fill null segmentation fields afterward; monitor results only after launch of `daily_receipts`.
- Append both datasets with unionByName instead of joining them; monitor results only after launch of `daily_receipts`.
- Retain all required primary records while documenting the dimension-match quality gap
- Use a cross join to preserve all customers without join keys; monitor results only after launch of `daily_receipts`.
Explanation
Correct Answer
C:Retain all required primary records while documenting the dimension-match quality gap. This explicitly bounds the compromise while still addressing an inner join discards left-side rows without a corresponding dimension match. Acceptance relies on include a customer with no segment and confirm that customer remains with a null or default segment rather than simply declaring the change complete.
Incorrect Answers
A:Keep the inner join and fill null segmentation fields afterward; monitor results only after launch of `daily_receipts`. Rows removed by the inner join cannot be recovered by a later fillna. Monitoring after deployment does not make this approach a safe bounded tradeoff for `daily_receipts`.
B:Append both datasets with unionByName instead of joining them; monitor results only after launch of `daily_receipts`. Union stacks rows; it does not enrich each customer with its dimension attributes. Monitoring after deployment does not make this approach a safe bounded tradeoff for `daily_receipts`.
D:Use a cross join to preserve all customers without join keys; monitor results only after launch of `daily_receipts`. A Cartesian product creates spurious combinations instead of valid matches. Monitoring after deployment does not make this approach a safe bounded tradeoff for `daily_receipts`.
Question 225
At a census statistics agency, a customer event enrichment drops customers without matching segmentation rows even though the report requires all customers. The dataset `daily_receipts` is in scope, and downstream dashboards cannot accept fabricated values. Which direct test could falsify the proposed fix instead of only confirming deployment?
- Check that the intended configuration exists for `daily_receipts` and that a healthy run succeeded, without inducing the earlier failure.
- Review a platform-level availability report for `daily_receipts` without testing whether an inner join discards left-side rows without a corresponding dimension match.
- Include a customer with no segment and confirm that customer remains with a null or default segment
- Run only a normal, unchanged `daily_receipts` sample and review left-side customer count.
Explanation
Correct Answer
C:Include a customer with no segment and confirm that customer remains with a null or default segment. This reproduces the scenario-dependent condition, verifies left-side customer count, unmatched key sample, join type, and missing segment rate, and directly tests whether an inner join discards left-side rows without a corresponding dimension match is resolved. Observed results, not intent, determine approval.
Incorrect Answers
A:Check that the intended configuration exists for `daily_receipts` and that a healthy run succeeded, without inducing the earlier failure. Installed configuration and a healthy run are insufficient to prove the failure mode is gone. The original issue was an inner join discards left-side rows without a corresponding dimension match.
B:Review a platform-level availability report for `daily_receipts` without testing whether an inner join discards left-side rows without a corresponding dimension match. Aggregate availability does not establish the behavior of affected `daily_receipts` records. It cannot replace the direct exercise: include a customer with no segment and confirm that customer remains with a null or default segment.
D:Run only a normal, unchanged `daily_receipts` sample and review left-side customer count. A normal sample does not challenge an inner join discards left-side rows without a corresponding dimension match.
Question 226
At a logistics carrier, a large fact DataFrame is joined to a very small reference dimension but shuffle volume dominates runtime. The dataset `viewer_sessions` is in scope, and raw records must remain available for replay. Which correlated observation is the strongest basis for the change decision?
- Correlate reference table size, Spark physical plan across affected and unaffected `R034-CASE-046` runs.
- Review completed run status and aggregate cluster health without record-level correlation
- Compare the current configuration and the signed change approval, without source-row evidence
- Examine only a normal successful run of `viewer_sessions` and trend reference table size without recreating the fault.
Explanation
Correct Answer
A:Correlate reference table size, Spark physical plan across affected and unaffected `R034-CASE-046` runs.. By joining reference table size, Spark physical plan, shuffle bytes, and joined row cardinality across the affected and unaffected conditions, the investigation directly tests the recorded mechanism: shuffling both sides is unnecessary when the small reference fits within safe broadcast memory. It also sets a measurable baseline for compare join plans and row counts with and without broadcasting the small dimension.
Incorrect Answers
B:Review completed run status and aggregate cluster health without record-level correlation. A successful aggregate platform state can coexist with shuffling both sides is unnecessary when the small reference fits within safe broadcast memory. This check never observes the affected `viewer_sessions` records or their causal path.
C:Compare the current configuration and the signed change approval, without source-row evidence. A desired configuration and approval ticket are not runtime proof. In this case they omit reference table size, Spark physical plan, shuffle bytes, and joined row cardinality, the direct means of distinguishing shuffling both sides is unnecessary when the small reference fits within safe broadcast memory.
D:Examine only a normal successful run of `viewer_sessions` and trend reference table size without recreating the fault. A healthy-run trend can be useful, but it does not expose shuffling both sides is unnecessary when the small reference fits within safe broadcast memory. The failing-path reference table size, Spark physical plan, shuffle bytes, and joined row cardinality has to be correlated before attributing cause.
Question 227
At a logistics carrier, a large fact DataFrame is joined to a very small reference dimension but shuffle volume dominates runtime. The dataset `viewer_sessions` is in scope, and raw records must remain available for replay. What implementation should the engineer select for the failing workflow?
- Use a cross join and filter the matches after multiplication; use that approach for `viewer_sessions`.
- Increase shuffle partitions without considering join distribution; use that approach for `viewer_sessions`.
- Broadcast the genuinely small dimension and join on the validated business key, checking output cardinality
- Broadcast the multi-terabyte fact table instead; use that approach for `viewer_sessions`.
Explanation
Correct Answer
C:Broadcast the genuinely small dimension and join on the validated business key, checking output cardinality. This directly addresses shuffling both sides is unnecessary when the small reference fits within safe broadcast memory. The design should be accepted only after compare join plans and row counts with and without broadcasting the small dimension; the related operational boundary is to use a targeted broadcast when the reference is safely small, not an unconditional broadcast policy.
Incorrect Answers
A:Use a cross join and filter the matches after multiplication; use that approach for `viewer_sessions`. A Cartesian product creates excessive intermediate rows and work. For `viewer_sessions`, this cannot remove the causal limitation: shuffling both sides is unnecessary when the small reference fits within safe broadcast memory.
B:Increase shuffle partitions without considering join distribution; use that approach for `viewer_sessions`. More partitions do not eliminate an avoidable shuffle of the small dimension. For `viewer_sessions`, this cannot remove the causal limitation: shuffling both sides is unnecessary when the small reference fits within safe broadcast memory.
D:Broadcast the multi-terabyte fact table instead; use that approach for `viewer_sessions`. Replicating the large side exceeds executor memory and can cause instability. For `viewer_sessions`, this cannot remove the causal limitation: shuffling both sides is unnecessary when the small reference fits within safe broadcast memory.
Question 228
At a logistics carrier, a large fact DataFrame is joined to a very small reference dimension but shuffle volume dominates runtime. The dataset `viewer_sessions` is in scope, and raw records must remain available for replay. A teammate proposes: "Broadcast the multi-terabyte fact table instead." Which technical limitation is the decisive reason not to approve that specific proposal?
- The dashboard cached the prior result; no DataFrame operation or underlying data requires correction.
- Replicating the large side exceeds executor memory and can cause instability. The proposal does not repair `viewer_sessions`.
- More partitions do not eliminate an avoidable shuffle of the small dimension. This would apply to another option for `viewer_sessions`.
- A Cartesian product creates excessive intermediate rows and work. This would apply to an alternative change for `viewer_sessions`.
Explanation
Correct Answer
B:Replicating the large side exceeds executor memory and can cause instability. The proposal does not repair `viewer_sessions`. This identifies the known limitation of the exact proposed action. Replicating the large side exceeds executor memory and can cause instability. The underlying problem remains shuffling both sides is unnecessary when the small reference fits within safe broadcast memory; Broadcast the genuinely small dimension and join on the validated business key, checking output cardinality instead targets that failure.
Incorrect Answers
A:The dashboard cached the prior result; no DataFrame operation or underlying data requires correction. The case provides a concrete failing data or control path; a mere alternate theory does not address shuffling both sides is unnecessary when the small reference fits within safe broadcast memory.
C:More partitions do not eliminate an avoidable shuffle of the small dimension. This would apply to another option for `viewer_sessions`. This limitation belongs to the different alternative "Increase shuffle partitions without considering join distribution". The stem calls for the specific failure of "Broadcast the multi-terabyte fact table instead".
D:A Cartesian product creates excessive intermediate rows and work. This would apply to an alternative change for `viewer_sessions`. This failure applies to the separate proposal "Use a cross join and filter the matches after multiplication".
Question 229
At a logistics carrier, a large fact DataFrame is joined to a very small reference dimension but shuffle volume dominates runtime. The dataset `viewer_sessions` is in scope, and raw records must remain available for replay. Which decision preserves the stated data contract while keeping the change scope proportionate?
- Broadcast the multi-terabyte fact table instead
- Increase shuffle partitions without considering join distribution
- Use a targeted broadcast when the reference is safely small, not an unconditional broadcast policy
- Use a cross join and filter the matches after multiplication; monitor results only after launch of `viewer_sessions`.
Explanation
Correct Answer
C:Use a targeted broadcast when the reference is safely small, not an unconditional broadcast policy. This explicitly bounds the compromise while still addressing shuffling both sides is unnecessary when the small reference fits within safe broadcast memory. Acceptance relies on compare join plans and row counts with and without broadcasting the small dimension rather than simply declaring the change complete.
Incorrect Answers
A:Broadcast the multi-terabyte fact table instead. Replicating the large side exceeds executor memory and can cause instability. Monitoring after deployment does not make this approach a safe bounded tradeoff for `viewer_sessions`.
B:Increase shuffle partitions without considering join distribution. More partitions do not eliminate an avoidable shuffle of the small dimension. Monitoring after deployment does not make this approach a safe bounded tradeoff for `viewer_sessions`.
D:Use a cross join and filter the matches after multiplication; monitor results only after launch of `viewer_sessions`. A Cartesian product creates excessive intermediate rows and work. Monitoring after deployment does not make this approach a safe bounded tradeoff for `viewer_sessions`.
Question 230
At a logistics carrier, a large fact DataFrame is joined to a very small reference dimension but shuffle volume dominates runtime. The dataset `viewer_sessions` is in scope, and raw records must remain available for replay. What verification provides evidence of correct behavior under the original adverse condition?
- Run only a normal, unchanged `viewer_sessions` sample and review reference table size.
- Review a platform-level availability report for `viewer_sessions` without testing whether shuffling both sides is unnecessary when the small reference fits within safe broadcast memory.
- Check that the intended configuration exists for `viewer_sessions` and that a healthy run succeeded, without inducing the earlier failure.
- Compare join plans and row counts with and without broadcasting the small dimension
Explanation
Correct Answer
D:Compare join plans and row counts with and without broadcasting the small dimension. This reproduces the scenario-dependent condition, verifies reference table size, Spark physical plan, shuffle bytes, and joined row cardinality, and directly tests whether shuffling both sides is unnecessary when the small reference fits within safe broadcast memory is resolved. Observed results, not intent, determine approval.
Incorrect Answers
A:Run only a normal, unchanged `viewer_sessions` sample and review reference table size. A normal sample does not challenge shuffling both sides is unnecessary when the small reference fits within safe broadcast memory.
B:Review a platform-level availability report for `viewer_sessions` without testing whether shuffling both sides is unnecessary when the small reference fits within safe broadcast memory. Aggregate availability does not establish the behavior of affected `viewer_sessions` records. It cannot replace the direct exercise: compare join plans and row counts with and without broadcasting the small dimension.
C:Check that the intended configuration exists for `viewer_sessions` and that a healthy run succeeded, without inducing the earlier failure. Installed configuration and a healthy run are insufficient to prove the failure mode is gone. The original issue was shuffling both sides is unnecessary when the small reference fits within safe broadcast memory.