Certified Data Engineer Associate Practice Test 11
Question 100
At a manufacturing supplier, a streaming JSON feed added a new optional field and Auto Loader stopped at a schema evolution exception. The dataset `wind_turbine_events` is in scope, and bronze tables must remain governed by Unity Catalog. Before release, which acceptance test would actually demonstrate that the root problem is controlled?
- Run only a normal, unchanged `wind_turbine_events` sample and review new file schema.
- Review a platform-level availability report for `wind_turbine_events` without testing whether Auto Loader records the new field and may stop the stream so its schema change can be applied on restart.
- Check that the intended configuration exists for `wind_turbine_events` and that a healthy run succeeded, without inducing the earlier failure.
- Inject a JSON field not present in the first batch and confirm the restarted stream retains it without losing records
Explanation
Correct Answer
D:Inject a JSON field not present in the first batch and confirm the restarted stream retains it without losing records. This reproduces the scenario-dependent condition, verifies new file schema, _schemas changes, stream exception, and post-restart bronze column count, and directly tests whether Auto Loader records the new field and may stop the stream so its schema change can be applied on restart is resolved. Observed results, not intent, determine approval.
Incorrect Answers
A:Run only a normal, unchanged `wind_turbine_events` sample and review new file schema. A normal sample does not challenge Auto Loader records the new field and may stop the stream so its schema change can be applied on restart.
B:Review a platform-level availability report for `wind_turbine_events` without testing whether Auto Loader records the new field and may stop the stream so its schema change can be applied on restart. Aggregate availability does not establish the behavior of affected `wind_turbine_events` records. It cannot replace the direct exercise: inject a JSON field not present in the first batch and confirm the restarted stream retains it without losing records.
C:Check that the intended configuration exists for `wind_turbine_events` and that a healthy run succeeded, without inducing the earlier failure. Installed configuration and a healthy run are insufficient to prove the failure mode is gone. The original issue was Auto Loader records the new field and may stop the stream so its schema change can be applied on restart.
Question 101
At a biotechnology lab, semi-structured feeds contain occasional type mismatches that must be retained for later triage rather than dropped. The dataset `maintenance_events` is in scope, and repeat deliveries cannot silently double-count business records. What evidence should the engineer collect to establish the cause rather than a coincidental symptom?
- Correlate _rescued_data contents, original source path across affected and unaffected `R034-CASE-021` runs.
- Examine only a normal successful run of `maintenance_events` and trend _rescued_data contents without recreating the fault.
- Review aggregate cluster availability and job completion for `maintenance_events` without correlating the observed error path.
- Compare the saved configuration with the signed change ticket for `maintenance_events` while omitting source and output records.
Explanation
Correct Answer
A:Correlate _rescued_data contents, original source path across affected and unaffected `R034-CASE-021` runs.. By joining _rescued_data contents, original source path, schema mismatch rate, and recovered values across the affected and unaffected conditions, the investigation directly tests the recorded mechanism: the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type. It also sets a measurable baseline for send a known type-mismatched JSON record and assert the original field survives in the rescued column.
Incorrect Answers
B:Examine only a normal successful run of `maintenance_events` and trend _rescued_data contents without recreating the fault. A healthy-run trend can be useful, but it does not expose the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type. The failing-path _rescued_data contents, original source path, schema mismatch rate, and recovered values has to be correlated before attributing cause.
C:Review aggregate cluster availability and job completion for `maintenance_events` without correlating the observed error path. A successful aggregate platform state can coexist with the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type. This check never observes the affected `maintenance_events` records or their causal path.
D:Compare the saved configuration with the signed change ticket for `maintenance_events` while omitting source and output records. A desired configuration and approval ticket are not runtime proof. In this case they omit _rescued_data contents, original source path, schema mismatch rate, and recovered values, the direct means of distinguishing the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type.
Question 102
At a biotechnology lab, semi-structured feeds contain occasional type mismatches that must be retained for later triage rather than dropped. The dataset `maintenance_events` is in scope, and repeat deliveries cannot silently double-count business records. Which platform configuration or data-engineering change is most defensible?
- Configure an Auto Loader rescued-data column and inspect its JSON payload for unparsed field values
- Use DROPMALFORMED and discard every row with an unexpected type; use that approach for `maintenance_events`.
- Coerce every nested field to an integer at ingestion without validation; use that approach for `maintenance_events`.
- Turn off schema inference and rescued data then accept null values; use that approach for `maintenance_events`.
Explanation
Correct Answer
A:Configure an Auto Loader rescued-data column and inspect its JSON payload for unparsed field values. This directly addresses the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type. The design should be accepted only after send a known type-mismatched JSON record and assert the original field survives in the rescued column; the related operational boundary is to retain anomalous source details in bronze while limiting automatic changes to trusted silver schemas.
Incorrect Answers
B:Use DROPMALFORMED and discard every row with an unexpected type; use that approach for `maintenance_events`. Dropping the record loses evidence needed for later data quality repair. For `maintenance_events`, this cannot remove the causal limitation: the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type.
C:Coerce every nested field to an integer at ingestion without validation; use that approach for `maintenance_events`. Invalid conversions either fail or misrepresent original payload content. For `maintenance_events`, this cannot remove the causal limitation: the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type.
D:Turn off schema inference and rescued data then accept null values; use that approach for `maintenance_events`. Unrecognized information is lost rather than available for triage. For `maintenance_events`, this cannot remove the causal limitation: the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type.
Question 103
At a biotechnology lab, semi-structured feeds contain occasional type mismatches that must be retained for later triage rather than dropped. The dataset `maintenance_events` is in scope, and repeat deliveries cannot silently double-count business records. A teammate proposes: "Use DROPMALFORMED and discard every row with an unexpected type." Which technical limitation is the decisive reason not to approve that specific proposal?
- Unrecognized information is lost rather than available for triage. This would apply to another option for `maintenance_events`.
- Invalid conversions either fail or misrepresent original payload content. This would apply to an alternative change for `maintenance_events`.
- Dropping the record loses evidence needed for later data quality repair. The proposal does not repair `maintenance_events`.
- Insufficient executor memory is the primary fault; source-file handling and ingestion checkpoints are correct.
Explanation
Correct Answer
C:Dropping the record loses evidence needed for later data quality repair. The proposal does not repair `maintenance_events`. This identifies the known limitation of the exact proposed action. Dropping the record loses evidence needed for later data quality repair. The underlying problem remains the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type; Configure an Auto Loader rescued-data column and inspect its JSON payload for unparsed field values instead targets that failure.
Incorrect Answers
A:Unrecognized information is lost rather than available for triage. This would apply to another option for `maintenance_events`. This limitation belongs to the different alternative "Turn off schema inference and rescued data then accept null values". The stem calls for the specific failure of "Use DROPMALFORMED and discard every row with an unexpected type".
B:Invalid conversions either fail or misrepresent original payload content. This would apply to an alternative change for `maintenance_events`. This failure applies to the separate proposal "Coerce every nested field to an integer at ingestion without validation".
D:Insufficient executor memory is the primary fault; source-file handling and ingestion checkpoints are correct. The case provides a concrete failing data or control path; a mere alternate theory does not address the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type.
Question 104
At a biotechnology lab, semi-structured feeds contain occasional type mismatches that must be retained for later triage rather than dropped. The dataset `maintenance_events` is in scope, and repeat deliveries cannot silently double-count business records. What risk-and-cost compromise should the owner accept instead of the proposed shortcuts?
- Use DROPMALFORMED and discard every row with an unexpected type; monitor results only after launch of `maintenance_events`.
- Retain anomalous source details in bronze while limiting automatic changes to trusted silver schemas
- Coerce every nested field to an integer at ingestion without validation; monitor results only after launch of `maintenance_events`.
- Turn off schema inference and rescued data then accept null values; monitor results only after launch of `maintenance_events`.
Explanation
Correct Answer
B:Retain anomalous source details in bronze while limiting automatic changes to trusted silver schemas. This explicitly bounds the compromise while still addressing the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type. Acceptance relies on send a known type-mismatched JSON record and assert the original field survives in the rescued column rather than simply declaring the change complete.
Incorrect Answers
A:Use DROPMALFORMED and discard every row with an unexpected type; monitor results only after launch of `maintenance_events`. Dropping the record loses evidence needed for later data quality repair. Monitoring after deployment does not make this approach a safe bounded tradeoff for `maintenance_events`.
C:Coerce every nested field to an integer at ingestion without validation; monitor results only after launch of `maintenance_events`. Invalid conversions either fail or misrepresent original payload content. Monitoring after deployment does not make this approach a safe bounded tradeoff for `maintenance_events`.
D:Turn off schema inference and rescued data then accept null values; monitor results only after launch of `maintenance_events`. Unrecognized information is lost rather than available for triage. Monitoring after deployment does not make this approach a safe bounded tradeoff for `maintenance_events`.
Question 105
At a biotechnology lab, semi-structured feeds contain occasional type mismatches that must be retained for later triage rather than dropped. The dataset `maintenance_events` is in scope, and repeat deliveries cannot silently double-count business records. Which direct test could falsify the proposed fix instead of only confirming deployment?
- Check that the intended configuration exists for `maintenance_events` and that a healthy run succeeded, without inducing the earlier failure.
- Review a platform-level availability report for `maintenance_events` without testing whether the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type.
- Send a known type-mismatched JSON record and assert the original field survives in the rescued column
- Run only a normal, unchanged `maintenance_events` sample and review _rescued_data contents.
Explanation
Correct Answer
C:Send a known type-mismatched JSON record and assert the original field survives in the rescued column. This reproduces the scenario-dependent condition, verifies _rescued_data contents, original source path, schema mismatch rate, and recovered values, and directly tests whether the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type is resolved. Observed results, not intent, determine approval.
Incorrect Answers
A:Check that the intended configuration exists for `maintenance_events` and that a healthy run succeeded, without inducing the earlier failure. Installed configuration and a healthy run are insufficient to prove the failure mode is gone. The original issue was the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type.
B:Review a platform-level availability report for `maintenance_events` without testing whether the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type. Aggregate availability does not establish the behavior of affected `maintenance_events` records. It cannot replace the direct exercise: send a known type-mismatched JSON record and assert the original field survives in the rescued column.
D:Run only a normal, unchanged `maintenance_events` sample and review _rescued_data contents. A normal sample does not challenge the rescue path preserves unexpected or mismatched fields without pretending they conform to the declared type.
Question 106
At a vehicle fleet operator, a streaming JSON feed added a new optional field and Auto Loader stopped at a schema evolution exception. The dataset `orders_bronze` is in scope, and bronze tables must remain governed by Unity Catalog. Which correlated observation is the strongest basis for the change decision?
- Compare the current configuration and the signed change approval, without source-row evidence
- Correlate new file schema, _schemas changes across affected and unaffected `R034-CASE-022` runs.
- Inspect normal-period success metrics for the stated dataset without reproducing the source error
- Review aggregate cluster availability and job completion for `orders_bronze` without correlating the observed error path.
Explanation
Correct Answer
B:Correlate new file schema, _schemas changes across affected and unaffected `R034-CASE-022` runs.. By joining new file schema, _schemas changes, stream exception, and post-restart bronze column count across the affected and unaffected conditions, the investigation directly tests the recorded mechanism: Auto Loader records the new field and may stop the stream so its schema change can be applied on restart. It also sets a measurable baseline for inject a JSON field not present in the first batch and confirm the restarted stream retains it without losing records.
Incorrect Answers
A:Compare the current configuration and the signed change approval, without source-row evidence. A desired configuration and approval ticket are not runtime proof. In this case they omit new file schema, _schemas changes, stream exception, and post-restart bronze column count, the direct means of distinguishing Auto Loader records the new field and may stop the stream so its schema change can be applied on restart.
C:Inspect normal-period success metrics for the stated dataset without reproducing the source error. A healthy-run trend can be useful, but it does not expose Auto Loader records the new field and may stop the stream so its schema change can be applied on restart. The failing-path new file schema, _schemas changes, stream exception, and post-restart bronze column count has to be correlated before attributing cause.
D:Review aggregate cluster availability and job completion for `orders_bronze` without correlating the observed error path. A successful aggregate platform state can coexist with Auto Loader records the new field and may stop the stream so its schema change can be applied on restart. This check never observes the affected `orders_bronze` records or their causal path.
Question 107
At a vehicle fleet operator, a streaming JSON feed added a new optional field and Auto Loader stopped at a schema evolution exception. The dataset `orders_bronze` is in scope, and bronze tables must remain governed by Unity Catalog. What implementation should the engineer select for the failing workflow?
- Continue to parse only the previous schema and silently discard the new field; use that approach for `orders_bronze`.
- Retain the schema location, use a supported add-new-columns evolution mode, and restart after Auto Loader updates the schema
- Delete both checkpoint and schema location and replay everything; use that approach for `orders_bronze`.
- Use failOnNewColumns indefinitely while demanding automatic schema evolution; use that approach for `orders_bronze`.
Explanation
Correct Answer
B:Retain the schema location, use a supported add-new-columns evolution mode, and restart after Auto Loader updates the schema. This directly addresses Auto Loader records the new field and may stop the stream so its schema change can be applied on restart. The design should be accepted only after inject a JSON field not present in the first batch and confirm the restarted stream retains it without losing records; the related operational boundary is to accept a controlled stream restart for schema evolution rather than losing newly arriving attributes.
Incorrect Answers
A:Continue to parse only the previous schema and silently discard the new field; use that approach for `orders_bronze`. Ignoring the new field violates the requirement to retain the source attribute. For `orders_bronze`, this cannot remove the causal limitation: Auto Loader records the new field and may stop the stream so its schema change can be applied on restart.
C:Delete both checkpoint and schema location and replay everything; use that approach for `orders_bronze`. Erasing progress and inference history risks replay and inconsistent schema discovery. For `orders_bronze`, this cannot remove the causal limitation: Auto Loader records the new field and may stop the stream so its schema change can be applied on restart.
D:Use failOnNewColumns indefinitely while demanding automatic schema evolution; use that approach for `orders_bronze`. Failing on every unknown field does not deliver the requested automatic adaptation. For `orders_bronze`, this cannot remove the causal limitation: Auto Loader records the new field and may stop the stream so its schema change can be applied on restart.
Question 108
At a vehicle fleet operator, a streaming JSON feed added a new optional field and Auto Loader stopped at a schema evolution exception. The dataset `orders_bronze` is in scope, and bronze tables must remain governed by Unity Catalog. A teammate proposes: "Continue to parse only the previous schema and silently discard the new field." Which technical limitation is the decisive reason not to approve that specific proposal?
- Erasing progress and inference history risks replay and inconsistent schema discovery. This would apply to an alternative change for `orders_bronze`.
- Insufficient executor memory is the primary fault; source-file handling and ingestion checkpoints are correct.
- Failing on every unknown field does not deliver the requested automatic adaptation. This would apply to another option for `orders_bronze`.
- Ignoring the new field violates the requirement to retain the source attribute. The proposal does not repair `orders_bronze`.
Explanation
Correct Answer
D:Ignoring the new field violates the requirement to retain the source attribute. The proposal does not repair `orders_bronze`. This identifies the known limitation of the exact proposed action. Ignoring the new field violates the requirement to retain the source attribute. The underlying problem remains Auto Loader records the new field and may stop the stream so its schema change can be applied on restart; Retain the schema location, use a supported add-new-columns evolution mode, and restart after Auto Loader updates the schema instead targets that failure.
Incorrect Answers
A:Erasing progress and inference history risks replay and inconsistent schema discovery. This would apply to an alternative change for `orders_bronze`. This failure applies to the separate proposal "Delete both checkpoint and schema location and replay everything".
B:Insufficient executor memory is the primary fault; source-file handling and ingestion checkpoints are correct. The case provides a concrete failing data or control path; a mere alternate theory does not address Auto Loader records the new field and may stop the stream so its schema change can be applied on restart.
C:Failing on every unknown field does not deliver the requested automatic adaptation. This would apply to another option for `orders_bronze`. This limitation belongs to the different alternative "Use failOnNewColumns indefinitely while demanding automatic schema evolution". The stem calls for the specific failure of "Continue to parse only the previous schema and silently discard the new field".
Question 109
At a vehicle fleet operator, a streaming JSON feed added a new optional field and Auto Loader stopped at a schema evolution exception. The dataset `orders_bronze` is in scope, and bronze tables must remain governed by Unity Catalog. Which decision preserves the stated data contract while keeping the change scope proportionate?
- Use failOnNewColumns indefinitely while demanding automatic schema evolution
- Accept a controlled stream restart for schema evolution rather than losing newly arriving attributes
- Delete both checkpoint and schema location and replay everything
- Continue to parse only the previous schema and silently discard the new field; monitor results only after launch of `orders_bronze`.
Explanation
Correct Answer
B:Accept a controlled stream restart for schema evolution rather than losing newly arriving attributes. This explicitly bounds the compromise while still addressing Auto Loader records the new field and may stop the stream so its schema change can be applied on restart. Acceptance relies on inject a JSON field not present in the first batch and confirm the restarted stream retains it without losing records rather than simply declaring the change complete.
Incorrect Answers
A:Use failOnNewColumns indefinitely while demanding automatic schema evolution. Failing on every unknown field does not deliver the requested automatic adaptation. Monitoring after deployment does not make this approach a safe bounded tradeoff for `orders_bronze`.
C:Delete both checkpoint and schema location and replay everything. Erasing progress and inference history risks replay and inconsistent schema discovery. Monitoring after deployment does not make this approach a safe bounded tradeoff for `orders_bronze`.
D:Continue to parse only the previous schema and silently discard the new field; monitor results only after launch of `orders_bronze`. Ignoring the new field violates the requirement to retain the source attribute. Monitoring after deployment does not make this approach a safe bounded tradeoff for `orders_bronze`.
Question 110
At a vehicle fleet operator, a streaming JSON feed added a new optional field and Auto Loader stopped at a schema evolution exception. The dataset `orders_bronze` is in scope, and bronze tables must remain governed by Unity Catalog. What verification provides evidence of correct behavior under the original adverse condition?
- Review a platform-level availability report for `orders_bronze` without testing whether Auto Loader records the new field and may stop the stream so its schema change can be applied on restart.
- Run only a normal, unchanged `orders_bronze` sample and review new file schema.
- Inject a JSON field not present in the first batch and confirm the restarted stream retains it without losing records
- Check that the intended configuration exists for `orders_bronze` and that a healthy run succeeded, without inducing the earlier failure.
Explanation
Correct Answer
C:Inject a JSON field not present in the first batch and confirm the restarted stream retains it without losing records. This reproduces the scenario-dependent condition, verifies new file schema, _schemas changes, stream exception, and post-restart bronze column count, and directly tests whether Auto Loader records the new field and may stop the stream so its schema change can be applied on restart is resolved. Observed results, not intent, determine approval.
Incorrect Answers
A:Review a platform-level availability report for `orders_bronze` without testing whether Auto Loader records the new field and may stop the stream so its schema change can be applied on restart. Aggregate availability does not establish the behavior of affected `orders_bronze` records. It cannot replace the direct exercise: inject a JSON field not present in the first batch and confirm the restarted stream retains it without losing records.
B:Run only a normal, unchanged `orders_bronze` sample and review new file schema. A normal sample does not challenge Auto Loader records the new field and may stop the stream so its schema change can be applied on restart.
D:Check that the intended configuration exists for `orders_bronze` and that a healthy run succeeded, without inducing the earlier failure. Installed configuration and a healthy run are insufficient to prove the failure mode is gone. The original issue was Auto Loader records the new field and may stop the stream so its schema change can be applied on restart.