ISACA Advanced in AI Audit AAIA Practice Test 19
Question 180
In a review of Data Management Specific to AI, investigators cannot establish the expected control proof: versioned data manifests, build ids and successful sample reconstruction. The underlying event was that the model-training platform overwrites yesterday's training extract without keeping a snapshot identifier. Which control weakness is the MOST likely cause requiring focused audit work?
- Representation imbalance hides subgroup error and instability
- In-transit protection and accountable data movement are missing
- Data reproducibility and provenance are absent from the ML pipeline
- Join normalization corrupts entity matching between data sources
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates data reproducibility and provenance are absent from the ml pipeline. A meaningful audit response is to verify this diagnosis and then recommend version immutable datasets and hashes linked to each model build.
Incorrect Answers
A:“Representation imbalance hides subgroup error and instability” concerns a separate audit weakness, not the missing assurance described by data reproducibility and provenance are absent from the ML pipeline. The correct cause is “Data reproducibility and provenance are absent from the ML pipeline”.
B:“In-transit protection and accountable data movement are missing” concerns a separate audit weakness, not the missing assurance described by data reproducibility and provenance are absent from the ML pipeline. The correct cause is “Data reproducibility and provenance are absent from the ML pipeline”.
D:“Join normalization corrupts entity matching between data sources” concerns a separate audit weakness, not the missing assurance described by data reproducibility and provenance are absent from the ML pipeline. The correct cause is “Data reproducibility and provenance are absent from the ML pipeline”.
Question 181
An image model learns to classify equipment faults from filenames encoding the defect label. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Review contractual source rights, permitted purpose and collection safeguards before use
- Strip outcome-bearing metadata and test performance on blinded image content
- Introduce contract tests for unit, range and meaning changes before retraining
- Enforce event-time feature availability and cutoff validation for every predictor
Explanation
Correct Answer
B:Correct. Strip outcome-bearing metadata and test performance on blinded image content is aimed at the actual failure: shortcut features leak labels and undermine production generalization. The audit should subsequently obtain metadata-ablated benchmark and unchanged test-label separation to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Review contractual source rights, permitted purpose and collection safeguards before use” is a control for external data sourcing controls omit restrictions on collection and use, but the audit finding here is shortcut features leak labels and undermine production generalization. The relevant measure is “Strip outcome-bearing metadata and test performance on blinded image content”.
C:“Introduce contract tests for unit, range and meaning changes before retraining” is a control for data-quality gates do not validate semantic feature definitions across pipeline versions, but the audit finding here is shortcut features leak labels and undermine production generalization. The relevant measure is “Strip outcome-bearing metadata and test performance on blinded image content”.
D:“Enforce event-time feature availability and cutoff validation for every predictor” is a control for outcome-period data leaks into predictors unavailable at scoring time, but the audit finding here is shortcut features leak labels and undermine production generalization. The relevant measure is “Strip outcome-bearing metadata and test performance on blinded image content”.
Question 182
In an audit of Data Management Specific to AI, managers say this measure is live: Strip outcome-bearing metadata and test performance on blinded image content. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Metadata-ablated benchmark and unchanged test-label separation
- Source inventory, collection logs, license terms and exception approval
- Schema diff, unit conversion assertions and rejected malformed batches
- Point-in-time feature store queries compared with historical scoring timestamps
Explanation
Correct Answer
A:Correct. The proposed records—metadata-ablated benchmark and unchanged test-label separation—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is shortcut features leak labels and undermine production generalization.
Incorrect Answers
B:“Source inventory, collection logs, license terms and exception approval” would test external data sourcing controls omit restrictions on collection and use, not the present safeguard against shortcut features leak labels and undermine production generalization. The expected audit evidence is “Metadata-ablated benchmark and unchanged test-label separation”.
C:“Schema diff, unit conversion assertions and rejected malformed batches” would test data-quality gates do not validate semantic feature definitions across pipeline versions, not the present safeguard against shortcut features leak labels and undermine production generalization. The expected audit evidence is “Metadata-ablated benchmark and unchanged test-label separation”.
D:“Point-in-time feature store queries compared with historical scoring timestamps” would test outcome-period data leaks into predictors unavailable at scoring time, not the present safeguard against shortcut features leak labels and undermine production generalization. The expected audit evidence is “Metadata-ablated benchmark and unchanged test-label separation”.
Question 183
In a review of Data Management Specific to AI, investigators cannot establish the expected control proof: metadata-ablated benchmark and unchanged test-label separation. The underlying event was that an image model learns to classify equipment faults from filenames encoding the defect label. Which control weakness is the MOST likely cause requiring focused audit work?
- External data sourcing controls omit restrictions on collection and use
- Data-quality gates do not validate semantic feature definitions across pipeline versions
- Outcome-period data leaks into predictors unavailable at scoring time
- Shortcut features leak labels and undermine production generalization
Explanation
Correct Answer
D:Correct. The observed condition specifically indicates shortcut features leak labels and undermine production generalization. A meaningful audit response is to verify this diagnosis and then recommend strip outcome-bearing metadata and test performance on blinded image content.
Incorrect Answers
A:“External data sourcing controls omit restrictions on collection and use” concerns a separate audit weakness, not the missing assurance described by shortcut features leak labels and undermine production generalization. The correct cause is “Shortcut features leak labels and undermine production generalization”.
B:“Data-quality gates do not validate semantic feature definitions across pipeline versions” concerns a separate audit weakness, not the missing assurance described by shortcut features leak labels and undermine production generalization. The correct cause is “Shortcut features leak labels and undermine production generalization”.
C:“Outcome-period data leaks into predictors unavailable at scoring time” concerns a separate audit weakness, not the missing assurance described by shortcut features leak labels and undermine production generalization. The correct cause is “Shortcut features leak labels and undermine production generalization”.
Question 184
A bank uses a training dataset where rare minority customer segments have almost no positive outcomes. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Reassess linkage risk and protect keys, quasi-identifiers and dataset releases
- Audit label completeness by merchant age and use independently confirmed outcome samples
- Quantify segment coverage and apply justified sampling with separate evaluation
- Version immutable datasets and hashes linked to each model build
Explanation
Correct Answer
C:Correct. Quantify segment coverage and apply justified sampling with separate evaluation is aimed at the actual failure: representation imbalance hides subgroup error and instability. The audit should subsequently obtain per-segment counts, sampling decisions and stable held-out subgroup metrics to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Reassess linkage risk and protect keys, quasi-identifiers and dataset releases” is a control for pseudonymization was treated as irreversibly anonymous, but the audit finding here is representation imbalance hides subgroup error and instability. The relevant measure is “Quantify segment coverage and apply justified sampling with separate evaluation”.
B:“Audit label completeness by merchant age and use independently confirmed outcome samples” is a control for missing labels and coverage create misleadingly low observed fraud rates, but the audit finding here is representation imbalance hides subgroup error and instability. The relevant measure is “Quantify segment coverage and apply justified sampling with separate evaluation”.
D:“Version immutable datasets and hashes linked to each model build” is a control for data reproducibility and provenance are absent from the ML pipeline, but the audit finding here is representation imbalance hides subgroup error and instability. The relevant measure is “Quantify segment coverage and apply justified sampling with separate evaluation”.
Question 185
In an audit of Data Management Specific to AI, managers say this measure is live: Quantify segment coverage and apply justified sampling with separate evaluation. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Reidentification tests, key access records and release-risk signoff
- Per-segment counts, sampling decisions and stable held-out subgroup metrics
- Label reconciliation by tenure cohort and corrected training-data completeness report
- Versioned data manifests, build IDs and successful sample reconstruction
Explanation
Correct Answer
B:Correct. The proposed records—per-segment counts, sampling decisions and stable held-out subgroup metrics—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is representation imbalance hides subgroup error and instability.
Incorrect Answers
A:“Reidentification tests, key access records and release-risk signoff” would test pseudonymization was treated as irreversibly anonymous, not the present safeguard against representation imbalance hides subgroup error and instability. The expected audit evidence is “Per-segment counts, sampling decisions and stable held-out subgroup metrics”.
C:“Label reconciliation by tenure cohort and corrected training-data completeness report” would test missing labels and coverage create misleadingly low observed fraud rates, not the present safeguard against representation imbalance hides subgroup error and instability. The expected audit evidence is “Per-segment counts, sampling decisions and stable held-out subgroup metrics”.
D:“Versioned data manifests, build IDs and successful sample reconstruction” would test data reproducibility and provenance are absent from the ML pipeline, not the present safeguard against representation imbalance hides subgroup error and instability. The expected audit evidence is “Per-segment counts, sampling decisions and stable held-out subgroup metrics”.
Question 186
In a review of Data Management Specific to AI, investigators cannot establish the expected control proof: per-segment counts, sampling decisions and stable held-out subgroup metrics. The underlying event was that a bank uses a training dataset where rare minority customer segments have almost no positive outcomes. Which control weakness is the MOST likely cause requiring focused audit work?
- Representation imbalance hides subgroup error and instability
- Pseudonymization was treated as irreversibly anonymous
- Missing labels and coverage create misleadingly low observed fraud rates
- Data reproducibility and provenance are absent from the ML pipeline
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates representation imbalance hides subgroup error and instability. A meaningful audit response is to verify this diagnosis and then recommend quantify segment coverage and apply justified sampling with separate evaluation.
Incorrect Answers
B:“Pseudonymization was treated as irreversibly anonymous” concerns a separate audit weakness, not the missing assurance described by representation imbalance hides subgroup error and instability. The correct cause is “Representation imbalance hides subgroup error and instability”.
C:“Missing labels and coverage create misleadingly low observed fraud rates” concerns a separate audit weakness, not the missing assurance described by representation imbalance hides subgroup error and instability. The correct cause is “Representation imbalance hides subgroup error and instability”.
D:“Data reproducibility and provenance are absent from the ML pipeline” concerns a separate audit weakness, not the missing assurance described by representation imbalance hides subgroup error and instability. The correct cause is “Representation imbalance hides subgroup error and instability”.
Question 187
A vendor delivers training data but cannot produce evidence of authorized web scraping limits. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Apply encrypted transport, integrity verification and attributable transfer logging
- Encode missingness and validate imputation against sensor-state evidence
- Strip outcome-bearing metadata and test performance on blinded image content
- Review contractual source rights, permitted purpose and collection safeguards before use
Explanation
Correct Answer
D:Correct. Review contractual source rights, permitted purpose and collection safeguards before use is aimed at the actual failure: external data sourcing controls omit restrictions on collection and use. The audit should subsequently obtain source inventory, collection logs, license terms and exception approval to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Apply encrypted transport, integrity verification and attributable transfer logging” is a control for in-transit protection and accountable data movement are missing, but the audit finding here is external data sourcing controls omit restrictions on collection and use. The relevant measure is “Review contractual source rights, permitted purpose and collection safeguards before use”.
B:“Encode missingness and validate imputation against sensor-state evidence” is a control for imputation logic confuses sensor outages with genuine zero measurements, but the audit finding here is external data sourcing controls omit restrictions on collection and use. The relevant measure is “Review contractual source rights, permitted purpose and collection safeguards before use”.
C:“Strip outcome-bearing metadata and test performance on blinded image content” is a control for shortcut features leak labels and undermine production generalization, but the audit finding here is external data sourcing controls omit restrictions on collection and use. The relevant measure is “Review contractual source rights, permitted purpose and collection safeguards before use”.
Question 188
In an audit of Data Management Specific to AI, managers say this measure is live: Review contractual source rights, permitted purpose and collection safeguards before use. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Transfer telemetry, cryptographic configuration and tamper-detection evidence
- Source telemetry and imputation trace on known outage intervals
- Source inventory, collection logs, license terms and exception approval
- Metadata-ablated benchmark and unchanged test-label separation
Explanation
Correct Answer
C:Correct. The proposed records—source inventory, collection logs, license terms and exception approval—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is external data sourcing controls omit restrictions on collection and use.
Incorrect Answers
A:“Transfer telemetry, cryptographic configuration and tamper-detection evidence” would test in-transit protection and accountable data movement are missing, not the present safeguard against external data sourcing controls omit restrictions on collection and use. The expected audit evidence is “Source inventory, collection logs, license terms and exception approval”.
B:“Source telemetry and imputation trace on known outage intervals” would test imputation logic confuses sensor outages with genuine zero measurements, not the present safeguard against external data sourcing controls omit restrictions on collection and use. The expected audit evidence is “Source inventory, collection logs, license terms and exception approval”.
D:“Metadata-ablated benchmark and unchanged test-label separation” would test shortcut features leak labels and undermine production generalization, not the present safeguard against external data sourcing controls omit restrictions on collection and use. The expected audit evidence is “Source inventory, collection logs, license terms and exception approval”.
Question 189
In a review of Data Management Specific to AI, investigators cannot establish the expected control proof: source inventory, collection logs, license terms and exception approval. The underlying event was that a vendor delivers training data but cannot produce evidence of authorized web scraping limits. Which control weakness is the MOST likely cause requiring focused audit work?
- In-transit protection and accountable data movement are missing
- External data sourcing controls omit restrictions on collection and use
- Imputation logic confuses sensor outages with genuine zero measurements
- Shortcut features leak labels and undermine production generalization
Explanation
Correct Answer
B:Correct. The observed condition specifically indicates external data sourcing controls omit restrictions on collection and use. A meaningful audit response is to verify this diagnosis and then recommend review contractual source rights, permitted purpose and collection safeguards before use.
Incorrect Answers
A:“In-transit protection and accountable data movement are missing” concerns a separate audit weakness, not the missing assurance described by external data sourcing controls omit restrictions on collection and use. The correct cause is “External data sourcing controls omit restrictions on collection and use”.
C:“Imputation logic confuses sensor outages with genuine zero measurements” concerns a separate audit weakness, not the missing assurance described by external data sourcing controls omit restrictions on collection and use. The correct cause is “External data sourcing controls omit restrictions on collection and use”.
D:“Shortcut features leak labels and undermine production generalization” concerns a separate audit weakness, not the missing assurance described by external data sourcing controls omit restrictions on collection and use. The correct cause is “External data sourcing controls omit restrictions on collection and use”.
Question 190
Sensitive fields are removed from a table, but a hashed identifier still links customers across public datasets. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Reassess linkage risk and protect keys, quasi-identifiers and dataset releases
- Introduce contract tests for unit, range and meaning changes before retraining
- Enforce typed identifiers and record-level reconciliation before feature merges
- Quantify segment coverage and apply justified sampling with separate evaluation
Explanation
Correct Answer
A:Correct. Reassess linkage risk and protect keys, quasi-identifiers and dataset releases is aimed at the actual failure: pseudonymization was treated as irreversibly anonymous. The audit should subsequently obtain reidentification tests, key access records and release-risk signoff to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Introduce contract tests for unit, range and meaning changes before retraining” is a control for data-quality gates do not validate semantic feature definitions across pipeline versions, but the audit finding here is pseudonymization was treated as irreversibly anonymous. The relevant measure is “Reassess linkage risk and protect keys, quasi-identifiers and dataset releases”.
C:“Enforce typed identifiers and record-level reconciliation before feature merges” is a control for join normalization corrupts entity matching between data sources, but the audit finding here is pseudonymization was treated as irreversibly anonymous. The relevant measure is “Reassess linkage risk and protect keys, quasi-identifiers and dataset releases”.
D:“Quantify segment coverage and apply justified sampling with separate evaluation” is a control for representation imbalance hides subgroup error and instability, but the audit finding here is pseudonymization was treated as irreversibly anonymous. The relevant measure is “Reassess linkage risk and protect keys, quasi-identifiers and dataset releases”.