ISACA Advanced in AI Audit AAIA Practice Test 18
Question 170
In an audit of Data Management Specific to AI, managers say this measure is live: Encode missingness and validate imputation against sensor-state evidence. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Source telemetry and imputation trace on known outage intervals
- Point-in-time feature store queries compared with historical scoring timestamps
- Per-segment counts, sampling decisions and stable held-out subgroup metrics
- Schema diff, unit conversion assertions and rejected malformed batches
Explanation
Correct Answer
A:Correct. The proposed records—source telemetry and imputation trace on known outage intervals—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is imputation logic confuses sensor outages with genuine zero measurements.
Incorrect Answers
B:“Point-in-time feature store queries compared with historical scoring timestamps” would test outcome-period data leaks into predictors unavailable at scoring time, not the present safeguard against imputation logic confuses sensor outages with genuine zero measurements. The expected audit evidence is “Source telemetry and imputation trace on known outage intervals”.
C:“Per-segment counts, sampling decisions and stable held-out subgroup metrics” would test representation imbalance hides subgroup error and instability, not the present safeguard against imputation logic confuses sensor outages with genuine zero measurements. The expected audit evidence is “Source telemetry and imputation trace on known outage intervals”.
D:“Schema diff, unit conversion assertions and rejected malformed batches” would test data-quality gates do not validate semantic feature definitions across pipeline versions, not the present safeguard against imputation logic confuses sensor outages with genuine zero measurements. The expected audit evidence is “Source telemetry and imputation trace on known outage intervals”.
Question 171
In a review of Data Management Specific to AI, investigators cannot establish the expected control proof: source telemetry and imputation trace on known outage intervals. The underlying event was that a predictive-maintenance model imputes every missing vibration reading as zero even when sensors were offline. Which control weakness is the MOST likely cause requiring focused audit work?
- Outcome-period data leaks into predictors unavailable at scoring time
- Representation imbalance hides subgroup error and instability
- Data-quality gates do not validate semantic feature definitions across pipeline versions
- Imputation logic confuses sensor outages with genuine zero measurements
Explanation
Correct Answer
D:Correct. The observed condition specifically indicates imputation logic confuses sensor outages with genuine zero measurements. A meaningful audit response is to verify this diagnosis and then recommend encode missingness and validate imputation against sensor-state evidence.
Incorrect Answers
A:“Outcome-period data leaks into predictors unavailable at scoring time” concerns a separate audit weakness, not the missing assurance described by imputation logic confuses sensor outages with genuine zero measurements. The correct cause is “Imputation logic confuses sensor outages with genuine zero measurements”.
B:“Representation imbalance hides subgroup error and instability” concerns a separate audit weakness, not the missing assurance described by imputation logic confuses sensor outages with genuine zero measurements. The correct cause is “Imputation logic confuses sensor outages with genuine zero measurements”.
C:“Data-quality gates do not validate semantic feature definitions across pipeline versions” concerns a separate audit weakness, not the missing assurance described by imputation logic confuses sensor outages with genuine zero measurements. The correct cause is “Imputation logic confuses sensor outages with genuine zero measurements”.
Question 172
Two training pipelines use a shared customer ID but one strips leading zeros. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Version immutable datasets and hashes linked to each model build
- Review contractual source rights, permitted purpose and collection safeguards before use
- Enforce typed identifiers and record-level reconciliation before feature merges
- Audit label completeness by merchant age and use independently confirmed outcome samples
Explanation
Correct Answer
C:Correct. Enforce typed identifiers and record-level reconciliation before feature merges is aimed at the actual failure: join normalization corrupts entity matching between data sources. The audit should subsequently obtain join exception statistics and sampled identifier matches against source systems to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Version immutable datasets and hashes linked to each model build” is a control for data reproducibility and provenance are absent from the ML pipeline, but the audit finding here is join normalization corrupts entity matching between data sources. The relevant measure is “Enforce typed identifiers and record-level reconciliation before feature merges”.
B:“Review contractual source rights, permitted purpose and collection safeguards before use” is a control for external data sourcing controls omit restrictions on collection and use, but the audit finding here is join normalization corrupts entity matching between data sources. The relevant measure is “Enforce typed identifiers and record-level reconciliation before feature merges”.
D:“Audit label completeness by merchant age and use independently confirmed outcome samples” is a control for missing labels and coverage create misleadingly low observed fraud rates, but the audit finding here is join normalization corrupts entity matching between data sources. The relevant measure is “Enforce typed identifiers and record-level reconciliation before feature merges”.
Question 173
In an audit of Data Management Specific to AI, managers say this measure is live: Enforce typed identifiers and record-level reconciliation before feature merges. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Versioned data manifests, build IDs and successful sample reconstruction
- Join exception statistics and sampled identifier matches against source systems
- Source inventory, collection logs, license terms and exception approval
- Label reconciliation by tenure cohort and corrected training-data completeness report
Explanation
Correct Answer
B:Correct. The proposed records—join exception statistics and sampled identifier matches against source systems—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is join normalization corrupts entity matching between data sources.
Incorrect Answers
A:“Versioned data manifests, build IDs and successful sample reconstruction” would test data reproducibility and provenance are absent from the ML pipeline, not the present safeguard against join normalization corrupts entity matching between data sources. The expected audit evidence is “Join exception statistics and sampled identifier matches against source systems”.
C:“Source inventory, collection logs, license terms and exception approval” would test external data sourcing controls omit restrictions on collection and use, not the present safeguard against join normalization corrupts entity matching between data sources. The expected audit evidence is “Join exception statistics and sampled identifier matches against source systems”.
D:“Label reconciliation by tenure cohort and corrected training-data completeness report” would test missing labels and coverage create misleadingly low observed fraud rates, not the present safeguard against join normalization corrupts entity matching between data sources. The expected audit evidence is “Join exception statistics and sampled identifier matches against source systems”.
Question 174
In a review of Data Management Specific to AI, investigators cannot establish the expected control proof: join exception statistics and sampled identifier matches against source systems. The underlying event was that two training pipelines use a shared customer ID but one strips leading zeros. Which control weakness is the MOST likely cause requiring focused audit work?
- Join normalization corrupts entity matching between data sources
- Data reproducibility and provenance are absent from the ML pipeline
- External data sourcing controls omit restrictions on collection and use
- Missing labels and coverage create misleadingly low observed fraud rates
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates join normalization corrupts entity matching between data sources. A meaningful audit response is to verify this diagnosis and then recommend enforce typed identifiers and record-level reconciliation before feature merges.
Incorrect Answers
B:“Data reproducibility and provenance are absent from the ML pipeline” concerns a separate audit weakness, not the missing assurance described by join normalization corrupts entity matching between data sources. The correct cause is “Join normalization corrupts entity matching between data sources”.
C:“External data sourcing controls omit restrictions on collection and use” concerns a separate audit weakness, not the missing assurance described by join normalization corrupts entity matching between data sources. The correct cause is “Join normalization corrupts entity matching between data sources”.
D:“Missing labels and coverage create misleadingly low observed fraud rates” concerns a separate audit weakness, not the missing assurance described by join normalization corrupts entity matching between data sources. The correct cause is “Join normalization corrupts entity matching between data sources”.
Question 175
A dataset used to predict employee attrition contains performance reviews written after the employee left. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Strip outcome-bearing metadata and test performance on blinded image content
- Reassess linkage risk and protect keys, quasi-identifiers and dataset releases
- Encode missingness and validate imputation against sensor-state evidence
- Enforce event-time feature availability and cutoff validation for every predictor
Explanation
Correct Answer
D:Correct. Enforce event-time feature availability and cutoff validation for every predictor is aimed at the actual failure: outcome-period data leaks into predictors unavailable at scoring time. The audit should subsequently obtain point-in-time feature store queries compared with historical scoring timestamps to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Strip outcome-bearing metadata and test performance on blinded image content” is a control for shortcut features leak labels and undermine production generalization, but the audit finding here is outcome-period data leaks into predictors unavailable at scoring time. The relevant measure is “Enforce event-time feature availability and cutoff validation for every predictor”.
B:“Reassess linkage risk and protect keys, quasi-identifiers and dataset releases” is a control for pseudonymization was treated as irreversibly anonymous, but the audit finding here is outcome-period data leaks into predictors unavailable at scoring time. The relevant measure is “Enforce event-time feature availability and cutoff validation for every predictor”.
C:“Encode missingness and validate imputation against sensor-state evidence” is a control for imputation logic confuses sensor outages with genuine zero measurements, but the audit finding here is outcome-period data leaks into predictors unavailable at scoring time. The relevant measure is “Enforce event-time feature availability and cutoff validation for every predictor”.
Question 176
In an audit of Data Management Specific to AI, managers say this measure is live: Enforce event-time feature availability and cutoff validation for every predictor. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Metadata-ablated benchmark and unchanged test-label separation
- Reidentification tests, key access records and release-risk signoff
- Point-in-time feature store queries compared with historical scoring timestamps
- Source telemetry and imputation trace on known outage intervals
Explanation
Correct Answer
C:Correct. The proposed records—point-in-time feature store queries compared with historical scoring timestamps—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is outcome-period data leaks into predictors unavailable at scoring time.
Incorrect Answers
A:“Metadata-ablated benchmark and unchanged test-label separation” would test shortcut features leak labels and undermine production generalization, not the present safeguard against outcome-period data leaks into predictors unavailable at scoring time. The expected audit evidence is “Point-in-time feature store queries compared with historical scoring timestamps”.
B:“Reidentification tests, key access records and release-risk signoff” would test pseudonymization was treated as irreversibly anonymous, not the present safeguard against outcome-period data leaks into predictors unavailable at scoring time. The expected audit evidence is “Point-in-time feature store queries compared with historical scoring timestamps”.
D:“Source telemetry and imputation trace on known outage intervals” would test imputation logic confuses sensor outages with genuine zero measurements, not the present safeguard against outcome-period data leaks into predictors unavailable at scoring time. The expected audit evidence is “Point-in-time feature store queries compared with historical scoring timestamps”.
Question 177
In a review of Data Management Specific to AI, investigators cannot establish the expected control proof: point-in-time feature store queries compared with historical scoring timestamps. The underlying event was that a dataset used to predict employee attrition contains performance reviews written after the employee left. Which control weakness is the MOST likely cause requiring focused audit work?
- Shortcut features leak labels and undermine production generalization
- Outcome-period data leaks into predictors unavailable at scoring time
- Pseudonymization was treated as irreversibly anonymous
- Imputation logic confuses sensor outages with genuine zero measurements
Explanation
Correct Answer
B:Correct. The observed condition specifically indicates outcome-period data leaks into predictors unavailable at scoring time. A meaningful audit response is to verify this diagnosis and then recommend enforce event-time feature availability and cutoff validation for every predictor.
Incorrect Answers
A:“Shortcut features leak labels and undermine production generalization” concerns a separate audit weakness, not the missing assurance described by outcome-period data leaks into predictors unavailable at scoring time. The correct cause is “Outcome-period data leaks into predictors unavailable at scoring time”.
C:“Pseudonymization was treated as irreversibly anonymous” concerns a separate audit weakness, not the missing assurance described by outcome-period data leaks into predictors unavailable at scoring time. The correct cause is “Outcome-period data leaks into predictors unavailable at scoring time”.
D:“Imputation logic confuses sensor outages with genuine zero measurements” concerns a separate audit weakness, not the missing assurance described by outcome-period data leaks into predictors unavailable at scoring time. The correct cause is “Outcome-period data leaks into predictors unavailable at scoring time”.
Question 178
The model-training platform overwrites yesterday's training extract without keeping a snapshot identifier. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Version immutable datasets and hashes linked to each model build
- Quantify segment coverage and apply justified sampling with separate evaluation
- Apply encrypted transport, integrity verification and attributable transfer logging
- Enforce typed identifiers and record-level reconciliation before feature merges
Explanation
Correct Answer
A:Correct. Version immutable datasets and hashes linked to each model build is aimed at the actual failure: data reproducibility and provenance are absent from the ml pipeline. The audit should subsequently obtain versioned data manifests, build ids and successful sample reconstruction to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Quantify segment coverage and apply justified sampling with separate evaluation” is a control for representation imbalance hides subgroup error and instability, but the audit finding here is data reproducibility and provenance are absent from the ML pipeline. The relevant measure is “Version immutable datasets and hashes linked to each model build”.
C:“Apply encrypted transport, integrity verification and attributable transfer logging” is a control for in-transit protection and accountable data movement are missing, but the audit finding here is data reproducibility and provenance are absent from the ML pipeline. The relevant measure is “Version immutable datasets and hashes linked to each model build”.
D:“Enforce typed identifiers and record-level reconciliation before feature merges” is a control for join normalization corrupts entity matching between data sources, but the audit finding here is data reproducibility and provenance are absent from the ML pipeline. The relevant measure is “Version immutable datasets and hashes linked to each model build”.
Question 179
In an audit of Data Management Specific to AI, managers say this measure is live: Version immutable datasets and hashes linked to each model build. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Per-segment counts, sampling decisions and stable held-out subgroup metrics
- Transfer telemetry, cryptographic configuration and tamper-detection evidence
- Join exception statistics and sampled identifier matches against source systems
- Versioned data manifests, build IDs and successful sample reconstruction
Explanation
Correct Answer
D:Correct. The proposed records—versioned data manifests, build ids and successful sample reconstruction—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is data reproducibility and provenance are absent from the ml pipeline.
Incorrect Answers
A:“Per-segment counts, sampling decisions and stable held-out subgroup metrics” would test representation imbalance hides subgroup error and instability, not the present safeguard against data reproducibility and provenance are absent from the ML pipeline. The expected audit evidence is “Versioned data manifests, build IDs and successful sample reconstruction”.
B:“Transfer telemetry, cryptographic configuration and tamper-detection evidence” would test in-transit protection and accountable data movement are missing, not the present safeguard against data reproducibility and provenance are absent from the ML pipeline. The expected audit evidence is “Versioned data manifests, build IDs and successful sample reconstruction”.
C:“Join exception statistics and sampled identifier matches against source systems” would test join normalization corrupts entity matching between data sources, not the present safeguard against data reproducibility and provenance are absent from the ML pipeline. The expected audit evidence is “Versioned data manifests, build IDs and successful sample reconstruction”.
Question 180
In a review of Data Management Specific to AI, investigators cannot establish the expected control proof: versioned data manifests, build ids and successful sample reconstruction. The underlying event was that the model-training platform overwrites yesterday's training extract without keeping a snapshot identifier. Which control weakness is the MOST likely cause requiring focused audit work?
- Representation imbalance hides subgroup error and instability
- In-transit protection and accountable data movement are missing
- Data reproducibility and provenance are absent from the ML pipeline
- Join normalization corrupts entity matching between data sources
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates data reproducibility and provenance are absent from the ml pipeline. A meaningful audit response is to verify this diagnosis and then recommend version immutable datasets and hashes linked to each model build.
Incorrect Answers
A:“Representation imbalance hides subgroup error and instability” concerns a separate audit weakness, not the missing assurance described by data reproducibility and provenance are absent from the ML pipeline. The correct cause is “Data reproducibility and provenance are absent from the ML pipeline”.
B:“In-transit protection and accountable data movement are missing” concerns a separate audit weakness, not the missing assurance described by data reproducibility and provenance are absent from the ML pipeline. The correct cause is “Data reproducibility and provenance are absent from the ML pipeline”.
D:“Join normalization corrupts entity matching between data sources” concerns a separate audit weakness, not the missing assurance described by data reproducibility and provenance are absent from the ML pipeline. The correct cause is “Data reproducibility and provenance are absent from the ML pipeline”.