ISACA Advanced in AI Audit AAIA Practice Test 17
Question 160
A public-sector AI decision is logged only as an opaque model score. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Provide understandable notice, accessible review and empowered human appeals
- Map binding obligations by use, geography and sector separately from voluntary guidance
- Retain decision rationale, responsible official and meaningful challenge information
- Obtain control-specific assurance, testing scope and material exception disclosure
Explanation
Correct Answer
C:Correct. Retain decision rationale, responsible official and meaningful challenge information is aimed at the actual failure: public accountability requires traceable reasons and reviewed decision ownership. The audit should subsequently obtain sampled decisions with provenance, reason codes and human signoff to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Provide understandable notice, accessible review and empowered human appeals” is a control for human rights and contestability were not incorporated into consequential AI decision design, but the audit finding here is public accountability requires traceable reasons and reviewed decision ownership. The relevant measure is “Retain decision rationale, responsible official and meaningful challenge information”.
B:“Map binding obligations by use, geography and sector separately from voluntary guidance” is a control for voluntary framework alignment was mistaken for jurisdiction-specific legal compliance, but the audit finding here is public accountability requires traceable reasons and reviewed decision ownership. The relevant measure is “Retain decision rationale, responsible official and meaningful challenge information”.
D:“Obtain control-specific assurance, testing scope and material exception disclosure” is a control for marketing attestations were accepted without independent evidence of risk controls, but the audit finding here is public accountability requires traceable reasons and reviewed decision ownership. The relevant measure is “Retain decision rationale, responsible official and meaningful challenge information”.
Question 161
In an audit of Leading Practices, Ethics, Regulations, and Standards for AI, managers say this measure is live: Retain decision rationale, responsible official and meaningful challenge information. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Sampled appeal cases, reversal authority and applicant notification records
- Sampled decisions with provenance, reason codes and human signoff
- Jurisdictional obligation matrix and legal-reviewed control coverage
- Supplier audit evidence, findings and accepted remediation plans
Explanation
Correct Answer
B:Correct. The proposed records—sampled decisions with provenance, reason codes and human signoff—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is public accountability requires traceable reasons and reviewed decision ownership.
Incorrect Answers
A:“Sampled appeal cases, reversal authority and applicant notification records” would test human rights and contestability were not incorporated into consequential AI decision design, not the present safeguard against public accountability requires traceable reasons and reviewed decision ownership. The expected audit evidence is “Sampled decisions with provenance, reason codes and human signoff”.
C:“Jurisdictional obligation matrix and legal-reviewed control coverage” would test voluntary framework alignment was mistaken for jurisdiction-specific legal compliance, not the present safeguard against public accountability requires traceable reasons and reviewed decision ownership. The expected audit evidence is “Sampled decisions with provenance, reason codes and human signoff”.
D:“Supplier audit evidence, findings and accepted remediation plans” would test marketing attestations were accepted without independent evidence of risk controls, not the present safeguard against public accountability requires traceable reasons and reviewed decision ownership. The expected audit evidence is “Sampled decisions with provenance, reason codes and human signoff”.
Question 162
In a review of Leading Practices, Ethics, Regulations, and Standards for AI, investigators cannot establish the expected control proof: sampled decisions with provenance, reason codes and human signoff. The underlying event was that a public-sector AI decision is logged only as an opaque model score. Which control weakness is the MOST likely cause requiring focused audit work?
- Public accountability requires traceable reasons and reviewed decision ownership
- Human rights and contestability were not incorporated into consequential AI decision design
- Voluntary framework alignment was mistaken for jurisdiction-specific legal compliance
- Marketing attestations were accepted without independent evidence of risk controls
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates public accountability requires traceable reasons and reviewed decision ownership. A meaningful audit response is to verify this diagnosis and then recommend retain decision rationale, responsible official and meaningful challenge information.
Incorrect Answers
B:“Human rights and contestability were not incorporated into consequential AI decision design” concerns a separate audit weakness, not the missing assurance described by public accountability requires traceable reasons and reviewed decision ownership. The correct cause is “Public accountability requires traceable reasons and reviewed decision ownership”.
C:“Voluntary framework alignment was mistaken for jurisdiction-specific legal compliance” concerns a separate audit weakness, not the missing assurance described by public accountability requires traceable reasons and reviewed decision ownership. The correct cause is “Public accountability requires traceable reasons and reviewed decision ownership”.
D:“Marketing attestations were accepted without independent evidence of risk controls” concerns a separate audit weakness, not the missing assurance described by public accountability requires traceable reasons and reviewed decision ownership. The correct cause is “Public accountability requires traceable reasons and reviewed decision ownership”.
Question 163
AI governance references the NIST AI RMF but teams treat 'govern' as a one-time compliance checkbox. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Evaluate subgroup errors, likely harms and risk treatment before deployment
- Determine applicable disclosure duties and implement timely understandable notices
- Evaluate necessity, less intrusive alternatives and employee safeguards
- Integrate govern-map-measure-manage activities into change and operating cycles
Explanation
Correct Answer
D:Correct. Integrate govern-map-measure-manage activities into change and operating cycles is aimed at the actual failure: continuous governance and measurement were reduced to static documentation. The audit should subsequently obtain versioned risk records, monitored outcomes and management-action closure to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Evaluate subgroup errors, likely harms and risk treatment before deployment” is a control for equity objectives and subgroup testing were excluded from ethical release criteria, but the audit finding here is continuous governance and measurement were reduced to static documentation. The relevant measure is “Integrate govern-map-measure-manage activities into change and operating cycles”.
B:“Determine applicable disclosure duties and implement timely understandable notices” is a control for required transparency and notice assessments were omitted from user experience, but the audit finding here is continuous governance and measurement were reduced to static documentation. The relevant measure is “Integrate govern-map-measure-manage activities into change and operating cycles”.
C:“Evaluate necessity, less intrusive alternatives and employee safeguards” is a control for workforce impacts and proportionality were excluded from deployment assessment, but the audit finding here is continuous governance and measurement were reduced to static documentation. The relevant measure is “Integrate govern-map-measure-manage activities into change and operating cycles”.
Question 164
In an audit of Leading Practices, Ethics, Regulations, and Standards for AI, managers say this measure is live: Integrate govern-map-measure-manage activities into change and operating cycles. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Group-disaggregated false-match rates and approved mitigation threshold results
- Approved notice language, delivery logs and complaints-handling evidence
- Versioned risk records, monitored outcomes and management-action closure
- Workforce consultation record, proportionality analysis and monitoring limits
Explanation
Correct Answer
C:Correct. The proposed records—versioned risk records, monitored outcomes and management-action closure—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is continuous governance and measurement were reduced to static documentation.
Incorrect Answers
A:“Group-disaggregated false-match rates and approved mitigation threshold results” would test equity objectives and subgroup testing were excluded from ethical release criteria, not the present safeguard against continuous governance and measurement were reduced to static documentation. The expected audit evidence is “Versioned risk records, monitored outcomes and management-action closure”.
B:“Approved notice language, delivery logs and complaints-handling evidence” would test required transparency and notice assessments were omitted from user experience, not the present safeguard against continuous governance and measurement were reduced to static documentation. The expected audit evidence is “Versioned risk records, monitored outcomes and management-action closure”.
D:“Workforce consultation record, proportionality analysis and monitoring limits” would test workforce impacts and proportionality were excluded from deployment assessment, not the present safeguard against continuous governance and measurement were reduced to static documentation. The expected audit evidence is “Versioned risk records, monitored outcomes and management-action closure”.
Question 165
In a review of Leading Practices, Ethics, Regulations, and Standards for AI, investigators cannot establish the expected control proof: versioned risk records, monitored outcomes and management-action closure. The underlying event was that aI governance references the NIST AI RMF but teams treat 'govern' as a one-time compliance checkbox. Which control weakness is the MOST likely cause requiring focused audit work?
- Equity objectives and subgroup testing were excluded from ethical release criteria
- Continuous governance and measurement were reduced to static documentation
- Required transparency and notice assessments were omitted from user experience
- Workforce impacts and proportionality were excluded from deployment assessment
Explanation
Correct Answer
B:Correct. The observed condition specifically indicates continuous governance and measurement were reduced to static documentation. A meaningful audit response is to verify this diagnosis and then recommend integrate govern-map-measure-manage activities into change and operating cycles.
Incorrect Answers
A:“Equity objectives and subgroup testing were excluded from ethical release criteria” concerns a separate audit weakness, not the missing assurance described by continuous governance and measurement were reduced to static documentation. The correct cause is “Continuous governance and measurement were reduced to static documentation”.
C:“Required transparency and notice assessments were omitted from user experience” concerns a separate audit weakness, not the missing assurance described by continuous governance and measurement were reduced to static documentation. The correct cause is “Continuous governance and measurement were reduced to static documentation”.
D:“Workforce impacts and proportionality were excluded from deployment assessment” concerns a separate audit weakness, not the missing assurance described by continuous governance and measurement were reduced to static documentation. The correct cause is “Continuous governance and measurement were reduced to static documentation”.
Question 166
The fraud team trains on a dataset where chargebacks are underreported for newly onboarded merchants. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Audit label completeness by merchant age and use independently confirmed outcome samples
- Enforce typed identifiers and record-level reconciliation before feature merges
- Strip outcome-bearing metadata and test performance on blinded image content
- Apply encrypted transport, integrity verification and attributable transfer logging
Explanation
Correct Answer
A:Correct. Audit label completeness by merchant age and use independently confirmed outcome samples is aimed at the actual failure: missing labels and coverage create misleadingly low observed fraud rates. The audit should subsequently obtain label reconciliation by tenure cohort and corrected training-data completeness report to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Enforce typed identifiers and record-level reconciliation before feature merges” is a control for join normalization corrupts entity matching between data sources, but the audit finding here is missing labels and coverage create misleadingly low observed fraud rates. The relevant measure is “Audit label completeness by merchant age and use independently confirmed outcome samples”.
C:“Strip outcome-bearing metadata and test performance on blinded image content” is a control for shortcut features leak labels and undermine production generalization, but the audit finding here is missing labels and coverage create misleadingly low observed fraud rates. The relevant measure is “Audit label completeness by merchant age and use independently confirmed outcome samples”.
D:“Apply encrypted transport, integrity verification and attributable transfer logging” is a control for in-transit protection and accountable data movement are missing, but the audit finding here is missing labels and coverage create misleadingly low observed fraud rates. The relevant measure is “Audit label completeness by merchant age and use independently confirmed outcome samples”.
Question 167
In an audit of Data Management Specific to AI, managers say this measure is live: Audit label completeness by merchant age and use independently confirmed outcome samples. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Join exception statistics and sampled identifier matches against source systems
- Metadata-ablated benchmark and unchanged test-label separation
- Transfer telemetry, cryptographic configuration and tamper-detection evidence
- Label reconciliation by tenure cohort and corrected training-data completeness report
Explanation
Correct Answer
D:Correct. The proposed records—label reconciliation by tenure cohort and corrected training-data completeness report—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is missing labels and coverage create misleadingly low observed fraud rates.
Incorrect Answers
A:“Join exception statistics and sampled identifier matches against source systems” would test join normalization corrupts entity matching between data sources, not the present safeguard against missing labels and coverage create misleadingly low observed fraud rates. The expected audit evidence is “Label reconciliation by tenure cohort and corrected training-data completeness report”.
B:“Metadata-ablated benchmark and unchanged test-label separation” would test shortcut features leak labels and undermine production generalization, not the present safeguard against missing labels and coverage create misleadingly low observed fraud rates. The expected audit evidence is “Label reconciliation by tenure cohort and corrected training-data completeness report”.
C:“Transfer telemetry, cryptographic configuration and tamper-detection evidence” would test in-transit protection and accountable data movement are missing, not the present safeguard against missing labels and coverage create misleadingly low observed fraud rates. The expected audit evidence is “Label reconciliation by tenure cohort and corrected training-data completeness report”.
Question 168
In a review of Data Management Specific to AI, investigators cannot establish the expected control proof: label reconciliation by tenure cohort and corrected training-data completeness report. The underlying event was that the fraud team trains on a dataset where chargebacks are underreported for newly onboarded merchants. Which control weakness is the MOST likely cause requiring focused audit work?
- Join normalization corrupts entity matching between data sources
- Shortcut features leak labels and undermine production generalization
- Missing labels and coverage create misleadingly low observed fraud rates
- In-transit protection and accountable data movement are missing
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates missing labels and coverage create misleadingly low observed fraud rates. A meaningful audit response is to verify this diagnosis and then recommend audit label completeness by merchant age and use independently confirmed outcome samples.
Incorrect Answers
A:“Join normalization corrupts entity matching between data sources” concerns a separate audit weakness, not the missing assurance described by missing labels and coverage create misleadingly low observed fraud rates. The correct cause is “Missing labels and coverage create misleadingly low observed fraud rates”.
B:“Shortcut features leak labels and undermine production generalization” concerns a separate audit weakness, not the missing assurance described by missing labels and coverage create misleadingly low observed fraud rates. The correct cause is “Missing labels and coverage create misleadingly low observed fraud rates”.
D:“In-transit protection and accountable data movement are missing” concerns a separate audit weakness, not the missing assurance described by missing labels and coverage create misleadingly low observed fraud rates. The correct cause is “Missing labels and coverage create misleadingly low observed fraud rates”.
Question 169
A predictive-maintenance model imputes every missing vibration reading as zero even when sensors were offline. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Enforce event-time feature availability and cutoff validation for every predictor
- Encode missingness and validate imputation against sensor-state evidence
- Quantify segment coverage and apply justified sampling with separate evaluation
- Introduce contract tests for unit, range and meaning changes before retraining
Explanation
Correct Answer
B:Correct. Encode missingness and validate imputation against sensor-state evidence is aimed at the actual failure: imputation logic confuses sensor outages with genuine zero measurements. The audit should subsequently obtain source telemetry and imputation trace on known outage intervals to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Enforce event-time feature availability and cutoff validation for every predictor” is a control for outcome-period data leaks into predictors unavailable at scoring time, but the audit finding here is imputation logic confuses sensor outages with genuine zero measurements. The relevant measure is “Encode missingness and validate imputation against sensor-state evidence”.
C:“Quantify segment coverage and apply justified sampling with separate evaluation” is a control for representation imbalance hides subgroup error and instability, but the audit finding here is imputation logic confuses sensor outages with genuine zero measurements. The relevant measure is “Encode missingness and validate imputation against sensor-state evidence”.
D:“Introduce contract tests for unit, range and meaning changes before retraining” is a control for data-quality gates do not validate semantic feature definitions across pipeline versions, but the audit finding here is imputation logic confuses sensor outages with genuine zero measurements. The relevant measure is “Encode missingness and validate imputation against sensor-state evidence”.
Question 170
In an audit of Data Management Specific to AI, managers say this measure is live: Encode missingness and validate imputation against sensor-state evidence. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Source telemetry and imputation trace on known outage intervals
- Point-in-time feature store queries compared with historical scoring timestamps
- Per-segment counts, sampling decisions and stable held-out subgroup metrics
- Schema diff, unit conversion assertions and rejected malformed batches
Explanation
Correct Answer
A:Correct. The proposed records—source telemetry and imputation trace on known outage intervals—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is imputation logic confuses sensor outages with genuine zero measurements.
Incorrect Answers
B:“Point-in-time feature store queries compared with historical scoring timestamps” would test outcome-period data leaks into predictors unavailable at scoring time, not the present safeguard against imputation logic confuses sensor outages with genuine zero measurements. The expected audit evidence is “Source telemetry and imputation trace on known outage intervals”.
C:“Per-segment counts, sampling decisions and stable held-out subgroup metrics” would test representation imbalance hides subgroup error and instability, not the present safeguard against imputation logic confuses sensor outages with genuine zero measurements. The expected audit evidence is “Source telemetry and imputation trace on known outage intervals”.
D:“Schema diff, unit conversion assertions and rejected malformed batches” would test data-quality gates do not validate semantic feature definitions across pipeline versions, not the present safeguard against imputation logic confuses sensor outages with genuine zero measurements. The expected audit evidence is “Source telemetry and imputation trace on known outage intervals”.