ISACA Advanced in AI Audit AAIA Practice Test 14
Question 130
A model output accidentally reveals confidential source text verbatim from training. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Test extraction risk and restrict sensitive content in model training and output handling
- Require documented lineage, rights basis and permitted-use checks before ingestion
- Perform linkage attacks and set release safeguards appropriate to residual identifiability
- Inventory processing locations, subprocessors and cross-border transfer safeguards
Explanation
Correct Answer
A:Correct. Test extraction risk and restrict sensitive content in model training and output handling is aimed at the actual failure: training memorization and output disclosure controls were unassessed. The audit should subsequently obtain memorization probe results, content filters and reviewed training-data exclusions to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Require documented lineage, rights basis and permitted-use checks before ingestion” is a control for data provenance and lawful-use assurance are not established, but the audit finding here is training memorization and output disclosure controls were unassessed. The relevant measure is “Test extraction risk and restrict sensitive content in model training and output handling”.
C:“Perform linkage attacks and set release safeguards appropriate to residual identifiability” is a control for reidentification risk was not tested for the actual release context, but the audit finding here is training memorization and output disclosure controls were unassessed. The relevant measure is “Test extraction risk and restrict sensitive content in model training and output handling”.
D:“Inventory processing locations, subprocessors and cross-border transfer safeguards” is a control for aI data transfers are missing location and processor transparency, but the audit finding here is training memorization and output disclosure controls were unassessed. The relevant measure is “Test extraction risk and restrict sensitive content in model training and output handling”.
Question 131
In an audit of Privacy and Data Governance Programs, managers say this measure is live: Test extraction risk and restrict sensitive content in model training and output handling. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Data source records, licensing terms and approval evidence for each training corpus
- Adversarial reidentification results, suppression thresholds and release decision
- Verified processor list, hosting configuration and applicable transfer assessments
- Memorization probe results, content filters and reviewed training-data exclusions
Explanation
Correct Answer
D:Correct. The proposed records—memorization probe results, content filters and reviewed training-data exclusions—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is training memorization and output disclosure controls were unassessed.
Incorrect Answers
A:“Data source records, licensing terms and approval evidence for each training corpus” would test data provenance and lawful-use assurance are not established, not the present safeguard against training memorization and output disclosure controls were unassessed. The expected audit evidence is “Memorization probe results, content filters and reviewed training-data exclusions”.
B:“Adversarial reidentification results, suppression thresholds and release decision” would test reidentification risk was not tested for the actual release context, not the present safeguard against training memorization and output disclosure controls were unassessed. The expected audit evidence is “Memorization probe results, content filters and reviewed training-data exclusions”.
C:“Verified processor list, hosting configuration and applicable transfer assessments” would test aI data transfers are missing location and processor transparency, not the present safeguard against training memorization and output disclosure controls were unassessed. The expected audit evidence is “Memorization probe results, content filters and reviewed training-data exclusions”.
Question 132
In a review of Privacy and Data Governance Programs, investigators cannot establish the expected control proof: memorization probe results, content filters and reviewed training-data exclusions. The underlying event was that a model output accidentally reveals confidential source text verbatim from training. Which control weakness is the MOST likely cause requiring focused audit work?
- Data provenance and lawful-use assurance are not established
- Reidentification risk was not tested for the actual release context
- Training memorization and output disclosure controls were unassessed
- AI data transfers are missing location and processor transparency
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates training memorization and output disclosure controls were unassessed. A meaningful audit response is to verify this diagnosis and then recommend test extraction risk and restrict sensitive content in model training and output handling.
Incorrect Answers
A:“Data provenance and lawful-use assurance are not established” concerns a separate audit weakness, not the missing assurance described by training memorization and output disclosure controls were unassessed. The correct cause is “Training memorization and output disclosure controls were unassessed”.
B:“Reidentification risk was not tested for the actual release context” concerns a separate audit weakness, not the missing assurance described by training memorization and output disclosure controls were unassessed. The correct cause is “Training memorization and output disclosure controls were unassessed”.
D:“AI data transfers are missing location and processor transparency” concerns a separate audit weakness, not the missing assurance described by training memorization and output disclosure controls were unassessed. The correct cause is “Training memorization and output disclosure controls were unassessed”.
Question 133
A benefits eligibility engine rejects applicants automatically and offers no meaningful appeal channel. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Map ethics principles to lifecycle controls with evidence owners and independent testing
- Provide understandable notice, accessible review and empowered human appeals
- Assess local explanation fidelity and suitability for users and reviewers
- Retain decision rationale, responsible official and meaningful challenge information
Explanation
Correct Answer
B:Correct. Provide understandable notice, accessible review and empowered human appeals is aimed at the actual failure: human rights and contestability were not incorporated into consequential ai decision design. The audit should subsequently obtain sampled appeal cases, reversal authority and applicant notification records to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Map ethics principles to lifecycle controls with evidence owners and independent testing” is a control for abstract principles lack assigned operational controls and measurable compliance, but the audit finding here is human rights and contestability were not incorporated into consequential AI decision design. The relevant measure is “Provide understandable notice, accessible review and empowered human appeals”.
C:“Assess local explanation fidelity and suitability for users and reviewers” is a control for explanations are not validated for individual decision context and audience, but the audit finding here is human rights and contestability were not incorporated into consequential AI decision design. The relevant measure is “Provide understandable notice, accessible review and empowered human appeals”.
D:“Retain decision rationale, responsible official and meaningful challenge information” is a control for public accountability requires traceable reasons and reviewed decision ownership, but the audit finding here is human rights and contestability were not incorporated into consequential AI decision design. The relevant measure is “Provide understandable notice, accessible review and empowered human appeals”.
Question 134
In an audit of Leading Practices, Ethics, Regulations, and Standards for AI, managers say this measure is live: Provide understandable notice, accessible review and empowered human appeals. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Sampled appeal cases, reversal authority and applicant notification records
- Control-to-principle trace and audited execution artifacts
- Reviewed case explanations, fidelity tests and comprehensibility feedback
- Sampled decisions with provenance, reason codes and human signoff
Explanation
Correct Answer
A:Correct. The proposed records—sampled appeal cases, reversal authority and applicant notification records—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is human rights and contestability were not incorporated into consequential ai decision design.
Incorrect Answers
B:“Control-to-principle trace and audited execution artifacts” would test abstract principles lack assigned operational controls and measurable compliance, not the present safeguard against human rights and contestability were not incorporated into consequential AI decision design. The expected audit evidence is “Sampled appeal cases, reversal authority and applicant notification records”.
C:“Reviewed case explanations, fidelity tests and comprehensibility feedback” would test explanations are not validated for individual decision context and audience, not the present safeguard against human rights and contestability were not incorporated into consequential AI decision design. The expected audit evidence is “Sampled appeal cases, reversal authority and applicant notification records”.
D:“Sampled decisions with provenance, reason codes and human signoff” would test public accountability requires traceable reasons and reviewed decision ownership, not the present safeguard against human rights and contestability were not incorporated into consequential AI decision design. The expected audit evidence is “Sampled appeal cases, reversal authority and applicant notification records”.
Question 135
In a review of Leading Practices, Ethics, Regulations, and Standards for AI, investigators cannot establish the expected control proof: sampled appeal cases, reversal authority and applicant notification records. The underlying event was that a benefits eligibility engine rejects applicants automatically and offers no meaningful appeal channel. Which control weakness is the MOST likely cause requiring focused audit work?
- Abstract principles lack assigned operational controls and measurable compliance
- Explanations are not validated for individual decision context and audience
- Public accountability requires traceable reasons and reviewed decision ownership
- Human rights and contestability were not incorporated into consequential AI decision design
Explanation
Correct Answer
D:Correct. The observed condition specifically indicates human rights and contestability were not incorporated into consequential ai decision design. A meaningful audit response is to verify this diagnosis and then recommend provide understandable notice, accessible review and empowered human appeals.
Incorrect Answers
A:“Abstract principles lack assigned operational controls and measurable compliance” concerns a separate audit weakness, not the missing assurance described by human rights and contestability were not incorporated into consequential AI decision design. The correct cause is “Human rights and contestability were not incorporated into consequential AI decision design”.
B:“Explanations are not validated for individual decision context and audience” concerns a separate audit weakness, not the missing assurance described by human rights and contestability were not incorporated into consequential AI decision design. The correct cause is “Human rights and contestability were not incorporated into consequential AI decision design”.
C:“Public accountability requires traceable reasons and reviewed decision ownership” concerns a separate audit weakness, not the missing assurance described by human rights and contestability were not incorporated into consequential AI decision design. The correct cause is “Human rights and contestability were not incorporated into consequential AI decision design”.
Question 136
A facial recognition pilot has higher false-match rates for certain demographics but approval uses overall accuracy. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Map binding obligations by use, geography and sector separately from voluntary guidance
- Conduct participatory impact review and establish accessible challenge mechanisms
- Evaluate subgroup errors, likely harms and risk treatment before deployment
- Integrate govern-map-measure-manage activities into change and operating cycles
Explanation
Correct Answer
C:Correct. Evaluate subgroup errors, likely harms and risk treatment before deployment is aimed at the actual failure: equity objectives and subgroup testing were excluded from ethical release criteria. The audit should subsequently obtain group-disaggregated false-match rates and approved mitigation threshold results to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Map binding obligations by use, geography and sector separately from voluntary guidance” is a control for voluntary framework alignment was mistaken for jurisdiction-specific legal compliance, but the audit finding here is equity objectives and subgroup testing were excluded from ethical release criteria. The relevant measure is “Evaluate subgroup errors, likely harms and risk treatment before deployment”.
B:“Conduct participatory impact review and establish accessible challenge mechanisms” is a control for vulnerable stakeholder impact was absent from responsible AI assessment, but the audit finding here is equity objectives and subgroup testing were excluded from ethical release criteria. The relevant measure is “Evaluate subgroup errors, likely harms and risk treatment before deployment”.
D:“Integrate govern-map-measure-manage activities into change and operating cycles” is a control for continuous governance and measurement were reduced to static documentation, but the audit finding here is equity objectives and subgroup testing were excluded from ethical release criteria. The relevant measure is “Evaluate subgroup errors, likely harms and risk treatment before deployment”.
Question 137
In an audit of Leading Practices, Ethics, Regulations, and Standards for AI, managers say this measure is live: Evaluate subgroup errors, likely harms and risk treatment before deployment. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Jurisdictional obligation matrix and legal-reviewed control coverage
- Group-disaggregated false-match rates and approved mitigation threshold results
- Impact-assessment minutes and targeted user testing of appeal paths
- Versioned risk records, monitored outcomes and management-action closure
Explanation
Correct Answer
B:Correct. The proposed records—group-disaggregated false-match rates and approved mitigation threshold results—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is equity objectives and subgroup testing were excluded from ethical release criteria.
Incorrect Answers
A:“Jurisdictional obligation matrix and legal-reviewed control coverage” would test voluntary framework alignment was mistaken for jurisdiction-specific legal compliance, not the present safeguard against equity objectives and subgroup testing were excluded from ethical release criteria. The expected audit evidence is “Group-disaggregated false-match rates and approved mitigation threshold results”.
C:“Impact-assessment minutes and targeted user testing of appeal paths” would test vulnerable stakeholder impact was absent from responsible AI assessment, not the present safeguard against equity objectives and subgroup testing were excluded from ethical release criteria. The expected audit evidence is “Group-disaggregated false-match rates and approved mitigation threshold results”.
D:“Versioned risk records, monitored outcomes and management-action closure” would test continuous governance and measurement were reduced to static documentation, not the present safeguard against equity objectives and subgroup testing were excluded from ethical release criteria. The expected audit evidence is “Group-disaggregated false-match rates and approved mitigation threshold results”.
Question 138
In a review of Leading Practices, Ethics, Regulations, and Standards for AI, investigators cannot establish the expected control proof: group-disaggregated false-match rates and approved mitigation threshold results. The underlying event was that a facial recognition pilot has higher false-match rates for certain demographics but approval uses overall accuracy. Which control weakness is the MOST likely cause requiring focused audit work?
- Equity objectives and subgroup testing were excluded from ethical release criteria
- Voluntary framework alignment was mistaken for jurisdiction-specific legal compliance
- Vulnerable stakeholder impact was absent from responsible AI assessment
- Continuous governance and measurement were reduced to static documentation
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates equity objectives and subgroup testing were excluded from ethical release criteria. A meaningful audit response is to verify this diagnosis and then recommend evaluate subgroup errors, likely harms and risk treatment before deployment.
Incorrect Answers
B:“Voluntary framework alignment was mistaken for jurisdiction-specific legal compliance” concerns a separate audit weakness, not the missing assurance described by equity objectives and subgroup testing were excluded from ethical release criteria. The correct cause is “Equity objectives and subgroup testing were excluded from ethical release criteria”.
C:“Vulnerable stakeholder impact was absent from responsible AI assessment” concerns a separate audit weakness, not the missing assurance described by equity objectives and subgroup testing were excluded from ethical release criteria. The correct cause is “Equity objectives and subgroup testing were excluded from ethical release criteria”.
D:“Continuous governance and measurement were reduced to static documentation” concerns a separate audit weakness, not the missing assurance described by equity objectives and subgroup testing were excluded from ethical release criteria. The correct cause is “Equity objectives and subgroup testing were excluded from ethical release criteria”.
Question 139
An AI ethics charter is displayed publicly but product teams have no documented control implementation. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Determine applicable disclosure duties and implement timely understandable notices
- Obtain control-specific assurance, testing scope and material exception disclosure
- Provide understandable notice, accessible review and empowered human appeals
- Map ethics principles to lifecycle controls with evidence owners and independent testing
Explanation
Correct Answer
D:Correct. Map ethics principles to lifecycle controls with evidence owners and independent testing is aimed at the actual failure: abstract principles lack assigned operational controls and measurable compliance. The audit should subsequently obtain control-to-principle trace and audited execution artifacts to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Determine applicable disclosure duties and implement timely understandable notices” is a control for required transparency and notice assessments were omitted from user experience, but the audit finding here is abstract principles lack assigned operational controls and measurable compliance. The relevant measure is “Map ethics principles to lifecycle controls with evidence owners and independent testing”.
B:“Obtain control-specific assurance, testing scope and material exception disclosure” is a control for marketing attestations were accepted without independent evidence of risk controls, but the audit finding here is abstract principles lack assigned operational controls and measurable compliance. The relevant measure is “Map ethics principles to lifecycle controls with evidence owners and independent testing”.
C:“Provide understandable notice, accessible review and empowered human appeals” is a control for human rights and contestability were not incorporated into consequential AI decision design, but the audit finding here is abstract principles lack assigned operational controls and measurable compliance. The relevant measure is “Map ethics principles to lifecycle controls with evidence owners and independent testing”.
Question 140
In an audit of Leading Practices, Ethics, Regulations, and Standards for AI, managers say this measure is live: Map ethics principles to lifecycle controls with evidence owners and independent testing. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Approved notice language, delivery logs and complaints-handling evidence
- Supplier audit evidence, findings and accepted remediation plans
- Control-to-principle trace and audited execution artifacts
- Sampled appeal cases, reversal authority and applicant notification records
Explanation
Correct Answer
C:Correct. The proposed records—control-to-principle trace and audited execution artifacts—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is abstract principles lack assigned operational controls and measurable compliance.
Incorrect Answers
A:“Approved notice language, delivery logs and complaints-handling evidence” would test required transparency and notice assessments were omitted from user experience, not the present safeguard against abstract principles lack assigned operational controls and measurable compliance. The expected audit evidence is “Control-to-principle trace and audited execution artifacts”.
B:“Supplier audit evidence, findings and accepted remediation plans” would test marketing attestations were accepted without independent evidence of risk controls, not the present safeguard against abstract principles lack assigned operational controls and measurable compliance. The expected audit evidence is “Control-to-principle trace and audited execution artifacts”.
D:“Sampled appeal cases, reversal authority and applicant notification records” would test human rights and contestability were not incorporated into consequential AI decision design, not the present safeguard against abstract principles lack assigned operational controls and measurable compliance. The expected audit evidence is “Control-to-principle trace and audited execution artifacts”.