ISACA Advanced in AI Audit AAIA Practice Test 29
Question 280
A loan denial model operates without an accessible method for a reviewer to reverse a decision. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Segment drift baselines by season and operational regime
- Introduce risk-sampled conversation review with privacy-safe escalation
- Enable authorized override with documented rationale and applicant notice
- Track override, appeal and adverse-outcome trends by use case
Explanation
Correct Answer
C:Correct. Enable authorized override with documented rationale and applicant notice is aimed at the actual failure: human intervention authority is not embedded in the workflow. The audit should subsequently obtain role-permission tests and sampled reversal workflow traces to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Segment drift baselines by season and operational regime” is a control for seasonal concept drift is obscured by static population benchmarks, but the audit finding here is human intervention authority is not embedded in the workflow. The relevant measure is “Enable authorized override with documented rationale and applicant notice”.
B:“Introduce risk-sampled conversation review with privacy-safe escalation” is a control for oversight fails to inspect high-risk content and privacy violations, but the audit finding here is human intervention authority is not embedded in the workflow. The relevant measure is “Enable authorized override with documented rationale and applicant notice”.
D:“Track override, appeal and adverse-outcome trends by use case” is a control for oversight metrics omit evidence of downstream correction and user challenge, but the audit finding here is human intervention authority is not embedded in the workflow. The relevant measure is “Enable authorized override with documented rationale and applicant notice”.
Question 281
In an audit of Supervision of AI Solutions, managers say this measure is live: Enable authorized override with documented rationale and applicant notice. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Season-specific drift dashboard and retraining investigation records
- Role-permission tests and sampled reversal workflow traces
- Redacted review samples, detected policy breaches and follow-up actions
- Audit logs of appeals, reversal reasons and trend-based actions
Explanation
Correct Answer
B:Correct. The proposed records—role-permission tests and sampled reversal workflow traces—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is human intervention authority is not embedded in the workflow.
Incorrect Answers
A:“Season-specific drift dashboard and retraining investigation records” would test seasonal concept drift is obscured by static population benchmarks, not the present safeguard against human intervention authority is not embedded in the workflow. The expected audit evidence is “Role-permission tests and sampled reversal workflow traces”.
C:“Redacted review samples, detected policy breaches and follow-up actions” would test oversight fails to inspect high-risk content and privacy violations, not the present safeguard against human intervention authority is not embedded in the workflow. The expected audit evidence is “Role-permission tests and sampled reversal workflow traces”.
D:“Audit logs of appeals, reversal reasons and trend-based actions” would test oversight metrics omit evidence of downstream correction and user challenge, not the present safeguard against human intervention authority is not embedded in the workflow. The expected audit evidence is “Role-permission tests and sampled reversal workflow traces”.
Question 282
In a review of Supervision of AI Solutions, investigators cannot establish the expected control proof: role-permission tests and sampled reversal workflow traces. The underlying event was that a loan denial model operates without an accessible method for a reviewer to reverse a decision. Which control weakness is the MOST likely cause requiring focused audit work?
- Human intervention authority is not embedded in the workflow
- Seasonal concept drift is obscured by static population benchmarks
- Oversight fails to inspect high-risk content and privacy violations
- Oversight metrics omit evidence of downstream correction and user challenge
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates human intervention authority is not embedded in the workflow. A meaningful audit response is to verify this diagnosis and then recommend enable authorized override with documented rationale and applicant notice.
Incorrect Answers
B:“Seasonal concept drift is obscured by static population benchmarks” concerns a separate audit weakness, not the missing assurance described by human intervention authority is not embedded in the workflow. The correct cause is “Human intervention authority is not embedded in the workflow”.
C:“Oversight fails to inspect high-risk content and privacy violations” concerns a separate audit weakness, not the missing assurance described by human intervention authority is not embedded in the workflow. The correct cause is “Human intervention authority is not embedded in the workflow”.
D:“Oversight metrics omit evidence of downstream correction and user challenge” concerns a separate audit weakness, not the missing assurance described by human intervention authority is not embedded in the workflow. The correct cause is “Human intervention authority is not embedded in the workflow”.
Question 283
An LLM summarizer fabricates citations in executive reports but no one checks quoted sources. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Enforce automated safety interlocks with effective emergency shutdown
- Surface relevant decision factors and enable sampled human challenge
- Size triage workflows using alert rates, severity, response targets and staffing
- Require source-link verification and abstention on ungrounded high-impact claims
Explanation
Correct Answer
D:Correct. Require source-link verification and abstention on ungrounded high-impact claims is aimed at the actual failure: output-grounding verification is absent from supervision controls. The audit should subsequently obtain sampled summaries with clicked citations, reviewer flags and corrections to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Enforce automated safety interlocks with effective emergency shutdown” is a control for human override design does not meet the system's action speed and safety needs, but the audit finding here is output-grounding verification is absent from supervision controls. The relevant measure is “Require source-link verification and abstention on ungrounded high-impact claims”.
B:“Surface relevant decision factors and enable sampled human challenge” is a control for reviewer visibility is insufficient to challenge automated adverse decisions, but the audit finding here is output-grounding verification is absent from supervision controls. The relevant measure is “Require source-link verification and abstention on ungrounded high-impact claims”.
C:“Size triage workflows using alert rates, severity, response targets and staffing” is a control for operational supervision capacity is not matched to alert generation, but the audit finding here is output-grounding verification is absent from supervision controls. The relevant measure is “Require source-link verification and abstention on ungrounded high-impact claims”.
Question 284
In an audit of Supervision of AI Solutions, managers say this measure is live: Require source-link verification and abstention on ungrounded high-impact claims. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Measured interlock activation and simulated unsafe action blocking
- Exclusion audit trails, reviewer annotations and remedied decisions
- Sampled summaries with clicked citations, reviewer flags and corrections
- Queue aging, staffing forecasts and sampled incident response times
Explanation
Correct Answer
C:Correct. The proposed records—sampled summaries with clicked citations, reviewer flags and corrections—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is output-grounding verification is absent from supervision controls.
Incorrect Answers
A:“Measured interlock activation and simulated unsafe action blocking” would test human override design does not meet the system's action speed and safety needs, not the present safeguard against output-grounding verification is absent from supervision controls. The expected audit evidence is “Sampled summaries with clicked citations, reviewer flags and corrections”.
B:“Exclusion audit trails, reviewer annotations and remedied decisions” would test reviewer visibility is insufficient to challenge automated adverse decisions, not the present safeguard against output-grounding verification is absent from supervision controls. The expected audit evidence is “Sampled summaries with clicked citations, reviewer flags and corrections”.
D:“Queue aging, staffing forecasts and sampled incident response times” would test operational supervision capacity is not matched to alert generation, not the present safeguard against output-grounding verification is absent from supervision controls. The expected audit evidence is “Sampled summaries with clicked citations, reviewer flags and corrections”.
Question 285
In a review of Supervision of AI Solutions, investigators cannot establish the expected control proof: sampled summaries with clicked citations, reviewer flags and corrections. The underlying event was that an LLM summarizer fabricates citations in executive reports but no one checks quoted sources. Which control weakness is the MOST likely cause requiring focused audit work?
- Human override design does not meet the system's action speed and safety needs
- Output-grounding verification is absent from supervision controls
- Reviewer visibility is insufficient to challenge automated adverse decisions
- Operational supervision capacity is not matched to alert generation
Explanation
Correct Answer
B:Correct. The observed condition specifically indicates output-grounding verification is absent from supervision controls. A meaningful audit response is to verify this diagnosis and then recommend require source-link verification and abstention on ungrounded high-impact claims.
Incorrect Answers
A:“Human override design does not meet the system's action speed and safety needs” concerns a separate audit weakness, not the missing assurance described by output-grounding verification is absent from supervision controls. The correct cause is “Output-grounding verification is absent from supervision controls”.
C:“Reviewer visibility is insufficient to challenge automated adverse decisions” concerns a separate audit weakness, not the missing assurance described by output-grounding verification is absent from supervision controls. The correct cause is “Output-grounding verification is absent from supervision controls”.
D:“Operational supervision capacity is not matched to alert generation” concerns a separate audit weakness, not the missing assurance described by output-grounding verification is absent from supervision controls. The correct cause is “Output-grounding verification is absent from supervision controls”.
Question 286
A field-operations AI model drifts during monsoon season but its monitoring baseline uses only annual averages. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Segment drift baselines by season and operational regime
- Implement severity-aware routing and time-bound specialist review
- Set site-level performance alerts and clinical escalation thresholds
- Enable authorized override with documented rationale and applicant notice
Explanation
Correct Answer
A:Correct. Segment drift baselines by season and operational regime is aimed at the actual failure: seasonal concept drift is obscured by static population benchmarks. The audit should subsequently obtain season-specific drift dashboard and retraining investigation records to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Implement severity-aware routing and time-bound specialist review” is a control for supervision lacks consequence-sensitive escalation tiers, but the audit finding here is seasonal concept drift is obscured by static population benchmarks. The relevant measure is “Segment drift baselines by season and operational regime”.
C:“Set site-level performance alerts and clinical escalation thresholds” is a control for monitoring aggregation hides operationally critical subgroup deterioration, but the audit finding here is seasonal concept drift is obscured by static population benchmarks. The relevant measure is “Segment drift baselines by season and operational regime”.
D:“Enable authorized override with documented rationale and applicant notice” is a control for human intervention authority is not embedded in the workflow, but the audit finding here is seasonal concept drift is obscured by static population benchmarks. The relevant measure is “Segment drift baselines by season and operational regime”.
Question 287
In an audit of Supervision of AI Solutions, managers say this measure is live: Segment drift baselines by season and operational regime. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Alert routing traces, critical-case drill and SLA outcomes
- Site-stratified outcomes, alerts and documented interventions
- Role-permission tests and sampled reversal workflow traces
- Season-specific drift dashboard and retraining investigation records
Explanation
Correct Answer
D:Correct. The proposed records—season-specific drift dashboard and retraining investigation records—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is seasonal concept drift is obscured by static population benchmarks.
Incorrect Answers
A:“Alert routing traces, critical-case drill and SLA outcomes” would test supervision lacks consequence-sensitive escalation tiers, not the present safeguard against seasonal concept drift is obscured by static population benchmarks. The expected audit evidence is “Season-specific drift dashboard and retraining investigation records”.
B:“Site-stratified outcomes, alerts and documented interventions” would test monitoring aggregation hides operationally critical subgroup deterioration, not the present safeguard against seasonal concept drift is obscured by static population benchmarks. The expected audit evidence is “Season-specific drift dashboard and retraining investigation records”.
C:“Role-permission tests and sampled reversal workflow traces” would test human intervention authority is not embedded in the workflow, not the present safeguard against seasonal concept drift is obscured by static population benchmarks. The expected audit evidence is “Season-specific drift dashboard and retraining investigation records”.
Question 288
In a review of Supervision of AI Solutions, investigators cannot establish the expected control proof: season-specific drift dashboard and retraining investigation records. The underlying event was that a field-operations AI model drifts during monsoon season but its monitoring baseline uses only annual averages. Which control weakness is the MOST likely cause requiring focused audit work?
- Supervision lacks consequence-sensitive escalation tiers
- Monitoring aggregation hides operationally critical subgroup deterioration
- Seasonal concept drift is obscured by static population benchmarks
- Human intervention authority is not embedded in the workflow
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates seasonal concept drift is obscured by static population benchmarks. A meaningful audit response is to verify this diagnosis and then recommend segment drift baselines by season and operational regime.
Incorrect Answers
A:“Supervision lacks consequence-sensitive escalation tiers” concerns a separate audit weakness, not the missing assurance described by seasonal concept drift is obscured by static population benchmarks. The correct cause is “Seasonal concept drift is obscured by static population benchmarks”.
B:“Monitoring aggregation hides operationally critical subgroup deterioration” concerns a separate audit weakness, not the missing assurance described by seasonal concept drift is obscured by static population benchmarks. The correct cause is “Seasonal concept drift is obscured by static population benchmarks”.
D:“Human intervention authority is not embedded in the workflow” concerns a separate audit weakness, not the missing assurance described by seasonal concept drift is obscured by static population benchmarks. The correct cause is “Seasonal concept drift is obscured by static population benchmarks”.
Question 289
An AI robot issues actions faster than operators can stop it during abnormal behavior. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Introduce risk-sampled conversation review with privacy-safe escalation
- Enforce automated safety interlocks with effective emergency shutdown
- Define meaningful review duties, challenge prompts and override authority
- Require source-link verification and abstention on ungrounded high-impact claims
Explanation
Correct Answer
B:Correct. Enforce automated safety interlocks with effective emergency shutdown is aimed at the actual failure: human override design does not meet the system's action speed and safety needs. The audit should subsequently obtain measured interlock activation and simulated unsafe action blocking to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Introduce risk-sampled conversation review with privacy-safe escalation” is a control for oversight fails to inspect high-risk content and privacy violations, but the audit finding here is human override design does not meet the system's action speed and safety needs. The relevant measure is “Enforce automated safety interlocks with effective emergency shutdown”.
C:“Define meaningful review duties, challenge prompts and override authority” is a control for human oversight exists only in name due to automation bias, but the audit finding here is human override design does not meet the system's action speed and safety needs. The relevant measure is “Enforce automated safety interlocks with effective emergency shutdown”.
D:“Require source-link verification and abstention on ungrounded high-impact claims” is a control for output-grounding verification is absent from supervision controls, but the audit finding here is human override design does not meet the system's action speed and safety needs. The relevant measure is “Enforce automated safety interlocks with effective emergency shutdown”.
Question 290
In an audit of Supervision of AI Solutions, managers say this measure is live: Enforce automated safety interlocks with effective emergency shutdown. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Measured interlock activation and simulated unsafe action blocking
- Redacted review samples, detected policy breaches and follow-up actions
- Reviewed decision samples showing reasoned human judgments and reversals
- Sampled summaries with clicked citations, reviewer flags and corrections
Explanation
Correct Answer
A:Correct. The proposed records—measured interlock activation and simulated unsafe action blocking—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is human override design does not meet the system's action speed and safety needs.
Incorrect Answers
B:“Redacted review samples, detected policy breaches and follow-up actions” would test oversight fails to inspect high-risk content and privacy violations, not the present safeguard against human override design does not meet the system's action speed and safety needs. The expected audit evidence is “Measured interlock activation and simulated unsafe action blocking”.
C:“Reviewed decision samples showing reasoned human judgments and reversals” would test human oversight exists only in name due to automation bias, not the present safeguard against human override design does not meet the system's action speed and safety needs. The expected audit evidence is “Measured interlock activation and simulated unsafe action blocking”.
D:“Sampled summaries with clicked citations, reviewer flags and corrections” would test output-grounding verification is absent from supervision controls, not the present safeguard against human override design does not meet the system's action speed and safety needs. The expected audit evidence is “Measured interlock activation and simulated unsafe action blocking”.