ISACA Advanced in AI Audit AAIA Practice Test 30
Question 290
In an audit of Supervision of AI Solutions, managers say this measure is live: Enforce automated safety interlocks with effective emergency shutdown. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Measured interlock activation and simulated unsafe action blocking
- Redacted review samples, detected policy breaches and follow-up actions
- Reviewed decision samples showing reasoned human judgments and reversals
- Sampled summaries with clicked citations, reviewer flags and corrections
Explanation
Correct Answer
A:Correct. The proposed records—measured interlock activation and simulated unsafe action blocking—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is human override design does not meet the system's action speed and safety needs.
Incorrect Answers
B:“Redacted review samples, detected policy breaches and follow-up actions” would test oversight fails to inspect high-risk content and privacy violations, not the present safeguard against human override design does not meet the system's action speed and safety needs. The expected audit evidence is “Measured interlock activation and simulated unsafe action blocking”.
C:“Reviewed decision samples showing reasoned human judgments and reversals” would test human oversight exists only in name due to automation bias, not the present safeguard against human override design does not meet the system's action speed and safety needs. The expected audit evidence is “Measured interlock activation and simulated unsafe action blocking”.
D:“Sampled summaries with clicked citations, reviewer flags and corrections” would test output-grounding verification is absent from supervision controls, not the present safeguard against human override design does not meet the system's action speed and safety needs. The expected audit evidence is “Measured interlock activation and simulated unsafe action blocking”.
Question 291
In a review of Supervision of AI Solutions, investigators cannot establish the expected control proof: measured interlock activation and simulated unsafe action blocking. The underlying event was that an AI robot issues actions faster than operators can stop it during abnormal behavior. Which control weakness is the MOST likely cause requiring focused audit work?
- Oversight fails to inspect high-risk content and privacy violations
- Human oversight exists only in name due to automation bias
- Output-grounding verification is absent from supervision controls
- Human override design does not meet the system's action speed and safety needs
Explanation
Correct Answer
D:Correct. The observed condition specifically indicates human override design does not meet the system's action speed and safety needs. A meaningful audit response is to verify this diagnosis and then recommend enforce automated safety interlocks with effective emergency shutdown.
Incorrect Answers
A:“Oversight fails to inspect high-risk content and privacy violations” concerns a separate audit weakness, not the missing assurance described by human override design does not meet the system's action speed and safety needs. The correct cause is “Human override design does not meet the system's action speed and safety needs”.
B:“Human oversight exists only in name due to automation bias” concerns a separate audit weakness, not the missing assurance described by human override design does not meet the system's action speed and safety needs. The correct cause is “Human override design does not meet the system's action speed and safety needs”.
C:“Output-grounding verification is absent from supervision controls” concerns a separate audit weakness, not the missing assurance described by human override design does not meet the system's action speed and safety needs. The correct cause is “Human override design does not meet the system's action speed and safety needs”.
Question 292
An automated triage model routes rare critical alerts to a general queue with no priority override. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Surface relevant decision factors and enable sampled human challenge
- Track override, appeal and adverse-outcome trends by use case
- Implement severity-aware routing and time-bound specialist review
- Segment drift baselines by season and operational regime
Explanation
Correct Answer
C:Correct. Implement severity-aware routing and time-bound specialist review is aimed at the actual failure: supervision lacks consequence-sensitive escalation tiers. The audit should subsequently obtain alert routing traces, critical-case drill and sla outcomes to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Surface relevant decision factors and enable sampled human challenge” is a control for reviewer visibility is insufficient to challenge automated adverse decisions, but the audit finding here is supervision lacks consequence-sensitive escalation tiers. The relevant measure is “Implement severity-aware routing and time-bound specialist review”.
B:“Track override, appeal and adverse-outcome trends by use case” is a control for oversight metrics omit evidence of downstream correction and user challenge, but the audit finding here is supervision lacks consequence-sensitive escalation tiers. The relevant measure is “Implement severity-aware routing and time-bound specialist review”.
D:“Segment drift baselines by season and operational regime” is a control for seasonal concept drift is obscured by static population benchmarks, but the audit finding here is supervision lacks consequence-sensitive escalation tiers. The relevant measure is “Implement severity-aware routing and time-bound specialist review”.
Question 293
In an audit of Supervision of AI Solutions, managers say this measure is live: Implement severity-aware routing and time-bound specialist review. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Exclusion audit trails, reviewer annotations and remedied decisions
- Alert routing traces, critical-case drill and SLA outcomes
- Audit logs of appeals, reversal reasons and trend-based actions
- Season-specific drift dashboard and retraining investigation records
Explanation
Correct Answer
B:Correct. The proposed records—alert routing traces, critical-case drill and sla outcomes—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is supervision lacks consequence-sensitive escalation tiers.
Incorrect Answers
A:“Exclusion audit trails, reviewer annotations and remedied decisions” would test reviewer visibility is insufficient to challenge automated adverse decisions, not the present safeguard against supervision lacks consequence-sensitive escalation tiers. The expected audit evidence is “Alert routing traces, critical-case drill and SLA outcomes”.
C:“Audit logs of appeals, reversal reasons and trend-based actions” would test oversight metrics omit evidence of downstream correction and user challenge, not the present safeguard against supervision lacks consequence-sensitive escalation tiers. The expected audit evidence is “Alert routing traces, critical-case drill and SLA outcomes”.
D:“Season-specific drift dashboard and retraining investigation records” would test seasonal concept drift is obscured by static population benchmarks, not the present safeguard against supervision lacks consequence-sensitive escalation tiers. The expected audit evidence is “Alert routing traces, critical-case drill and SLA outcomes”.
Question 294
In a review of Supervision of AI Solutions, investigators cannot establish the expected control proof: alert routing traces, critical-case drill and sla outcomes. The underlying event was that an automated triage model routes rare critical alerts to a general queue with no priority override. Which control weakness is the MOST likely cause requiring focused audit work?
- Supervision lacks consequence-sensitive escalation tiers
- Reviewer visibility is insufficient to challenge automated adverse decisions
- Oversight metrics omit evidence of downstream correction and user challenge
- Seasonal concept drift is obscured by static population benchmarks
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates supervision lacks consequence-sensitive escalation tiers. A meaningful audit response is to verify this diagnosis and then recommend implement severity-aware routing and time-bound specialist review.
Incorrect Answers
B:“Reviewer visibility is insufficient to challenge automated adverse decisions” concerns a separate audit weakness, not the missing assurance described by supervision lacks consequence-sensitive escalation tiers. The correct cause is “Supervision lacks consequence-sensitive escalation tiers”.
C:“Oversight metrics omit evidence of downstream correction and user challenge” concerns a separate audit weakness, not the missing assurance described by supervision lacks consequence-sensitive escalation tiers. The correct cause is “Supervision lacks consequence-sensitive escalation tiers”.
D:“Seasonal concept drift is obscured by static population benchmarks” concerns a separate audit weakness, not the missing assurance described by supervision lacks consequence-sensitive escalation tiers. The correct cause is “Supervision lacks consequence-sensitive escalation tiers”.
Question 295
A customer support agent collects sensitive documents while supervisors view only aggregate conversation ratings. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Set site-level performance alerts and clinical escalation thresholds
- Size triage workflows using alert rates, severity, response targets and staffing
- Enforce automated safety interlocks with effective emergency shutdown
- Introduce risk-sampled conversation review with privacy-safe escalation
Explanation
Correct Answer
D:Correct. Introduce risk-sampled conversation review with privacy-safe escalation is aimed at the actual failure: oversight fails to inspect high-risk content and privacy violations. The audit should subsequently obtain redacted review samples, detected policy breaches and follow-up actions to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Set site-level performance alerts and clinical escalation thresholds” is a control for monitoring aggregation hides operationally critical subgroup deterioration, but the audit finding here is oversight fails to inspect high-risk content and privacy violations. The relevant measure is “Introduce risk-sampled conversation review with privacy-safe escalation”.
B:“Size triage workflows using alert rates, severity, response targets and staffing” is a control for operational supervision capacity is not matched to alert generation, but the audit finding here is oversight fails to inspect high-risk content and privacy violations. The relevant measure is “Introduce risk-sampled conversation review with privacy-safe escalation”.
C:“Enforce automated safety interlocks with effective emergency shutdown” is a control for human override design does not meet the system's action speed and safety needs, but the audit finding here is oversight fails to inspect high-risk content and privacy violations. The relevant measure is “Introduce risk-sampled conversation review with privacy-safe escalation”.
Question 296
In an audit of Supervision of AI Solutions, managers say this measure is live: Introduce risk-sampled conversation review with privacy-safe escalation. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Site-stratified outcomes, alerts and documented interventions
- Queue aging, staffing forecasts and sampled incident response times
- Redacted review samples, detected policy breaches and follow-up actions
- Measured interlock activation and simulated unsafe action blocking
Explanation
Correct Answer
C:Correct. The proposed records—redacted review samples, detected policy breaches and follow-up actions—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is oversight fails to inspect high-risk content and privacy violations.
Incorrect Answers
A:“Site-stratified outcomes, alerts and documented interventions” would test monitoring aggregation hides operationally critical subgroup deterioration, not the present safeguard against oversight fails to inspect high-risk content and privacy violations. The expected audit evidence is “Redacted review samples, detected policy breaches and follow-up actions”.
B:“Queue aging, staffing forecasts and sampled incident response times” would test operational supervision capacity is not matched to alert generation, not the present safeguard against oversight fails to inspect high-risk content and privacy violations. The expected audit evidence is “Redacted review samples, detected policy breaches and follow-up actions”.
D:“Measured interlock activation and simulated unsafe action blocking” would test human override design does not meet the system's action speed and safety needs, not the present safeguard against oversight fails to inspect high-risk content and privacy violations. The expected audit evidence is “Redacted review samples, detected policy breaches and follow-up actions”.
Question 297
In a review of Supervision of AI Solutions, investigators cannot establish the expected control proof: redacted review samples, detected policy breaches and follow-up actions. The underlying event was that a customer support agent collects sensitive documents while supervisors view only aggregate conversation ratings. Which control weakness is the MOST likely cause requiring focused audit work?
- Monitoring aggregation hides operationally critical subgroup deterioration
- Oversight fails to inspect high-risk content and privacy violations
- Operational supervision capacity is not matched to alert generation
- Human override design does not meet the system's action speed and safety needs
Explanation
Correct Answer
B:Correct. The observed condition specifically indicates oversight fails to inspect high-risk content and privacy violations. A meaningful audit response is to verify this diagnosis and then recommend introduce risk-sampled conversation review with privacy-safe escalation.
Incorrect Answers
A:“Monitoring aggregation hides operationally critical subgroup deterioration” concerns a separate audit weakness, not the missing assurance described by oversight fails to inspect high-risk content and privacy violations. The correct cause is “Oversight fails to inspect high-risk content and privacy violations”.
C:“Operational supervision capacity is not matched to alert generation” concerns a separate audit weakness, not the missing assurance described by oversight fails to inspect high-risk content and privacy violations. The correct cause is “Oversight fails to inspect high-risk content and privacy violations”.
D:“Human override design does not meet the system's action speed and safety needs” concerns a separate audit weakness, not the missing assurance described by oversight fails to inspect high-risk content and privacy violations. The correct cause is “Oversight fails to inspect high-risk content and privacy violations”.
Question 298
An AI-assisted hiring tool filters candidates without supervisors seeing the criteria that drive exclusions. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Surface relevant decision factors and enable sampled human challenge
- Define meaningful review duties, challenge prompts and override authority
- Enable authorized override with documented rationale and applicant notice
- Implement severity-aware routing and time-bound specialist review
Explanation
Correct Answer
A:Correct. Surface relevant decision factors and enable sampled human challenge is aimed at the actual failure: reviewer visibility is insufficient to challenge automated adverse decisions. The audit should subsequently obtain exclusion audit trails, reviewer annotations and remedied decisions to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Define meaningful review duties, challenge prompts and override authority” is a control for human oversight exists only in name due to automation bias, but the audit finding here is reviewer visibility is insufficient to challenge automated adverse decisions. The relevant measure is “Surface relevant decision factors and enable sampled human challenge”.
C:“Enable authorized override with documented rationale and applicant notice” is a control for human intervention authority is not embedded in the workflow, but the audit finding here is reviewer visibility is insufficient to challenge automated adverse decisions. The relevant measure is “Surface relevant decision factors and enable sampled human challenge”.
D:“Implement severity-aware routing and time-bound specialist review” is a control for supervision lacks consequence-sensitive escalation tiers, but the audit finding here is reviewer visibility is insufficient to challenge automated adverse decisions. The relevant measure is “Surface relevant decision factors and enable sampled human challenge”.
Question 299
In an audit of Supervision of AI Solutions, managers say this measure is live: Surface relevant decision factors and enable sampled human challenge. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Reviewed decision samples showing reasoned human judgments and reversals
- Role-permission tests and sampled reversal workflow traces
- Alert routing traces, critical-case drill and SLA outcomes
- Exclusion audit trails, reviewer annotations and remedied decisions
Explanation
Correct Answer
D:Correct. The proposed records—exclusion audit trails, reviewer annotations and remedied decisions—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is reviewer visibility is insufficient to challenge automated adverse decisions.
Incorrect Answers
A:“Reviewed decision samples showing reasoned human judgments and reversals” would test human oversight exists only in name due to automation bias, not the present safeguard against reviewer visibility is insufficient to challenge automated adverse decisions. The expected audit evidence is “Exclusion audit trails, reviewer annotations and remedied decisions”.
B:“Role-permission tests and sampled reversal workflow traces” would test human intervention authority is not embedded in the workflow, not the present safeguard against reviewer visibility is insufficient to challenge automated adverse decisions. The expected audit evidence is “Exclusion audit trails, reviewer annotations and remedied decisions”.
C:“Alert routing traces, critical-case drill and SLA outcomes” would test supervision lacks consequence-sensitive escalation tiers, not the present safeguard against reviewer visibility is insufficient to challenge automated adverse decisions. The expected audit evidence is “Exclusion audit trails, reviewer annotations and remedied decisions”.
Question 300
In a review of Supervision of AI Solutions, investigators cannot establish the expected control proof: exclusion audit trails, reviewer annotations and remedied decisions. The underlying event was that an AI-assisted hiring tool filters candidates without supervisors seeing the criteria that drive exclusions. Which control weakness is the MOST likely cause requiring focused audit work?
- Human oversight exists only in name due to automation bias
- Human intervention authority is not embedded in the workflow
- Reviewer visibility is insufficient to challenge automated adverse decisions
- Supervision lacks consequence-sensitive escalation tiers
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates reviewer visibility is insufficient to challenge automated adverse decisions. A meaningful audit response is to verify this diagnosis and then recommend surface relevant decision factors and enable sampled human challenge.
Incorrect Answers
A:“Human oversight exists only in name due to automation bias” concerns a separate audit weakness, not the missing assurance described by reviewer visibility is insufficient to challenge automated adverse decisions. The correct cause is “Reviewer visibility is insufficient to challenge automated adverse decisions”.
B:“Human intervention authority is not embedded in the workflow” concerns a separate audit weakness, not the missing assurance described by reviewer visibility is insufficient to challenge automated adverse decisions. The correct cause is “Reviewer visibility is insufficient to challenge automated adverse decisions”.
D:“Supervision lacks consequence-sensitive escalation tiers” concerns a separate audit weakness, not the missing assurance described by reviewer visibility is insufficient to challenge automated adverse decisions. The correct cause is “Reviewer visibility is insufficient to challenge automated adverse decisions”.