ISACA Advanced in AI Audit AAIA Practice Test 27
Question 260
In an audit of Change Management Specific to AI, managers say this measure is live: Evaluate tail latency, throughput, fault behavior and recovery under new workload. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- End-to-end rollback rehearsal with matching artifact manifest
- Signed prompt diffs, reviewer approval and deployment traces
- Load test profiles, SLO approval and failure-injection observations
- Exception log, authorization evidence and completed postchange review
Explanation
Correct Answer
C:Correct. The proposed records—load test profiles, slo approval and failure-injection observations—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is architecture change did not trigger performance and availability reassessment.
Incorrect Answers
A:“End-to-end rollback rehearsal with matching artifact manifest” would test recovery plan is incomplete across the full inference dependency bundle, not the present safeguard against architecture change did not trigger performance and availability reassessment. The expected audit evidence is “Load test profiles, SLO approval and failure-injection observations”.
B:“Signed prompt diffs, reviewer approval and deployment traces” would test prompt configuration changes bypass change authorization and traceability, not the present safeguard against architecture change did not trigger performance and availability reassessment. The expected audit evidence is “Load test profiles, SLO approval and failure-injection observations”.
D:“Exception log, authorization evidence and completed postchange review” would test emergency change procedure lacks audit trails and postimplementation review, not the present safeguard against architecture change did not trigger performance and availability reassessment. The expected audit evidence is “Load test profiles, SLO approval and failure-injection observations”.
Question 261
In a review of Change Management Specific to AI, investigators cannot establish the expected control proof: load test profiles, slo approval and failure-injection observations. The underlying event was that a deployment shifts from batch inference to real-time scoring without recalculating capacity risks. Which control weakness is the MOST likely cause requiring focused audit work?
- Recovery plan is incomplete across the full inference dependency bundle
- Architecture change did not trigger performance and availability reassessment
- Prompt configuration changes bypass change authorization and traceability
- Emergency change procedure lacks audit trails and postimplementation review
Explanation
Correct Answer
B:Correct. The observed condition specifically indicates architecture change did not trigger performance and availability reassessment. A meaningful audit response is to verify this diagnosis and then recommend evaluate tail latency, throughput, fault behavior and recovery under new workload.
Incorrect Answers
A:“Recovery plan is incomplete across the full inference dependency bundle” concerns a separate audit weakness, not the missing assurance described by architecture change did not trigger performance and availability reassessment. The correct cause is “Architecture change did not trigger performance and availability reassessment”.
C:“Prompt configuration changes bypass change authorization and traceability” concerns a separate audit weakness, not the missing assurance described by architecture change did not trigger performance and availability reassessment. The correct cause is “Architecture change did not trigger performance and availability reassessment”.
D:“Emergency change procedure lacks audit trails and postimplementation review” concerns a separate audit weakness, not the missing assurance described by architecture change did not trigger performance and availability reassessment. The correct cause is “Architecture change did not trigger performance and availability reassessment”.
Question 262
Two teams deploy different versions of the same feature encoder to training and serving. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Pin compatible preprocessing artifacts and enforce training-serving parity tests
- Maintain supplier change notification and test significant version updates before production
- Use controlled canaries with guardrail metrics and automatic pause thresholds
- Assess output-policy changes with workflow owners and revalidate impacted processes
Explanation
Correct Answer
A:Correct. Pin compatible preprocessing artifacts and enforce training-serving parity tests is aimed at the actual failure: configuration drift between training and production was not detected. The audit should subsequently obtain registry hash comparison and parity gate in the release pipeline to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Maintain supplier change notification and test significant version updates before production” is a control for vendor-driven model updates are not captured as material changes, but the audit finding here is configuration drift between training and production was not detected. The relevant measure is “Pin compatible preprocessing artifacts and enforce training-serving parity tests”.
C:“Use controlled canaries with guardrail metrics and automatic pause thresholds” is a control for progressive delivery and stop criteria were not included in rollout governance, but the audit finding here is configuration drift between training and production was not detected. The relevant measure is “Pin compatible preprocessing artifacts and enforce training-serving parity tests”.
D:“Assess output-policy changes with workflow owners and revalidate impacted processes” is a control for vendor policy revisions bypass business-impact communication and acceptance, but the audit finding here is configuration drift between training and production was not detected. The relevant measure is “Pin compatible preprocessing artifacts and enforce training-serving parity tests”.
Question 263
In an audit of Change Management Specific to AI, managers say this measure is live: Pin compatible preprocessing artifacts and enforce training-serving parity tests. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Supplier release notices linked to impact assessments and regression approvals
- Canary dashboard, pause trigger history and deployment decision log
- Vendor policy diff, business test results and signed readiness decision
- Registry hash comparison and parity gate in the release pipeline
Explanation
Correct Answer
D:Correct. The proposed records—registry hash comparison and parity gate in the release pipeline—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is configuration drift between training and production was not detected.
Incorrect Answers
A:“Supplier release notices linked to impact assessments and regression approvals” would test vendor-driven model updates are not captured as material changes, not the present safeguard against configuration drift between training and production was not detected. The expected audit evidence is “Registry hash comparison and parity gate in the release pipeline”.
B:“Canary dashboard, pause trigger history and deployment decision log” would test progressive delivery and stop criteria were not included in rollout governance, not the present safeguard against configuration drift between training and production was not detected. The expected audit evidence is “Registry hash comparison and parity gate in the release pipeline”.
C:“Vendor policy diff, business test results and signed readiness decision” would test vendor policy revisions bypass business-impact communication and acceptance, not the present safeguard against configuration drift between training and production was not detected. The expected audit evidence is “Registry hash comparison and parity gate in the release pipeline”.
Question 264
In a review of Change Management Specific to AI, investigators cannot establish the expected control proof: registry hash comparison and parity gate in the release pipeline. The underlying event was that two teams deploy different versions of the same feature encoder to training and serving. Which control weakness is the MOST likely cause requiring focused audit work?
- Vendor-driven model updates are not captured as material changes
- Progressive delivery and stop criteria were not included in rollout governance
- Configuration drift between training and production was not detected
- Vendor policy revisions bypass business-impact communication and acceptance
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates configuration drift between training and production was not detected. A meaningful audit response is to verify this diagnosis and then recommend pin compatible preprocessing artifacts and enforce training-serving parity tests.
Incorrect Answers
A:“Vendor-driven model updates are not captured as material changes” concerns a separate audit weakness, not the missing assurance described by configuration drift between training and production was not detected. The correct cause is “Configuration drift between training and production was not detected”.
B:“Progressive delivery and stop criteria were not included in rollout governance” concerns a separate audit weakness, not the missing assurance described by configuration drift between training and production was not detected. The correct cause is “Configuration drift between training and production was not detected”.
D:“Vendor policy revisions bypass business-impact communication and acceptance” concerns a separate audit weakness, not the missing assurance described by configuration drift between training and production was not detected. The correct cause is “Configuration drift between training and production was not detected”.
Question 265
A rollback restores old model weights but leaves new prompt templates and feature schemas active. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Gate promotion on disaggregated thresholds and risk-owner review
- Version and roll back model, data contract, prompt and dependent services together
- Treat preprocessing/schema modifications as model-affecting changes requiring retest
- Evaluate tail latency, throughput, fault behavior and recovery under new workload
Explanation
Correct Answer
B:Correct. Version and roll back model, data contract, prompt and dependent services together is aimed at the actual failure: recovery plan is incomplete across the full inference dependency bundle. The audit should subsequently obtain end-to-end rollback rehearsal with matching artifact manifest to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Gate promotion on disaggregated thresholds and risk-owner review” is a control for aggregate improvement is masking high-impact regression during model change, but the audit finding here is recovery plan is incomplete across the full inference dependency bundle. The relevant measure is “Version and roll back model, data contract, prompt and dependent services together”.
C:“Treat preprocessing/schema modifications as model-affecting changes requiring retest” is a control for change impact assessment excludes feature engineering dependencies, but the audit finding here is recovery plan is incomplete across the full inference dependency bundle. The relevant measure is “Version and roll back model, data contract, prompt and dependent services together”.
D:“Evaluate tail latency, throughput, fault behavior and recovery under new workload” is a control for architecture change did not trigger performance and availability reassessment, but the audit finding here is recovery plan is incomplete across the full inference dependency bundle. The relevant measure is “Version and roll back model, data contract, prompt and dependent services together”.
Question 266
In an audit of Change Management Specific to AI, managers say this measure is live: Version and roll back model, data contract, prompt and dependent services together. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- End-to-end rollback rehearsal with matching artifact manifest
- Candidate/champion cohort comparison and blocked release record
- Feature contract diff, replayed cases and approved impact analysis
- Load test profiles, SLO approval and failure-injection observations
Explanation
Correct Answer
A:Correct. The proposed records—end-to-end rollback rehearsal with matching artifact manifest—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is recovery plan is incomplete across the full inference dependency bundle.
Incorrect Answers
B:“Candidate/champion cohort comparison and blocked release record” would test aggregate improvement is masking high-impact regression during model change, not the present safeguard against recovery plan is incomplete across the full inference dependency bundle. The expected audit evidence is “End-to-end rollback rehearsal with matching artifact manifest”.
C:“Feature contract diff, replayed cases and approved impact analysis” would test change impact assessment excludes feature engineering dependencies, not the present safeguard against recovery plan is incomplete across the full inference dependency bundle. The expected audit evidence is “End-to-end rollback rehearsal with matching artifact manifest”.
D:“Load test profiles, SLO approval and failure-injection observations” would test architecture change did not trigger performance and availability reassessment, not the present safeguard against recovery plan is incomplete across the full inference dependency bundle. The expected audit evidence is “End-to-end rollback rehearsal with matching artifact manifest”.
Question 267
In a review of Change Management Specific to AI, investigators cannot establish the expected control proof: end-to-end rollback rehearsal with matching artifact manifest. The underlying event was that a rollback restores old model weights but leaves new prompt templates and feature schemas active. Which control weakness is the MOST likely cause requiring focused audit work?
- Aggregate improvement is masking high-impact regression during model change
- Change impact assessment excludes feature engineering dependencies
- Architecture change did not trigger performance and availability reassessment
- Recovery plan is incomplete across the full inference dependency bundle
Explanation
Correct Answer
D:Correct. The observed condition specifically indicates recovery plan is incomplete across the full inference dependency bundle. A meaningful audit response is to verify this diagnosis and then recommend version and roll back model, data contract, prompt and dependent services together.
Incorrect Answers
A:“Aggregate improvement is masking high-impact regression during model change” concerns a separate audit weakness, not the missing assurance described by recovery plan is incomplete across the full inference dependency bundle. The correct cause is “Recovery plan is incomplete across the full inference dependency bundle”.
B:“Change impact assessment excludes feature engineering dependencies” concerns a separate audit weakness, not the missing assurance described by recovery plan is incomplete across the full inference dependency bundle. The correct cause is “Recovery plan is incomplete across the full inference dependency bundle”.
C:“Architecture change did not trigger performance and availability reassessment” concerns a separate audit weakness, not the missing assurance described by recovery plan is incomplete across the full inference dependency bundle. The correct cause is “Recovery plan is incomplete across the full inference dependency bundle”.
Question 268
A hospital model's confidence falls on rural clinics, but monitoring reports only national averages. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Track override, appeal and adverse-outcome trends by use case
- Require source-link verification and abstention on ungrounded high-impact claims
- Set site-level performance alerts and clinical escalation thresholds
- Introduce risk-sampled conversation review with privacy-safe escalation
Explanation
Correct Answer
C:Correct. Set site-level performance alerts and clinical escalation thresholds is aimed at the actual failure: monitoring aggregation hides operationally critical subgroup deterioration. The audit should subsequently obtain site-stratified outcomes, alerts and documented interventions to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Track override, appeal and adverse-outcome trends by use case” is a control for oversight metrics omit evidence of downstream correction and user challenge, but the audit finding here is monitoring aggregation hides operationally critical subgroup deterioration. The relevant measure is “Set site-level performance alerts and clinical escalation thresholds”.
B:“Require source-link verification and abstention on ungrounded high-impact claims” is a control for output-grounding verification is absent from supervision controls, but the audit finding here is monitoring aggregation hides operationally critical subgroup deterioration. The relevant measure is “Set site-level performance alerts and clinical escalation thresholds”.
D:“Introduce risk-sampled conversation review with privacy-safe escalation” is a control for oversight fails to inspect high-risk content and privacy violations, but the audit finding here is monitoring aggregation hides operationally critical subgroup deterioration. The relevant measure is “Set site-level performance alerts and clinical escalation thresholds”.
Question 269
In an audit of Supervision of AI Solutions, managers say this measure is live: Set site-level performance alerts and clinical escalation thresholds. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Audit logs of appeals, reversal reasons and trend-based actions
- Site-stratified outcomes, alerts and documented interventions
- Sampled summaries with clicked citations, reviewer flags and corrections
- Redacted review samples, detected policy breaches and follow-up actions
Explanation
Correct Answer
B:Correct. The proposed records—site-stratified outcomes, alerts and documented interventions—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is monitoring aggregation hides operationally critical subgroup deterioration.
Incorrect Answers
A:“Audit logs of appeals, reversal reasons and trend-based actions” would test oversight metrics omit evidence of downstream correction and user challenge, not the present safeguard against monitoring aggregation hides operationally critical subgroup deterioration. The expected audit evidence is “Site-stratified outcomes, alerts and documented interventions”.
C:“Sampled summaries with clicked citations, reviewer flags and corrections” would test output-grounding verification is absent from supervision controls, not the present safeguard against monitoring aggregation hides operationally critical subgroup deterioration. The expected audit evidence is “Site-stratified outcomes, alerts and documented interventions”.
D:“Redacted review samples, detected policy breaches and follow-up actions” would test oversight fails to inspect high-risk content and privacy violations, not the present safeguard against monitoring aggregation hides operationally critical subgroup deterioration. The expected audit evidence is “Site-stratified outcomes, alerts and documented interventions”.
Question 270
In a review of Supervision of AI Solutions, investigators cannot establish the expected control proof: site-stratified outcomes, alerts and documented interventions. The underlying event was that a hospital model's confidence falls on rural clinics, but monitoring reports only national averages. Which control weakness is the MOST likely cause requiring focused audit work?
- Monitoring aggregation hides operationally critical subgroup deterioration
- Oversight metrics omit evidence of downstream correction and user challenge
- Output-grounding verification is absent from supervision controls
- Oversight fails to inspect high-risk content and privacy violations
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates monitoring aggregation hides operationally critical subgroup deterioration. A meaningful audit response is to verify this diagnosis and then recommend set site-level performance alerts and clinical escalation thresholds.
Incorrect Answers
B:“Oversight metrics omit evidence of downstream correction and user challenge” concerns a separate audit weakness, not the missing assurance described by monitoring aggregation hides operationally critical subgroup deterioration. The correct cause is “Monitoring aggregation hides operationally critical subgroup deterioration”.
C:“Output-grounding verification is absent from supervision controls” concerns a separate audit weakness, not the missing assurance described by monitoring aggregation hides operationally critical subgroup deterioration. The correct cause is “Monitoring aggregation hides operationally critical subgroup deterioration”.
D:“Oversight fails to inspect high-risk content and privacy violations” concerns a separate audit weakness, not the missing assurance described by monitoring aggregation hides operationally critical subgroup deterioration. The correct cause is “Monitoring aggregation hides operationally critical subgroup deterioration”.