ISACA Advanced in AI Audit AAIA Practice Test 23
Question 220
The AI platform uses unreviewed open-source packages in an inference container. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Separate training from deployment and require validation plus appropriate approval
- Define monitoring thresholds, decommission owners and safe replacement routes
- Apply SBOM review, dependency pinning, vulnerability gates and artifact signing
- Require staged deployment, operational go/no-go criteria and tested rollback
Explanation
Correct Answer
C:Correct. Apply SBOM review, dependency pinning, vulnerability gates and artifact signing is aimed at the actual failure: software and model supply-chain integrity are excluded from the development pipeline. The audit should subsequently obtain ci gate logs, signed image digest and dependency exception register to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Separate training from deployment and require validation plus appropriate approval” is a control for autonomous retraining lacks risk-tiered reauthorization of changed behavior, but the audit finding here is software and model supply-chain integrity are excluded from the development pipeline. The relevant measure is “Apply SBOM review, dependency pinning, vulnerability gates and artifact signing”.
B:“Define monitoring thresholds, decommission owners and safe replacement routes” is a control for lifecycle documentation lacks explicit retirement and sunset criteria, but the audit finding here is software and model supply-chain integrity are excluded from the development pipeline. The relevant measure is “Apply SBOM review, dependency pinning, vulnerability gates and artifact signing”.
D:“Require staged deployment, operational go/no-go criteria and tested rollback” is a control for operational acceptance gates and fail-safe recovery are omitted from release design, but the audit finding here is software and model supply-chain integrity are excluded from the development pipeline. The relevant measure is “Apply SBOM review, dependency pinning, vulnerability gates and artifact signing”.
Question 221
In an audit of AI Solution Development Methodologies and Lifecycle, managers say this measure is live: Apply SBOM review, dependency pinning, vulnerability gates and artifact signing. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Retrain job trace, candidate-vs-champion tests and release approval
- CI gate logs, signed image digest and dependency exception register
- Approved retirement plan and a tested rollback/decommission exercise
- Canary decision log, failed deployment drill and rollback telemetry
Explanation
Correct Answer
B:Correct. The proposed records—ci gate logs, signed image digest and dependency exception register—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is software and model supply-chain integrity are excluded from the development pipeline.
Incorrect Answers
A:“Retrain job trace, candidate-vs-champion tests and release approval” would test autonomous retraining lacks risk-tiered reauthorization of changed behavior, not the present safeguard against software and model supply-chain integrity are excluded from the development pipeline. The expected audit evidence is “CI gate logs, signed image digest and dependency exception register”.
C:“Approved retirement plan and a tested rollback/decommission exercise” would test lifecycle documentation lacks explicit retirement and sunset criteria, not the present safeguard against software and model supply-chain integrity are excluded from the development pipeline. The expected audit evidence is “CI gate logs, signed image digest and dependency exception register”.
D:“Canary decision log, failed deployment drill and rollback telemetry” would test operational acceptance gates and fail-safe recovery are omitted from release design, not the present safeguard against software and model supply-chain integrity are excluded from the development pipeline. The expected audit evidence is “CI gate logs, signed image digest and dependency exception register”.
Question 222
In a review of AI Solution Development Methodologies and Lifecycle, investigators cannot establish the expected control proof: ci gate logs, signed image digest and dependency exception register. The underlying event was that the AI platform uses unreviewed open-source packages in an inference container. Which control weakness is the MOST likely cause requiring focused audit work?
- Software and model supply-chain integrity are excluded from the development pipeline
- Autonomous retraining lacks risk-tiered reauthorization of changed behavior
- Lifecycle documentation lacks explicit retirement and sunset criteria
- Operational acceptance gates and fail-safe recovery are omitted from release design
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates software and model supply-chain integrity are excluded from the development pipeline. A meaningful audit response is to verify this diagnosis and then recommend apply sbom review, dependency pinning, vulnerability gates and artifact signing.
Incorrect Answers
B:“Autonomous retraining lacks risk-tiered reauthorization of changed behavior” concerns a separate audit weakness, not the missing assurance described by software and model supply-chain integrity are excluded from the development pipeline. The correct cause is “Software and model supply-chain integrity are excluded from the development pipeline”.
C:“Lifecycle documentation lacks explicit retirement and sunset criteria” concerns a separate audit weakness, not the missing assurance described by software and model supply-chain integrity are excluded from the development pipeline. The correct cause is “Software and model supply-chain integrity are excluded from the development pipeline”.
D:“Operational acceptance gates and fail-safe recovery are omitted from release design” concerns a separate audit weakness, not the missing assurance described by software and model supply-chain integrity are excluded from the development pipeline. The correct cause is “Software and model supply-chain integrity are excluded from the development pipeline”.
Question 223
A team tests a chatbot only with scripted happy-path conversations before a public launch. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Rotate service credentials and transfer documented operations ownership at contract exit
- Obtain risk-proportionate independent validation before production authorization
- Rebaseline intended-use requirements and update safety, privacy and validation tests
- Add human-centered scenario tests and misuse cases to release acceptance
Explanation
Correct Answer
D:Correct. Add human-centered scenario tests and misuse cases to release acceptance is aimed at the actual failure: lifecycle testing omitted realistic failures and adversarial interaction. The audit should subsequently obtain negative-path test library, severity triage and corrected regression evidence to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Rotate service credentials and transfer documented operations ownership at contract exit” is a control for knowledge transfer and ownership transition are absent from lifecycle controls, but the audit finding here is lifecycle testing omitted realistic failures and adversarial interaction. The relevant measure is “Add human-centered scenario tests and misuse cases to release acceptance”.
B:“Obtain risk-proportionate independent validation before production authorization” is a control for segregation of model development and challenge is missing, but the audit finding here is lifecycle testing omitted realistic failures and adversarial interaction. The relevant measure is “Add human-centered scenario tests and misuse cases to release acceptance”.
C:“Rebaseline intended-use requirements and update safety, privacy and validation tests” is a control for requirements traceability failed across scope change, but the audit finding here is lifecycle testing omitted realistic failures and adversarial interaction. The relevant measure is “Add human-centered scenario tests and misuse cases to release acceptance”.
Question 224
In an audit of AI Solution Development Methodologies and Lifecycle, managers say this measure is live: Add human-centered scenario tests and misuse cases to release acceptance. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Offboarding checklist, credential revocation evidence and runbook acceptance
- Separate validator signoff, challenge findings and closure evidence
- Negative-path test library, severity triage and corrected regression evidence
- Signed requirements diff and refreshed trace from risks to test cases
Explanation
Correct Answer
C:Correct. The proposed records—negative-path test library, severity triage and corrected regression evidence—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is lifecycle testing omitted realistic failures and adversarial interaction.
Incorrect Answers
A:“Offboarding checklist, credential revocation evidence and runbook acceptance” would test knowledge transfer and ownership transition are absent from lifecycle controls, not the present safeguard against lifecycle testing omitted realistic failures and adversarial interaction. The expected audit evidence is “Negative-path test library, severity triage and corrected regression evidence”.
B:“Separate validator signoff, challenge findings and closure evidence” would test segregation of model development and challenge is missing, not the present safeguard against lifecycle testing omitted realistic failures and adversarial interaction. The expected audit evidence is “Negative-path test library, severity triage and corrected regression evidence”.
D:“Signed requirements diff and refreshed trace from risks to test cases” would test requirements traceability failed across scope change, not the present safeguard against lifecycle testing omitted realistic failures and adversarial interaction. The expected audit evidence is “Negative-path test library, severity triage and corrected regression evidence”.
Question 225
In a review of AI Solution Development Methodologies and Lifecycle, investigators cannot establish the expected control proof: negative-path test library, severity triage and corrected regression evidence. The underlying event was that a team tests a chatbot only with scripted happy-path conversations before a public launch. Which control weakness is the MOST likely cause requiring focused audit work?
- Knowledge transfer and ownership transition are absent from lifecycle controls
- Lifecycle testing omitted realistic failures and adversarial interaction
- Segregation of model development and challenge is missing
- Requirements traceability failed across scope change
Explanation
Correct Answer
B:Correct. The observed condition specifically indicates lifecycle testing omitted realistic failures and adversarial interaction. A meaningful audit response is to verify this diagnosis and then recommend add human-centered scenario tests and misuse cases to release acceptance.
Incorrect Answers
A:“Knowledge transfer and ownership transition are absent from lifecycle controls” concerns a separate audit weakness, not the missing assurance described by lifecycle testing omitted realistic failures and adversarial interaction. The correct cause is “Lifecycle testing omitted realistic failures and adversarial interaction”.
C:“Segregation of model development and challenge is missing” concerns a separate audit weakness, not the missing assurance described by lifecycle testing omitted realistic failures and adversarial interaction. The correct cause is “Lifecycle testing omitted realistic failures and adversarial interaction”.
D:“Requirements traceability failed across scope change” concerns a separate audit weakness, not the missing assurance described by lifecycle testing omitted realistic failures and adversarial interaction. The correct cause is “Lifecycle testing omitted realistic failures and adversarial interaction”.
Question 226
Retraining is triggered automatically by a metric threshold but the release pipeline has no business approval gate. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Separate training from deployment and require validation plus appropriate approval
- Model full prompt-to-tool flows, authorization gates and human-stop mechanisms
- Record immutable base-model identifiers, hyperparameters, code and data lineage
- Apply SBOM review, dependency pinning, vulnerability gates and artifact signing
Explanation
Correct Answer
A:Correct. Separate training from deployment and require validation plus appropriate approval is aimed at the actual failure: autonomous retraining lacks risk-tiered reauthorization of changed behavior. The audit should subsequently obtain retrain job trace, candidate-vs-champion tests and release approval to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Model full prompt-to-tool flows, authorization gates and human-stop mechanisms” is a control for agentic system interfaces and permissions are outside the documented solution boundary, but the audit finding here is autonomous retraining lacks risk-tiered reauthorization of changed behavior. The relevant measure is “Separate training from deployment and require validation plus appropriate approval”.
C:“Record immutable base-model identifiers, hyperparameters, code and data lineage” is a control for model configuration and dependencies are not reproducible, but the audit finding here is autonomous retraining lacks risk-tiered reauthorization of changed behavior. The relevant measure is “Separate training from deployment and require validation plus appropriate approval”.
D:“Apply SBOM review, dependency pinning, vulnerability gates and artifact signing” is a control for software and model supply-chain integrity are excluded from the development pipeline, but the audit finding here is autonomous retraining lacks risk-tiered reauthorization of changed behavior. The relevant measure is “Separate training from deployment and require validation plus appropriate approval”.
Question 227
In an audit of AI Solution Development Methodologies and Lifecycle, managers say this measure is live: Separate training from deployment and require validation plus appropriate approval. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Reviewed sequence diagrams, negative tool-call tests and approved authority map
- Rebuilt model hash comparison and versioned training configuration
- CI gate logs, signed image digest and dependency exception register
- Retrain job trace, candidate-vs-champion tests and release approval
Explanation
Correct Answer
D:Correct. The proposed records—retrain job trace, candidate-vs-champion tests and release approval—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is autonomous retraining lacks risk-tiered reauthorization of changed behavior.
Incorrect Answers
A:“Reviewed sequence diagrams, negative tool-call tests and approved authority map” would test agentic system interfaces and permissions are outside the documented solution boundary, not the present safeguard against autonomous retraining lacks risk-tiered reauthorization of changed behavior. The expected audit evidence is “Retrain job trace, candidate-vs-champion tests and release approval”.
B:“Rebuilt model hash comparison and versioned training configuration” would test model configuration and dependencies are not reproducible, not the present safeguard against autonomous retraining lacks risk-tiered reauthorization of changed behavior. The expected audit evidence is “Retrain job trace, candidate-vs-champion tests and release approval”.
C:“CI gate logs, signed image digest and dependency exception register” would test software and model supply-chain integrity are excluded from the development pipeline, not the present safeguard against autonomous retraining lacks risk-tiered reauthorization of changed behavior. The expected audit evidence is “Retrain job trace, candidate-vs-champion tests and release approval”.
Question 228
In a review of AI Solution Development Methodologies and Lifecycle, investigators cannot establish the expected control proof: retrain job trace, candidate-vs-champion tests and release approval. The underlying event was that retraining is triggered automatically by a metric threshold but the release pipeline has no business approval gate. Which control weakness is the MOST likely cause requiring focused audit work?
- Agentic system interfaces and permissions are outside the documented solution boundary
- Model configuration and dependencies are not reproducible
- Autonomous retraining lacks risk-tiered reauthorization of changed behavior
- Software and model supply-chain integrity are excluded from the development pipeline
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates autonomous retraining lacks risk-tiered reauthorization of changed behavior. A meaningful audit response is to verify this diagnosis and then recommend separate training from deployment and require validation plus appropriate approval.
Incorrect Answers
A:“Agentic system interfaces and permissions are outside the documented solution boundary” concerns a separate audit weakness, not the missing assurance described by autonomous retraining lacks risk-tiered reauthorization of changed behavior. The correct cause is “Autonomous retraining lacks risk-tiered reauthorization of changed behavior”.
B:“Model configuration and dependencies are not reproducible” concerns a separate audit weakness, not the missing assurance described by autonomous retraining lacks risk-tiered reauthorization of changed behavior. The correct cause is “Autonomous retraining lacks risk-tiered reauthorization of changed behavior”.
D:“Software and model supply-chain integrity are excluded from the development pipeline” concerns a separate audit weakness, not the missing assurance described by autonomous retraining lacks risk-tiered reauthorization of changed behavior. The correct cause is “Autonomous retraining lacks risk-tiered reauthorization of changed behavior”.
Question 229
A data science vendor leaves and all model-serving credentials remain embedded in shared scripts. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Require reproducible build, independent review and signed model registry promotion
- Rotate service credentials and transfer documented operations ownership at contract exit
- Pin service versions where possible and gate supplier changes through compatibility tests
- Add human-centered scenario tests and misuse cases to release acceptance
Explanation
Correct Answer
B:Correct. Rotate service credentials and transfer documented operations ownership at contract exit is aimed at the actual failure: knowledge transfer and ownership transition are absent from lifecycle controls. The audit should subsequently obtain offboarding checklist, credential revocation evidence and runbook acceptance to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Require reproducible build, independent review and signed model registry promotion” is a control for build and release provenance is bypassed by ad hoc notebook deployment, but the audit finding here is knowledge transfer and ownership transition are absent from lifecycle controls. The relevant measure is “Rotate service credentials and transfer documented operations ownership at contract exit”.
C:“Pin service versions where possible and gate supplier changes through compatibility tests” is a control for upstream dependency versions are not pinned or monitored, but the audit finding here is knowledge transfer and ownership transition are absent from lifecycle controls. The relevant measure is “Rotate service credentials and transfer documented operations ownership at contract exit”.
D:“Add human-centered scenario tests and misuse cases to release acceptance” is a control for lifecycle testing omitted realistic failures and adversarial interaction, but the audit finding here is knowledge transfer and ownership transition are absent from lifecycle controls. The relevant measure is “Rotate service credentials and transfer documented operations ownership at contract exit”.
Question 230
In an audit of AI Solution Development Methodologies and Lifecycle, managers say this measure is live: Rotate service credentials and transfer documented operations ownership at contract exit. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Offboarding checklist, credential revocation evidence and runbook acceptance
- Promotion approvals, immutable build hashes and traced artifact lineage
- Dependency lock, vendor change alert and executed regression suite
- Negative-path test library, severity triage and corrected regression evidence
Explanation
Correct Answer
A:Correct. The proposed records—offboarding checklist, credential revocation evidence and runbook acceptance—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is knowledge transfer and ownership transition are absent from lifecycle controls.
Incorrect Answers
B:“Promotion approvals, immutable build hashes and traced artifact lineage” would test build and release provenance is bypassed by ad hoc notebook deployment, not the present safeguard against knowledge transfer and ownership transition are absent from lifecycle controls. The expected audit evidence is “Offboarding checklist, credential revocation evidence and runbook acceptance”.
C:“Dependency lock, vendor change alert and executed regression suite” would test upstream dependency versions are not pinned or monitored, not the present safeguard against knowledge transfer and ownership transition are absent from lifecycle controls. The expected audit evidence is “Offboarding checklist, credential revocation evidence and runbook acceptance”.
D:“Negative-path test library, severity triage and corrected regression evidence” would test lifecycle testing omitted realistic failures and adversarial interaction, not the present safeguard against knowledge transfer and ownership transition are absent from lifecycle controls. The expected audit evidence is “Offboarding checklist, credential revocation evidence and runbook acceptance”.