ISACA Advanced in AI Audit AAIA Practice Test 21
Question 200
In an audit of AI Solution Development Methodologies and Lifecycle, managers say this measure is live: Require reproducible build, independent review and signed model registry promotion. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Separate validator signoff, challenge findings and closure evidence
- Signed requirements diff and refreshed trace from risks to test cases
- Promotion approvals, immutable build hashes and traced artifact lineage
- Offboarding checklist, credential revocation evidence and runbook acceptance
Explanation
Correct Answer
C:Correct. The proposed records—promotion approvals, immutable build hashes and traced artifact lineage—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is build and release provenance is bypassed by ad hoc notebook deployment.
Incorrect Answers
A:“Separate validator signoff, challenge findings and closure evidence” would test segregation of model development and challenge is missing, not the present safeguard against build and release provenance is bypassed by ad hoc notebook deployment. The expected audit evidence is “Promotion approvals, immutable build hashes and traced artifact lineage”.
B:“Signed requirements diff and refreshed trace from risks to test cases” would test requirements traceability failed across scope change, not the present safeguard against build and release provenance is bypassed by ad hoc notebook deployment. The expected audit evidence is “Promotion approvals, immutable build hashes and traced artifact lineage”.
D:“Offboarding checklist, credential revocation evidence and runbook acceptance” would test knowledge transfer and ownership transition are absent from lifecycle controls, not the present safeguard against build and release provenance is bypassed by ad hoc notebook deployment. The expected audit evidence is “Promotion approvals, immutable build hashes and traced artifact lineage”.
Question 201
In a review of AI Solution Development Methodologies and Lifecycle, investigators cannot establish the expected control proof: promotion approvals, immutable build hashes and traced artifact lineage. The underlying event was that a team can deploy a model directly from a notebook without peer review or signed lineage. Which control weakness is the MOST likely cause requiring focused audit work?
- Segregation of model development and challenge is missing
- Build and release provenance is bypassed by ad hoc notebook deployment
- Requirements traceability failed across scope change
- Knowledge transfer and ownership transition are absent from lifecycle controls
Explanation
Correct Answer
B:Correct. The observed condition specifically indicates build and release provenance is bypassed by ad hoc notebook deployment. A meaningful audit response is to verify this diagnosis and then recommend require reproducible build, independent review and signed model registry promotion.
Incorrect Answers
A:“Segregation of model development and challenge is missing” concerns a separate audit weakness, not the missing assurance described by build and release provenance is bypassed by ad hoc notebook deployment. The correct cause is “Build and release provenance is bypassed by ad hoc notebook deployment”.
C:“Requirements traceability failed across scope change” concerns a separate audit weakness, not the missing assurance described by build and release provenance is bypassed by ad hoc notebook deployment. The correct cause is “Build and release provenance is bypassed by ad hoc notebook deployment”.
D:“Knowledge transfer and ownership transition are absent from lifecycle controls” concerns a separate audit weakness, not the missing assurance described by build and release provenance is bypassed by ad hoc notebook deployment. The correct cause is “Build and release provenance is bypassed by ad hoc notebook deployment”.
Question 202
An AI product has a model card but no stated conditions under which it must be withdrawn. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Define monitoring thresholds, decommission owners and safe replacement routes
- Record immutable base-model identifiers, hyperparameters, code and data lineage
- Apply SBOM review, dependency pinning, vulnerability gates and artifact signing
- Model full prompt-to-tool flows, authorization gates and human-stop mechanisms
Explanation
Correct Answer
A:Correct. Define monitoring thresholds, decommission owners and safe replacement routes is aimed at the actual failure: lifecycle documentation lacks explicit retirement and sunset criteria. The audit should subsequently obtain approved retirement plan and a tested rollback/decommission exercise to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Record immutable base-model identifiers, hyperparameters, code and data lineage” is a control for model configuration and dependencies are not reproducible, but the audit finding here is lifecycle documentation lacks explicit retirement and sunset criteria. The relevant measure is “Define monitoring thresholds, decommission owners and safe replacement routes”.
C:“Apply SBOM review, dependency pinning, vulnerability gates and artifact signing” is a control for software and model supply-chain integrity are excluded from the development pipeline, but the audit finding here is lifecycle documentation lacks explicit retirement and sunset criteria. The relevant measure is “Define monitoring thresholds, decommission owners and safe replacement routes”.
D:“Model full prompt-to-tool flows, authorization gates and human-stop mechanisms” is a control for agentic system interfaces and permissions are outside the documented solution boundary, but the audit finding here is lifecycle documentation lacks explicit retirement and sunset criteria. The relevant measure is “Define monitoring thresholds, decommission owners and safe replacement routes”.
Question 203
In an audit of AI Solution Development Methodologies and Lifecycle, managers say this measure is live: Define monitoring thresholds, decommission owners and safe replacement routes. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Rebuilt model hash comparison and versioned training configuration
- CI gate logs, signed image digest and dependency exception register
- Reviewed sequence diagrams, negative tool-call tests and approved authority map
- Approved retirement plan and a tested rollback/decommission exercise
Explanation
Correct Answer
D:Correct. The proposed records—approved retirement plan and a tested rollback/decommission exercise—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is lifecycle documentation lacks explicit retirement and sunset criteria.
Incorrect Answers
A:“Rebuilt model hash comparison and versioned training configuration” would test model configuration and dependencies are not reproducible, not the present safeguard against lifecycle documentation lacks explicit retirement and sunset criteria. The expected audit evidence is “Approved retirement plan and a tested rollback/decommission exercise”.
B:“CI gate logs, signed image digest and dependency exception register” would test software and model supply-chain integrity are excluded from the development pipeline, not the present safeguard against lifecycle documentation lacks explicit retirement and sunset criteria. The expected audit evidence is “Approved retirement plan and a tested rollback/decommission exercise”.
C:“Reviewed sequence diagrams, negative tool-call tests and approved authority map” would test agentic system interfaces and permissions are outside the documented solution boundary, not the present safeguard against lifecycle documentation lacks explicit retirement and sunset criteria. The expected audit evidence is “Approved retirement plan and a tested rollback/decommission exercise”.
Question 204
In a review of AI Solution Development Methodologies and Lifecycle, investigators cannot establish the expected control proof: approved retirement plan and a tested rollback/decommission exercise. The underlying event was that an AI product has a model card but no stated conditions under which it must be withdrawn. Which control weakness is the MOST likely cause requiring focused audit work?
- Model configuration and dependencies are not reproducible
- Software and model supply-chain integrity are excluded from the development pipeline
- Lifecycle documentation lacks explicit retirement and sunset criteria
- Agentic system interfaces and permissions are outside the documented solution boundary
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates lifecycle documentation lacks explicit retirement and sunset criteria. A meaningful audit response is to verify this diagnosis and then recommend define monitoring thresholds, decommission owners and safe replacement routes.
Incorrect Answers
A:“Model configuration and dependencies are not reproducible” concerns a separate audit weakness, not the missing assurance described by lifecycle documentation lacks explicit retirement and sunset criteria. The correct cause is “Lifecycle documentation lacks explicit retirement and sunset criteria”.
B:“Software and model supply-chain integrity are excluded from the development pipeline” concerns a separate audit weakness, not the missing assurance described by lifecycle documentation lacks explicit retirement and sunset criteria. The correct cause is “Lifecycle documentation lacks explicit retirement and sunset criteria”.
D:“Agentic system interfaces and permissions are outside the documented solution boundary” concerns a separate audit weakness, not the missing assurance described by lifecycle documentation lacks explicit retirement and sunset criteria. The correct cause is “Lifecycle documentation lacks explicit retirement and sunset criteria”.
Question 205
A medical-risk model passes development tests but is never validated independently from its authors. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Pin service versions where possible and gate supplier changes through compatibility tests
- Obtain risk-proportionate independent validation before production authorization
- Add human-centered scenario tests and misuse cases to release acceptance
- Require reproducible build, independent review and signed model registry promotion
Explanation
Correct Answer
B:Correct. Obtain risk-proportionate independent validation before production authorization is aimed at the actual failure: segregation of model development and challenge is missing. The audit should subsequently obtain separate validator signoff, challenge findings and closure evidence to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Pin service versions where possible and gate supplier changes through compatibility tests” is a control for upstream dependency versions are not pinned or monitored, but the audit finding here is segregation of model development and challenge is missing. The relevant measure is “Obtain risk-proportionate independent validation before production authorization”.
C:“Add human-centered scenario tests and misuse cases to release acceptance” is a control for lifecycle testing omitted realistic failures and adversarial interaction, but the audit finding here is segregation of model development and challenge is missing. The relevant measure is “Obtain risk-proportionate independent validation before production authorization”.
D:“Require reproducible build, independent review and signed model registry promotion” is a control for build and release provenance is bypassed by ad hoc notebook deployment, but the audit finding here is segregation of model development and challenge is missing. The relevant measure is “Obtain risk-proportionate independent validation before production authorization”.
Question 206
In an audit of AI Solution Development Methodologies and Lifecycle, managers say this measure is live: Obtain risk-proportionate independent validation before production authorization. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Separate validator signoff, challenge findings and closure evidence
- Dependency lock, vendor change alert and executed regression suite
- Negative-path test library, severity triage and corrected regression evidence
- Promotion approvals, immutable build hashes and traced artifact lineage
Explanation
Correct Answer
A:Correct. The proposed records—separate validator signoff, challenge findings and closure evidence—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is segregation of model development and challenge is missing.
Incorrect Answers
B:“Dependency lock, vendor change alert and executed regression suite” would test upstream dependency versions are not pinned or monitored, not the present safeguard against segregation of model development and challenge is missing. The expected audit evidence is “Separate validator signoff, challenge findings and closure evidence”.
C:“Negative-path test library, severity triage and corrected regression evidence” would test lifecycle testing omitted realistic failures and adversarial interaction, not the present safeguard against segregation of model development and challenge is missing. The expected audit evidence is “Separate validator signoff, challenge findings and closure evidence”.
D:“Promotion approvals, immutable build hashes and traced artifact lineage” would test build and release provenance is bypassed by ad hoc notebook deployment, not the present safeguard against segregation of model development and challenge is missing. The expected audit evidence is “Separate validator signoff, challenge findings and closure evidence”.
Question 207
In a review of AI Solution Development Methodologies and Lifecycle, investigators cannot establish the expected control proof: separate validator signoff, challenge findings and closure evidence. The underlying event was that a medical-risk model passes development tests but is never validated independently from its authors. Which control weakness is the MOST likely cause requiring focused audit work?
- Upstream dependency versions are not pinned or monitored
- Lifecycle testing omitted realistic failures and adversarial interaction
- Build and release provenance is bypassed by ad hoc notebook deployment
- Segregation of model development and challenge is missing
Explanation
Correct Answer
D:Correct. The observed condition specifically indicates segregation of model development and challenge is missing. A meaningful audit response is to verify this diagnosis and then recommend obtain risk-proportionate independent validation before production authorization.
Incorrect Answers
A:“Upstream dependency versions are not pinned or monitored” concerns a separate audit weakness, not the missing assurance described by segregation of model development and challenge is missing. The correct cause is “Segregation of model development and challenge is missing”.
B:“Lifecycle testing omitted realistic failures and adversarial interaction” concerns a separate audit weakness, not the missing assurance described by segregation of model development and challenge is missing. The correct cause is “Segregation of model development and challenge is missing”.
C:“Build and release provenance is bypassed by ad hoc notebook deployment” concerns a separate audit weakness, not the missing assurance described by segregation of model development and challenge is missing. The correct cause is “Segregation of model development and challenge is missing”.
Question 208
An engineering team fine-tunes a base model without capturing the original model version or training hyperparameters. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Require staged deployment, operational go/no-go criteria and tested rollback
- Separate training from deployment and require validation plus appropriate approval
- Record immutable base-model identifiers, hyperparameters, code and data lineage
- Define monitoring thresholds, decommission owners and safe replacement routes
Explanation
Correct Answer
C:Correct. Record immutable base-model identifiers, hyperparameters, code and data lineage is aimed at the actual failure: model configuration and dependencies are not reproducible. The audit should subsequently obtain rebuilt model hash comparison and versioned training configuration to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Require staged deployment, operational go/no-go criteria and tested rollback” is a control for operational acceptance gates and fail-safe recovery are omitted from release design, but the audit finding here is model configuration and dependencies are not reproducible. The relevant measure is “Record immutable base-model identifiers, hyperparameters, code and data lineage”.
B:“Separate training from deployment and require validation plus appropriate approval” is a control for autonomous retraining lacks risk-tiered reauthorization of changed behavior, but the audit finding here is model configuration and dependencies are not reproducible. The relevant measure is “Record immutable base-model identifiers, hyperparameters, code and data lineage”.
D:“Define monitoring thresholds, decommission owners and safe replacement routes” is a control for lifecycle documentation lacks explicit retirement and sunset criteria, but the audit finding here is model configuration and dependencies are not reproducible. The relevant measure is “Record immutable base-model identifiers, hyperparameters, code and data lineage”.
Question 209
In an audit of AI Solution Development Methodologies and Lifecycle, managers say this measure is live: Record immutable base-model identifiers, hyperparameters, code and data lineage. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Canary decision log, failed deployment drill and rollback telemetry
- Rebuilt model hash comparison and versioned training configuration
- Retrain job trace, candidate-vs-champion tests and release approval
- Approved retirement plan and a tested rollback/decommission exercise
Explanation
Correct Answer
B:Correct. The proposed records—rebuilt model hash comparison and versioned training configuration—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is model configuration and dependencies are not reproducible.
Incorrect Answers
A:“Canary decision log, failed deployment drill and rollback telemetry” would test operational acceptance gates and fail-safe recovery are omitted from release design, not the present safeguard against model configuration and dependencies are not reproducible. The expected audit evidence is “Rebuilt model hash comparison and versioned training configuration”.
C:“Retrain job trace, candidate-vs-champion tests and release approval” would test autonomous retraining lacks risk-tiered reauthorization of changed behavior, not the present safeguard against model configuration and dependencies are not reproducible. The expected audit evidence is “Rebuilt model hash comparison and versioned training configuration”.
D:“Approved retirement plan and a tested rollback/decommission exercise” would test lifecycle documentation lacks explicit retirement and sunset criteria, not the present safeguard against model configuration and dependencies are not reproducible. The expected audit evidence is “Rebuilt model hash comparison and versioned training configuration”.
Question 210
In a review of AI Solution Development Methodologies and Lifecycle, investigators cannot establish the expected control proof: rebuilt model hash comparison and versioned training configuration. The underlying event was that an engineering team fine-tunes a base model without capturing the original model version or training hyperparameters. Which control weakness is the MOST likely cause requiring focused audit work?
- Model configuration and dependencies are not reproducible
- Operational acceptance gates and fail-safe recovery are omitted from release design
- Autonomous retraining lacks risk-tiered reauthorization of changed behavior
- Lifecycle documentation lacks explicit retirement and sunset criteria
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates model configuration and dependencies are not reproducible. A meaningful audit response is to verify this diagnosis and then recommend record immutable base-model identifiers, hyperparameters, code and data lineage.
Incorrect Answers
B:“Operational acceptance gates and fail-safe recovery are omitted from release design” concerns a separate audit weakness, not the missing assurance described by model configuration and dependencies are not reproducible. The correct cause is “Model configuration and dependencies are not reproducible”.
C:“Autonomous retraining lacks risk-tiered reauthorization of changed behavior” concerns a separate audit weakness, not the missing assurance described by model configuration and dependencies are not reproducible. The correct cause is “Model configuration and dependencies are not reproducible”.
D:“Lifecycle documentation lacks explicit retirement and sunset criteria” concerns a separate audit weakness, not the missing assurance described by model configuration and dependencies are not reproducible. The correct cause is “Model configuration and dependencies are not reproducible”.