ISACA Advanced in AI Audit AAIA Practice Test 22
Question 210
In a review of AI Solution Development Methodologies and Lifecycle, investigators cannot establish the expected control proof: rebuilt model hash comparison and versioned training configuration. The underlying event was that an engineering team fine-tunes a base model without capturing the original model version or training hyperparameters. Which control weakness is the MOST likely cause requiring focused audit work?
- Model configuration and dependencies are not reproducible
- Operational acceptance gates and fail-safe recovery are omitted from release design
- Autonomous retraining lacks risk-tiered reauthorization of changed behavior
- Lifecycle documentation lacks explicit retirement and sunset criteria
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates model configuration and dependencies are not reproducible. A meaningful audit response is to verify this diagnosis and then recommend record immutable base-model identifiers, hyperparameters, code and data lineage.
Incorrect Answers
B:“Operational acceptance gates and fail-safe recovery are omitted from release design” concerns a separate audit weakness, not the missing assurance described by model configuration and dependencies are not reproducible. The correct cause is “Model configuration and dependencies are not reproducible”.
C:“Autonomous retraining lacks risk-tiered reauthorization of changed behavior” concerns a separate audit weakness, not the missing assurance described by model configuration and dependencies are not reproducible. The correct cause is “Model configuration and dependencies are not reproducible”.
D:“Lifecycle documentation lacks explicit retirement and sunset criteria” concerns a separate audit weakness, not the missing assurance described by model configuration and dependencies are not reproducible. The correct cause is “Model configuration and dependencies are not reproducible”.
Question 211
A recommendation engine depends on a vendor embedding endpoint whose version can change silently. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Rebaseline intended-use requirements and update safety, privacy and validation tests
- Rotate service credentials and transfer documented operations ownership at contract exit
- Obtain risk-proportionate independent validation before production authorization
- Pin service versions where possible and gate supplier changes through compatibility tests
Explanation
Correct Answer
D:Correct. Pin service versions where possible and gate supplier changes through compatibility tests is aimed at the actual failure: upstream dependency versions are not pinned or monitored. The audit should subsequently obtain dependency lock, vendor change alert and executed regression suite to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Rebaseline intended-use requirements and update safety, privacy and validation tests” is a control for requirements traceability failed across scope change, but the audit finding here is upstream dependency versions are not pinned or monitored. The relevant measure is “Pin service versions where possible and gate supplier changes through compatibility tests”.
B:“Rotate service credentials and transfer documented operations ownership at contract exit” is a control for knowledge transfer and ownership transition are absent from lifecycle controls, but the audit finding here is upstream dependency versions are not pinned or monitored. The relevant measure is “Pin service versions where possible and gate supplier changes through compatibility tests”.
C:“Obtain risk-proportionate independent validation before production authorization” is a control for segregation of model development and challenge is missing, but the audit finding here is upstream dependency versions are not pinned or monitored. The relevant measure is “Pin service versions where possible and gate supplier changes through compatibility tests”.
Question 212
In an audit of AI Solution Development Methodologies and Lifecycle, managers say this measure is live: Pin service versions where possible and gate supplier changes through compatibility tests. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Signed requirements diff and refreshed trace from risks to test cases
- Offboarding checklist, credential revocation evidence and runbook acceptance
- Dependency lock, vendor change alert and executed regression suite
- Separate validator signoff, challenge findings and closure evidence
Explanation
Correct Answer
C:Correct. The proposed records—dependency lock, vendor change alert and executed regression suite—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is upstream dependency versions are not pinned or monitored.
Incorrect Answers
A:“Signed requirements diff and refreshed trace from risks to test cases” would test requirements traceability failed across scope change, not the present safeguard against upstream dependency versions are not pinned or monitored. The expected audit evidence is “Dependency lock, vendor change alert and executed regression suite”.
B:“Offboarding checklist, credential revocation evidence and runbook acceptance” would test knowledge transfer and ownership transition are absent from lifecycle controls, not the present safeguard against upstream dependency versions are not pinned or monitored. The expected audit evidence is “Dependency lock, vendor change alert and executed regression suite”.
D:“Separate validator signoff, challenge findings and closure evidence” would test segregation of model development and challenge is missing, not the present safeguard against upstream dependency versions are not pinned or monitored. The expected audit evidence is “Dependency lock, vendor change alert and executed regression suite”.
Question 213
In a review of AI Solution Development Methodologies and Lifecycle, investigators cannot establish the expected control proof: dependency lock, vendor change alert and executed regression suite. The underlying event was that a recommendation engine depends on a vendor embedding endpoint whose version can change silently. Which control weakness is the MOST likely cause requiring focused audit work?
- Requirements traceability failed across scope change
- Upstream dependency versions are not pinned or monitored
- Knowledge transfer and ownership transition are absent from lifecycle controls
- Segregation of model development and challenge is missing
Explanation
Correct Answer
B:Correct. The observed condition specifically indicates upstream dependency versions are not pinned or monitored. A meaningful audit response is to verify this diagnosis and then recommend pin service versions where possible and gate supplier changes through compatibility tests.
Incorrect Answers
A:“Requirements traceability failed across scope change” concerns a separate audit weakness, not the missing assurance described by upstream dependency versions are not pinned or monitored. The correct cause is “Upstream dependency versions are not pinned or monitored”.
C:“Knowledge transfer and ownership transition are absent from lifecycle controls” concerns a separate audit weakness, not the missing assurance described by upstream dependency versions are not pinned or monitored. The correct cause is “Upstream dependency versions are not pinned or monitored”.
D:“Segregation of model development and challenge is missing” concerns a separate audit weakness, not the missing assurance described by upstream dependency versions are not pinned or monitored. The correct cause is “Upstream dependency versions are not pinned or monitored”.
Question 214
A high-impact AI solution is launched with only laboratory accuracy testing and no rollback approach. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Require staged deployment, operational go/no-go criteria and tested rollback
- Apply SBOM review, dependency pinning, vulnerability gates and artifact signing
- Model full prompt-to-tool flows, authorization gates and human-stop mechanisms
- Record immutable base-model identifiers, hyperparameters, code and data lineage
Explanation
Correct Answer
A:Correct. Require staged deployment, operational go/no-go criteria and tested rollback is aimed at the actual failure: operational acceptance gates and fail-safe recovery are omitted from release design. The audit should subsequently obtain canary decision log, failed deployment drill and rollback telemetry to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Apply SBOM review, dependency pinning, vulnerability gates and artifact signing” is a control for software and model supply-chain integrity are excluded from the development pipeline, but the audit finding here is operational acceptance gates and fail-safe recovery are omitted from release design. The relevant measure is “Require staged deployment, operational go/no-go criteria and tested rollback”.
C:“Model full prompt-to-tool flows, authorization gates and human-stop mechanisms” is a control for agentic system interfaces and permissions are outside the documented solution boundary, but the audit finding here is operational acceptance gates and fail-safe recovery are omitted from release design. The relevant measure is “Require staged deployment, operational go/no-go criteria and tested rollback”.
D:“Record immutable base-model identifiers, hyperparameters, code and data lineage” is a control for model configuration and dependencies are not reproducible, but the audit finding here is operational acceptance gates and fail-safe recovery are omitted from release design. The relevant measure is “Require staged deployment, operational go/no-go criteria and tested rollback”.
Question 215
In an audit of AI Solution Development Methodologies and Lifecycle, managers say this measure is live: Require staged deployment, operational go/no-go criteria and tested rollback. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- CI gate logs, signed image digest and dependency exception register
- Reviewed sequence diagrams, negative tool-call tests and approved authority map
- Rebuilt model hash comparison and versioned training configuration
- Canary decision log, failed deployment drill and rollback telemetry
Explanation
Correct Answer
D:Correct. The proposed records—canary decision log, failed deployment drill and rollback telemetry—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is operational acceptance gates and fail-safe recovery are omitted from release design.
Incorrect Answers
A:“CI gate logs, signed image digest and dependency exception register” would test software and model supply-chain integrity are excluded from the development pipeline, not the present safeguard against operational acceptance gates and fail-safe recovery are omitted from release design. The expected audit evidence is “Canary decision log, failed deployment drill and rollback telemetry”.
B:“Reviewed sequence diagrams, negative tool-call tests and approved authority map” would test agentic system interfaces and permissions are outside the documented solution boundary, not the present safeguard against operational acceptance gates and fail-safe recovery are omitted from release design. The expected audit evidence is “Canary decision log, failed deployment drill and rollback telemetry”.
C:“Rebuilt model hash comparison and versioned training configuration” would test model configuration and dependencies are not reproducible, not the present safeguard against operational acceptance gates and fail-safe recovery are omitted from release design. The expected audit evidence is “Canary decision log, failed deployment drill and rollback telemetry”.
Question 216
In a review of AI Solution Development Methodologies and Lifecycle, investigators cannot establish the expected control proof: canary decision log, failed deployment drill and rollback telemetry. The underlying event was that a high-impact AI solution is launched with only laboratory accuracy testing and no rollback approach. Which control weakness is the MOST likely cause requiring focused audit work?
- Software and model supply-chain integrity are excluded from the development pipeline
- Agentic system interfaces and permissions are outside the documented solution boundary
- Operational acceptance gates and fail-safe recovery are omitted from release design
- Model configuration and dependencies are not reproducible
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates operational acceptance gates and fail-safe recovery are omitted from release design. A meaningful audit response is to verify this diagnosis and then recommend require staged deployment, operational go/no-go criteria and tested rollback.
Incorrect Answers
A:“Software and model supply-chain integrity are excluded from the development pipeline” concerns a separate audit weakness, not the missing assurance described by operational acceptance gates and fail-safe recovery are omitted from release design. The correct cause is “Operational acceptance gates and fail-safe recovery are omitted from release design”.
B:“Agentic system interfaces and permissions are outside the documented solution boundary” concerns a separate audit weakness, not the missing assurance described by operational acceptance gates and fail-safe recovery are omitted from release design. The correct cause is “Operational acceptance gates and fail-safe recovery are omitted from release design”.
D:“Model configuration and dependencies are not reproducible” concerns a separate audit weakness, not the missing assurance described by operational acceptance gates and fail-safe recovery are omitted from release design. The correct cause is “Operational acceptance gates and fail-safe recovery are omitted from release design”.
Question 217
A product owner changes the intended use after a successful prototype but leaves the original validation plan unchanged. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Add human-centered scenario tests and misuse cases to release acceptance
- Rebaseline intended-use requirements and update safety, privacy and validation tests
- Require reproducible build, independent review and signed model registry promotion
- Pin service versions where possible and gate supplier changes through compatibility tests
Explanation
Correct Answer
B:Correct. Rebaseline intended-use requirements and update safety, privacy and validation tests is aimed at the actual failure: requirements traceability failed across scope change. The audit should subsequently obtain signed requirements diff and refreshed trace from risks to test cases to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Add human-centered scenario tests and misuse cases to release acceptance” is a control for lifecycle testing omitted realistic failures and adversarial interaction, but the audit finding here is requirements traceability failed across scope change. The relevant measure is “Rebaseline intended-use requirements and update safety, privacy and validation tests”.
C:“Require reproducible build, independent review and signed model registry promotion” is a control for build and release provenance is bypassed by ad hoc notebook deployment, but the audit finding here is requirements traceability failed across scope change. The relevant measure is “Rebaseline intended-use requirements and update safety, privacy and validation tests”.
D:“Pin service versions where possible and gate supplier changes through compatibility tests” is a control for upstream dependency versions are not pinned or monitored, but the audit finding here is requirements traceability failed across scope change. The relevant measure is “Rebaseline intended-use requirements and update safety, privacy and validation tests”.
Question 218
In an audit of AI Solution Development Methodologies and Lifecycle, managers say this measure is live: Rebaseline intended-use requirements and update safety, privacy and validation tests. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Signed requirements diff and refreshed trace from risks to test cases
- Negative-path test library, severity triage and corrected regression evidence
- Promotion approvals, immutable build hashes and traced artifact lineage
- Dependency lock, vendor change alert and executed regression suite
Explanation
Correct Answer
A:Correct. The proposed records—signed requirements diff and refreshed trace from risks to test cases—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is requirements traceability failed across scope change.
Incorrect Answers
B:“Negative-path test library, severity triage and corrected regression evidence” would test lifecycle testing omitted realistic failures and adversarial interaction, not the present safeguard against requirements traceability failed across scope change. The expected audit evidence is “Signed requirements diff and refreshed trace from risks to test cases”.
C:“Promotion approvals, immutable build hashes and traced artifact lineage” would test build and release provenance is bypassed by ad hoc notebook deployment, not the present safeguard against requirements traceability failed across scope change. The expected audit evidence is “Signed requirements diff and refreshed trace from risks to test cases”.
D:“Dependency lock, vendor change alert and executed regression suite” would test upstream dependency versions are not pinned or monitored, not the present safeguard against requirements traceability failed across scope change. The expected audit evidence is “Signed requirements diff and refreshed trace from risks to test cases”.
Question 219
In a review of AI Solution Development Methodologies and Lifecycle, investigators cannot establish the expected control proof: signed requirements diff and refreshed trace from risks to test cases. The underlying event was that a product owner changes the intended use after a successful prototype but leaves the original validation plan unchanged. Which control weakness is the MOST likely cause requiring focused audit work?
- Lifecycle testing omitted realistic failures and adversarial interaction
- Build and release provenance is bypassed by ad hoc notebook deployment
- Upstream dependency versions are not pinned or monitored
- Requirements traceability failed across scope change
Explanation
Correct Answer
D:Correct. The observed condition specifically indicates requirements traceability failed across scope change. A meaningful audit response is to verify this diagnosis and then recommend rebaseline intended-use requirements and update safety, privacy and validation tests.
Incorrect Answers
A:“Lifecycle testing omitted realistic failures and adversarial interaction” concerns a separate audit weakness, not the missing assurance described by requirements traceability failed across scope change. The correct cause is “Requirements traceability failed across scope change”.
B:“Build and release provenance is bypassed by ad hoc notebook deployment” concerns a separate audit weakness, not the missing assurance described by requirements traceability failed across scope change. The correct cause is “Requirements traceability failed across scope change”.
C:“Upstream dependency versions are not pinned or monitored” concerns a separate audit weakness, not the missing assurance described by requirements traceability failed across scope change. The correct cause is “Requirements traceability failed across scope change”.
Question 220
The AI platform uses unreviewed open-source packages in an inference container. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Separate training from deployment and require validation plus appropriate approval
- Define monitoring thresholds, decommission owners and safe replacement routes
- Apply SBOM review, dependency pinning, vulnerability gates and artifact signing
- Require staged deployment, operational go/no-go criteria and tested rollback
Explanation
Correct Answer
C:Correct. Apply SBOM review, dependency pinning, vulnerability gates and artifact signing is aimed at the actual failure: software and model supply-chain integrity are excluded from the development pipeline. The audit should subsequently obtain ci gate logs, signed image digest and dependency exception register to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Separate training from deployment and require validation plus appropriate approval” is a control for autonomous retraining lacks risk-tiered reauthorization of changed behavior, but the audit finding here is software and model supply-chain integrity are excluded from the development pipeline. The relevant measure is “Apply SBOM review, dependency pinning, vulnerability gates and artifact signing”.
B:“Define monitoring thresholds, decommission owners and safe replacement routes” is a control for lifecycle documentation lacks explicit retirement and sunset criteria, but the audit finding here is software and model supply-chain integrity are excluded from the development pipeline. The relevant measure is “Apply SBOM review, dependency pinning, vulnerability gates and artifact signing”.
D:“Require staged deployment, operational go/no-go criteria and tested rollback” is a control for operational acceptance gates and fail-safe recovery are omitted from release design, but the audit finding here is software and model supply-chain integrity are excluded from the development pipeline. The relevant measure is “Apply SBOM review, dependency pinning, vulnerability gates and artifact signing”.