ISACA Advanced in AI Audit AAIA Practice Test 46
Question 450
Investigators cannot reproduce an AI decision because only the output score was preserved. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Capture privacy-safe decision lineage sufficient for controlled replay
- Collect external gateway logs and independently execute negative control tests
- Obtain timestamped native exports and reconcile to authoritative system logs
- Trace assurance report boundaries to the contracted service and bridge gaps with direct evidence
Explanation
Correct Answer
A:Correct. Capture privacy-safe decision lineage sufficient for controlled replay is aimed at the actual failure: evidence capture omits input, feature, model version and inference context. The audit should subsequently obtain replay record linking inputs, model digest, decision time and postprocessing to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Collect external gateway logs and independently execute negative control tests” is a control for self-generated attestation is not independent evidence of control effectiveness, but the audit finding here is evidence capture omits input, feature, model version and inference context. The relevant measure is “Capture privacy-safe decision lineage sufficient for controlled replay”.
C:“Obtain timestamped native exports and reconcile to authoritative system logs” is a control for evidence provenance and completeness cannot be established, but the audit finding here is evidence capture omits input, feature, model version and inference context. The relevant measure is “Capture privacy-safe decision lineage sufficient for controlled replay”.
D:“Trace assurance report boundaries to the contracted service and bridge gaps with direct evidence” is a control for third-party assurance scope does not cover the actual AI service used, but the audit finding here is evidence capture omits input, feature, model version and inference context. The relevant measure is “Capture privacy-safe decision lineage sufficient for controlled replay”.
Question 451
In an audit of Audit Evidence Collection Techniques, managers say this measure is live: Capture privacy-safe decision lineage sufficient for controlled replay. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Gateway block evidence and independent test results mapped to the control
- Read-only export hashes, log lineage and reconciled report totals
- Control mapping, report scope analysis and supplier gap responses
- Replay record linking inputs, model digest, decision time and postprocessing
Explanation
Correct Answer
D:Correct. The proposed records—replay record linking inputs, model digest, decision time and postprocessing—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is evidence capture omits input, feature, model version and inference context.
Incorrect Answers
A:“Gateway block evidence and independent test results mapped to the control” would test self-generated attestation is not independent evidence of control effectiveness, not the present safeguard against evidence capture omits input, feature, model version and inference context. The expected audit evidence is “Replay record linking inputs, model digest, decision time and postprocessing”.
B:“Read-only export hashes, log lineage and reconciled report totals” would test evidence provenance and completeness cannot be established, not the present safeguard against evidence capture omits input, feature, model version and inference context. The expected audit evidence is “Replay record linking inputs, model digest, decision time and postprocessing”.
C:“Control mapping, report scope analysis and supplier gap responses” would test third-party assurance scope does not cover the actual AI service used, not the present safeguard against evidence capture omits input, feature, model version and inference context. The expected audit evidence is “Replay record linking inputs, model digest, decision time and postprocessing”.
Question 452
In a review of Audit Evidence Collection Techniques, investigators cannot establish the expected control proof: replay record linking inputs, model digest, decision time and postprocessing. The underlying event was that investigators cannot reproduce an AI decision because only the output score was preserved. Which control weakness is the MOST likely cause requiring focused audit work?
- Self-generated attestation is not independent evidence of control effectiveness
- Evidence provenance and completeness cannot be established
- Evidence capture omits input, feature, model version and inference context
- Third-party assurance scope does not cover the actual AI service used
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates evidence capture omits input, feature, model version and inference context. A meaningful audit response is to verify this diagnosis and then recommend capture privacy-safe decision lineage sufficient for controlled replay.
Incorrect Answers
A:“Self-generated attestation is not independent evidence of control effectiveness” concerns a separate audit weakness, not the missing assurance described by evidence capture omits input, feature, model version and inference context. The correct cause is “Evidence capture omits input, feature, model version and inference context”.
B:“Evidence provenance and completeness cannot be established” concerns a separate audit weakness, not the missing assurance described by evidence capture omits input, feature, model version and inference context. The correct cause is “Evidence capture omits input, feature, model version and inference context”.
D:“Third-party assurance scope does not cover the actual AI service used” concerns a separate audit weakness, not the missing assurance described by evidence capture omits input, feature, model version and inference context. The correct cause is “Evidence capture omits input, feature, model version and inference context”.
Question 453
A dataset's quality report is based on transformed data with no record of excluded source rows. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Obtain timestamped native exports and reconcile to authoritative system logs
- Obtain raw-to-processed lineage and reconcile dropped, transformed and sampled records
- Correlate signed model registry versions with runtime configuration and approvals
- Use approved restricted evidence environments with minimization and access logs
Explanation
Correct Answer
B:Correct. Obtain raw-to-processed lineage and reconcile dropped, transformed and sampled records is aimed at the actual failure: source-to-report completeness cannot be substantiated. The audit should subsequently obtain etl execution trace and count reconciliation across pipeline stages to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Obtain timestamped native exports and reconcile to authoritative system logs” is a control for evidence provenance and completeness cannot be established, but the audit finding here is source-to-report completeness cannot be substantiated. The relevant measure is “Obtain raw-to-processed lineage and reconcile dropped, transformed and sampled records”.
C:“Correlate signed model registry versions with runtime configuration and approvals” is a control for design documentation is not linked to deployed model artifacts, but the audit finding here is source-to-report completeness cannot be substantiated. The relevant measure is “Obtain raw-to-processed lineage and reconcile dropped, transformed and sampled records”.
D:“Use approved restricted evidence environments with minimization and access logs” is a control for audit collection breaches confidentiality and least-privilege safeguards, but the audit finding here is source-to-report completeness cannot be substantiated. The relevant measure is “Obtain raw-to-processed lineage and reconcile dropped, transformed and sampled records”.
Question 454
In an audit of Audit Evidence Collection Techniques, managers say this measure is live: Obtain raw-to-processed lineage and reconcile dropped, transformed and sampled records. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- ETL execution trace and count reconciliation across pipeline stages
- Read-only export hashes, log lineage and reconciled report totals
- Runtime digest, registry metadata and deployment attestations
- Controlled export records, DLP events and access approvals
Explanation
Correct Answer
A:Correct. The proposed records—etl execution trace and count reconciliation across pipeline stages—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is source-to-report completeness cannot be substantiated.
Incorrect Answers
B:“Read-only export hashes, log lineage and reconciled report totals” would test evidence provenance and completeness cannot be established, not the present safeguard against source-to-report completeness cannot be substantiated. The expected audit evidence is “ETL execution trace and count reconciliation across pipeline stages”.
C:“Runtime digest, registry metadata and deployment attestations” would test design documentation is not linked to deployed model artifacts, not the present safeguard against source-to-report completeness cannot be substantiated. The expected audit evidence is “ETL execution trace and count reconciliation across pipeline stages”.
D:“Controlled export records, DLP events and access approvals” would test audit collection breaches confidentiality and least-privilege safeguards, not the present safeguard against source-to-report completeness cannot be substantiated. The expected audit evidence is “ETL execution trace and count reconciliation across pipeline stages”.
Question 455
In a review of Audit Evidence Collection Techniques, investigators cannot establish the expected control proof: etl execution trace and count reconciliation across pipeline stages. The underlying event was that a dataset's quality report is based on transformed data with no record of excluded source rows. Which control weakness is the MOST likely cause requiring focused audit work?
- Evidence provenance and completeness cannot be established
- Design documentation is not linked to deployed model artifacts
- Audit collection breaches confidentiality and least-privilege safeguards
- Source-to-report completeness cannot be substantiated
Explanation
Correct Answer
D:Correct. The observed condition specifically indicates source-to-report completeness cannot be substantiated. A meaningful audit response is to verify this diagnosis and then recommend obtain raw-to-processed lineage and reconcile dropped, transformed and sampled records.
Incorrect Answers
A:“Evidence provenance and completeness cannot be established” concerns a separate audit weakness, not the missing assurance described by source-to-report completeness cannot be substantiated. The correct cause is “Source-to-report completeness cannot be substantiated”.
B:“Design documentation is not linked to deployed model artifacts” concerns a separate audit weakness, not the missing assurance described by source-to-report completeness cannot be substantiated. The correct cause is “Source-to-report completeness cannot be substantiated”.
C:“Audit collection breaches confidentiality and least-privilege safeguards” concerns a separate audit weakness, not the missing assurance described by source-to-report completeness cannot be substantiated. The correct cause is “Source-to-report completeness cannot be substantiated”.
Question 456
A reviewer relies on an LLM's own statement that its safety filter worked. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Correlate signed model registry versions with runtime configuration and approvals
- Trace assurance report boundaries to the contracted service and bridge gaps with direct evidence
- Collect external gateway logs and independently execute negative control tests
- Capture privacy-safe decision lineage sufficient for controlled replay
Explanation
Correct Answer
C:Correct. Collect external gateway logs and independently execute negative control tests is aimed at the actual failure: self-generated attestation is not independent evidence of control effectiveness. The audit should subsequently obtain gateway block evidence and independent test results mapped to the control to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Correlate signed model registry versions with runtime configuration and approvals” is a control for design documentation is not linked to deployed model artifacts, but the audit finding here is self-generated attestation is not independent evidence of control effectiveness. The relevant measure is “Collect external gateway logs and independently execute negative control tests”.
B:“Trace assurance report boundaries to the contracted service and bridge gaps with direct evidence” is a control for third-party assurance scope does not cover the actual AI service used, but the audit finding here is self-generated attestation is not independent evidence of control effectiveness. The relevant measure is “Collect external gateway logs and independently execute negative control tests”.
D:“Capture privacy-safe decision lineage sufficient for controlled replay” is a control for evidence capture omits input, feature, model version and inference context, but the audit finding here is self-generated attestation is not independent evidence of control effectiveness. The relevant measure is “Collect external gateway logs and independently execute negative control tests”.
Question 457
In an audit of Audit Evidence Collection Techniques, managers say this measure is live: Collect external gateway logs and independently execute negative control tests. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Runtime digest, registry metadata and deployment attestations
- Gateway block evidence and independent test results mapped to the control
- Control mapping, report scope analysis and supplier gap responses
- Replay record linking inputs, model digest, decision time and postprocessing
Explanation
Correct Answer
B:Correct. The proposed records—gateway block evidence and independent test results mapped to the control—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is self-generated attestation is not independent evidence of control effectiveness.
Incorrect Answers
A:“Runtime digest, registry metadata and deployment attestations” would test design documentation is not linked to deployed model artifacts, not the present safeguard against self-generated attestation is not independent evidence of control effectiveness. The expected audit evidence is “Gateway block evidence and independent test results mapped to the control”.
C:“Control mapping, report scope analysis and supplier gap responses” would test third-party assurance scope does not cover the actual AI service used, not the present safeguard against self-generated attestation is not independent evidence of control effectiveness. The expected audit evidence is “Gateway block evidence and independent test results mapped to the control”.
D:“Replay record linking inputs, model digest, decision time and postprocessing” would test evidence capture omits input, feature, model version and inference context, not the present safeguard against self-generated attestation is not independent evidence of control effectiveness. The expected audit evidence is “Gateway block evidence and independent test results mapped to the control”.
Question 458
In a review of Audit Evidence Collection Techniques, investigators cannot establish the expected control proof: gateway block evidence and independent test results mapped to the control. The underlying event was that a reviewer relies on an LLM's own statement that its safety filter worked. Which control weakness is the MOST likely cause requiring focused audit work?
- Self-generated attestation is not independent evidence of control effectiveness
- Design documentation is not linked to deployed model artifacts
- Third-party assurance scope does not cover the actual AI service used
- Evidence capture omits input, feature, model version and inference context
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates self-generated attestation is not independent evidence of control effectiveness. A meaningful audit response is to verify this diagnosis and then recommend collect external gateway logs and independently execute negative control tests.
Incorrect Answers
B:“Design documentation is not linked to deployed model artifacts” concerns a separate audit weakness, not the missing assurance described by self-generated attestation is not independent evidence of control effectiveness. The correct cause is “Self-generated attestation is not independent evidence of control effectiveness”.
C:“Third-party assurance scope does not cover the actual AI service used” concerns a separate audit weakness, not the missing assurance described by self-generated attestation is not independent evidence of control effectiveness. The correct cause is “Self-generated attestation is not independent evidence of control effectiveness”.
D:“Evidence capture omits input, feature, model version and inference context” concerns a separate audit weakness, not the missing assurance described by self-generated attestation is not independent evidence of control effectiveness. The correct cause is “Self-generated attestation is not independent evidence of control effectiveness”.
Question 459
An audit dashboard shows fewer harmful responses after a filter upgrade, but usage volume fell by 80 percent. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Define canonical decision keys and reconcile retries before calculating impacts
- Treat missing labels separately and perform missingness sensitivity analysis
- Report severity-weighted tails, percentiles and confidence intervals with event counts
- Analyze incident rates per relevant exposure with cohort and confidence controls
Explanation
Correct Answer
D:Correct. Analyze incident rates per relevant exposure with cohort and confidence controls is aimed at the actual failure: counts are compared without normalizing for exposure and mix changes. The audit should subsequently obtain rate-based trend analysis with traffic and case-mix reconciliation to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Define canonical decision keys and reconcile retries before calculating impacts” is a control for audit analytics did not deduplicate decision events using business identifiers, but the audit finding here is counts are compared without normalizing for exposure and mix changes. The relevant measure is “Analyze incident rates per relevant exposure with cohort and confidence controls”.
B:“Treat missing labels separately and perform missingness sensitivity analysis” is a control for missing ground truth is silently biasing performance results, but the audit finding here is counts are compared without normalizing for exposure and mix changes. The relevant measure is “Analyze incident rates per relevant exposure with cohort and confidence controls”.
C:“Report severity-weighted tails, percentiles and confidence intervals with event counts” is a control for aggregation masks extreme risk and severity distribution, but the audit finding here is counts are compared without normalizing for exposure and mix changes. The relevant measure is “Analyze incident rates per relevant exposure with cohort and confidence controls”.
Question 460
In an audit of Audit Data Quality and Data Analytics, managers say this measure is live: Analyze incident rates per relevant exposure with cohort and confidence controls. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Deduplication logic and source-to-unique-decision count bridge
- Labeled/unlabeled cohort counts and bounded performance estimates
- Rate-based trend analysis with traffic and case-mix reconciliation
- Tail-event series and independently checked severity classification
Explanation
Correct Answer
C:Correct. The proposed records—rate-based trend analysis with traffic and case-mix reconciliation—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is counts are compared without normalizing for exposure and mix changes.
Incorrect Answers
A:“Deduplication logic and source-to-unique-decision count bridge” would test audit analytics did not deduplicate decision events using business identifiers, not the present safeguard against counts are compared without normalizing for exposure and mix changes. The expected audit evidence is “Rate-based trend analysis with traffic and case-mix reconciliation”.
B:“Labeled/unlabeled cohort counts and bounded performance estimates” would test missing ground truth is silently biasing performance results, not the present safeguard against counts are compared without normalizing for exposure and mix changes. The expected audit evidence is “Rate-based trend analysis with traffic and case-mix reconciliation”.
D:“Tail-event series and independently checked severity classification” would test aggregation masks extreme risk and severity distribution, not the present safeguard against counts are compared without normalizing for exposure and mix changes. The expected audit evidence is “Rate-based trend analysis with traffic and case-mix reconciliation”.