ISACA Advanced in AI Audit AAIA Practice Test 44
Question 430
In an audit of Audit Testing and Sampling Methodologies, managers say this measure is live: Apply correct weights and report group-specific intervals separately. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Weighted estimator worksheet and sample-design documentation
- Versioned adversarial sample corpus and observed defense outcomes
- Sampling frame and documented coverage of high-severity scenarios
- Date-stratified sample list and observed control execution
Explanation
Correct Answer
A:Correct. The proposed records—weighted estimator worksheet and sample-design documentation—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is stratified sample estimates were aggregated without sampling weights.
Incorrect Answers
B:“Versioned adversarial sample corpus and observed defense outcomes” would test test selection does not reflect adaptable AI attack patterns, not the present safeguard against stratified sample estimates were aggregated without sampling weights. The expected audit evidence is “Weighted estimator worksheet and sample-design documentation”.
C:“Sampling frame and documented coverage of high-severity scenarios” would test sampling design misses severity-stratified tail events, not the present safeguard against stratified sample estimates were aggregated without sampling weights. The expected audit evidence is “Weighted estimator worksheet and sample-design documentation”.
D:“Date-stratified sample list and observed control execution” would test selection method omits temporal operational risk windows, not the present safeguard against stratified sample estimates were aggregated without sampling weights. The expected audit evidence is “Weighted estimator worksheet and sample-design documentation”.
Question 431
In a review of Audit Testing and Sampling Methodologies, investigators cannot establish the expected control proof: weighted estimator worksheet and sample-design documentation. The underlying event was that a fairness review selects the same number from each group but reports unweighted population error rates. Which control weakness is the MOST likely cause requiring focused audit work?
- Test selection does not reflect adaptable AI attack patterns
- Sampling design misses severity-stratified tail events
- Selection method omits temporal operational risk windows
- Stratified sample estimates were aggregated without sampling weights
Explanation
Correct Answer
D:Correct. The observed condition specifically indicates stratified sample estimates were aggregated without sampling weights. A meaningful audit response is to verify this diagnosis and then recommend apply correct weights and report group-specific intervals separately.
Incorrect Answers
A:“Test selection does not reflect adaptable AI attack patterns” concerns a separate audit weakness, not the missing assurance described by stratified sample estimates were aggregated without sampling weights. The correct cause is “Stratified sample estimates were aggregated without sampling weights”.
B:“Sampling design misses severity-stratified tail events” concerns a separate audit weakness, not the missing assurance described by stratified sample estimates were aggregated without sampling weights. The correct cause is “Stratified sample estimates were aggregated without sampling weights”.
C:“Selection method omits temporal operational risk windows” concerns a separate audit weakness, not the missing assurance described by stratified sample estimates were aggregated without sampling weights. The correct cause is “Stratified sample estimates were aggregated without sampling weights”.
Question 432
An auditor uses only management-selected success stories to evaluate human overrides. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Select risk-stratified cases including high-impact exceptions and adversarial inputs
- Set sample size and confidence limits appropriate to the test objective
- Draw independent samples from complete decision and override logs
- Preserve deterministic sample selection, population snapshots and trace IDs
Explanation
Correct Answer
C:Correct. Draw independent samples from complete decision and override logs is aimed at the actual failure: selection bias undermines audit conclusions about control performance. The audit should subsequently obtain source-population reconciliation and independent random seed record to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Select risk-stratified cases including high-impact exceptions and adversarial inputs” is a control for sampling design misses severity-stratified tail events, but the audit finding here is selection bias undermines audit conclusions about control performance. The relevant measure is “Draw independent samples from complete decision and override logs”.
B:“Set sample size and confidence limits appropriate to the test objective” is a control for sampling uncertainty and population representativeness were ignored, but the audit finding here is selection bias undermines audit conclusions about control performance. The relevant measure is “Draw independent samples from complete decision and override logs”.
D:“Preserve deterministic sample selection, population snapshots and trace IDs” is a control for sampling reproducibility and evidence chain are absent, but the audit finding here is selection bias undermines audit conclusions about control performance. The relevant measure is “Draw independent samples from complete decision and override logs”.
Question 433
In an audit of Audit Testing and Sampling Methodologies, managers say this measure is live: Draw independent samples from complete decision and override logs. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Sampling frame and documented coverage of high-severity scenarios
- Source-population reconciliation and independent random seed record
- Sampling calculation, inclusion probabilities and confidence intervals
- Seeded sampling script, frozen population and replayable sample manifests
Explanation
Correct Answer
B:Correct. The proposed records—source-population reconciliation and independent random seed record—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is selection bias undermines audit conclusions about control performance.
Incorrect Answers
A:“Sampling frame and documented coverage of high-severity scenarios” would test sampling design misses severity-stratified tail events, not the present safeguard against selection bias undermines audit conclusions about control performance. The expected audit evidence is “Source-population reconciliation and independent random seed record”.
C:“Sampling calculation, inclusion probabilities and confidence intervals” would test sampling uncertainty and population representativeness were ignored, not the present safeguard against selection bias undermines audit conclusions about control performance. The expected audit evidence is “Source-population reconciliation and independent random seed record”.
D:“Seeded sampling script, frozen population and replayable sample manifests” would test sampling reproducibility and evidence chain are absent, not the present safeguard against selection bias undermines audit conclusions about control performance. The expected audit evidence is “Source-population reconciliation and independent random seed record”.
Question 434
In a review of Audit Testing and Sampling Methodologies, investigators cannot establish the expected control proof: source-population reconciliation and independent random seed record. The underlying event was that an auditor uses only management-selected success stories to evaluate human overrides. Which control weakness is the MOST likely cause requiring focused audit work?
- Selection bias undermines audit conclusions about control performance
- Sampling design misses severity-stratified tail events
- Sampling uncertainty and population representativeness were ignored
- Sampling reproducibility and evidence chain are absent
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates selection bias undermines audit conclusions about control performance. A meaningful audit response is to verify this diagnosis and then recommend draw independent samples from complete decision and override logs.
Incorrect Answers
B:“Sampling design misses severity-stratified tail events” concerns a separate audit weakness, not the missing assurance described by selection bias undermines audit conclusions about control performance. The correct cause is “Selection bias undermines audit conclusions about control performance”.
C:“Sampling uncertainty and population representativeness were ignored” concerns a separate audit weakness, not the missing assurance described by selection bias undermines audit conclusions about control performance. The correct cause is “Selection bias undermines audit conclusions about control performance”.
D:“Sampling reproducibility and evidence chain are absent” concerns a separate audit weakness, not the missing assurance described by selection bias undermines audit conclusions about control performance. The correct cause is “Selection bias undermines audit conclusions about control performance”.
Question 435
A penetration test covers known prompt injection strings but not adversarial mutations of retrieved content. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Set sample size and confidence limits appropriate to the test objective
- Stratify samples by weekday, weekend and peak processing conditions
- Apply correct weights and report group-specific intervals separately
- Include evolving attack strategies, threat-informed mutations and retesting
Explanation
Correct Answer
D:Correct. Include evolving attack strategies, threat-informed mutations and retesting is aimed at the actual failure: test selection does not reflect adaptable ai attack patterns. The audit should subsequently obtain versioned adversarial sample corpus and observed defense outcomes to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Set sample size and confidence limits appropriate to the test objective” is a control for sampling uncertainty and population representativeness were ignored, but the audit finding here is test selection does not reflect adaptable AI attack patterns. The relevant measure is “Include evolving attack strategies, threat-informed mutations and retesting”.
B:“Stratify samples by weekday, weekend and peak processing conditions” is a control for selection method omits temporal operational risk windows, but the audit finding here is test selection does not reflect adaptable AI attack patterns. The relevant measure is “Include evolving attack strategies, threat-informed mutations and retesting”.
C:“Apply correct weights and report group-specific intervals separately” is a control for stratified sample estimates were aggregated without sampling weights, but the audit finding here is test selection does not reflect adaptable AI attack patterns. The relevant measure is “Include evolving attack strategies, threat-informed mutations and retesting”.
Question 436
In an audit of Audit Testing and Sampling Methodologies, managers say this measure is live: Include evolving attack strategies, threat-informed mutations and retesting. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Sampling calculation, inclusion probabilities and confidence intervals
- Date-stratified sample list and observed control execution
- Versioned adversarial sample corpus and observed defense outcomes
- Weighted estimator worksheet and sample-design documentation
Explanation
Correct Answer
C:Correct. The proposed records—versioned adversarial sample corpus and observed defense outcomes—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is test selection does not reflect adaptable ai attack patterns.
Incorrect Answers
A:“Sampling calculation, inclusion probabilities and confidence intervals” would test sampling uncertainty and population representativeness were ignored, not the present safeguard against test selection does not reflect adaptable AI attack patterns. The expected audit evidence is “Versioned adversarial sample corpus and observed defense outcomes”.
B:“Date-stratified sample list and observed control execution” would test selection method omits temporal operational risk windows, not the present safeguard against test selection does not reflect adaptable AI attack patterns. The expected audit evidence is “Versioned adversarial sample corpus and observed defense outcomes”.
D:“Weighted estimator worksheet and sample-design documentation” would test stratified sample estimates were aggregated without sampling weights, not the present safeguard against test selection does not reflect adaptable AI attack patterns. The expected audit evidence is “Versioned adversarial sample corpus and observed defense outcomes”.
Question 437
In a review of Audit Testing and Sampling Methodologies, investigators cannot establish the expected control proof: versioned adversarial sample corpus and observed defense outcomes. The underlying event was that a penetration test covers known prompt injection strings but not adversarial mutations of retrieved content. Which control weakness is the MOST likely cause requiring focused audit work?
- Sampling uncertainty and population representativeness were ignored
- Test selection does not reflect adaptable AI attack patterns
- Selection method omits temporal operational risk windows
- Stratified sample estimates were aggregated without sampling weights
Explanation
Correct Answer
B:Correct. The observed condition specifically indicates test selection does not reflect adaptable ai attack patterns. A meaningful audit response is to verify this diagnosis and then recommend include evolving attack strategies, threat-informed mutations and retesting.
Incorrect Answers
A:“Sampling uncertainty and population representativeness were ignored” concerns a separate audit weakness, not the missing assurance described by test selection does not reflect adaptable AI attack patterns. The correct cause is “Test selection does not reflect adaptable AI attack patterns”.
C:“Selection method omits temporal operational risk windows” concerns a separate audit weakness, not the missing assurance described by test selection does not reflect adaptable AI attack patterns. The correct cause is “Test selection does not reflect adaptable AI attack patterns”.
D:“Stratified sample estimates were aggregated without sampling weights” concerns a separate audit weakness, not the missing assurance described by test selection does not reflect adaptable AI attack patterns. The correct cause is “Test selection does not reflect adaptable AI attack patterns”.
Question 438
An auditor receives screenshots of an AI ethics dashboard with no dates or underlying source records. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Obtain timestamped native exports and reconcile to authoritative system logs
- Trace assurance report boundaries to the contracted service and bridge gaps with direct evidence
- Use approved restricted evidence environments with minimization and access logs
- Obtain raw-to-processed lineage and reconcile dropped, transformed and sampled records
Explanation
Correct Answer
A:Correct. Obtain timestamped native exports and reconcile to authoritative system logs is aimed at the actual failure: evidence provenance and completeness cannot be established. The audit should subsequently obtain read-only export hashes, log lineage and reconciled report totals to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Trace assurance report boundaries to the contracted service and bridge gaps with direct evidence” is a control for third-party assurance scope does not cover the actual AI service used, but the audit finding here is evidence provenance and completeness cannot be established. The relevant measure is “Obtain timestamped native exports and reconcile to authoritative system logs”.
C:“Use approved restricted evidence environments with minimization and access logs” is a control for audit collection breaches confidentiality and least-privilege safeguards, but the audit finding here is evidence provenance and completeness cannot be established. The relevant measure is “Obtain timestamped native exports and reconcile to authoritative system logs”.
D:“Obtain raw-to-processed lineage and reconcile dropped, transformed and sampled records” is a control for source-to-report completeness cannot be substantiated, but the audit finding here is evidence provenance and completeness cannot be established. The relevant measure is “Obtain timestamped native exports and reconcile to authoritative system logs”.
Question 439
In an audit of Audit Evidence Collection Techniques, managers say this measure is live: Obtain timestamped native exports and reconcile to authoritative system logs. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Control mapping, report scope analysis and supplier gap responses
- Controlled export records, DLP events and access approvals
- ETL execution trace and count reconciliation across pipeline stages
- Read-only export hashes, log lineage and reconciled report totals
Explanation
Correct Answer
D:Correct. The proposed records—read-only export hashes, log lineage and reconciled report totals—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is evidence provenance and completeness cannot be established.
Incorrect Answers
A:“Control mapping, report scope analysis and supplier gap responses” would test third-party assurance scope does not cover the actual AI service used, not the present safeguard against evidence provenance and completeness cannot be established. The expected audit evidence is “Read-only export hashes, log lineage and reconciled report totals”.
B:“Controlled export records, DLP events and access approvals” would test audit collection breaches confidentiality and least-privilege safeguards, not the present safeguard against evidence provenance and completeness cannot be established. The expected audit evidence is “Read-only export hashes, log lineage and reconciled report totals”.
C:“ETL execution trace and count reconciliation across pipeline stages” would test source-to-report completeness cannot be substantiated, not the present safeguard against evidence provenance and completeness cannot be established. The expected audit evidence is “Read-only export hashes, log lineage and reconciled report totals”.
Question 440
In a review of Audit Evidence Collection Techniques, investigators cannot establish the expected control proof: read-only export hashes, log lineage and reconciled report totals. The underlying event was that an auditor receives screenshots of an AI ethics dashboard with no dates or underlying source records. Which control weakness is the MOST likely cause requiring focused audit work?
- Third-party assurance scope does not cover the actual AI service used
- Audit collection breaches confidentiality and least-privilege safeguards
- Evidence provenance and completeness cannot be established
- Source-to-report completeness cannot be substantiated
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates evidence provenance and completeness cannot be established. A meaningful audit response is to verify this diagnosis and then recommend obtain timestamped native exports and reconcile to authoritative system logs.
Incorrect Answers
A:“Third-party assurance scope does not cover the actual AI service used” concerns a separate audit weakness, not the missing assurance described by evidence provenance and completeness cannot be established. The correct cause is “Evidence provenance and completeness cannot be established”.
B:“Audit collection breaches confidentiality and least-privilege safeguards” concerns a separate audit weakness, not the missing assurance described by evidence provenance and completeness cannot be established. The correct cause is “Evidence provenance and completeness cannot be established”.
D:“Source-to-report completeness cannot be substantiated” concerns a separate audit weakness, not the missing assurance described by evidence provenance and completeness cannot be established. The correct cause is “Evidence provenance and completeness cannot be established”.