ISACA Advanced in AI Audit AAIA Practice Test 47
Question 460
In an audit of Audit Data Quality and Data Analytics, managers say this measure is live: Analyze incident rates per relevant exposure with cohort and confidence controls. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Deduplication logic and source-to-unique-decision count bridge
- Labeled/unlabeled cohort counts and bounded performance estimates
- Rate-based trend analysis with traffic and case-mix reconciliation
- Tail-event series and independently checked severity classification
Explanation
Correct Answer
C:Correct. The proposed records—rate-based trend analysis with traffic and case-mix reconciliation—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is counts are compared without normalizing for exposure and mix changes.
Incorrect Answers
A:“Deduplication logic and source-to-unique-decision count bridge” would test audit analytics did not deduplicate decision events using business identifiers, not the present safeguard against counts are compared without normalizing for exposure and mix changes. The expected audit evidence is “Rate-based trend analysis with traffic and case-mix reconciliation”.
B:“Labeled/unlabeled cohort counts and bounded performance estimates” would test missing ground truth is silently biasing performance results, not the present safeguard against counts are compared without normalizing for exposure and mix changes. The expected audit evidence is “Rate-based trend analysis with traffic and case-mix reconciliation”.
D:“Tail-event series and independently checked severity classification” would test aggregation masks extreme risk and severity distribution, not the present safeguard against counts are compared without normalizing for exposure and mix changes. The expected audit evidence is “Rate-based trend analysis with traffic and case-mix reconciliation”.
Question 461
In a review of Audit Data Quality and Data Analytics, investigators cannot establish the expected control proof: rate-based trend analysis with traffic and case-mix reconciliation. The underlying event was that an audit dashboard shows fewer harmful responses after a filter upgrade, but usage volume fell by 80 percent. Which control weakness is the MOST likely cause requiring focused audit work?
- Audit analytics did not deduplicate decision events using business identifiers
- Counts are compared without normalizing for exposure and mix changes
- Missing ground truth is silently biasing performance results
- Aggregation masks extreme risk and severity distribution
Explanation
Correct Answer
B:Correct. The observed condition specifically indicates counts are compared without normalizing for exposure and mix changes. A meaningful audit response is to verify this diagnosis and then recommend analyze incident rates per relevant exposure with cohort and confidence controls.
Incorrect Answers
A:“Audit analytics did not deduplicate decision events using business identifiers” concerns a separate audit weakness, not the missing assurance described by counts are compared without normalizing for exposure and mix changes. The correct cause is “Counts are compared without normalizing for exposure and mix changes”.
C:“Missing ground truth is silently biasing performance results” concerns a separate audit weakness, not the missing assurance described by counts are compared without normalizing for exposure and mix changes. The correct cause is “Counts are compared without normalizing for exposure and mix changes”.
D:“Aggregation masks extreme risk and severity distribution” concerns a separate audit weakness, not the missing assurance described by counts are compared without normalizing for exposure and mix changes. The correct cause is “Counts are compared without normalizing for exposure and mix changes”.
Question 462
An auditor combines logs from two scoring services without reconciling time zones. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Normalize to a common time standard and verify clock synchronization
- Treat missing labels separately and perform missingness sensitivity analysis
- Validate category definitions, missing values and mapping consistency before testing
- Validate alerts against a blinded reference sample and measure false positives/negatives
Explanation
Correct Answer
A:Correct. Normalize to a common time standard and verify clock synchronization is aimed at the actual failure: timestamp inconsistency distorts event sequence and control-delay estimates. The audit should subsequently obtain source timestamp mappings and reconstructed independent event timeline to test execution, not merely the existence of a policy.
Incorrect Answers
B:“Treat missing labels separately and perform missingness sensitivity analysis” is a control for missing ground truth is silently biasing performance results, but the audit finding here is timestamp inconsistency distorts event sequence and control-delay estimates. The relevant measure is “Normalize to a common time standard and verify clock synchronization”.
C:“Validate category definitions, missing values and mapping consistency before testing” is a control for input data classification and harmonization are inadequate for subgroup analytics, but the audit finding here is timestamp inconsistency distorts event sequence and control-delay estimates. The relevant measure is “Normalize to a common time standard and verify clock synchronization”.
D:“Validate alerts against a blinded reference sample and measure false positives/negatives” is a control for analytic model error is unquantified within the audit process, but the audit finding here is timestamp inconsistency distorts event sequence and control-delay estimates. The relevant measure is “Normalize to a common time standard and verify clock synchronization”.
Question 463
In an audit of Audit Data Quality and Data Analytics, managers say this measure is live: Normalize to a common time standard and verify clock synchronization. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Labeled/unlabeled cohort counts and bounded performance estimates
- Data dictionary, mapping exceptions and subgroup quality report
- Reviewer-labeled calibration set, confusion matrix and quality threshold signoff
- Source timestamp mappings and reconstructed independent event timeline
Explanation
Correct Answer
D:Correct. The proposed records—source timestamp mappings and reconstructed independent event timeline—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is timestamp inconsistency distorts event sequence and control-delay estimates.
Incorrect Answers
A:“Labeled/unlabeled cohort counts and bounded performance estimates” would test missing ground truth is silently biasing performance results, not the present safeguard against timestamp inconsistency distorts event sequence and control-delay estimates. The expected audit evidence is “Source timestamp mappings and reconstructed independent event timeline”.
B:“Data dictionary, mapping exceptions and subgroup quality report” would test input data classification and harmonization are inadequate for subgroup analytics, not the present safeguard against timestamp inconsistency distorts event sequence and control-delay estimates. The expected audit evidence is “Source timestamp mappings and reconstructed independent event timeline”.
C:“Reviewer-labeled calibration set, confusion matrix and quality threshold signoff” would test analytic model error is unquantified within the audit process, not the present safeguard against timestamp inconsistency distorts event sequence and control-delay estimates. The expected audit evidence is “Source timestamp mappings and reconstructed independent event timeline”.
Question 464
In a review of Audit Data Quality and Data Analytics, investigators cannot establish the expected control proof: source timestamp mappings and reconstructed independent event timeline. The underlying event was that an auditor combines logs from two scoring services without reconciling time zones. Which control weakness is the MOST likely cause requiring focused audit work?
- Missing ground truth is silently biasing performance results
- Input data classification and harmonization are inadequate for subgroup analytics
- Timestamp inconsistency distorts event sequence and control-delay estimates
- Analytic model error is unquantified within the audit process
Explanation
Correct Answer
C:Correct. The observed condition specifically indicates timestamp inconsistency distorts event sequence and control-delay estimates. A meaningful audit response is to verify this diagnosis and then recommend normalize to a common time standard and verify clock synchronization.
Incorrect Answers
A:“Missing ground truth is silently biasing performance results” concerns a separate audit weakness, not the missing assurance described by timestamp inconsistency distorts event sequence and control-delay estimates. The correct cause is “Timestamp inconsistency distorts event sequence and control-delay estimates”.
B:“Input data classification and harmonization are inadequate for subgroup analytics” concerns a separate audit weakness, not the missing assurance described by timestamp inconsistency distorts event sequence and control-delay estimates. The correct cause is “Timestamp inconsistency distorts event sequence and control-delay estimates”.
D:“Analytic model error is unquantified within the audit process” concerns a separate audit weakness, not the missing assurance described by timestamp inconsistency distorts event sequence and control-delay estimates. The correct cause is “Timestamp inconsistency distorts event sequence and control-delay estimates”.
Question 465
An AI model's exception reports contain duplicate retries counted as separate customer denials. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Validate category definitions, missing values and mapping consistency before testing
- Define canonical decision keys and reconcile retries before calculating impacts
- Report severity-weighted tails, percentiles and confidence intervals with event counts
- Analyze incident rates per relevant exposure with cohort and confidence controls
Explanation
Correct Answer
B:Correct. Define canonical decision keys and reconcile retries before calculating impacts is aimed at the actual failure: audit analytics did not deduplicate decision events using business identifiers. The audit should subsequently obtain deduplication logic and source-to-unique-decision count bridge to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Validate category definitions, missing values and mapping consistency before testing” is a control for input data classification and harmonization are inadequate for subgroup analytics, but the audit finding here is audit analytics did not deduplicate decision events using business identifiers. The relevant measure is “Define canonical decision keys and reconcile retries before calculating impacts”.
C:“Report severity-weighted tails, percentiles and confidence intervals with event counts” is a control for aggregation masks extreme risk and severity distribution, but the audit finding here is audit analytics did not deduplicate decision events using business identifiers. The relevant measure is “Define canonical decision keys and reconcile retries before calculating impacts”.
D:“Analyze incident rates per relevant exposure with cohort and confidence controls” is a control for counts are compared without normalizing for exposure and mix changes, but the audit finding here is audit analytics did not deduplicate decision events using business identifiers. The relevant measure is “Define canonical decision keys and reconcile retries before calculating impacts”.
Question 466
In an audit of Audit Data Quality and Data Analytics, managers say this measure is live: Define canonical decision keys and reconcile retries before calculating impacts. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Deduplication logic and source-to-unique-decision count bridge
- Data dictionary, mapping exceptions and subgroup quality report
- Tail-event series and independently checked severity classification
- Rate-based trend analysis with traffic and case-mix reconciliation
Explanation
Correct Answer
A:Correct. The proposed records—deduplication logic and source-to-unique-decision count bridge—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is audit analytics did not deduplicate decision events using business identifiers.
Incorrect Answers
B:“Data dictionary, mapping exceptions and subgroup quality report” would test input data classification and harmonization are inadequate for subgroup analytics, not the present safeguard against audit analytics did not deduplicate decision events using business identifiers. The expected audit evidence is “Deduplication logic and source-to-unique-decision count bridge”.
C:“Tail-event series and independently checked severity classification” would test aggregation masks extreme risk and severity distribution, not the present safeguard against audit analytics did not deduplicate decision events using business identifiers. The expected audit evidence is “Deduplication logic and source-to-unique-decision count bridge”.
D:“Rate-based trend analysis with traffic and case-mix reconciliation” would test counts are compared without normalizing for exposure and mix changes, not the present safeguard against audit analytics did not deduplicate decision events using business identifiers. The expected audit evidence is “Deduplication logic and source-to-unique-decision count bridge”.
Question 467
In a review of Audit Data Quality and Data Analytics, investigators cannot establish the expected control proof: deduplication logic and source-to-unique-decision count bridge. The underlying event was that an AI model's exception reports contain duplicate retries counted as separate customer denials. Which control weakness is the MOST likely cause requiring focused audit work?
- Input data classification and harmonization are inadequate for subgroup analytics
- Aggregation masks extreme risk and severity distribution
- Counts are compared without normalizing for exposure and mix changes
- Audit analytics did not deduplicate decision events using business identifiers
Explanation
Correct Answer
D:Correct. The observed condition specifically indicates audit analytics did not deduplicate decision events using business identifiers. A meaningful audit response is to verify this diagnosis and then recommend define canonical decision keys and reconcile retries before calculating impacts.
Incorrect Answers
A:“Input data classification and harmonization are inadequate for subgroup analytics” concerns a separate audit weakness, not the missing assurance described by audit analytics did not deduplicate decision events using business identifiers. The correct cause is “Audit analytics did not deduplicate decision events using business identifiers”.
B:“Aggregation masks extreme risk and severity distribution” concerns a separate audit weakness, not the missing assurance described by audit analytics did not deduplicate decision events using business identifiers. The correct cause is “Audit analytics did not deduplicate decision events using business identifiers”.
C:“Counts are compared without normalizing for exposure and mix changes” concerns a separate audit weakness, not the missing assurance described by audit analytics did not deduplicate decision events using business identifiers. The correct cause is “Audit analytics did not deduplicate decision events using business identifiers”.
Question 468
Model monitoring labels every missing outcome as a correct prediction. From an AI audit perspective, which corrective action is the MOST direct response to the weakness described, rather than a useful control for some different AI risk?
- Report severity-weighted tails, percentiles and confidence intervals with event counts
- Validate alerts against a blinded reference sample and measure false positives/negatives
- Treat missing labels separately and perform missingness sensitivity analysis
- Normalize to a common time standard and verify clock synchronization
Explanation
Correct Answer
C:Correct. Treat missing labels separately and perform missingness sensitivity analysis is aimed at the actual failure: missing ground truth is silently biasing performance results. The audit should subsequently obtain labeled/unlabeled cohort counts and bounded performance estimates to test execution, not merely the existence of a policy.
Incorrect Answers
A:“Report severity-weighted tails, percentiles and confidence intervals with event counts” is a control for aggregation masks extreme risk and severity distribution, but the audit finding here is missing ground truth is silently biasing performance results. The relevant measure is “Treat missing labels separately and perform missingness sensitivity analysis”.
B:“Validate alerts against a blinded reference sample and measure false positives/negatives” is a control for analytic model error is unquantified within the audit process, but the audit finding here is missing ground truth is silently biasing performance results. The relevant measure is “Treat missing labels separately and perform missingness sensitivity analysis”.
D:“Normalize to a common time standard and verify clock synchronization” is a control for timestamp inconsistency distorts event sequence and control-delay estimates, but the audit finding here is missing ground truth is silently biasing performance results. The relevant measure is “Treat missing labels separately and perform missingness sensitivity analysis”.
Question 469
In an audit of Audit Data Quality and Data Analytics, managers say this measure is live: Treat missing labels separately and perform missingness sensitivity analysis. They claim it corrected a material audit gap. What direct operating evidence would allow the auditor to substantiate that particular claim across a defensible sample?
- Tail-event series and independently checked severity classification
- Labeled/unlabeled cohort counts and bounded performance estimates
- Reviewer-labeled calibration set, confusion matrix and quality threshold signoff
- Source timestamp mappings and reconstructed independent event timeline
Explanation
Correct Answer
B:Correct. The proposed records—labeled/unlabeled cohort counts and bounded performance estimates—are directly tied to the asserted safeguard and allow its actual operation to be corroborated. The gap being tested is missing ground truth is silently biasing performance results.
Incorrect Answers
A:“Tail-event series and independently checked severity classification” would test aggregation masks extreme risk and severity distribution, not the present safeguard against missing ground truth is silently biasing performance results. The expected audit evidence is “Labeled/unlabeled cohort counts and bounded performance estimates”.
C:“Reviewer-labeled calibration set, confusion matrix and quality threshold signoff” would test analytic model error is unquantified within the audit process, not the present safeguard against missing ground truth is silently biasing performance results. The expected audit evidence is “Labeled/unlabeled cohort counts and bounded performance estimates”.
D:“Source timestamp mappings and reconstructed independent event timeline” would test timestamp inconsistency distorts event sequence and control-delay estimates, not the present safeguard against missing ground truth is silently biasing performance results. The expected audit evidence is “Labeled/unlabeled cohort counts and bounded performance estimates”.
Question 470
In a review of Audit Data Quality and Data Analytics, investigators cannot establish the expected control proof: labeled/unlabeled cohort counts and bounded performance estimates. The underlying event was that model monitoring labels every missing outcome as a correct prediction. Which control weakness is the MOST likely cause requiring focused audit work?
- Missing ground truth is silently biasing performance results
- Aggregation masks extreme risk and severity distribution
- Analytic model error is unquantified within the audit process
- Timestamp inconsistency distorts event sequence and control-delay estimates
Explanation
Correct Answer
A:Correct. The observed condition specifically indicates missing ground truth is silently biasing performance results. A meaningful audit response is to verify this diagnosis and then recommend treat missing labels separately and perform missingness sensitivity analysis.
Incorrect Answers
B:“Aggregation masks extreme risk and severity distribution” concerns a separate audit weakness, not the missing assurance described by missing ground truth is silently biasing performance results. The correct cause is “Missing ground truth is silently biasing performance results”.
C:“Analytic model error is unquantified within the audit process” concerns a separate audit weakness, not the missing assurance described by missing ground truth is silently biasing performance results. The correct cause is “Missing ground truth is silently biasing performance results”.
D:“Timestamp inconsistency distorts event sequence and control-delay estimates” concerns a separate audit weakness, not the missing assurance described by missing ground truth is silently biasing performance results. The correct cause is “Missing ground truth is silently biasing performance results”.