Question 390
A quality team sees new errors after a process change but samples only yesterday's familiar files. What should it change?
Explanation
Correct Answer
Representative coverage is needed to detect distribution shifts. The appropriate implementation is “Stratify sampling across new and old workflows and important document types”. Use source validation, representative sampling or independent checks appropriate to the risk, and never disregard a concrete contradiction because of a confidence score. An alternative, “Sample only familiar pre-change records after new failures emerge”, fails here. The changed workflow remains untested. “Treat one successful new record as evidence all new layouts work” does not establish the required outcome. A representative sample is needed. By comparison, “Drop older workflows from evaluation even when they remain in production” is inadequate. Both cohorts can carry risk.
Incorrect Answers
The changed workflow remains untested. This alternative suggests “Sample only familiar pre-change records after new failures emerge”, whereas the supported approach is “Stratify sampling across new and old workflows and important document types”. Representative coverage is needed to detect distribution shifts. Confidence estimates need empirical calibration and suitable quality checks; apparent certainty alone does not guarantee correctness on rare, ambiguous or high-impact cases.
“Treat one successful new record as evidence all new layouts work” is the wrong action for this case. A representative sample is needed. The supported choice is “Stratify sampling across new and old workflows and important document types”. Representative coverage is needed to detect distribution shifts. Use source validation, representative sampling or independent checks appropriate to the risk, and never disregard a concrete contradiction because of a confidence score.
Both cohorts can carry risk. The proposal “Drop older workflows from evaluation even when they remain in production” conflicts with the required decision. “Stratify sampling across new and old workflows and important document types” addresses the actual condition. Representative coverage is needed to detect distribution shifts. Confidence estimates need empirical calibration and suitable quality checks; apparent certainty alone does not guarantee correctness on rare, ambiguous or high-impact cases.