Topic 15 Practice Test 1 covers GenAI Troubleshooting, Trace Analysis, Failure Isolation, and Remediation in Domain 5 (Task 5.2). For broader exam preparation, review the AIP-C01 Exam Dumps. Every option includes focused technical reasoning tied to the tested decision.
Question 1
A RAG prompt intermittently omits the final retrieved passages when long conversations are included. Which approach should the developer use? Choose ONE.
- Truncation-error analysis
- Prompt design optimization
- Context-window overflow diagnostics
- Dynamic chunking
Correct Answer: C
Correct Answer
Answer C is correct because Context-window overflow diagnostics — in this scenario, checks token counts, truncation behavior,, in this case, and model context limits to, in this case, confirm whether necessary information is, in this case, being dropped. Useful when responses lose information when prompts, at this layer, or retrieved context become too, at this layer, large. Need: verify whether context limits are dropping required information. Context-window overflow diagnostics matches it directly.
Incorrect Answers
Answer A is incorrect because Truncation-error analysis — in this scenario, inspects where input or output, in this case, was cut off and correlates, in this case, it with token or payload, in this case, limits. Useful when the symptom is incomplete content, at this layer, at a boundary. Need: verify whether context limits are dropping required information. Truncation-error analysis solves another problem.
Answer B is incorrect because Prompt design optimization — in this scenario, restructures instructions, context ordering, delimiters,, in this case, or examples to make model, in this case, behavior more reliable. Useful when the model receives all necessary, at this layer, data but interprets the prompt, at this layer, poorly. Need: verify whether context limits are dropping required information. Prompt design optimization solves another problem.
Answer D is incorrect because Dynamic chunking — in this scenario, adjusts document or context chunk, in this case, sizes so relevant content fits, in this case, model and retrieval constraints more, in this case, effectively. Useful when fixed chunks are causing important, at this layer, passages to be split or, at this layer, overflow context. Need: verify whether context limits are dropping required information. Dynamic chunking solves another problem.
Question 2
A fixed 4,000-token chunk often splits procedures so the step containing the answer lands in a different chunk from its heading. What should the team implement? Choose ONE.
- Context-window overflow diagnostics
- Top-k and metadata-filter review
- Dynamic chunking
- Chunking/preprocessing remediation
Correct Answer: C
Correct Answer
Answer C is correct because Dynamic chunking — for this design, adjusts document or context chunk, for this workload, sizes so relevant content fits, for this workload, model and retrieval constraints more, for this workload, effectively. Fits when fixed chunks are causing important, for the stated need, passages to be split or, for the stated need, overflow context. Constraint: adapt chunk boundaries so related evidence stays retrievable. Dynamic chunking is the closest fit.
Incorrect Answers
Answer A is incorrect because Context-window overflow diagnostics — for this design, checks token counts, truncation behavior,, for this workload, and model context limits to, for this workload, confirm whether necessary information is, for this workload, being dropped. Fits when responses lose information when prompts, for the stated need, or retrieved context become too, for the stated need, large. Constraint: adapt chunk boundaries so related evidence stays retrievable. Context-window overflow diagnostics targets another layer.
Answer B is incorrect because Top-k and metadata-filter review — in this scenario, checks retrieval count and filters, in this case, to confirm that relevant documents, in this case, are not excluded or drowned, in this case, out. Useful when the index is healthy but, at this layer, query configuration suppresses useful context. Need: adapt chunk boundaries so related evidence stays retrievable. Top-k and metadata-filter review solves another problem.
Answer D is incorrect because Chunking/preprocessing remediation — in this scenario, fixes document splitting, normalization, metadata,, in this case, or preprocessing that prevents useful, in this case, passages from being retrieved. Useful when source text is present but, at this layer, indexing preparation damages retrieval. Need: adapt chunk boundaries so related evidence stays retrievable. Chunking/preprocessing remediation solves another problem.
Question 3
All required facts are present in the prompt, but the model ignores a constraint buried between examples and user content. Which option most directly addresses the requirement? Choose ONE.
- Prompt design optimization
- Prompt-version comparison
- Context-window overflow diagnostics
- Request-schema validation
Correct Answer: A
Correct Answer
Answer A is correct because Prompt design optimization — for this design, restructures instructions, context ordering, delimiters,, for this workload, or examples to make model, for this workload, behavior more reliable. Fits when the model receives all necessary, for the stated need, data but interprets the prompt, for the stated need, poorly. Constraint: improve instruction structure rather than data retrieval. Prompt design optimization is the closest fit.
Incorrect Answers
Answer B is incorrect because Prompt-version comparison — in this scenario, compares outputs and metrics from, in this case, two prompt versions against the, in this case, same evaluation cases. Useful when a recent prompt change is, at this layer, suspected of causing quality degradation. Need: improve instruction structure rather than data retrieval. Prompt-version comparison solves another problem.
Answer C is incorrect because Context-window overflow diagnostics — at this decision point, checks token counts, truncation behavior,, within this design, and model context limits to, within this design, confirm whether necessary information is, within this design, being dropped. Typical use: responses lose information when prompts, in the described flow, or retrieved context become too, in the described flow, large. Goal: improve instruction structure rather than data retrieval. Context-window overflow diagnostics does not provide it.
Answer D is incorrect because Request-schema validation — in this scenario, checks required fields, types, ranges,, in this case, and payload structure before sending, in this case, an FM request. Useful when invalid or malformed requests are, at this layer, reaching the model API. Need: improve instruction structure rather than data retrieval. Request-schema validation solves another problem.
Question 4
A response consistently stops halfway through a required JSON array near the same token count. Which design best satisfies this requirement? Choose ONE.
- Truncation-error analysis
- JSON Schema output validation
- Parser/serialization check
- Context-window overflow diagnostics
Correct Answer: A
Correct Answer
Answer A is correct because Truncation-error analysis — for this design, inspects where input or output, for this workload, was cut off and correlates, for this workload, it with token or payload, for this workload, limits. Fits when the symptom is incomplete content, for the stated need, at a boundary. Constraint: determine whether output is being cut off at a limit. Truncation-error analysis is the closest fit.
Incorrect Answers
Answer B is incorrect because JSON Schema output validation — in this scenario, verifies that generated structured output, in this case, conforms to the required fields, in this case, and types before downstream processing. Useful when the model response must meet, at this layer, a deterministic machine-readable contract. Need: determine whether output is being cut off at a limit. JSON Schema output validation solves another problem.
Answer C is incorrect because Parser/serialization check — in this scenario, verifies that the application correctly, in this case, parses streamed or structured model, in this case, responses without dropping fields or, in this case, chunks. Useful when the service returns data but, at this layer, the application presents incomplete output. Need: determine whether output is being cut off at a limit. Parser/serialization check solves another problem.
Answer D is incorrect because Context-window diagnostics are useful when accumulated input, conversation history, or retrieved context may exceed the model’s available context. Here the response itself repeatedly stops near the same token count. Truncation-error analysis is more direct because it investigates output cutoffs and configured generation limits rather than missing input context.
Question 5
Bedrock calls return different 4xx and 5xx errors, but the application currently logs only ‘model call failed’. What is the most appropriate implementation choice? Choose ONE.
- Response-shape analysis
- Request-schema validation
- Structured API error logging
- CloudWatch Logs correlation
Correct Answer: C
Correct Answer
Answer C is correct because Structured API error logging — in this scenario, captures request identifiers, status codes,, in this case, exception details, and relevant non-sensitive, in this case, metadata for failed calls. Useful when integration failures must be diagnosed, at this layer, from concrete API evidence. Need: capture concrete API failure evidence. Structured API error logging matches it directly.
Incorrect Answers
Answer A is incorrect because Response-shape analysis — in this scenario, examines returned status, metadata, stop, in this case, reason, and payload structure to, in this case, distinguish model behavior from integration, in this case, parsing errors. Useful when the caller receives a response, at this layer, but handles it incorrectly. Need: capture concrete API failure evidence. Response-shape analysis solves another problem.
Answer B is incorrect because Request-schema validation — for this design, checks required fields, types, ranges,, for this workload, and payload structure before sending, for this workload, an FM request. Fits when invalid or malformed requests are, for the stated need, reaching the model API. Constraint: capture concrete API failure evidence. Request-schema validation targets another layer.
Answer D is incorrect because CloudWatch Logs correlation — in this scenario, centralizes application and service logs, in this case, so request IDs and events, in this case, can be correlated across failed, in this case, interactions. Useful when troubleshooting requires a timeline of, at this layer, logged events. Need: capture concrete API failure evidence. CloudWatch Logs correlation solves another problem.
Question 6
A client occasionally sends the model ID in the wrong field and omits a required inference parameter. Which approach should the developer use? Choose ONE.
- Structured API error logging
- JSON Schema output validation
- Request-schema validation
- Prompt test harness
Correct Answer: C
Correct Answer
Answer C is correct because Request-schema validation — at this decision point, checks required fields, types, ranges,, within this design, and payload structure before sending, within this design, an FM request. Typical use: invalid or malformed requests are, in the described flow, reaching the model API. Goal: reject malformed requests before invocation. Request-schema validation provides that behavior.
Incorrect Answers
Answer A is incorrect because Structured API error logging — for this design, captures request identifiers, status codes,, for this workload, exception details, and relevant non-sensitive, for this workload, metadata for failed calls. Fits when integration failures must be diagnosed, for the stated need, from concrete API evidence. Constraint: reject malformed requests before invocation. Structured API error logging targets another layer.
Answer B is incorrect because JSON Schema output validation — for this design, verifies that generated structured output, for this workload, conforms to the required fields, for this workload, and types before downstream processing. Fits when the model response must meet, for the stated need, a deterministic machine-readable contract. Constraint: reject malformed requests before invocation. JSON Schema output validation targets another layer.
Answer D is incorrect because Prompt test harness — in this scenario, runs controlled prompt cases repeatedly, in this case, so changes can be compared, in this case, against expected behaviors. Useful when prompt quality regressions need reproducible, at this layer, testing rather than ad hoc, at this layer, editing. Need: reject malformed requests before invocation. Prompt test harness solves another problem.
Question 7
The service returns a valid response with a stop reason and usage metadata, but the application treats it as empty. What should the team implement? Choose ONE.
- Response-shape analysis
- Structured API error logging
- Parser/serialization check
- JSON Schema output validation
Correct Answer: A
Correct Answer
Answer A is correct because Response-shape analysis — for this design, examines returned status, metadata, stop, for this workload, reason, and payload structure to, for this workload, distinguish model behavior from integration, for this workload, parsing errors. Fits when the caller receives a response, for the stated need, but handles it incorrectly. Constraint: inspect the returned response structure and caller handling. Response-shape analysis is the closest fit.
Incorrect Answers
Answer B is incorrect because Structured API error logging — at this decision point, captures request identifiers, status codes,, within this design, exception details, and relevant non-sensitive, within this design, metadata for failed calls. Typical use: integration failures must be diagnosed, in the described flow, from concrete API evidence. Goal: inspect the returned response structure and caller handling. Structured API error logging does not provide it.
Answer C is incorrect because Parser/serialization check — for this design, verifies that the application correctly, for this workload, parses streamed or structured model, for this workload, responses without dropping fields or, for this workload, chunks. Fits when the service returns data but, for the stated need, the application presents incomplete output. Constraint: inspect the returned response structure and caller handling. Parser/serialization check targets another layer.
Answer D is incorrect because JSON Schema output validation — at this decision point, verifies that generated structured output, within this design, conforms to the required fields, within this design, and types before downstream processing. Typical use: the model response must meet, in the described flow, a deterministic machine-readable contract. Goal: inspect the returned response structure and caller handling. JSON Schema output validation does not provide it.
Question 8
A prompt change should be tested against 200 known cases before deployment, with pass/fail expectations for each case. Which option most directly addresses the requirement? Choose ONE.
- Prompt-version comparison
- Systematic prompt refinement
- Prompt test harness
- CloudWatch Logs correlation
Correct Answer: C
Correct Answer
Answer C is correct because Prompt test harness — for this design, runs controlled prompt cases repeatedly, for this workload, so changes can be compared, for this workload, against expected behaviors. Fits when prompt quality regressions need reproducible, for the stated need, testing rather than ad hoc, for the stated need, editing. Constraint: reproduce prompt behavior against controlled test cases. Prompt test harness is the closest fit.
Incorrect Answers
Answer A is incorrect because Prompt-version comparison — for this design, compares outputs and metrics from, for this workload, two prompt versions against the, for this workload, same evaluation cases. Fits when a recent prompt change is, for the stated need, suspected of causing quality degradation. Constraint: reproduce prompt behavior against controlled test cases. Prompt-version comparison targets another layer.
Answer B is incorrect because Systematic prompt refinement — in this scenario, changes one prompt factor at, in this case, a time using measured results, in this case, to isolate the cause of, in this case, poor responses. Useful when the prompt is suspected but, at this layer, the failure source is not, at this layer, yet isolated. Need: reproduce prompt behavior against controlled test cases. Systematic prompt refinement solves another problem.
Answer D is incorrect because CloudWatch Logs correlation — for this design, centralizes application and service logs, for this workload, so request IDs and events, for this workload, can be correlated across failed, for this workload, interactions. Fits when troubleshooting requires a timeline of, for the stated need, logged events. Constraint: reproduce prompt behavior against controlled test cases. CloudWatch Logs correlation targets another layer.
Question 9
Quality dropped after a prompt-template edit. The team needs to run the old and new versions on the same cases. Which design best satisfies this requirement? Choose ONE.
- Prompt test harness
- Prompt-version comparison
- Systematic prompt refinement
- Model/configuration comparison
Correct Answer: B
Correct Answer
Answer B is correct because Prompt-version comparison — at this decision point, compares outputs and metrics from, within this design, two prompt versions against the, within this design, same evaluation cases. Typical use: a recent prompt change is, in the described flow, suspected of causing quality degradation. Goal: isolate the effect of a specific prompt revision. Prompt-version comparison provides that behavior.
Incorrect Answers
Answer A is incorrect because Prompt test harness — at this decision point, runs controlled prompt cases repeatedly, within this design, so changes can be compared, within this design, against expected behaviors. Typical use: prompt quality regressions need reproducible, in the described flow, testing rather than ad hoc, in the described flow, editing. Goal: isolate the effect of a specific prompt revision. Prompt test harness does not provide it.
Answer C is incorrect because Systematic prompt refinement — for this design, changes one prompt factor at, for this workload, a time using measured results, for this workload, to isolate the cause of, for this workload, poor responses. Fits when the prompt is suspected but, for the stated need, the failure source is not, for the stated need, yet isolated. Constraint: isolate the effect of a specific prompt revision. Systematic prompt refinement targets another layer.
Answer D is incorrect because Model/configuration comparison — in this scenario, reproduces the same case with, in this case, a known-good model configuration to, in this case, isolate whether parameters or model, in this case, choice cause the symptom. Useful when quality changed after model or, at this layer, inference configuration changes. Need: isolate the effect of a specific prompt revision. Model/configuration comparison solves another problem.
Question 10
A team suspects the instruction wording causes inconsistent answers and wants to change one factor at a time while measuring results. What is the most appropriate implementation choice? Choose ONE.
- Systematic prompt refinement
- Prompt-version comparison
- Prompt test harness
- Prompt design optimization
Correct Answer: A
Correct Answer
Answer A is correct because Systematic prompt refinement — at this decision point, changes one prompt factor at, within this design, a time using measured results, within this design, to isolate the cause of, within this design, poor responses. Typical use: the prompt is suspected but, in the described flow, the failure source is not, in the described flow, yet isolated. Goal: isolate prompt causes through controlled incremental changes. Systematic prompt refinement provides that behavior.
Incorrect Answers
Answer B is incorrect because Prompt-version comparison — under these constraints, compares outputs and metrics from, at this layer, two prompt versions against the, at this layer, same evaluation cases. Choose it when a recent prompt change is, under these constraints, suspected of causing quality degradation. Requirement: isolate prompt causes through controlled incremental changes. Prompt-version comparison is not the direct fit.
Answer C is incorrect because Prompt test harness — under these constraints, runs controlled prompt cases repeatedly, at this layer, so changes can be compared, at this layer, against expected behaviors. Choose it when prompt quality regressions need reproducible, under these constraints, testing rather than ad hoc, under these constraints, editing. Requirement: isolate prompt causes through controlled incremental changes. Prompt test harness is not the direct fit.
Answer D is incorrect because Prompt design optimization — at this decision point, restructures instructions, context ordering, delimiters,, within this design, or examples to make model, within this design, behavior more reliable. Typical use: the model receives all necessary, in the described flow, data but interprets the prompt, in the described flow, poorly. Goal: isolate prompt causes through controlled incremental changes. Prompt design optimization does not provide it.
Question 11
RAG answers cite unrelated documents even though the source repository contains the correct policy. Which approach should the developer use? Choose ONE.
- Top-k and metadata-filter review
- Source-document freshness check
- Embedding-quality diagnostics
- Retrieval relevance analysis
Correct Answer: D
Correct Answer
Answer D is correct because Retrieval relevance analysis — in this scenario, checks whether retrieved chunks actually, in this case, match the question and contain, in this case, the evidence the model needs. Useful when RAG answers are wrong because, at this layer, supporting context may be irrelevant. Need: check whether retrieved chunks actually contain relevant evidence. Retrieval relevance analysis matches it directly.
Incorrect Answers
Answer A is incorrect because Top-k and metadata-filter review — for this design, checks retrieval count and filters, for this workload, to confirm that relevant documents, for this workload, are not excluded or drowned, for this workload, out. Fits when the index is healthy but, for the stated need, query configuration suppresses useful context. Constraint: check whether retrieved chunks actually contain relevant evidence. Top-k and metadata-filter review targets another layer.
Answer B is incorrect because Source-document freshness check — in this scenario, confirms the retrieval corpus contains, in this case, the current source content and, in this case, that updates were indexed. Useful when answers use outdated facts because, at this layer, the knowledge source may be, at this layer, stale. Need: check whether retrieved chunks actually contain relevant evidence. Source-document freshness check solves another problem.
Answer C is incorrect because Embedding-quality diagnostics — in this scenario, examines whether semantically related content, in this case, is represented and retrieved as, in this case, expected by the embedding process. Useful when retrieval failure points to poor, at this layer, vector representations rather than generation. Need: check whether retrieved chunks actually contain relevant evidence. Embedding-quality diagnostics solves another problem.
Question 12
Semantically similar questions retrieve unrelated chunks after a new embedding pipeline was introduced. What should the team implement? Choose ONE.
- Retrieval relevance analysis
- Embedding-quality diagnostics
- Embedding drift monitoring
- Re-embed with consistent model
Correct Answer: B
Correct Answer
Answer B is correct because Embedding-quality diagnostics — for this design, examines whether semantically related content, for this workload, is represented and retrieved as, for this workload, expected by the embedding process. Fits when retrieval failure points to poor, for the stated need, vector representations rather than generation. Constraint: test whether vector representations preserve expected semantic relationships. Embedding-quality diagnostics is the closest fit.
Incorrect Answers
Answer A is incorrect because Retrieval relevance analysis — for this design, checks whether retrieved chunks actually, for this workload, match the question and contain, for this workload, the evidence the model needs. Fits when RAG answers are wrong because, for the stated need, supporting context may be irrelevant. Constraint: test whether vector representations preserve expected semantic relationships. Retrieval relevance analysis targets another layer.
Answer C is incorrect because Embedding drift monitoring — in this scenario, detects changes in retrieval behavior, in this case, or vector distributions after model,, in this case, corpus, or preprocessing changes. Useful when retrieval quality degrades gradually after, at this layer, upstream changes. Need: test whether vector representations preserve expected semantic relationships. Embedding drift monitoring solves another problem.
Answer D is incorrect because Re-embed with consistent model — in this scenario, recomputes vectors using the intended, in this case, embedding model and preprocessing so, in this case, query and corpus vectors are, in this case, compatible. Useful when the index contains vectors built, at this layer, with mismatched embedding settings. Need: test whether vector representations preserve expected semantic relationships. Re-embed with consistent model solves another problem.
Question 13
Retrieval recall slowly degrades after monthly corpus and preprocessing updates, with no single deployment failure. Which option most directly addresses the requirement? Choose ONE.
- Source-document freshness check
- Vector-search performance tuning
- Embedding-quality diagnostics
- Embedding drift monitoring
Correct Answer: D
Correct Answer
Answer D is correct because Embedding drift monitoring — for this design, detects changes in retrieval behavior, for this workload, or vector distributions after model,, for this workload, corpus, or preprocessing changes. Fits when retrieval quality degrades gradually after, for the stated need, upstream changes. Constraint: detect gradual changes in embedding or retrieval behavior. Embedding drift monitoring is the closest fit.
Incorrect Answers
Answer A is incorrect because Source-document freshness check — for this design, confirms the retrieval corpus contains, for this workload, the current source content and, for this workload, that updates were indexed. Fits when answers use outdated facts because, for the stated need, the knowledge source may be, for the stated need, stale. Constraint: detect gradual changes in embedding or retrieval behavior. Source-document freshness check targets another layer.
Answer B is incorrect because Vector-search performance tuning — in this scenario, optimizes vector index/search parameters and, in this case, query patterns to reduce latency, in this case, while preserving acceptable recall. Useful when retrieval is relevant but too slow. Need: detect gradual changes in embedding or retrieval behavior. Vector-search performance tuning solves another problem.
Answer C is incorrect because Embedding-quality diagnostics — at this decision point, examines whether semantically related content, within this design, is represented and retrieved as, within this design, expected by the embedding process. Typical use: retrieval failure points to poor, in the described flow, vector representations rather than generation. Goal: detect gradual changes in embedding or retrieval behavior. Embedding-quality diagnostics does not provide it.
Question 14
Half the index was created with one embedding model and new documents with another incompatible model. Which design best satisfies this requirement? Choose ONE.
- Embedding-quality diagnostics
- Re-embed with consistent model
- Chunking/preprocessing remediation
- Embedding drift monitoring
Correct Answer: B
Correct Answer
Answer B is correct because Re-embed with consistent model — for this design, recomputes vectors using the intended, for this workload, embedding model and preprocessing so, for this workload, query and corpus vectors are, for this workload, compatible. Fits when the index contains vectors built, for the stated need, with mismatched embedding settings. Constraint: restore embedding compatibility across the corpus. Re-embed with consistent model is the closest fit.
Incorrect Answers
Answer A is incorrect because Embedding-quality diagnostics — under these constraints, examines whether semantically related content, at this layer, is represented and retrieved as, at this layer, expected by the embedding process. Choose it when retrieval failure points to poor, under these constraints, vector representations rather than generation. Requirement: restore embedding compatibility across the corpus. Embedding-quality diagnostics is not the direct fit.
Answer C is incorrect because Chunking/preprocessing remediation — for this design, fixes document splitting, normalization, metadata,, for this workload, or preprocessing that prevents useful, for this workload, passages from being retrieved. Fits when source text is present but, for the stated need, indexing preparation damages retrieval. Constraint: restore embedding compatibility across the corpus. Chunking/preprocessing remediation targets another layer.
Answer D is incorrect because Embedding drift monitoring — at this decision point, detects changes in retrieval behavior, within this design, or vector distributions after model,, within this design, corpus, or preprocessing changes. Typical use: retrieval quality degrades gradually after, in the described flow, upstream changes. Goal: restore embedding compatibility across the corpus. Embedding drift monitoring does not provide it.
Question 15
OCR cleanup removed headings and merged unrelated paragraphs before vectorization, reducing answer quality. What is the most appropriate implementation choice? Choose ONE.
- Embedding-quality diagnostics
- Dynamic chunking
- Source-document freshness check
- Chunking/preprocessing remediation
Correct Answer: D
Correct Answer
Answer D is correct because Chunking/preprocessing remediation — at this decision point, fixes document splitting, normalization, metadata,, within this design, or preprocessing that prevents useful, within this design, passages from being retrieved. Typical use: source text is present but, in the described flow, indexing preparation damages retrieval. Goal: fix source preparation that damages retrieval structure. Chunking/preprocessing remediation provides that behavior.
Incorrect Answers
Answer A is incorrect because Embedding-quality diagnostics — for the described workload, examines whether semantically related content, for the stated need, is represented and retrieved as, for the stated need, expected by the embedding process. Best suited when retrieval failure points to poor, for this decision, vector representations rather than generation. Deciding factor: fix source preparation that damages retrieval structure. Embedding-quality diagnostics addresses something else.
Answer B is incorrect because Dynamic chunking — at this decision point, adjusts document or context chunk, within this design, sizes so relevant content fits, within this design, model and retrieval constraints more, within this design, effectively. Typical use: fixed chunks are causing important, in the described flow, passages to be split or, in the described flow, overflow context. Goal: fix source preparation that damages retrieval structure. Dynamic chunking does not provide it.
Answer C is incorrect because Source-document freshness check — at this decision point, confirms the retrieval corpus contains, within this design, the current source content and, within this design, that updates were indexed. Typical use: answers use outdated facts because, in the described flow, the knowledge source may be, in the described flow, stale. Goal: fix source preparation that damages retrieval structure. Source-document freshness check does not provide it.
Question 16
Retrieved chunks are relevant, but vector search latency has doubled as the index grew. Which approach should the developer use? Choose ONE.
- Top-k and metadata-filter review
- AWS X-Ray tracing
- Embedding drift monitoring
- Vector-search performance tuning
Correct Answer: D
Correct Answer
Answer D is correct because Vector-search performance tuning — for this design, optimizes vector index/search parameters and, for this workload, query patterns to reduce latency, for this workload, while preserving acceptable recall. Fits when retrieval is relevant but too slow. Constraint: optimize retrieval performance rather than model quality. Vector-search performance tuning is the closest fit.
Incorrect Answers
Answer A is incorrect because Top-k and metadata-filter review — at this decision point, checks retrieval count and filters, within this design, to confirm that relevant documents, within this design, are not excluded or drowned, within this design, out. Typical use: the index is healthy but, in the described flow, query configuration suppresses useful context. Goal: optimize retrieval performance rather than model quality. Top-k and metadata-filter review does not provide it.
Answer B is incorrect because AWS X-Ray tracing — in this scenario, traces distributed request paths and, in this case, timing so latency or failures, in this case, can be isolated across application, in this case, components. Useful when a multi-service GenAI request has, at this layer, an unclear latency or failure, at this layer, bottleneck. Need: optimize retrieval performance rather than model quality. AWS X-Ray tracing solves another problem.
Answer C is incorrect because Embedding drift monitoring — under these constraints, detects changes in retrieval behavior, at this layer, or vector distributions after model,, at this layer, corpus, or preprocessing changes. Choose it when retrieval quality degrades gradually after, under these constraints, upstream changes. Requirement: optimize retrieval performance rather than model quality. Embedding drift monitoring is not the direct fit.
Question 17
A maintained prompt template sometimes leaves an unresolved placeholder. The team needs reproducible tests plus logs showing substitutions. What should the team implement? Choose ONE.
- Prompt test harness
- Template testing with CloudWatch Logs
- JSON Schema output validation
- CloudWatch Logs correlation
Correct Answer: B
Correct Answer
Answer B is correct because Template testing with CloudWatch Logs — in this scenario, uses repeatable prompt-template tests and, in this case, logs to identify confusion, missing, in this case, variables, or inconsistent substitutions. Useful when a maintained template behaves differently, at this layer, across inputs. Need: diagnose template-variable and substitution problems. Template testing with CloudWatch Logs matches it directly.
Incorrect Answers
Answer A is incorrect because Prompt test harness — for the described workload, runs controlled prompt cases repeatedly, for the stated need, so changes can be compared, for the stated need, against expected behaviors. Best suited when prompt quality regressions need reproducible, for this decision, testing rather than ad hoc, for this decision, editing. Deciding factor: diagnose template-variable and substitution problems. Prompt test harness addresses something else.
Answer C is incorrect because JSON Schema output validation — under these constraints, verifies that generated structured output, at this layer, conforms to the required fields, at this layer, and types before downstream processing. Choose it when the model response must meet, under these constraints, a deterministic machine-readable contract. Requirement: diagnose template-variable and substitution problems. JSON Schema output validation is not the direct fit.
Answer D is incorrect because CloudWatch Logs correlation — at this decision point, centralizes application and service logs, within this design, so request IDs and events, within this design, can be correlated across failed, within this design, interactions. Typical use: troubleshooting requires a timeline of, in the described flow, logged events. Goal: diagnose template-variable and substitution problems. CloudWatch Logs correlation does not provide it.
Question 18
A request traverses API Gateway, retrieval, Bedrock, and a tool Lambda; end-to-end latency is high but logs do not show which hop is slow. Which option most directly addresses the requirement? Choose ONE.
- AWS X-Ray tracing
- Structured API error logging
- Timeout isolation
- CloudWatch Logs correlation
Correct Answer: A
Correct Answer
Answer A is correct because AWS X-Ray tracing — for this design, traces distributed request paths and, for this workload, timing so latency or failures, for this workload, can be isolated across application, for this workload, components. Fits when a multi-service GenAI request has, for the stated need, an unclear latency or failure, for the stated need, bottleneck. Constraint: trace distributed request timing across services. AWS X-Ray tracing is the closest fit.
Incorrect Answers
Answer B is incorrect because Structured API error logging — under these constraints, captures request identifiers, status codes,, at this layer, exception details, and relevant non-sensitive, at this layer, metadata for failed calls. Choose it when integration failures must be diagnosed, under these constraints, from concrete API evidence. Requirement: trace distributed request timing across services. Structured API error logging is not the direct fit.
Answer C is incorrect because Timeout isolation — in this scenario, separates model, retrieval, and tool, in this case, timing so the component exceeding, in this case, its time budget can be, in this case, identified. Useful when the overall request times out, at this layer, but the slow stage is, at this layer, unknown. Need: trace distributed request timing across services. Timeout isolation solves another problem.
Answer D is incorrect because CloudWatch Logs correlation — under these constraints, centralizes application and service logs, at this layer, so request IDs and events, at this layer, can be correlated across failed, at this layer, interactions. Choose it when troubleshooting requires a timeline of, under these constraints, logged events. Requirement: trace distributed request timing across services. CloudWatch Logs correlation is not the direct fit.
Question 19
The model must return fields name, riskScore, and rationale with fixed types before a downstream workflow can continue. Which design best satisfies this requirement? Choose ONE.
- Response-shape analysis
- Prompt design optimization
- Request-schema validation
- JSON Schema output validation
Correct Answer: D
Correct Answer
Answer D is correct because JSON Schema output validation — for the described workload, verifies that generated structured output, for the stated need, conforms to the required fields, for the stated need, and types before downstream processing. Best suited when the model response must meet, for this decision, a deterministic machine-readable contract. Deciding factor: validate deterministic structured output contracts. JSON Schema output validation fits this case.
Incorrect Answers
Answer A is incorrect because Response-shape analysis — at this decision point, examines returned status, metadata, stop, within this design, reason, and payload structure to, within this design, distinguish model behavior from integration, within this design, parsing errors. Typical use: the caller receives a response, in the described flow, but handles it incorrectly. Goal: validate deterministic structured output contracts. Response-shape analysis does not provide it.
Answer B is incorrect because Prompt design optimization — under these constraints, restructures instructions, context ordering, delimiters,, at this layer, or examples to make model, at this layer, behavior more reliable. Choose it when the model receives all necessary, under these constraints, data but interprets the prompt, under these constraints, poorly. Requirement: validate deterministic structured output contracts. Prompt design optimization is not the direct fit.
Answer C is incorrect because Request-schema validation — under these constraints, checks required fields, types, ranges,, at this layer, and payload structure before sending, at this layer, an FM request. Choose it when invalid or malformed requests are, under these constraints, reaching the model API. Requirement: validate deterministic structured output contracts. Request-schema validation is not the direct fit.
Question 20
Several services log the same request ID, and operations needs to reconstruct the event sequence that led to a failed answer. What is the most appropriate implementation choice? Choose ONE.
- AWS X-Ray tracing
- CloudWatch Logs correlation
- Tool-call trace analysis
- Structured API error logging
Correct Answer: B
Correct Answer
Answer B is correct because CloudWatch Logs correlation — for the described workload, centralizes application and service logs, for the stated need, so request IDs and events, for the stated need, can be correlated across failed, for the stated need, interactions. Best suited when troubleshooting requires a timeline of, for this decision, logged events. Deciding factor: correlate logged events across components. CloudWatch Logs correlation fits this case.
Incorrect Answers
Answer A is incorrect because AWS X-Ray tracing — at this decision point, traces distributed request paths and, within this design, timing so latency or failures, within this design, can be isolated across application, within this design, components. Typical use: a multi-service GenAI request has, in the described flow, an unclear latency or failure, in the described flow, bottleneck. Goal: correlate logged events across components. AWS X-Ray tracing does not provide it.
Answer C is incorrect because Tool-call trace analysis — in this scenario, inspects agent/tool traces to determine, in this case, whether the model chose the, in this case, correct tool, supplied valid parameters,, in this case, and received a usable result. Useful when an agent fails during multi-step, at this layer, tool execution. Need: correlate logged events across components. Tool-call trace analysis solves another problem.
Answer D is incorrect because Structured API error logging — for the described workload, captures request identifiers, status codes,, for the stated need, exception details, and relevant non-sensitive, for the stated need, metadata for failed calls. Best suited when integration failures must be diagnosed, for this decision, from concrete API evidence. Deciding factor: correlate logged events across components. Structured API error logging addresses something else.
