Topic 13 Practice Test 2 covers GenAI Monitoring, Model Invocation Logging, Tool Observability, and Vector Operations in Domain 4 (Task 4.3). For broader exam preparation, review the AIP-C01 Exam Dumps. Every option includes focused technical reasoning tied to the tested decision.
Question 1
A support engineer has Bedrock invocation logging enabled and needs to compare prompt families by token use and error patterns using fields already recorded in the logs. Which monitoring approach is most appropriate? Choose ONE.
- CloudWatch token usage metric
- CloudWatch Logs Insights query over invocation logs
- CloudWatch Model Invocation dashboard
- CloudTrail API auditing
Correct Answer: B
Correct Answer
Answer B is correct because CloudWatch Logs Insights query over invocation logs is the best fit because it queries structured invocation-log fields and can aggregate values such as token counts, request metadata, model IDs, and errors. The engineer needs to query fields inside detailed invocation records, which is exactly the analysis this mechanism supports. Aggregate alarms or unrelated audit signals would not answer that field-level question.
Incorrect Answers
Answer A is incorrect because CloudWatch token usage metric aggregates model input and output token consumption over time for supported Bedrock runtime traffic. That helps with tracking consumption trends and capacity or cost drivers. The stem instead requires the team to query fields inside detailed invocation records, so this option monitors the wrong condition.
Answer C is incorrect because CloudWatch Model Invocation dashboard works because it summarizes invocation volume, tokens, latency, errors, and throttling in a curated CloudWatch view. It is intended for broad Bedrock operations monitoring. Here the required outcome is to query fields inside detailed invocation records, which this mechanism does not provide.
Answer D is incorrect because The normal role of CloudTrail API auditing is determining who called an API and when; it records supported AWS API activity with caller identity, operation, and request metadata for audit purposes. This scenario is controlled by a different requirement: the team must query fields inside detailed invocation records.
Question 2
The operations team enabled model invocation logging, but some expected records never arrive in either CloudWatch Logs or Amazon S3. Which signal should they monitor first to determine whether the logging pipeline itself is failing? Choose ONE.
- Invocation server-error rate
- CloudWatch Logs Insights query over invocation logs
- CloudWatch throttling-rate metric
- Model invocation log delivery failure metrics
Correct Answer: D
Correct Answer
Answer D is correct because Model invocation log delivery failure metrics is the decisive signal because it uses the AWS/Bedrock ModelInvocationLogsCloudWatchDeliveryFailure or ModelInvocationLogsS3DeliveryFailure metrics emitted for failed delivery attempts. The team must detect invocation-log delivery failure before blaming inference. This isolates the logging pipeline itself rather than model behavior.
Incorrect Answers
Answer A is incorrect because Invocation server-error rate is valid for detecting inference service failures because it tracks service-side invocation errors for Bedrock runtime requests. The question asks the team to detect invocation-log delivery failure; those are separate observability decisions.
Answer B is incorrect because CloudWatch Logs Insights query over invocation logs contributes useful evidence by the fact that it queries structured invocation-log fields and can aggregate values such as token counts, request metadata, model IDs, and errors. Its best use is detailed ad hoc analysis of already-delivered invocation records. It does not satisfy the need to detect invocation-log delivery failure.
Answer C is incorrect because Choosing CloudWatch throttling-rate metric supports capacity and quota pressure; specifically, it measures requests constrained by service limits. The controlling condition is to detect invocation-log delivery failure, so this cannot replace the required mechanism.
Question 3
Two product features share the same Bedrock model. Engineers must filter detailed invocation logs by feature and experiment without creating separate endpoints. Which design best supports that analysis? Choose ONE.
- Per-request Bedrock requestMetadata tags
- CloudWatch log group per feature
- IAM principal grouping in invocation logs
- Application inference profile cost tags
Correct Answer: A
Correct Answer
Answer A is correct because Use Per-request Bedrock requestMetadata tags: it adds caller-defined key-value metadata to supported Bedrock runtime requests and records those tags in model invocation logs when logging is enabled. The two features share one endpoint, so the team must segment records by feature and experiment. Caller-supplied metadata provides that dimension without inventing separate endpoints.
Incorrect Answers
Answer B is incorrect because CloudWatch log group per feature is oriented toward isolation when different workloads truly require distinct destinations and physically separates application logging destinations by feature. The system must instead segment records by feature and experiment; those outcomes can vary independently.
Answer C is incorrect because The value of IAM principal grouping in invocation logs is in attributing activity to distinct callers or roles, where it groups invocation-log records by the automatically captured identity ARN. It does not directly help the team segment records by feature and experiment, which determines the answer.
Answer D is incorrect because Application inference profile cost tags would be reasonable for chargeback and cost allocation across applications because it attributes Bedrock usage costs to tagged application inference profiles for cost reporting. The evidence points elsewhere: the team must segment records by feature and experiment.
Question 4
Daily token consumption has strong weekday patterns, so a fixed threshold creates either false alarms or missed spikes. The team still needs proactive notification when usage departs sharply from its learned normal range. What should they implement? Choose ONE.
- Static CloudWatch token threshold
- AWS Budgets monthly cost threshold
- CloudWatch anomaly detection on token usage
- Invocation latency percentile alarm
Correct Answer: C
Correct Answer
Answer C is correct because CloudWatch anomaly detection on token usage fits because it builds an expected-value band from historical metric behavior and detects values that depart from the learned pattern. The workload is seasonal, so the team needs to find token spikes against a learned baseline. A learned band is more appropriate than one fixed threshold.
Incorrect Answers
Answer A is incorrect because Static CloudWatch token threshold provides a different operational view because it alarms when a token metric crosses one fixed configured value. That matters for workloads with stable limits and little seasonality, but it cannot establish the required ability to find token spikes against a learned baseline.
Answer B is incorrect because AWS Budgets monthly cost threshold is a sound control for budget governance over a billing period; it notifies when actual or forecast spend approaches configured budget values. The stem asks for a mechanism to find token spikes against a learned baseline, leaving this option insufficient.
Answer D is incorrect because With Invocation latency percentile alarm, the team can address performance degradation because it alarms on tail latency such as P90 or P99. The actual decision is how to find token spikes against a learned baseline, so the two controls are not interchangeable.
Question 5
An on-call engineer wants an out-of-box view of Bedrock invocation count, token consumption, latency percentiles, errors, and throttling before opening individual logs. Which capability best fits? Choose ONE.
- CloudWatch Logs Insights query over invocation logs
- CloudTrail API auditing
- CloudWatch GenAI Model Invocations dashboard
- AWS Cost Explorer
Correct Answer: C
Correct Answer
Answer C is correct because CloudWatch GenAI Model Invocations dashboard supplies the requested starting point because it provides a preconfigured CloudWatch view of Bedrock invocation volume, token consumption, latency percentiles, errors, throttles, cost-related dimensions, and invocation-log data. The on-call engineer wants to open a curated Bedrock invocation operations view before drilling into raw records, making the curated dashboard the appropriate first view.
Incorrect Answers
Answer A is incorrect because CloudWatch Logs Insights query over invocation logs has a legitimate role in detailed ad hoc analysis of already-delivered invocation records: it queries structured invocation-log fields and can aggregate values such as token counts, request metadata, model IDs, and errors. In contrast, this case requires the team to open a curated Bedrock invocation operations view, so the option acts on the wrong signal.
Answer B is incorrect because CloudTrail API auditing helps with determining who called an API and when since it records supported AWS API activity with caller identity, operation, and request metadata for audit purposes. That is not the failure described here; the team needs to open a curated Bedrock invocation operations view.
Answer D is incorrect because AWS Cost Explorer supports retrospective spend analysis and allocation; its mechanism is to analyzes historical AWS cost and usage dimensions. The requested outcome is to open a curated Bedrock invocation operations view, which this option does not deliver.
Question 6
A self-hosted agent uses Bedrock, custom tools, a database, and an internal API. Service-native metrics do not show the complete request path. Which monitoring change provides end-to-end telemetry across those components? Choose ONE.
- ADOT/OpenTelemetry instrumentation across the agent path
- Bedrock runtime service metrics only
- Application log correlation ID only
- CloudTrail management-event history
Correct Answer: A
Correct Answer
Answer A is correct because ADOT/OpenTelemetry instrumentation across the agent path is appropriate because it emits structured OpenTelemetry traces, metrics, and logs from application code and downstream components so custom agent activity appears in a shared observability path. Managed model metrics cannot reconstruct custom dependencies; the requirement is to trace the complete custom agent execution path. OpenTelemetry instrumentation provides that cross-component evidence.
Incorrect Answers
Answer B is incorrect because Bedrock runtime service metrics only exposes service-level Bedrock invocation, latency, token, error, and throttle metrics. That helps with monitoring the managed model endpoint. The stem instead requires the team to trace the complete custom agent execution path, so this option monitors the wrong condition.
Answer C is incorrect because Application log correlation ID only works because it adds an application identifier that can be searched across compatible logs. It is intended for manual log correlation when all components reliably emit the identifier. Here the required outcome is to trace the complete custom agent execution path, which this mechanism does not provide.
Answer D is incorrect because The normal role of CloudTrail management-event history is reviewing configuration changes and administrative calls; it captures AWS control-plane API activity for governance and audit. This scenario is controlled by a different requirement: the team must trace the complete custom agent execution path.
Question 7
One customer conversation can trigger several separate agent traces over many turns. Support needs to view those traces as belonging to the same logical conversation. What should the application add to its telemetry? Choose ONE.
- Distributed trace-context propagation
- Per-request Bedrock requestMetadata tags
- CloudWatch log retention policy
- Session identifier carried in agent telemetry
Correct Answer: D
Correct Answer
Answer D is correct because Session identifier carried in agent telemetry solves the grouping need because it labels related traces with a stable conversation or session identifier so multiple request-response traces can be grouped into one logical interaction. Support must group multiple traces into one conversation; a session-level identifier groups related request traces without confusing them with a single distributed trace.
Incorrect Answers
Answer A is incorrect because Distributed trace-context propagation is valid for requests that cross API, service, and tool boundaries and currently fragment into unrelated traces because it passes W3C Trace Context or X-Ray trace headers across calls so downstream spans inherit the originating trace and parent-child relationships remain connected. The question asks the team to group multiple traces into one conversation; those are separate observability decisions.
Answer B is incorrect because Per-request Bedrock requestMetadata tags contributes useful evidence by the fact that it adds caller-defined key-value metadata to supported Bedrock runtime requests and records those tags in model invocation logs when logging is enabled. Its best use is segmenting invocation records by feature, experiment, tenant, or other caller-controlled dimension. It does not satisfy the need to group multiple traces into one conversation.
Answer C is incorrect because Choosing CloudWatch log retention policy supports data lifecycle and storage governance; specifically, it expires log events according to a configured retention period. The controlling condition is to group multiple traces into one conversation, so this cannot replace the required mechanism.
Question 8
An agent request moves from API Gateway to a service, then to a tool running elsewhere. Each component emits spans, but the trace breaks at the service boundary. Which change most directly fixes the correlation problem? Choose ONE.
- Session identifier carried in agent telemetry
- Distributed trace-context propagation
- Application log correlation ID only
- CloudWatch Logs Insights query
Correct Answer: B
Correct Answer
Answer B is correct because Distributed trace-context propagation is the missing mechanism because it passes W3C Trace Context or X-Ray trace headers across calls so downstream spans inherit the originating trace and parent-child relationships remain connected. The requirement is to keep downstream spans in one distributed trace; carrying trace context across every service boundary preserves the parent-child relationships.
Incorrect Answers
Answer A is incorrect because Session identifier carried in agent telemetry is oriented toward multi-turn agents where support must inspect all traces that belong to one conversation and labels related traces with a stable conversation or session identifier so multiple request-response traces can be grouped into one logical interaction. The system must instead keep downstream spans in one distributed trace; those outcomes can vary independently.
Answer C is incorrect because The value of Application log correlation ID only is in manual log correlation when all components reliably emit the identifier, where it adds an application identifier that can be searched across compatible logs. It does not directly help the team keep downstream spans in one distributed trace, which determines the answer.
Answer D is incorrect because CloudWatch Logs Insights query would be reasonable for investigating records already present in logs because it queries structured CloudWatch log events using fields, filters, parsing, and aggregations. The evidence points elsewhere: the team must keep downstream spans in one distributed trace.
Question 9
A single agent request takes nine seconds. Operators know the request succeeded but need to see whether the delay occurred in model inference, retrieval, or a tool call. Which view is the best next diagnostic step? Choose ONE.
- Invocation latency percentile metric
- Per-tool latency metric
- CloudWatch GenAI model dashboard
- CloudWatch trace timeline and trajectory view
Correct Answer: D
Correct Answer
Answer D is correct because CloudWatch trace timeline and trajectory view is the best diagnostic because it shows spans for a selected trace with durations and relationships, letting operators identify which invocation, retrieval, or tool segment consumed time. Operators need to localize latency inside one request, and the trace view exposes duration and relationships for the specific request rather than a fleet-wide trend.
Incorrect Answers
Answer A is incorrect because Invocation latency percentile metric provides a different operational view because it tracks tail inference latency such as P90 or P99 for Bedrock model calls. That matters for detecting slow model responses and latency regressions, but it cannot establish the required ability to localize latency inside one request.
Answer B is incorrect because Per-tool latency metric is a sound control for finding slow external actions in aggregate; it measures the duration of calls to a particular tool. The stem asks for a mechanism to localize latency inside one request, leaving this option insufficient.
Answer C is incorrect because With CloudWatch GenAI model dashboard, the team can address broad model-operation monitoring because it summarizes curated model invocation metrics, token counts, latency, errors, throttles, and related observability data. The actual decision is how to localize latency inside one request, so the two controls are not interchangeable.
Question 10
A calendar tool normally handles about 8% of agent actions with a low failure rate. After a release, its invocation share triples and failures rise even though model latency is unchanged. Which monitoring design best detects this type of problem? Choose ONE.
- Model invocation throttling alarm
- Tool-call failure ratio
- Per-tool usage baseline with anomaly alerting
- CloudWatch server-error rate
Correct Answer: C
Correct Answer
Answer C is correct because Per-tool usage baseline with anomaly alerting matches the evidence because it tracks tool invocation share, latency, failures, and related usage patterns against an established normal baseline and flags material deviations. Model latency is normal, while the requirement is to detect abnormal behavior of one agent tool. A per-tool baseline can reveal the changed invocation mix and failure behavior.
Incorrect Answers
Answer A is incorrect because Model invocation throttling alarm has a legitimate role in detecting quota or capacity pressure at the model invocation layer: it alarms when Bedrock requests are throttled because limits or capacity constraints are reached. In contrast, this case requires the team to detect abnormal behavior of one agent tool, so the option acts on the wrong signal.
Answer B is incorrect because Tool-call failure ratio helps with finding unstable or unavailable tools since it measures the proportion of external tool invocations that fail. That is not the failure described here; the team needs to detect abnormal behavior of one agent tool.
Answer D is incorrect because CloudWatch server-error rate supports availability and reliability monitoring because it tracks service-side failures. The requested outcome is to detect abnormal behavior of one agent tool, which this option does not deliver.
Question 11
A supervisor agent repeatedly delegates the same task between two specialist agents until the workflow times out. Individual model calls look normal. Which observability artifact is most useful for diagnosing the failure? Choose ONE.
- Per-tool usage baseline with anomaly alerting
- Multi-agent coordination trace
- End-to-end correlation ID only
- Model invocation latency dashboard
Correct Answer: B
Correct Answer
Answer B is correct because Multi-agent coordination trace exposes the orchestration defect because it records supervisor and subagent handoffs, spans, and decision relationships so repeated delegation and orchestration loops are visible. Individual calls succeed, but operators must expose supervisor-to-subagent delegation loops. The coordination trace makes repeated handoffs visible.
Incorrect Answers
Answer A is incorrect because Per-tool usage baseline with anomaly alerting tracks tool invocation share, latency, failures, and related usage patterns against an established normal baseline and flags material deviations. That helps with detecting unexpected tool behavior even when model metrics remain normal. The stem instead requires the team to expose supervisor-to-subagent delegation loops, so this option monitors the wrong condition.
Answer C is incorrect because End-to-end correlation ID only works because it associates records from several components with one request identifier. It is intended for manual correlation across logs. Here the required outcome is to expose supervisor-to-subagent delegation loops, which this mechanism does not provide.
Answer D is incorrect because The normal role of Model invocation latency dashboard is monitoring the model service broadly; it shows model invocation latency and related model metrics over time. This scenario is controlled by a different requirement: the team must expose supervisor-to-subagent delegation loops.
Question 12
An OpenSearch vector workload starts rejecting new vector indexing when approximate k-NN graph memory becomes constrained. Which metric most directly warns that the circuit-breaker limit is being approached? Choose ONE.
- OpenSearch KNNGraphMemoryUsagePercentage
- Vector retrieval latency metric
- OpenSearch CPUUtilization
- OpenSearch FreeStorageSpace
Correct Answer: A
Correct Answer
Answer A is correct because OpenSearch KNNGraphMemoryUsagePercentage is the native pressure signal because it reports k-NN graph memory use as a percentage of the circuit-breaker allowance for an OpenSearch domain. The problem is specifically to measure k-NN graph memory circuit-breaker pressure, which CPU, disk, or retrieval-latency metrics do not measure.
Incorrect Answers
Answer B is incorrect because Vector retrieval latency metric is valid for diagnosing slow vector search because it measures retrieval-stage response time separately from model generation. The question asks the team to measure k-NN graph memory circuit-breaker pressure; those are separate observability decisions.
Answer C is incorrect because OpenSearch CPUUtilization contributes useful evidence by the fact that it reports processor utilization on OpenSearch data nodes. Its best use is detecting CPU saturation. It does not satisfy the need to measure k-NN graph memory circuit-breaker pressure.
Answer D is incorrect because Choosing OpenSearch FreeStorageSpace supports detecting disk-capacity pressure; specifically, it tracks available storage on OpenSearch nodes. The controlling condition is to measure k-NN graph memory circuit-breaker pressure, so this cannot replace the required mechanism.
Question 13
A knowledge corpus contains 50,000 approved records. Retrieval latency is normal, but a reconciliation job shows only 48,200 records represented in the searchable vector index. Which monitoring control best detects this operational defect? Choose ONE.
- Vector index freshness metric
- Ingestion failure counter
- Vector corpus-to-index coverage validation
- Vector retrieval latency metric
Correct Answer: C
Correct Answer
Answer C is correct because Vector corpus-to-index coverage validation tests the observed defect because it reconciles expected source records or chunks with successfully indexed vector records and raises a data-quality defect when coverage is incomplete. The team must detect missing corpus coverage in the vector index despite normal latency. Coverage reconciliation finds silent omissions that performance metrics can miss.
Incorrect Answers
Answer A is incorrect because Vector index freshness metric is oriented toward ensuring updates become searchable within a time objective and measures lag between source changes and their appearance in the searchable vector index. The system must instead detect missing corpus coverage in the vector index; those outcomes can vary independently.
Answer B is incorrect because The value of Ingestion failure counter is in surfacing known failed records or chunks, where it counts explicit parse, embedding, or index failures during ingestion. It does not directly help the team detect missing corpus coverage in the vector index, which determines the answer.
Answer D is incorrect because Vector retrieval latency metric would be reasonable for diagnosing slow vector search because it measures retrieval-stage response time separately from model generation. The evidence points elsewhere: the team must detect missing corpus coverage in the vector index.
Question 14
A vector store’s query latency and maintenance indicators deteriorate gradually over several days. The platform team wants remediation to begin automatically when sustained health thresholds are crossed. Which design best meets the requirement? Choose ONE.
- Manual weekly index review
- CloudWatch fixed latency threshold only
- Increase OpenSearch storage capacity
- Automated vector-index maintenance workflow
Correct Answer: D
Correct Answer
Answer D is correct because Automated vector-index maintenance workflow meets the automation requirement because it uses sustained vector-store health indicators to trigger controlled optimization or maintenance actions through an automated workflow. The platform must start index maintenance automatically from sustained health signals, not simply notify an operator. A controlled maintenance workflow closes that operational loop.
Incorrect Answers
Answer A is incorrect because Manual weekly index review provides a different operational view because it has an operator periodically inspect index health and decide whether maintenance is needed. That matters for low-scale environments where delayed manual intervention is acceptable, but it cannot establish the required ability to start index maintenance automatically from sustained health signals.
Answer B is incorrect because CloudWatch fixed latency threshold only is a sound control for simple notification about a single metric; it raises an alarm when vector query latency crosses one configured value. The stem asks for a mechanism to start index maintenance automatically from sustained health signals, leaving this option insufficient.
Answer C is incorrect because With Increase OpenSearch storage capacity, the team can address low free-storage conditions because it adds disk capacity for index data. The actual decision is how to start index maintenance automatically from sustained health signals, so the two controls are not interchangeable.
Question 15
A model upgrade passes availability checks, but the team must know whether grounded answers regress on a stable set of known-answer business questions. Which monitoring practice is most appropriate? Choose ONE.
- Golden-dataset hallucination regression test
- Groundedness sample from live traffic
- Service availability canary
- Token-usage regression
Correct Answer: A
Correct Answer
Answer A is correct because Golden-dataset hallucination regression test gives a repeatable quality gate because it runs a curated set of known-answer prompts and scores the new model or configuration against expected facts or grounded answers. Availability is healthy; the team needs to measure factual regression on stable known-answer cases. A fixed golden set makes release-to-release comparison controlled.
Incorrect Answers
Answer B is incorrect because Groundedness sample from live traffic has a legitimate role in ongoing production quality surveillance: it samples production answers and scores whether claims are supported by context. In contrast, this case requires the team to measure factual regression on stable known-answer cases, so the option acts on the wrong signal.
Answer C is incorrect because Service availability canary helps with availability monitoring since it sends synthetic requests to verify that an endpoint is reachable and returns acceptable service-level responses. That is not the failure described here; the team needs to measure factual regression on stable known-answer cases.
Answer D is incorrect because Token-usage regression supports cost or prompt-efficiency analysis because it compares token consumption before and after a release. The requested outcome is to measure factual regression on stable known-answer cases, which this option does not deliver.
Question 16
Developers suspect a prompt-template change made answers less consistent. They want a repeatable signal based on comparing outputs from controlled prompts before and after the change. Which technique best fits? Choose ONE.
- Golden-dataset hallucination regression test
- Controlled output diffing across releases
- Latency percentile comparison
- Retrieval relevance evaluation
Correct Answer: B
Correct Answer
Answer B is correct because Controlled output diffing across releases isolates the suspected release effect because it replays controlled prompts against two versions and compares outputs or derived quality measures to expose consistency and behavior changes. The goal is to compare controlled outputs across releases; keeping prompts fixed makes the response change attributable and measurable.
Incorrect Answers
Answer A is incorrect because Golden-dataset hallucination regression test runs a curated set of known-answer prompts and scores the new model or configuration against expected facts or grounded answers. That helps with repeatable detection of quality or hallucination regression after model, prompt, or retrieval changes. The stem instead requires the team to compare controlled outputs across releases, so this option monitors the wrong condition.
Answer C is incorrect because Latency percentile comparison works because it compares response-time distributions across releases. It is intended for performance regression analysis. Here the required outcome is to compare controlled outputs across releases, which this mechanism does not provide.
Answer D is incorrect because The normal role of Retrieval relevance evaluation is RAG retrieval quality; it scores whether retrieved passages match the query’s information need. This scenario is controlled by a different requirement: the team must compare controlled outputs across releases.
Question 17
An agent chooses the wrong sequence of actions even though every downstream tool returns success. Engineers need to identify the planning step where the logic went off course. Which monitoring method is most useful? Choose ONE.
- Reasoning-path trace inspection
- Tool-call failure ratio
- CloudWatch service map only
- End-user interaction tracking
Correct Answer: A
Correct Answer
Answer A is correct because Reasoning-path trace inspection targets the failure layer because it examines the agent’s planning and decision steps together with downstream actions so the point that selected an incorrect sequence can be identified. Tool execution succeeded, so engineers must locate the incorrect planning decision. Inspecting the reasoning path reveals where the action sequence went wrong.
Incorrect Answers
Answer B is incorrect because Tool-call failure ratio is valid for finding unstable or unavailable tools because it measures the proportion of external tool invocations that fail. The question asks the team to locate the incorrect planning decision; those are separate observability decisions.
Answer C is incorrect because CloudWatch service map only contributes useful evidence by the fact that it visualizes traced services and dependencies. Its best use is high-level topology and service relationships. It does not satisfy the need to locate the incorrect planning decision.
Answer D is incorrect because Choosing End-user interaction tracking supports understanding production user experience; specifically, it records user actions and quality feedback around generated responses. The controlling condition is to locate the incorrect planning decision, so this cannot replace the required mechanism.
Question 18
A product team improved latency, yet customer task completion did not improve. Leadership wants one dashboard that shows technical GenAI health beside task completion, escalation, and abandonment. Which design addresses this need? Choose ONE.
- Service dashboard by model and workflow
- Per-model cost attribution
- Business-impact dashboard linked to GenAI health
- Invocation latency percentile metric
Correct Answer: C
Correct Answer
Answer C is correct because Business-impact dashboard linked to GenAI health satisfies the requirement because it places operational AI signals beside business outcomes such as task completion, escalation, conversion, or abandonment in the same monitoring view. Leadership wants to connect technical health with business outcomes, so technical metrics must appear beside completion, escalation, or abandonment outcomes.
Incorrect Answers
Answer A is incorrect because Service dashboard by model and workflow is oriented toward technical service monitoring and compares operational health across models and application workflows. The system must instead connect technical health with business outcomes; those outcomes can vary independently.
Answer B is incorrect because The value of Per-model cost attribution is in identifying expensive models or workloads, where it separates spend by model, application, or tagged usage dimension. It does not directly help the team connect technical health with business outcomes, which determines the answer.
Answer D is incorrect because Invocation latency percentile metric would be reasonable for detecting slow model responses and latency regressions because it tracks tail inference latency such as P90 or P99 for Bedrock model calls. The evidence points elsewhere: the team must connect technical health with business outcomes.
Question 19
Infrastructure metrics remain stable after a model update, but users begin retrying answers and abandoning sessions more often. Which monitoring signal is most likely to expose the behavior change early? Choose ONE.
- CloudWatch server-error rate
- Token usage metric
- Service availability canary
- End-user interaction tracking by model version
Correct Answer: D
Correct Answer
Answer D is correct because End-user interaction tracking by model version is the strongest signal because it captures production interaction signals such as retries, ratings, corrections, or abandonment and associates them with the model or release serving the session. Infrastructure is stable, but the team needs to detect user-experience regression by model version. Interaction behavior can reveal degradation that errors and latency do not.
Incorrect Answers
Answer A is incorrect because CloudWatch server-error rate provides a different operational view because it tracks service-side failures. That matters for availability and reliability monitoring, but it cannot establish the required ability to detect user-experience regression by model version.
Answer B is incorrect because Token usage metric is a sound control for usage and cost monitoring; it tracks model token consumption over time. The stem asks for a mechanism to detect user-experience regression by model version, leaving this option insufficient.
Answer C is incorrect because With Service availability canary, the team can address availability monitoring because it sends synthetic requests to verify that an endpoint is reachable and returns acceptable service-level responses. The actual decision is how to detect user-experience regression by model version, so the two controls are not interchangeable.
Question 20
Bedrock spend varies with traffic and model mix, making one static monthly threshold too coarse. Finance and operations need proactive notice when current spend becomes unusually high relative to learned historical patterns. What should they configure? Choose ONE.
- AWS Budgets static monthly threshold
- AWS Cost Anomaly Detection
- Per-model cost attribution
- CloudWatch token usage anomaly detection
Correct Answer: B
Correct Answer
Answer B is correct because AWS Cost Anomaly Detection fits the variable-spend pattern because it uses machine-learning-based spend monitoring to identify unusual cost patterns relative to historical behavior and can notify subscribed teams. Finance needs to detect unusual spend relative to historical behavior; learned cost behavior is more suitable than one static monthly ceiling.
Incorrect Answers
Answer A is incorrect because AWS Budgets static monthly threshold has a legitimate role in financial guardrails against a known budget ceiling: it compares actual or forecast spending with a predefined budget amount. In contrast, this case requires the team to detect unusual spend relative to historical behavior, so the option acts on the wrong signal.
Answer C is incorrect because Per-model cost attribution helps with identifying expensive models or workloads since it separates spend by model, application, or tagged usage dimension. That is not the failure described here; the team needs to detect unusual spend relative to historical behavior.
Answer D is incorrect because CloudWatch token usage anomaly detection supports operational token-spike monitoring because it detects unusual token-consumption patterns with a learned CloudWatch metric band. The requested outcome is to detect unusual spend relative to historical behavior, which this option does not deliver.
