Category Archives: AI & Machine Learning
Amazon Bedrock Guardrails sensitive-information filters detect personally identifiable information and can either block the entire prompt/response or mask recognized values before they pass through the conversational flow. Current AWS documentation describes this as a context-dependent machine-learning classifier over text, with built-in PII categories plus custom regular expressions for deterministic patterns. Within Generative AI on AWS, […]
NUMA matters for GPU workloads because CPUs, system memory, GPUs, NICs, and NVMe controllers are physically attached to specific sockets and PCIe root complexes. A process can run correctly while repeatedly crossing the inter-socket link to reach memory or a device, adding latency and consuming bandwidth before data even reaches the GPU. Within NVIDIA AI […]
Amazon Bedrock Guardrails is a policy layer for evaluating model input and output against configurable safety, privacy, and grounding rules. Current AWS documentation includes content filters, denied topics, word filters, sensitive-information filters, image content filters, contextual-grounding checks, and automated-reasoning checks. Guardrails can be attached to supported model-inference flows or invoked independently through the Guardrails runtime […]
GPU scheduling is the process of deciding which workload gets which accelerator, for how long, and with what degree of sharing or isolation. In Kubernetes, GPUs are usually exposed as extended resources. NVIDIA GPU Operator can add drivers, device plugin, feature discovery, DCGM components, MIG management, and sharing configuration so the scheduler has enough information […]
Amazon Bedrock inference profiles are model-invocation resources that either route requests across AWS Regions or give an application a named resource for tracking model usage and cost. Current Bedrock documentation separates system-defined cross-Region inference profiles from application inference profiles that customers create for single- or multi-Region model use. Within Generative AI on AWS, inference profiles […]
Choosing GPU versus CPU is a workload decomposition problem, not a brand or benchmark contest. GPUs excel when the application exposes large amounts of parallel arithmetic with regular data movement, while CPUs remain strong at latency-sensitive control flow, serial work, branch-heavy logic, orchestration, I/O handling, and tasks too small to amortize accelerator launch and transfer […]
Amazon Bedrock intelligent prompt routing sends each request to one of several foundation models based on a predicted response-quality difference and cost/quality trade-off. Instead of hard-coding one model for every prompt or building a custom classifier, an application calls a prompt router and Bedrock selects the model according to the router’s configuration. Within Generative AI […]
NVIDIA GPUDirect RDMA enables a third-party PCIe peer device such as a network interface to access GPU memory directly, avoiding a staging copy through host CPU memory. In distributed AI systems, this shortens the data path between RDMA NICs and GPU memory and is a key building block for high-performance communication over InfiniBand and RoCE. […]
An inference latency budget breaks the user-facing SLO into components that engineering teams can measure and optimize separately. A request might spend time in the client/network, ingress/load balancer, inference queue, input preprocessing, model execution, output processing, token streaming, and downstream application logic. If the product promises p95 under 500 ms, the infrastructure team cannot spend […]
Filtered vector search in Azure AI Search combines similarity retrieval with ordinary filterable metadata such as tenant, category, language, security label, date, or document type. The important design choice is not simply whether a filter exists, but when it is applied relative to vector search. Current Azure AI Search supports preFilter, postFilter, and preview strictPostFilter, […]
AI policy exceptions are temporary, documented approvals to operate outside a defined governance requirement under controlled conditions. They are necessary because delivery realities do not always align perfectly with policy timing, but unmanaged exceptions can quickly become permanent bypasses that undermine the whole control framework. Within AI Governance, the exception process should make deviation visible, […]
AI red teaming is an adversarial evaluation practice designed to uncover harmful behaviors, security weaknesses, misuse pathways, failure modes, and gaps that ordinary testing may miss. Governance determines which systems are red-teamed, who performs the testing, how realistic attacks can be, what data or users may be involved, how findings are handled, and which results […]
AI risk acceptance is the deliberate decision by an authorized owner to proceed with a defined residual risk after controls, evidence, and alternatives have been considered. It is not the absence of remediation, a missed deadline, or a statement that “all AI has risk.” NIST AI RMF treats risk tolerance as contextual and does not […]
An AI risk taxonomy is a common classification system for describing the sources, scenarios, impacts, affected parties, and control domains associated with AI risk. Its purpose is to let product, engineering, security, privacy, legal, audit, and leadership teams discuss the same risk portfolio without using the same word to mean different things. Within AI Governance, […]
AI governance is the operating system around how an organization decides where AI can be used, who is accountable for it, what evidence must exist before deployment, how risk is measured, how exceptions are approved, and when a system should be changed or retired. It is broader than a policy document and narrower than “ethics” […]
