Generative AI on Databricks now spans far more than model serving. Production applications can combine Foundation Model APIs, Unity Gateway model services, Databricks Apps, AI Search and Delta Sync indexes, online features, MLflow Prompt Registry, inference tables, tracing, scorers, and service-policy guardrails inside one governed platform. The engineering task is deciding which layer owns model access, application identity, retrieval, prompt versioning, quality monitoring, routing, and safety.<\/p>\n
This hub organizes that system for the Databricks<\/a> ecosystem. It is the parent for the new cluster covering RAG chunking, Databricks Apps authentication, Foundation Model APIs, inference tables, model-serving routes, AI Search synchronization, feature stores for GenAI, guardrails, MLflow Prompt Registry, and MLflow scorers. Later child pages extend the cluster into Agent Evaluation, Agent Framework, Model Serving, Vector Search, online feature stores, prompt management, query rewriting, and reranking.<\/p>\n The existing Databricks data and AI stack behind production RAG<\/a> article provides a broad production context, while MLflow for GenAI<\/a> provides the observability\/evaluation perspective. This hub focuses on how the pieces fit into a modern GenAI application lifecycle.<\/p>\n Foundation Model APIs gives Databricks users access to hosted foundation models without running their own deployment. Pay-per-token endpoints are the easiest entry point, while current Databricks guidance recommends priority pay-per-token for latency-sensitive production workloads that need more consistent performance, and provisioned throughput when the workload requires dedicated performance characteristics.<\/p>\n Databricks Foundation Model APIs<\/a> goes deeper into the serving modes, API surface, quotas, model availability, and compliance considerations. The important platform decision is whether the application should call a ready-to-use Databricks-hosted model, a provisioned Databricks model, or an external provider governed through Unity Gateway.<\/p>\n That model choice should remain swappable behind a stable service contract where possible. Hard-coding provider-specific details throughout application code makes migration, routing, cost control, and fallback harder.<\/p>\n Current Databricks documentation positions Unity Gateway as the central AI gateway for models, providers, MCP services, tools, skills, and guardrails. It can route inference traffic, apply access controls, attach service policies, log requests and responses, manage rate limits and budgets, and configure traffic splitting or fallbacks across model services.<\/p>\n Databricks Model Serving Routes<\/a> focuses on the routing layer. Current terminology increasingly uses model services<\/em> and Unity Gateway rather than treating every routing decision as a property of one legacy model-serving endpoint.<\/p>\n The architecture should therefore distinguish the stable application-facing service name from the model destinations behind it. That creates room for A\/B testing, provider migration, capacity failover, and session affinity without forcing every client to change its model identifier.<\/p>\n Databricks Apps has two complementary authorization modes. Every app has its own dedicated service principal for app-level operations. User authorization can also pass the signed-in user\u2019s Databricks identity through so the application can enforce existing Unity Catalog permissions, row filters, and column masks.<\/p>\n Databricks Apps Authentication<\/a> explains when to use the app service principal, when to use user authorization, and where the workspace boundary matters. A background task should not impersonate a user merely because the app has access to their session, and a user-specific query should not automatically run under one shared app identity if per-user governance matters.<\/p>\n Identity should remain explicit through the full request path, including model calls, SQL, feature lookups, and downstream tools.<\/p>\n Vector search and retrieval quality depend heavily on how source documents are split and represented before indexing. Chunking for Databricks RAG<\/a> focuses on structure-aware chunking, overlap, metadata, identifiers, update behavior, and the interaction between chunk size and embedding context.<\/p>\n The existing Databricks AI Search<\/a> article reinforces the same principle: query-time tuning cannot recover document relationships discarded during ingestion.<\/p>\n Later cluster pages on query rewriting and reranking extend that retrieval pipeline, but chunking remains the first major quality decision because every later stage operates on the units created at ingestion time.<\/p>\n Databricks AI Search supports Delta Sync indexes that incrementally follow source Delta tables and Direct Vector Access indexes where the application manages updates explicitly. Delta Sync can run in continuous mode for low-latency freshness or triggered mode when the application wants controlled refresh windows.<\/p>\n Databricks Vector Search Sync<\/a> covers the operational differences between standard and storage-optimized endpoints, continuous and triggered sync, source-column selection, managed embeddings, and the consequences of source-table change patterns.<\/p>\n Freshness should be treated as a retrieval SLO. A healthy endpoint with a stale index can still produce wrong answers.<\/p>\n Current Unity Gateway inference tables log model-service requests and responses into Unity Catalog Delta tables. They are billed Unity Gateway features and can support debugging, monitoring, optimization, and compliance use cases.<\/p>\n Databricks Inference Tables<\/a> explains the current experience and the retirement of the old legacy inference-table path, which Databricks stopped supporting in April 2026. Modern implementations should use the Unity Gateway inference-table model.<\/p>\n Inference logs can contain sensitive prompts and outputs, so access, sampling, retention, and redaction should be designed with the same care as application data.<\/p>\n MLflow Prompt Registry treats prompts as versioned assets in Unity Catalog. Versions are immutable, aliases are mutable pointers, tags and commit messages support change tracking, and deployed applications can resolve aliases such as MLflow Prompt Registry<\/a> explains that lifecycle. The current feature is Beta, but the architectural principle is mature: prompts should have versions, lineage, evaluation evidence, and rollback rather than being edited silently inside application source or dashboards.<\/p>\n The existing prompt management at application scale<\/a> article provides the broader cross-platform context.<\/p>\n MLflow 3 scorers provide a common interface for evaluation. Built-in LLM judges cover common dimensions such as relevance, safety, retrieval relevance, groundedness, and correctness; custom LLM judges and code-based scorers provide domain-specific or deterministic checks.<\/p>\nModel access should be governed as a platform service<\/h3>\n
Unity Gateway is now the AI control plane<\/h3>\n
Applications need both app identity and user identity<\/h3>\n
RAG quality starts before AI Search<\/h3>\n
AI Search sync turns Delta tables into governed retrieval indexes<\/h3>\n
Inference tables make runtime behavior queryable<\/h3>\n
Prompt management should become a versioned release artifact<\/h3>\n
production<\/code> without hard-coding one version.<\/p>\nQuality evaluation should use the same criteria before and after deployment<\/h3>\n