Category Archives: Data & Analytics
When preparing for Databricks GenAI Engineer Associate, the Databricks feature historically called Mosaic AI Agent Evaluation now sits inside the broader MLflow 3 generative-AI workflow. That change is more than branding. Current Databricks guidance exposes evaluation through the mlflow.genai namespace, combines offline testing with tracing and production monitoring, and treats evaluation datasets, scorers, human feedback, […]
For Databricks Data Engineer Associate candidates, streaming tables in Databricks provide a declarative way to maintain incrementally updated datasets from continuously arriving data. In the current Lakeflow Spark Declarative Pipelines model, a developer defines the target table and its transformation while the pipeline manages execution, dependencies, checkpointing, retries, and incremental progress. The result is different […]
MCP Toolbox for Databases is the current name for the open-source project previously called Gen AI Toolbox for Databases. It acts as a Model Context Protocol server between AI clients and data systems, handling authentication, connection pooling, tool definitions, and observability so agents do not need a custom integration for every database. In Generative AI […]
Microsoft OneLake shortcuts let Fabric expose data that already lives elsewhere without first copying it into a new lakehouse. A shortcut behaves like a symbolic link: the object appears as a folder or table inside OneLake, but the target remains in its original location. Fabric workloads can then access the target through OneLake’s unified namespace […]
Unity Catalog is the governance layer built into Azure Databricks for data and AI assets. It centralizes object ownership, access control, lineage, auditing, and discovery across workspaces, which makes it valuable when a platform grows beyond one team or one lakehouse. The design challenge is turning that control plane into clear ownership and enforceable boundaries […]
Adaptive Query Execution (AQE) re-optimizes Spark SQL physical plans while a query is already running. For Data Engineer Associate, the important consequence is that runtime statistics collected after exchanges can change join strategy, partition sizing, skew handling, and empty-relation behavior when the initial plan was based on incomplete estimates. From a Databricks Data Engineering design […]
Generative AI on Databricks now spans far more than model serving. Those serving, retrieval, evaluation, governance, and application-identity decisions are central to GenAI Engineer Associate preparation. Production applications can combine Foundation Model APIs, Unity Gateway model services, Databricks Apps, AI Search and Delta Sync indexes, online features, MLflow Prompt Registry, inference tables, tracing, scorers, and […]
Chunking turns source documents into the retrieval units an AI Search index can return. For GenAI Engineer Associate, the production decision is how to preserve the source relationships an answer needs while creating passages small enough to retrieve and rank precisely. Within Databricks GenAI, chunking sits before AI Search and model prompting. The cross-platform principles […]
Databricks Apps uses OAuth 2.0 and separates the app’s dedicated service principal from the signed-in user. For GenAI Engineer Associate, the important operating boundary is that app authorization lets the application act as itself, while user authorization lets supported calls preserve the interacting user’s Databricks identity and existing permissions. As part of the Databricks GenAI […]
Databricks Foundation Model APIs gives applications hosted access to foundation models without requiring each team to operate inference infrastructure. For GenAI Engineer Associate, the architectural decision is how serving mode, model capability, routing, identity, evaluation, and cost controls fit together once model access becomes a managed platform service. At the Databricks GenAI layer, Foundation Model […]
Databricks inference tables make runtime AI traffic queryable by logging model-service requests and responses into Unity Catalog Delta tables. In GenAI Engineer Associate, that evidence matters because debugging, cost attribution, routing analysis, compliance, and production evaluation all depend on knowing what actually traversed the serving layer. Current designs should use Unity Gateway inference logging rather […]
Databricks routing now belongs to the Unity Gateway model-service control plane rather than only to legacy endpoint configuration. For GenAI Engineer Associate, the important boundary is that clients can call one governed service name while platform teams control traffic splitting, session affinity, destination changes, and fallback behind that contract. For teams working in Databricks GenAI, […]
Databricks AI Search Delta Sync indexes keep retrieval state aligned with a governed source table. For GenAI Engineer Associate, the operational decision is whether index freshness should be maintained continuously or through triggered synchronization, and how embedding ownership, metadata, source changes, and recovery affect that contract. The Databricks GenAI architecture puts this in context: vector-search […]
Feature stores are traditionally associated with predictive ML, but governed structured features can also supply GenAI applications with reusable business context such as customer tier, risk state, inventory status, or recent activity. In GenAI Engineer Associate, the key distinction is that these deterministic facts should complement retrieval and model reasoning rather than be recreated inside […]
Databricks AI guardrails are implemented through Unity Gateway service policies, currently a Beta capability that can govern model, model-provider, and MCP service interactions. For GenAI Engineer Associate, the architectural distinction is between access control—who may invoke a service—and content policy—whether a particular request or response should be allowed, denied, redacted, or otherwise constrained. In practical […]
