Unity Catalog row filters restrict which rows a user can see from a table at query time. A row-filter UDF evaluates attributes from each row and the current identity or group context; rows that evaluate to false are removed from the result. Current Databricks supports manual table-level row filters and the newer recommended ABAC policy model, where governed tags and hierarchical policy scope apply filtering consistently across many tables.
Within Databricks Data Engineering, row filters are the primary query-time mechanism for region, department, tenant, legal-entity, account, or consent-based row-level security without creating one physical table per audience.
The policy must remain understandable because a logically wrong filter can either hide legitimate records or expose another tenant’s data while every query still succeeds technically.
Manual row filters attach one SQL UDF to one table
The table owner can bind a SQL UDF as the row filter for a specific table.
The function can take one or more table columns as arguments and use identity/group functions to return true or false.
This is practical for isolated tables but creates duplicated policy when the same rule applies throughout a catalog.
ABAC policies scale row security through governed tags
ABAC row-filter policies can be attached at catalog or schema level and apply automatically to matching tagged tables.
Current Databricks recommends ABAC for policies that should follow data consistently rather than be manually attached one table at a time.
Use a small governed taxonomy and central policy ownership so producers cannot accidentally create unfiltered replicas simply by adding another table.
Identity and group membership are evaluated at query time
Different users can run the same SQL and receive different row sets based on their identity, group memberships, tags, and policy.
This makes directory/SCIM group lifecycle part of data security.
Test access after adding and removing users from entitlement groups, not only the SQL logic under an administrator account.
Mapping tables can model complex entitlements
When access cannot be expressed by one group-to-region rule, a UDF can consult an entitlement mapping table.
Keep the mapping small, indexed/optimized appropriately, and owned by the security/data-governance process because every query may depend on it.
A stale entitlement table creates stale data access even if the row-filter code is perfect.
Filter functions should be simple enough for the optimizer
Complex UDF logic can reduce query optimization opportunities and add runtime cost.
Databricks guidance includes best practices and limitations for row-filter/mask UDFs; use deterministic SQL expressions and avoid unnecessary external complexity.
Measure query plans and latency on high-volume tables before applying one policy broadly across a catalog.
Compute requirements are part of the enforcement boundary
Current ABAC row-filter policies require serverless compute, Standard compute on Databricks Runtime 16.4+, or supported Dedicated 16.4+ compute with fine-grained access control.
Older runtimes and unsupported access paths should not be allowed to become the way users bypass policy.
Use compute policies and workspace governance to keep sensitive datasets on enforcement-capable compute.
Row filters can affect joins and aggregates in non-obvious ways
A filtered fact table changes counts, sums, joins, and distinct values before the user’s query logic sees the rows.
Business dashboards should be validated under each major access role because totals can differ intentionally by audience.
Document filtered metrics so users do not mistake access-scope differences for data-quality inconsistency.
Filters should be fail-closed for missing entitlement metadata
If a row has a null or unrecognized tenant/region classification, the safest policy is usually to hide it from ordinary users until ownership is resolved.
Do not treat missing security metadata as “public.”
Data-quality pipelines should quarantine or alert on rows that lack the attributes required by the security policy.
Row filters and column masks can be combined
A user may be allowed to see all rows in their region but only masked email/SSN values within those rows.
Unity Catalog Column Masks covers the value-level control.
Design both controls from one entitlement/classification model so policy combinations do not produce contradictory behavior.
Audit and policy visibility should support incident response
Keep the policy, governed-tag assignments, UDF code, group membership, table ownership, and access logs available for investigation.
If a user reports seeing another region’s data, responders should be able to reconstruct which policy and identity attributes were effective at the query timestamp.
Version security UDFs and policy changes through controlled deployment rather than ad hoc SQL in production.
Row filters succeed when tenant and business boundaries remain invisible to application SQL
The mature platform centralizes reusable ABAC policy, keeps entitlement data current, uses supported compute, tests all role combinations, monitors policy performance, and audits changes.
Applications should query the same governed table while Unity Catalog deterministically narrows the result to the rows the caller is entitled to see.
Tenant filters should be based on immutable tenant identifiers, not display names or mutable attributes. A mapping from user/group to tenant ID should be centrally governed and tested for uniqueness. Free-form strings invite case mismatches and can become security defects when two business entities have similar names.
ABAC scope should be chosen carefully. A catalog-level row policy can be powerful for an entire regulated domain, but one incorrectly tagged table can suddenly inherit logic it was not designed for. Start at schema or narrower scope, verify coverage/performance, then expand once the governed-tag model is reliable.
Policy UDFs should default deny on errors where security requires it. If an entitlement lookup returns null or a mapping table is unavailable, returning true to preserve availability can expose data. Define failure semantics explicitly and test them; secure row filtering is as much about error behavior as the normal boolean expression.
Performance can improve when entitlement logic is reduced to simple comparisons against columns already present in the row. Repeated joins to a large entitlement table on every query add complexity. Consider materializing stable classification keys or using compact lookup tables maintained by the identity pipeline.
Data producers should not be able to rewrite the protected tenant/region column casually. If the row-filter key is just another mutable field, an incorrect ETL update can move records into another user’s visibility. Add data-quality checks, constraints where applicable, and ownership separation around security-classification columns.
Views can still be useful above filtered tables. Business views can simplify schemas while row filters enforce the underlying security boundary. Test that filters remain effective through views, joins, and derived queries and that users cannot reach the unfiltered base table through an alternate schema or external path.
Aggregates can leak information even when raw rows are filtered correctly. Very small cohorts, repeated queries, or differencing can reveal sensitive attributes. For privacy-sensitive analytics, combine row filters with output aggregation thresholds or clean-room-style controls rather than assuming row-level security solves inference risk.
Policy reviews should include ‘who sees zero rows?’ as well as ‘who sees rows.’ A user accidentally omitted from all entitlement mappings may interpret an empty dashboard as no business activity. Monitoring and support should distinguish legitimate empty scope from identity/policy misconfiguration.
Row-level policy should be validated with joins across multiple filtered tables. If customer and transaction tables use different tenant keys or entitlement logic, joins can create confusing partial results. Standardize tenant/security keys across the domain and test multi-table analytics under each major role.
Service principals need the same scrutiny as users. ETL/BI tools often query as machine identities with broad groups or ownership. Decide whether those identities should see all rows for data processing or remain tenant-scoped, and prevent broad service identities from becoming accidental bypass channels for end-user applications.
ABAC policies can dynamically respond to governed-tag and group changes at query time, which is powerful but increases change sensitivity. Tag-management and group-membership changes should be audited as security configuration and, for high-risk data, reviewed like code changes.
Row-filter incident drills should include a deliberate mis-tagged table or entitlement mapping error. Verify that monitoring catches unexpected row-count/access changes and that rollback can restore the previous policy quickly. Fine-grained governance should have operational recovery just like any other production control.
Testing should include table owners, metastore admins, service principals, ordinary users, and users who belong to multiple entitlement groups. Mixed membership can produce broader access than one-role tests reveal. Build a matrix of expected row counts or known keys by persona and run it whenever policy UDFs, governed tags, or group mappings change.
Documentation should state whether the filter is intended for security isolation or merely convenience. Security-critical tenant isolation demands fail-closed behavior, strict change control, and monitoring; a convenience filter for regional dashboards may tolerate different failure semantics. Using the same mechanism for both does not make the risk identical.
Row-filter policy should be part of disaster-recovery and cross-workspace testing. Replicated or promoted catalogs, jobs, and user groups must retain the same governed tags and policy objects in the recovery environment. A DR copy that contains all rows but loses tenant filtering is not a valid recovery target.
Keep row-filter coverage in automated governance checks so newly created tables and security-key columns are validated before users receive access.
Row filters need the same rigor as authorization code. Test positive access, denied access, role changes, group membership delays, and administrative exceptions, because a filter that is logically correct can still fail if the identity context feeding it is stale or incomplete.
