Category Archives: Technology
Unity Catalog external locations make the connection between cloud object-storage paths and authorized credentials explicit. An external location references a storage credential and a path; the catalog then governs which principals may use that location to register or access managed and external data objects. The architecture is powerful precisely because an identity’s Databricks privileges can […]
Databricks SQL warehouses provide compute for interactive analytics, dashboards, and SQL-driven applications. Choosing their size is not simply a matter of finding the largest available compute tier. Query complexity, concurrency, start-up latency, serverless capabilities, cost controls, and workload isolation determine whether users experience a responsive service. A warehouse that looks powerful on paper can spend […]
Lakeflow Declarative Pipelines can apply data-quality expectations while transforming arriving records into governed tables. Expectations express conditions that data should satisfy, but their action matters as much as their predicate. A policy may record quality metrics, drop invalid records, or fail a pipeline update; those outcomes create different operational and business consequences. Quarantining suspect records […]
A Structured Streaming checkpoint preserves the state a query needs to resume processing, including its progress through supported sources and stateful operations. In Databricks, the checkpoint is an operational dependency rather than a disposable temporary folder. Deleting or reusing it blindly can cause duplicate processing, invalid state restoration, or a replay that no longer corresponds […]
DHCP snooping allows a Catalyst switch to distinguish trusted DHCP server-facing paths from client-facing ports, helping prevent unauthorized devices from distributing addresses and gateway settings. The control is straightforward in principle but fragile in complex access networks. An uplink that is trusted too broadly can permit rogue DHCP replies; an uplink left untrusted can block […]
Delta Change Data Feed (CDF) can expose row-level changes from a Delta table to incremental downstream consumers. Changes include information that distinguishes inserts, deletes, and update images, alongside commit metadata. This makes CDF useful for propagating changes into analytical stores and services, but a consumer’s recovery plan must respect the source’s history and retention. CDF […]
Amazon RDS Performance Insights helped database teams see where active database sessions were spending time and which SQL statements, waits, and hosts contributed to load. Its legacy name remains common in runbooks, but the service reached its announced end-of-life date on July 31, 2026. AWS now directs operators to CloudWatch Database Insights. A current troubleshooting […]
Amazon S3 Object Lock can protect individual object versions against deletion or overwriting, using time-bound retention or a legal hold. A legal hold does not automatically expire on a predetermined date; an authorized actor must remove it. That flexibility is useful during litigation, investigation, and preservation requests, but it also requires careful identity, scope, and […]
Amazon EKS Pod Identity lets workloads running in an EKS cluster obtain AWS credentials through a supported agent and IAM role association. It reduces the need to distribute long-lived access keys into pods, but it does not erase the distinction between a Kubernetes ServiceAccount and the AWS permissions assumed on behalf of the workload. A […]
AWS Key Management Service encryption context is a set of non-secret key-value pairs supplied with eligible cryptographic operations using symmetric encryption KMS keys. AWS KMS binds this context to ciphertext as additional authenticated data. A decryption request must supply matching context when it was used during encryption, so the context helps associate protected data with […]
An Amazon SQS dead-letter queue isolates messages that a consumer could not process within the source queue’s receive-attempt policy. The messages are valuable diagnostic evidence, but returning them to production processing without fixing the failure can restart the same incident. A redrive operation is therefore a controlled data reintroduction, not simply a convenient button for […]
OSPF distributes link-state information so routers within an area can build a consistent view of topology. Filtering routes at a redistribution boundary is different from preventing an internal link-state advertisement from being flooded. Confusing these operations can produce unintended reachability, black holes, or the false belief that a prefix is absent from every router simply […]
BGP communities attach policy information to routes so networks can influence routing treatment without creating a unique prefix rule for every destination. A community is not a direct forwarding instruction; routers must interpret it through configured import, export, and route-selection policies. The same tag can be useful at one boundary and meaningless at another if […]
Service control policies (SCPs) define permission guardrails across accounts in AWS Organizations. They set the maximum permissions that principals in affected accounts may use, but do not grant permissions on their own. That distinction is crucial when a new organizational security boundary is deployed. An SCP can be syntactically valid and still deny a legitimate […]
Modern AWS workloads rarely fit within one account. Development boundaries, business units, security controls, and application components may be separated for governance and isolation. CloudWatch cross-account observability allows a monitoring account to view supported metrics, logs, traces, and related telemetry from linked source accounts within a Region. This improves incident investigation across organizational boundaries, but […]
