-
Unity Catalog, RBAC, and ABAC — Data Governance that Scales
A practical look at how Databricks Unity Catalog unifies data governance across catalogs, schemas, and external storage — from role-based access control (RBAC) to attribute-based access control (AB...
-
Medallion Architecture: Bronze → Silver → Gold in Practice
A practical, code-backed guide to implementing the Medallion Architecture—how Bronze staging, Silver refinement, and Gold analytics fit together in a real lakehouse with governance and time travel ...
-
Time Travel with Delta-RS: Local Experiments with MinIO, Azurite, and Databricks
Exploring Delta Lake’s Time Travel feature across environments — MinIO over HTTPS, Azurite over HTTPS, and Databricks SQL — using Python’s delta-rs library for local reproducibility.
-
Delta-RS Proof: Why Late Data Matters More Than You Think
How Delta-RS proves its value when late-arriving data threatens the integrity of business reporting — a story from the NYC Taxi data pipeline.
-
Emulating Cloud Storage and Orchestration Locally with MinIO, Airflow, and Delta-RS
Building a full-fledged, ___zero-cost data-engineering lab___ using MinIO for S3 storage, Airflow for orchestration, and Delta-RS for ACID transactional integrity in local Delta Lake tables.
-
Defining the NYC Taxi OLTP Core — A Modern ER Skeleton for Analytics-Ready Data
A blueprint for transforming the raw NYC Taxi dataset into a clean, normalized OLTP schema—built for integrity, extensibility, and analytical evolution. This is where transactional truth meets anal...
-
Parquet: _The Everywhere File Format_—What It Is, Why, When and How? (_Draft_)
In this post, I unpack what Parquet actually is—why it’s everywhere across Azure, Databricks, AWS, GCP, Snowflake, Power BI, and SQL Server—and how tools like DuckDB, Python, and even S...
-
Mastering Databricks — Day 1 (Parquet, SQL Warehouse, Volumes, and Delta)
A fast, DBA-rigorous walk from first login to a real Delta table on Databricks Free Edition. Validate the SQL Warehouse, inspect sample data, upload and ingest Parquet files—two NYC Taxi file...
-
Azure Migration & Databricks Acronyms Reference
A concise glossary of essential Azure, SQL, and Databricks acronyms.
-
Airflow Setup Lab – Pages 1–6 Recap (Draft)
Step-by-step setup of an Apache Airflow lab with CeleryExecutor: baseline cluster running, volume mounts, smoke tests, Flower monitoring, and preparation for the first DAG run.
-
Windows Server FCI Lab – Rolling SQL Server CU on FCI
Rolling installation of the latest SQL Server Cumulative Update across a two-node WSFC Failover Cluster Instance: planned failover to SQLNODE3, patching SQLNODE2 then SQLNODE3, validation of connec...
-
Windows Server FCI Lab – WSFC Achieved
Documenting the pivotal step of bringing Windows Server Failover Clustering (WSFC) online: from domain health, SMB shares, and virtual disks to cluster validation and creation. This post stops just...
-
NYC Taxi Pick-ups Choropleth in D3.js using GeoJSON MultiPolygon
Interactive NYC Taxi choropleth built from the official Taxi Zone GeoJSON (MultiPolygon). This post renders one of four quadrants, coloring zones by trip counts with a sequential ramp. Legend uses ...
-
NYC Taxi Pick-ups Choropleth in D3.js using GeoJSON MultiPolygon
Interactive NYC Taxi choropleth built from the official Taxi Zone GeoJSON (MultiPolygon). This post renders one of four quadrants, coloring zones by trip counts with a sequential ramp. Legend uses ...
-
Using GeoJSON MultiPolygon and D3.js
GeoJSON scaffolding
-
Windows Server FCI Lab
Building a step-by-step Windows Server 2022 Failover Cluster Instance (FCI) lab: from VM setup, patching, networking, and PowerShell automation, edits and verification, to creating a reusable playb...
-
Word-clouds - randomized
Grid Word‑Cloud Cards
-
Azure Migration Playbook: SQL Server On-Prem to Azure (First Draft)
A practical first-draft playbook for planning, executing, and validating SQL Server migrations from on-premises to Azure — with emphasis on HA/DR, rollback, cost-control, and documentation.
-
Phase 3 — First Data Landed in PostgreSQL
I’ve loaded a large public dataset into PostgreSQL through a clean, repeatable workflow. The process included acquiring the raw data in Parquet format, converting it to PSV with Python, generating ...
-
SQL Server JSON SPs → Minimal .NET API → Postman
Stored procedures emit JSON via OUT params; a tiny .NET 8 Minimal API exposes them as endpoints you can validate with curl and Postman.
-
SQL Server Naming Conventions — PascalCase, camelCase, and API Standards
The standardized naming approach for SQL Server objects, columns, and API outputs—balancing database clarity with application developer conventions.
-
NYC Taxi: Advanced SQL — Analytics to JSON for APIs
Turn dashboard metrics into clean, nested JSON payloads using FOR JSON PATH, without leaving SQL Server.
-
Stored Procedures That Write Stored Procedures (CRUD Generator)
Production-safe T-SQL code generators that emit INSERT/UPDATE/DELETE/SELECT procedures from live table metadata—parameterized, TRY/CATCH-wrapped, and QUOTENAME-hardened.
- Job Titles Flatten Careers
- Eight Essential Queries for NYC Yellow Taxi Data
-
AWS, Microsoft, and Google Back Linux Foundation DocumentDB to Reduce Lock-In
AWS, Microsoft, and Google are backing a Linux Foundation DocumentDB initiative to make document databases more portable and community-governed. The move targets vendor lock-in by standardizing API...
-
NYC Taxi Database Project
The NYC Taxi Database Project demonstrates a full data-engineering pipeline built from messy real-world trip data. Starting with Parquet files, we generate PSV conversions, schema validations, and ...
-
NYC Taxi Data: First Steps in Validation
Every data pipeline starts with trust in the data. In this first step with the NYC Taxi dataset, We step through schema checks, column profiling, and early validation techniques that catch errors b...
-
Kicking Off My Technical Blog
Welcome! 🎉
