Skip to content
kc@kumarChandrachooda.com:~$ cd /blog/archive && read --section="top" 0%
Archive

Every article

Newest first, grouped by year. Search or filter by topic.

  • 556 articles
Data Warehousing 19 Aug 2025

Running the Warehouse — the V7 Runbook

A pile of subsystems isn't an operable system until you can run it. KC Star's docs distil into a runbook - daily health checks, weekly index hygiene, monthly reindex and security review, PostgreSQL tuning, backups, and one consistency check that catches a whole class of corruption.

Data Warehousing 18 Aug 2025

Partitioning as a Backup Strategy

KC Star V7 ships two overlapping ways to partition and age out cold data - a static backup-partitioning subsystem and a dynamic one - both monthly range partitions on CreatedDate, both gated on one flag. The honest read on why a feature ended up implemented twice.

Data Warehousing 18 Aug 2025

Two Fix Scripts and What They Confess

Refactoring KC Star V6's monolith into V7's subfolders broke the schema - so badly it took two passes of fix scripts to repair. Missing columns, duplicate triggers, a trigger literally named _Corrected, and a live test INSERT embedded in a migration. What the scars teach.

Data Warehousing 17 Aug 2025

The Scheduler That Reads Its Own Health

KC Star V7 scores its own health from queue depth, processing time and success rate, then maps that score to a batch cadence - from 30 seconds when healthy down to emergency mode when critical - and writes the decision back into its own settings. Clever, and honestly limited.

Data Warehousing 16 Aug 2025

A Batch Has a Name, a Log, and a Deadline

In KC Star V7 every batch run gets a generated id, a log row, a processed count and a time budget it must finish inside. Here is the orchestrator that ties it together, the per-batch exception isolation - and a variable-shadows-column bug I shipped in the middle of it.

Data Warehousing 16 Aug 2025

Feature Flags Inside the Database

KC Star V7 ships a six-table feature-flag system so you can turn capabilities on and retune settings without a deploy. Here is the normalized flag model, the twenty-one seeded flags, and the honest gap between what the changelog documents and what the code ships.

Data Warehousing 15 Aug 2025

Turning Off the Triggers at Twenty Thousand Sales a Minute

The per-row triggers that ran KC Star for six versions stop scaling at high volume. V7 turns them off and moves fact generation into an explicit batch pipeline. Kicking off the ultra-scale series - batch processing, feature flags, automated monitoring.

Data Warehousing 14 Aug 2025

Backfilling the Facts the Pipeline Skipped

Two fix scripts in KC Star repair a V6 database where dimensions were hand-seeded but the fact-generation half of the pipeline never ran. A war story about half-run pipelines, the DO-block backfills that rescue them, and what a MANUAL_SETUP batch id confesses.

Data Warehousing 14 Aug 2025

The Changelog That Describes a Different System

The KC Star V6 changelog documents an entire RBAC system - tables, functions, triggers, password columns - that appears nowhere in the actual SQL. Here is how I found the drift, why it happened, and why I treat the folder code as ground truth and the changelog as fiction.

Data Warehousing 13 Aug 2025

The Aggregate That Censors Itself

KC Star V6's GetFilteredAnalyticalData returns different rows to different callers - each row self-selects by the caller's clearance. Two personas, one query, two answers. Here is how permission-filtered aggregates work, and where the filtering really lives.

Data Warehousing 12 Aug 2025

Hierarchy as an Ordinal

KC Star V6 decides access by comparing a single integer. An employee's highest LevelOrder against the level being requested - Global sees everything below it, an employee sees only themselves. Coarse, cascading, and a deliberate trade against per-object ACLs.

Data Warehousing 12 Aug 2025

Who Gets to See the Numbers

KC Star V6 layers role-based access control onto the org hierarchy - eight permission types, five hierarchy levels, and grants that expire. Kicking off the series where the warehouse learns that not everyone should see every number.

Data Warehousing 11 Aug 2025

The Analytical Star and Its Update Log

KC Star V5's third star is keyed by a polymorphic ReferenceId and a hierarchy-level string, fed by real-time triggers, and tracked by an update log with a PENDING to COMPLETED status - a queue that isn't actually a queue. Closing the V5 series.

Data Warehousing 10 Aug 2025

One Version, Two Implementations

KC Star V5 exists in two materially different builds under one version number - the folder tells a batch/SCD2 story, the root file tells a real-time story, and they don't contain the same objects. What I learned from letting variants drift instead of forcing them to agree.

Data Warehousing 10 Aug 2025

Rollups at Every Altitude

KC Star V5 climbs the org chart in a single view - employee to office to region to organisation - and a stored procedure that materialises the rollups. Commission analysis and quote-to-delivery ratios fall out of the same hierarchy.

Data Warehousing 09 Aug 2025

The Bridge Table with Two Passports

KC Star V5 pretends one PostgreSQL instance is two databases, joined by a single un-enforced integer. CrossDatabaseFact carries two reference keys at once, and eight windowed measures value a single sale at every altitude of the org chart.

Data Warehousing 08 Aug 2025

An Org Chart Walks into a Warehouse

KC Star V5 gives the sales star a sibling: a full organizational hierarchy — organisation, regions, offices, personnel — with its own dimension rows, its own fact rows, and its own SCD2. Kicking off the series on the version where one star became three.

Data Warehousing 08 Aug 2025

One Compose File, Four Databases

KC Star's dev stack stands up PostgreSQL and SQL Server with the schema auto-loaded through V4, pgAdmin pre-wired to the servers, and one command to a working warehouse. Here is how the compose file, the init scripts, and the pgAdmin auto-registration fit together.

Data Warehousing 07 Aug 2025

Facts That Show Their Work

A stored number that can't explain itself is a number you can only trust. KC Star V4 promotes CalculationContext to JSONB and adds statistical columns to aggregates, so every fact carries a queryable record of the inputs and formula that produced it.

Data Warehousing 06 Aug 2025

Two Ways to Remember Where a Fact Came From

KC Star V4 tracks lineage two incompatible ways depending on which build you run - a normalized FactLineage graph in the root file, a denormalized DataLineage string in the version folder. The fork is real, and it is a clean case study in when a graph pays and when a string is enough.

Data Warehousing 05 Aug 2025

A Quality Score on Every Row

KC Star V4 puts a DataQualityScore on every dimension and fact, defaulting to a perfect 1.0 and downgrading only on evidence. Here is the opt-out philosophy, the assessment procedures behind it, and the one query that finds every row the warehouse doesn't quite trust.

Data Warehousing 05 Aug 2025

The Version Where the Warehouse Grew a Conscience

Through V3, KC Star assumed every number it was fed was true. V4 stops assuming - adding data quality scores, lineage, and performance monitoring so the warehouse can say how much it trusts its own data. Kicking off the data governance series.

Data Warehousing 04 Aug 2025

Twenty Queries Against a Dual Star

V3 ships the largest usage-example set of the early versions - twenty queries against base facts and aggregates living in two tables. A tour of what a dual-star warehouse lets you ask, from base-vs-aggregate joins to watching a rollup evolve over time.

Data Warehousing 03 Aug 2025

Expire Everything, Rebuild Everything

V3's rewritten CreateAggregatedFacts expires every aggregate and rebuilds them all from scratch on each run. It is blunt, it is correct, and the trade between rebuild-it-all simplicity and incremental cleverness is one KC Star kept relitigating all the way to V8.

Data Warehousing 03 Aug 2025

FactCount and the Audit Trail of an Average

An average with no denominator is a number you can't defend. KC Star V3 adds a FactCount column to every aggregate - how many facts went into this rollup - turning each stored aggregate into something you can audit rather than just trust.