```json { "table_name": "pipeline_stage_status", "primary_key": "id", "candidates": [ { "rank": 0, "partition_key": "dataset_id", "partition_key_label": "E", "partition_key_reasoning": "RPUV=12.83 is in the ideal range (2-100). Top 3 coverage=23.75% is well below 50%. Semantically, dataset_id is the primary entity whose pipeline execution timeline we trace. Each dataset goes through multiple pipeline stages across multiple batches.", "clustering_key": "started_at", "clustering_key_label": "S", "clustering_key_reasoning": "Uniqueness=1.000 (each event has a distinct start time). This is the temporal ordering axis for pipeline stage events. Sorting by started_at within each dataset_id groups all pipeline stages chronologically, enabling analysis of pipeline execution flow.", "world_line": "dataset_id", "session_column": "batch_id", "confidence": "HIGH", "notes": "Primary perspective: dataset-centric pipeline execution log. Each dataset (E) has a timeline of pipeline stages (S=started_at). batch_id serves as a natural session boundary grouping related stages within a single pipeline run. The id column is a unique surrogate key for each stage execution event." }, { "rank": 1, "partition_key": "batch_id", "partition_key_label": "E", "partition_key_reasoning": "RPUV=6.95 is in the acceptable range. Top 3 coverage=3.06% is very low. Semantically, batch_id represents a single pipeline execution run. This perspective groups all stages that occurred within the same batch/run, useful for analyzing pipeline run completeness and stage sequencing.", "clustering_key": "started_at", "clustering_key_label": "S", "clustering_key_reasoning": "Same temporal ordering as rank=0. Within each batch, started_at orders the pipeline stages in execution sequence (download → bronze → classify → tags → analytics → search → catalog → gold → sample_queries → dashboards → reconcile).", "world_line": "batch_id", "session_column": null, "confidence": "MEDIUM", "notes": "Alternative perspective: batch-centric pipeline execution log. Useful for analyzing individual pipeline runs, tracking stage progression within a single batch, and identifying failed or incomplete runs. Less natural than dataset_id because batches are ephemeral execution contexts rather than persistent entities." } ] } ```