# Columns in epoch_ai_benchmarks Dataset Based on the evidence gathered, the `epoch_ai_benchmarks` table contains **69 columns total**. Here's what I can document from the data: ## Complete Column List (30 of 69 shown in detail) ### Identifiers - **id_runs** (VARCHAR) - Unique identifier for benchmark runs; ~976 distinct values ranging from "298stsbTPpZp7toZsS5cp8" to "pre-release-gpt-5-4-xhigh-fm-t4" - **id_model_version** (VARCHAR) - Model version identifier; ~325 distinct values ranging from "DeepSeek-R1" to "zai-org/GLM-4.7" - **job** (VARCHAR) - GitHub Actions job URL; ~899 distinct values ### Core Benchmark Data - **task** (VARCHAR) - Benchmark task name; ~15 distinct values ranging from "Chess Puzzles" to "SimpleQA Verified" - **model** (VARCHAR) - Model identifier; ~325 distinct values ranging from "DeepSeek-R1" to "zai-org/GLM-4.7" - **Best score (across scorers)** (DOUBLE) - Primary performance metric; ~797 distinct values ranging from 0.0 to 1.0 - **Scores** (VARCHAR) - Detailed scorer results; ~529 distinct values (e.g., "agent_submitted_next_move_scorer:0.17±0.04") - **Status** (VARCHAR) - Run status; only 1 distinct value: "Success" ### Temporal Data - **started_at** (TIMESTAMP) - Benchmark run start time; ~1,171 distinct values, 4 nulls, ranging from 2025-01-27 00:00:00 to 2026-08-07 01:06:53 - **Version release date** (DATE) - Model release date; ~149 distinct values ranging from 2023-03-14 to 2026-08-02 ### Model Metadata - **Model name** (VARCHAR) - Human-readable model name; ~186 distinct values ranging from "AI Co-Mathematician" to "phi-3-medium 14B" - **Display name** (VARCHAR) - Display name for UI; ~177 distinct values - **Unique display name** (VARCHAR) - Unique display identifier; ~195 distinct values - **Organization** (VARCHAR) - Model creator organization; ~21 distinct values ranging from "01.AI" to "xAI" - **Domain** (VARCHAR) - Model domain(s); ~20 distinct values (e.g., "Multimodal,Vision,Language,Image generation") - **Model task** (VARCHAR) - Specific model capabilities; ~75 distinct values (e.g., "Visual question answering,Image captioning,Object detection") ### Token Usage Metrics - **billable_input_tokens** (VARCHAR) - Input token count; only 1 distinct value (empty string) - **billable_output_tokens** (VARCHAR) - Output token count; only 1 distinct value (empty string) - **cache_read_tokens** (VARCHAR) - Cache read tokens; only 1 distinct value (empty string) - **cache_write_tokens** (VARCHAR) - Cache write tokens; only 1 distinct value (empty string) ### Additional Metadata - **Manual evaluation** (BOOLEAN) - Whether manually evaluated; 1 distinct value ("true"), 1,305 nulls - **task version** (VARCHAR) - Version of the benchmark task; ~32 distinct values up to "2.0.8" - **comments** (VARCHAR) - Run-specific notes; ~25 distinct values - **Hugging Face developer id** (VARCHAR) - HuggingFace organization; ~14 distinct values up to "moonshotai" - **Link** (VARCHAR) - Reference URL; ~69 distinct values - **Notes** (VARCHAR) - Model-specific notes; ~66 distinct values - **Authors** (VARCHAR) - Model authors list; ~41 distinct values - **benchmarks/runs** (VARCHAR) - Related benchmark run IDs; ~305 distinct values ### URLs and References - **log viewer** (VARCHAR) - Link to log viewer interface; ~844 distinct values - **logs** (VARCHAR) - Direct log file URLs; ~1,078 distinct values ## Additional Columns (39 remaining) From step-2 evidence, additional columns include: - **Abstract**, **Accessibility notes**, **Base model**, **Batch size**, **Batch size notes** - **Citations**, **Confidence**, **Country (of organization)**, **Dataset size notes** - **Epochs**, **Finetune compute (FLOP)**, **Finetune compute notes** - **Hardware quantity**, **Hardware utilization**, **Link_model** - **Model accessibility**, **Notability criteria**, **Notability criteria notes** - **Parameters**, **Parameters notes**, **Publication date**, **Reference** - **Training code accessibility**, **Training compute (FLOP)**, **Training compute cost (2023 USD)** - **Training compute estimation method**, **Training compute notes** - **Training dataset**, **Training dataset notes**, **Training dataset size (datapoints)** - **Training hardware**, **best_score** ## Data Quality Notes - Most columns have 0 nulls except: **started_at** (4 nulls), **Manual evaluation** (1,305 nulls) - Token-related columns appear to be empty (only empty string values) - The dataset spans benchmark runs from January 2025 through August 2026 **Limitation**: Detailed statistics for 39 of the 69 columns were not fully captured in the investigation.