A Relational Model of Data for Large Shared Data Banks
Data as time-varying n-ary relations queried by predicate calculus, so programs survive changes in storage, indexes, and access paths.
Organization and Maintenance of Large Ordered Indices
The B-tree: a page-sized, always-balanced ordered index that stays cheap to search and to update on disc.
Retrospection on a Database System
A candid engineering retrospective on building INGRES: what a working relational DBMS actually cost, and which design choices were mistakes.
The Transaction Concept: Virtues and Limitations
Names atomicity, consistency and durability as the transaction's defining properties, then shows exactly where nested and long-lived transactions break them.
Eight Transaction Papers by Jim Gray
A retrospective walk through eight Jim Gray papers that built the transaction abstraction, from two-phase locking to Paxos Commit.
Looking Back at Postgres
A retrospective on Berkeley Postgres, showing how one extensible object-relational design seeded PostgreSQL and a generation of database systems.
GAMMA - A High Performance Dataflow Database Machine
The first working shared-nothing parallel database: one processor per disk, every relation partitioned, queries run as self-scheduling dataflow.
Parallel Database Systems: The Future of High Performance Database Processing
Shared-nothing hardware plus partitioned data and a split/merge dataflow runtime give relational queries near-linear speedup and scaleup.
ARIES: A Transaction Recovery Method Supporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging
ARIES made crash recovery correct and fast under record-level locking by repeating history, then undoing losers with redo-only compensation records.
Data Cube: A Relational Aggregation Operator Generalizing Group-By, Cross-Tab, and Sub-Totals
The CUBE operator: one SQL clause that computes every group-by over N dimensions at once, and returns it as a relation.
The Log-Structured Merge-Tree (LSM-Tree)
A disk index that defers and batches inserts into cascading sorted merges, cutting disk-arm cost by nearly two orders of magnitude.
The Part-Time Parliament
Paxos: a majority-quorum protocol that keeps replicated logs consistent through crashes and lost messages, and progresses when the network settles.
Paxos Made Simple
A plain-English derivation of Paxos, showing fault-tolerant consensus follows almost unavoidably from wanting a majority of acceptors to agree.
Perspectives on the CAP Theorem
The definitive restatement of CAP: not pick two of three, but the impossibility of safety plus liveness on an unreliable network.
The Google File System
A cluster file system that makes commodity failure routine, files enormous, and concurrent append a first-class atomic operation.
MapReduce: Simplified Data Processing on Large Clusters
A programming model that hides parallelization, fault tolerance, locality and load balancing behind two user-written functions: map and reduce.
C-Store: A Column-oriented DBMS
A read-optimized column store built from overlapping sorted projections, compressed columns, and a hybrid write store with snapshot isolation.
The Vertica Analytic Database: C-Store 7 Years Later
The engineering post-mortem of C-Store: which column-store research ideas survived seven years of paying customers, and which were dropped.
Cassandra - A Decentralized Structured Storage System
A production store that fused Dynamo's leaderless ring with Bigtable's column families to absorb billions of writes a day.
Hive - A Warehousing Solution Over a Map-Reduce Framework
A SQL-like warehouse over Hadoop: HiveQL compiles into map-reduce DAGs, backed by a catalog, partitions, buckets and pluggable SerDes.
Spark: Cluster Computing with Working Sets
Resilient distributed datasets: cached, lineage-recoverable collections that let clusters reuse a working set in memory across many operations.
Kafka: a Distributed Messaging System for Log Processing
A distributed commit log for high-volume event data: partitioned append-only segments, pull-based consumers holding their own offsets, zero-copy delivery.
Spanner: Google's Globally-Distributed Database
The first database to give globally distributed transactions external consistency, by exposing clock uncertainty in the time API and waiting it out.
Calvin: Fast Distributed Transactions for Partitioned Database Systems
Order transactions deterministically before executing them, and a partitioned database can drop two-phase commit entirely.
Presto: SQL on Everything
One adaptive distributed SQL engine that serves sub-second dashboards and multi-hour ETL over dozens of pluggable data sources.
Evolution of Development Priorities in Key-value Stores Serving Large-scale Applications: The RocksDB Experience
Eight years of running RocksDB at Facebook scale, and how that moved its optimization target from write amplification to space to CPU.
In Search of an Understandable Consensus Algorithm (Extended Version)
A leader-based consensus algorithm designed for understandability, equivalent to multi-Paxos in safety and efficiency but teachable and implementable.
The Snowflake Elastic Data Warehouse
Split a warehouse into blob storage, ephemeral compute clusters and a shared metadata brain, making elasticity an architectural property.
Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores
ACID tables on plain cloud object stores, built from a Parquet-checkpointed write-ahead log that needs no always-on metadata service.
Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics
A blueprint for running warehouse-grade transactions, indexing and SQL performance directly over open Parquet files in cloud object storage.
DuckDB: an Embeddable Analytical Database
An in-process SQL engine that brings vectorized OLAP execution to the embedded niche SQLite left empty.
FoundationDB: A Distributed Unbundled Transactional Key Value Store
Serializable ACID transactions at NoSQL scale, built by unbundling the database and proving every feature correct in deterministic simulation.
Disaggregated State Management in Apache Flink 2.0
Flink 2.0 makes remote storage the primary home of streaming state, hiding its latency with asynchronous, out-of-order record execution.