Dead simple design with Reddit's database

Reddit’s database has two tables

Steve Huffman talks about Reddit’s approach to data storage in a High Scalability post from 2010. I was surprised to learn that they only have two tables in their database.

Lesson: Don’t worry about the schema.

[Reddit] used to spend a lot of time worrying about the database, keeping everthing nice and normalized. You shouldn’t have to worry about the database. Schema updates are very slow when you get bigger. Adding a column to 10 million rows takes locks and doesn’t work. They used replication for backup and for scaling. Schema updates and maintaining replication is a pain. They would have to restart replication and could go a day without backups. Deployments are a pain because you have to orchestrate how new software and new database upgrades happen together.

Instead, they keep a Thing Table and a Data Table. Everything in Reddit is a Thing: users, links, comments, subreddits, awards, etc. Things keep common attribute like up/down votes, a type, and creation date. The Data table has three columns: thing id, key, value. There’s a row for every attribute. There’s a row for title, url, author, spam votes, etc. When they add new features they didn’t have to worry about the database anymore. They didn’t have to add new tables for new things or worry about upgrades. Easier for development, deployment, maintenance.

The price is you can’t use cool relational features. There are no joins in the database and you must manually enforce consistency. No joins means it’s really easy to distribute data to different machines. You don’t have to worry about foreign keys are doing joins or how to split the data up. Worked out really well. Worries of using a relational database are a thing of the past.

This fits with a piece I read the other day about how MongoDB has high adoption for small projects because it lets you just start storing things, without worrying about what the schema or indexes need to be. Reddit’s approach lets them easily add more data to existing objects, without the pain of schema updates or database pivots. Of course, your mileage is going to vary, and you should think closely about your data model and what relationships you need.

Event sourcing – mindset shift

Stop focusing on the final data state , and start focusing on the list of events which as lead to this final state(result).

The mindset shift behind Event Sourcing is to look at your database not as a stock but as a series of events that can be used to build the current stock. This is more obvious if you think of your bank account. The stock is your balance : how much money you have. The events are all the operations (credit or debit) that occured and “created” the balance.

Didn’t guess the relationship with NoSQL yet ? Check out the full article here

Picked up from the MongoDB manual: schema design

Data modeling is definitely one the main issue,  MongoDB’s manual provide a short, simple, well documented sample: here.

Quoting:

The key question in Mongo schema design is “does this object merit its own collection, or rather should it embed in objects in other collections?” In relational databases, each sub-item of interest typically becomes a separate table