Big Tech Coach

What is a Key Value Store?

titleImagePath
components/Key-Value-Store-Article.png
date
May 23, 2024
slug
what-is-a-key-value-store
status
Published
tags
summary
Explore what a Key Value Store is and its role in system design interviews, understanding how it manages data efficiently for scalable and high-performance applications.
type
SystemComponent
systemType
probability

What is a Key-Value Store?

notion image
The simplest NoSQL database is the key-value store. Every data element in the database is stored as a key value pair consisting of an index (or "key") and a value. In a sense, you can think about a key-value store is like a relational database with only two columns: the key (such as "Country") and the value (such as "Population").
And that's really the whole idea behind a key-value store. Let's talk about the technical details to get a better idea how such a database can be leveraged.
The simplest implementations, support only basic data types like integers, strings and or booleans as values. More advanced implementations can also support semi-structured data like arrays or nested dictionaries also called objects in other languages.
So much about how to store data, but how can data be retrieved?
No matter the implementation complexity, it's advised to systematically name the keys to allow for sorting.
Thereby, key-value stores can support some common data retrieval use cases like grabbing all keys that...
  • start with a certain letter
  • are within a range of numbers
  • that are less than or greater than a certain number
  • or keys that are within a certain period of time if the key is a timestamp
 
In contrast, looking up values is much more tricky. In pure key-value store implementations it's simply not possible to query by value. It would be very inefficient as in the worse case scenario you would have to iterate over the entire database. That's why key-value stores don't come with query languages like SQL.
Interacting with a key-value store is very simple. In general it's like you would interact with an in-memory data structures.
Often you are limited to a Put a Get and a Delete method.
So much about the functional features of a key-value store, but what's about the non-functional once?
💡
No-SQL in a Nutshell: The majority of non-functional features descend from their belonging to the class of No-SQL databases. In contrast, no-sql databases have been developed during a time when large distributed systems were already common. That's why they are designed to abstract away all implementation details of how to deploy and scale the database. This is convenient for us developers as we can focus on writing code that directly creates business impact.
Besides the inherited ability to scale well, key-value stores are also known for their excellent performance and their almost instantaneous access.
Of course, the absence of complex queries is a major factor in this, but also many key-value stores keeping an in-memory data structure that is mapped to the data stored on disk. Thereby, they automatically keep frequently requested data in RAM as is much faster than accessing data on disk.

When to use

Some key-value store implementations take access speed to an extreme and only store data in memory. Those implementations are then used to implement caching layers on top of other databases. We get to those later in the course.
For now, let's talk about use cases classic key-value stores are often used for.
By now you probably already know why I am introducing key-value stores in this section about URL shorteners.
In general key-value stores are great when your application needs to handle lots of small continuous reads and writes, that might even be volatile.
This could mean to look-up long URLs based on short handles there is a much wide area of application for that. You cloud use a key-value store to store
  • shopping carts for online shops
  • personalized Product recommendations for individual customers
  • user preference of specific users.
  • Session details
 
The importance of key-value stores is illustrated by the fact that many big tech company have implemented their own key-value store solution to solve their own storage needs before they open sourced the implementations.
Level DB is a very influential implementation, written at google. It's github repo states that it's "a fast key-value storage library...that provides an ordered mapping from string keys to string values."
Facebook, or Meta by now, build their own implementation RocksDB based on LevelDB and eventually build ZippyDB as another abstraction layer on top of it, to address the needs of various teams, to solving similar challenges such as consistency, fault tolerance, failure recovery. Thereby, ZippyDB allowed products to move a lot faster by offloading all this individual development effort to the Zippy team.
Amazon also has their own key-value store implementation - DynamoDB. It's probably the most widely used key-value store database, in fact, it was the research into DynamoDB that really started making NoSQL really popular. However DynamoDB is not open-source but runs excessively on AWS.
  • LevelDB
    • LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.
  • RocksDB
    • RocksDB is developed and maintained by Facebook Database Engineering Team. Build-upon Level DB
  • ZippyDB based on rocksDB
    • To address the needs of these various teams, we (facebook) built ZippyDB to provide a highly durable and consistent key-value data store that allowed products to move a lot faster by offloading all the data and the challenges associated with managing this data at scale to ZippyDB.
  • Amazon DynamoDB:
    • The most widely used key-value store database, in fact, it was the research into DynamoDB that really started making NoSQL really popular.

Summary

Let’s wrap up with summarizing the advantages and disadvantages of Key-Value stores.

Advantages

  • The response time is fast, due to simplicity and in-memory storage.
  • The simplicity is also the reason why key-value stores are easy to use.
  • Key-value store databases are also highly scalable due to their no-sql properties.

Disadvantages

  • The only way to look up values is by key. This makes it difficult and extremely inefficient to search through the database.
  • There is also no standard query language like SQL. In consequence queries are written on the application later and are not portable to other key-value stores.
  • There is no data normalisation like in relational databases which can lead to duplicate data when implemented inefficiently.
 
Well done you completed the theory lectures of this section. Now time to jump into the Mock Interview and finally design TinyURL!
/