Redis
Introduction
Redis, one of the most-used software for caching is an in-memory data structure store that can also work like a database or even a message broker. It has support for various data types such as string, list, hash, and so on.
Persistence
One of the major concerns with caching your data is persistence, for sure, you don’t want to lose all your data when your machine is down. Fortunately, Redis provides different levels of persistence so you can also write your data into storage.
RDB(Redis Database): This level of persistence backup your database repeatedly at some predefined time. You can also choose to backup your database if there are m changes in n time. By default, Redis would dump a binary file called dump.rdb. You can transfer the backup file into cloud storage, which is ideal for the case of disaster recovery.
AOF(Append Only File): record all the write commands in the Redis. You can see it as the process of appending a file with all the commands, so when you start up a new server, Redis would run all the commands in the file to reconstruct the database. Also, if accidentally remove something or even all the keys, you can simply remove the last command and rerun the log file to get back your data.
Trade-offs: RDB would be better in terms of performance but there is still a chance of data loss. AOF would be much more robust in terms of data persistence with some sacrificing in performance.
Further reading: https://redis.io/docs/manual/persistence/#how-durable-is-the-append-only-file
Replication
Replication is an important topic in the distributed system. Just like other databases, Redis does come with leader-follower feature. You want the followers to handle read requests to ease the load on your database system. While the master node would handle the write request and send a stream of the commands to followers so that they can be updated.
In case there is a disconnection between leader and followers. A follower would try to do a partial resynchronization first and if it fails to do so, the follower would proceed with full resynchronization.
partial resynchronization: obtain the part of the stream of commands it missed during the disconnection
full resynchronization: get a copy of the database from the leader node.
Redis is using asynchronous replication by default, which mean the follower would acknowledge the command sent by the master. The master node doesn’t wait for the replica to process the data, in order to achieve low latency and high performance. You can still opt for synchronous replication by using the wait command
further reading: https://redis.io/docs/manual/replication/
Data Structure
Key in Redis is unique, binary safe and can be up to 512MB in size.
Key naming should be consistent, Colon is a typical community convention. Plurals are nice for Lists, and sets.
key Name:
event: Eating
event: Swimming
Naming is also case-sensitive.
Sushi is different from sushi
key expiration: how long the keys are retained. You can reset the expiration time anytime. By default, all keys are retained.
list
an ordered collection of strings, duplication is allowed, elements added from left, right, or by position
You can use the list to create queues, stacks, or capped lists. Typical use case: interprocess communication.
many operations are 0(1) e.g,
- LPOP, LPUSH, LPUSHX
- RPOP,RPUSH,
- BLPOP,BRPOP,
- LLEN
- RPOPLUSH,BRPOPLPUSH
Sets
an unordered collection of unique Strings.
set operations:
- Difference
- intersect
- Union
sorted sets
an ordered collection of unique strings
manipulation by value, score, position, or lexicography
set operations:
- intersect
- union
typical use cases: leaderBoard, priority queue
Many operation is o(1) or O(logN):
- ZADD
- ZCOUNT, ZINCRBY
- ZSCORE, ZRANK
Hashes
A mini key-value store in the key, field, and value pairs, dynamically adds remove fields
key: JudoScore: venue
value: super DomeScore: capacity
value: 32000
Hashes operations: Intersection and union
typical use case: rate limiting, session cache
Many operations are o(1):
- HGET, HSET, HSETNX
- HINCRBY, HINCRBYFLOAT
- HLEN, HSTRLEN
- HEXISTS
Redis Command
set string as value
>> set name SF //set key value
>> get name
"SF">> getrange name 0 0
"S">>set name "sfsf"
>> get name
"sfsf"
set multiple values
>> mset lang EN lang2 CN //set key value key value>> mget lang lang2
1) "EN"
2) "CN">> strlen lang
2
increment/decrement
>> set count 1
>> incr count
2
>> incrby count 10
12
>> decr count
11
>> decr count 2
9>> set count 1.234
>> incrbyfloat count 0.001
1.235
Expire a key
>> set a 1
>> expire a 10 //delete after 10 second
>> ttl a
4 // 4 second is remaining >> get a
(nil)>> setx b 10 val // expire b in 10 second
>> ttl b
5 //5 secodn is remaining
List
>> keys * //list all the keys
"a"
"b">> flushall //remove all the keys>> lpush country india
1
>>lpush country my
2
>> lrange country 0 -1
1)"india"
2)"my">>rpush county ph //right push
3>> llen country //len of list
3>> lpop country // left pop
2>> linsert country before my "nz" //insert nz at position before my
3>>lindex country 0
"nz">>lpushx movies Av //push value into key if key exist
0>>rpushx movies Av //push value into key if key exist
0>>sort country ALPHA //sort by alphabel>> sort country desc ALPHA // sort by alphabel in descending order//pop the value from key before timeout
>> blpop movies 1 //try to popleft from movie within 1 second
(nil)
(1.07s)
Set
>> sadd tech java py go //add to set
3>> scard tech //get count
4>> sismber tech java //check if in set
1>> smembers tech
1) "java"
2) "py"
3) "go">> sdiff set1 set2 //different in 2 sets
>> sinter set1 set2 // common element in 2 sets>>sdiffstore newset set1 set2 // store the different in set1 and set2 into a newset
>>sinterstore newset set1 set2 // store the common in set1 and set2 into a newset>>sunion set1 set2 // get the union of 2 set
sorted set
>> zadd users 1 sf 2 sfsf 3 sfsfsf
3
>> zrange users 0 1
1)"sf"
2)"sfsf"
>>zcard users
3>>zrem key value //remove a value from key>>zrange users 0 -1 withscores // get the score associate with the value>> zrevrangebyscore users 5 0 withscores //reverse by score in range of 0 to 5>> zincrby users 2 sf // increase the score of user
Hash
>> hset myhash name sf
1>> hkeys myhash //return all the key
1)"name">> hvals myhash //return all the value
1) "sf">> hgetall myhash //return all the key and value
1)"name"
2)"value">>hexists myhash name // check if the value is present in the key or not
1
>>hlen myhash // return len of has
1>>hset myhash age 24
1>> hincrby myhash age 2 //increase by value>> hincrbyfloat myhash age 1.4>>hdel myhash age //remove key>> hstrlen myhash name //find the len of string
2>> hsetnx myhash name Daniel // only add value if field is not exist
Redis Transaction
ensure commands are atomic, use the multi and exec keyword.
e.g.
>>multi
>> set name sf
queue
>> set age 25
queue
>> exec
1)OK
2)OKuse the discard keyword to discard a transaction.
>>multi
>> set name sf
queue
>> set age 25
queue
>> discard
OK