Will the expired data of redis be deleted immediately? Great mystery

Margo , When key Expiration time reached ,Redis Will it be deleted immediately ？

Say first conclusion ,  It will not be deleted immediately  ,Redis There are two strategies for deleting expired data ：

• Periodically delete some data ;

• Lazy deletion ;

The command is in Redis 2.4 edition , The expiration time is not very accurate , It could be between zero and a second .

from Redis 2.6 Start , The expiration error is 0 To 1 millisecond .

EXPIRE key seconds [ NX | XX | GT | LT]  The instruction can set the specified key Set expiration time , If the expiration time is not set , key There will always be , Unless we explicitly delete it , Such as execution  DEL  Instructions .

So-called ” The cunning rabbit died , Running dog cooks “, Kill when it's useless , Follow 35 At the age of “ graduation ” It's one thing .

So flustered ……

from Redis edition 7.0.0 Start ： EXPIRE  Added options ： NX 、 XX  and  GT 、 LT  Options .

• NX： When key Set the expiration time only when there is no expiration ;

• XX： Only key Set the expiration time only when it has expired ;

• GT： Only when the   New expiration date   The expiration time is set only when it is greater than the current expiration time ;

• LT： The expiration time is set only when the new expiration time is less than the current expiration time .

Expiration and persistence

In the master-slave or cluster architecture , The clocks of the two machines are seriously out of sync , Is there going to be a problem ？

key Expired information is in  Unix Absolute timestamp   It means .

In order for the expiration operation to work properly , The time between machines must ensure stable synchronization , Otherwise, the expiration time will be inaccurate .

For example, two machines whose clocks are seriously out of sync RDB transmission , slave The time of is set to the future 2000 second , If be in, master One of the key Set up 1000 Second survival , When Slave load RDB When key Think it's time to key Be overdue （ because slave The machine time is set to the future 2000 s）, Not waiting 1000 s It's overdue .

Lazy deletion

Inert deletion is simple , That is, when there is a request from the client to query the  key  When , Check  key  Is it overdue , If expired , Delete the  key .

For example, when Redis Received... From client  GET movie: Ozawa # m …… Leah .rmvb  request , I'll check first  key = movie: Ozawa # m …… Leah .rmvb  Has it expired , If it expires, delete .

The initiative to delete expired data is given to each access request .

The implementation is implemented through  expireIfNeeded  Function implementation , Source path ： src/db.c .

int expireIfNeeded(redisDb *db, robj *key, int force_delete_expired) {
       // key  No expired ,return 0    if (!keyIsExpired(db,key)) return 0;    if (server.masterhost != NULL) {
           if (server.current_client == server.master) return 0;        if (!force_delete_expired) return 1;    }
    if (checkClientPauseTimeoutAndReturnIfPaused()) return 1;
    /* Delete the key */    deleteExpiredKeyAndPropagate(db,key);    return 1;}

Copy code

Delete periodically

Just rely on client access to judge key Whether to delete after expiration is certainly not enough , Because there are key Out of date , But no one will visit in the future , How to delete these data ？

You can't let these data 「 It's not shit to occupy the manger 」.

So called periodic deletion , That is to say Redis The default for each 1 Seconds to run 10 Time （ Every time 100 ms Do it once ）, Randomly select some with expiration time each time key, Check if it's overdue , If it is found that it has expired, it will be deleted directly .

Be careful ： Not all libraries are checked in one run , All the keys , It's a random number of keys .

The specific steps are as follows ：

1. From all with expiration time set key Random in the set   choice 20 individual key ;

2. Delete 「 step 1」 All found expired key data ;

3. 「 step 2 」 end , overdue key exceed 25%, Then continue 「 step 1」.

Delete the source code  expire.c Of activeExpireCycle Function implementation  .

This means that at any time , Be overdue key The maximum number of writes per second is equal to the maximum number of writes per second divided by 4.

Why not check all settings for expiration time key？

You want to , hypothesis Redis There's... In it 100 w individual key, Expiration time is set , every other 100 Check in milliseconds 100 w individual key,CPU All wasted on inspection expiration key Yes ,Redis It's useless .

Pay attention ：  Whether it's scheduled deletion , Or inert deletion . When the data is deleted  , master  Delete instructions will be generated and recorded to  AOF  and  slave  node  .

Margo , If there is too much expired data , Scheduled deletion cannot delete completely （ Every time you delete the expired key Or more than 25%）, At the same time these key It will never be requested by the client again , That is, you can't go , What will happen ？

Will it lead to Redis Out of memory , How to deal with it ？

That's a good question , The answer is to go   Memory obsolescence mechanism  .

That's it today , Say too much , It's easy for everyone to choke in the mass of knowledge , It's important to keep your life , As for the details of memory elimination mechanism , Please look at the next breakdown .