Catalog

One 、 Different concepts

1、Journal journal

2、Oplog journal

Two 、 The data range is different

         I'm learning mongoDB When it comes to basic knowledge , Some concepts may be confused , such as Journal Journal and oplog What's the difference between logs .

        The following content is mainly extracted from 《MongoDB Principle and practice 》 Written by Zhang Youdong

One 、 Different concepts

1、Journal journal

        Journal The journal is MongoDB The mechanism of the storage engine layer , ⽬ front MongoDB All storage engines ⽀ Holding configuration Journal, since 2.0 Start default on . stay MongoDB Write all the data in ⼊、 Reading is ultimately implemented by calling the interface of the storage engine layer , and Journal Logs are used to record write operations , To achieve the purpose of data failure recovery and persistent data .

         With wiredtiger For example , If you don't configure Journal, Write ⼊ wiredtiger The data of , It's not ⽴ Persistent storage ; ⽽ I do it every minute ⼀ Sub total checkpoint（storage.syncPeriodSecs Configuration item , The default is 1 minute ） , Make all data persistent . If there is an outage in the middle , Then the data can only be restored to the latest ⼀ Time checkpoint, In this way, you may lose the most recent 1 Minute data . And when you turn on Journal After the log , Every operation written to the database will be recorded , In the event of downtime , After recovering to the latest checkpoint After the point , You can continue to replay the subsequent operation logs , Restore data .

2、Oplog journal

        Oplog The log defaults to mongoDB Enabled in replica set mode .Oplog Logs are mainly used for Primary Nodes and Secondary Data synchronization of nodes ,Oplog Recorded Primary All write operations of the node ,Secondary Nodes are copied on a regular basis Primary Node Oplog Achieve incremental backup .

        since mongoDB3.6 Newly added changestream The mechanism is also based on oplog Realized .ChangeStreams stay Oplog A layer of application is wrapped on it , Provide a API Interface , Push the data in real time .

  FIG selected 《mongodb Introduction of actual combat 》

Two 、 The data range is different

        With mongoDB For example, insert data once in the replica set , At the data level, it is divided into the following three steps ：

                1. take ⽂ File data write ⼊ Corresponding set

                2. Update all index information of the collection

                3. Write ⼊⼀ strip oplog⽤ In sync

         On ⾯3 Modification operations , You need to make sure that it's either successful , Or they all failed .MongoDB Writing ⼊ Data time , Will put the above 3 Put operations into ⼀ individual wiredtiger The business of ⾥, Make sure 「 primary ⼦ sex 」 . As shown below ：

//  Start business 
beginTransaction();
//  Three steps to insert data 
writeDataToColleciton();
writeCollectionIndex();
writeOplog();
// Commit transaction 
commitTransaction();

            The execution sequence is shown in the figure below ：

         wiredtiger When committing a transaction , All modifications will be ⽤, And put the above 3 Operation write ⼊ To ⼀ strip journal In the operation log ; The backstage will be cyclical checkpoint, Persist changes , And remove the useless journal. From the data layout , oplog And journal The relationship is as follows ：