Files in a WiredTiger database

The WiredTiger storage engine

WiredTiger is a high performance, key-value storage engine that is the default storage engine in MongoDB 3.2 and newer.
In a MongoDB deployment using the WiredTiger engine, the storage.dbpath directory is managed by WiredTiger. That is, WiredTiger needs the content of the directory as a whole to function properly. Removing files from this directory could render WiredTiger (and thus MongoDB) unable to start. A notable exception is the diagnostic.data directory, which contains non-personally identifiable information regarding the status of the database for troubleshooting purposes.

Files in a WiredTiger database

A newly created MongoDB 3.2 database started with the default settings typically consists of:
WiredTiger
WiredTigerLAS.wt
WiredTiger.lock
WiredTiger.turtle
WiredTiger.wt
_mdb_catalog.wt
mongod.lock
sizeStorer.wt
collection-0-601483983079373625.wt
index-1-601483983079373625.wt
diagnostic.data
storage.bson
Important
Do not remove or modify any of these files. Doing so may leave the database in an inoperable state.
Descriptions of each of the files and its functions are as follows:
  • WiredTiger: the WiredTiger version file
    This text file contains the version of WiredTiger used to create the database, and is used by WiredTiger on startup to determine whether a database is present.
  • WiredTiger.wt: the WiredTiger metadata file
    The main metadata file that tracks information about which collections and indexes exist in the database, as well as the most recent durable update (checkpoint) for each of them.
  • WiredTiger.turtle: engine configuration file for the WiredTiger.wt file
    This text file contains the WiredTiger engine configuration for the WiredTiger.wt metadata file described above.
  • WiredTigerLAS.wt: the WiredTiger lookaside file
    MongoDB configures WiredTiger in-memory cache to contain no more than a specified percentage of dirty data (defaults to 20% in MongoDB 3.2.10). If the dirty data exceeds the configured number and WiredTiger is required to keep the dirty data for operational reasons, this file contains the disk spill of the dirty data.
  • _mdb_catalog.wt: the MongoDB metadata file
    This file contains the metadata and catalog of MongoDB databases, collections, and indexes, including which WiredTiger data file maps to each MongoDB collection or index.
  • WiredTiger.lock and mongod.lock: lock files
    MongoDB and WiredTiger use these lock files to ensure that only a single process accesses the database at any particular time.
  • sizeStorer.wt: the MongoDB collection size tracking file
    This file contains information about the size and document count of each MongoDB collection.
  • collection-X-XXX.wt and index-X-XXX.wt: MongoDB collection and index files
    Each MongoDB collection is stored in a collection-X-XXX.wt file, and each MongoDB index is stored in a index-X-XXX.wt file. The X-XXXportion of the filename represents a random number used by WiredTiger as a unique internal identifier. If MongoDB is started with the --directoryPerDB option these files may appear in a subdirectory of the dbpath.

Files created by optional settings

  • journal directory: WiredTiger journal files
    If a database is started without the --nojournal option, the database directory also contain a journal subdirectory that has the following contents:
    WiredTigerLog.0000000001
    WiredTigerPreplog.0000000001
    WiredTigerPreplog.0000000002
    
    Of importance here is the WiredTigerLog.XX file, which contains the Write Ahead Log (WAL) maintained by WiredTiger.

Comments

  1. This comment has been removed by the author.

    ReplyDelete
  2. Thanks for this great explanation.. but do we know what storage.bson is used for? I dont see any explanation on this anywhere on google of mongo itself.

    ReplyDelete
  3. Thank you for the post. this is really helpful for my work. May I ask the reason why collection-X-XXX.wt and index-X-XXX.wt files are divided into several files, even if I create one collection?

    ReplyDelete
  4. What if i remove all three folders as i'm not using mongodb on my machine?

    ReplyDelete
  5. I have removed data from the collections and ran the db.runcommand() with compact

    but disk is not reclaimed..

    i did some digging in the machine.. found many collection-xxx.wt files taking up all the space..
    I want to reclaim the disk.. how do I do that.. pls help.
    and I dont have enough space to run repair command.

    ReplyDelete

Post a Comment