Metadata Mirroring

Warning

Mirroring is not a replacement for backups. If files are accidentally deleted or overwritten by a user or process, mirroring won’t help you to bring the old file back. So you are still responsible to do regular backups of your important bits.

Activating Metadata Mirroring

After metadata server buddy groups have been defined as described under Management of Mirror Buddy Groups, metadata mirroring can be activated on the file system.

Warning

In order to synchronize all information across the various components of BeeGFS correctly, when activating metadata mirroring, no clients may be mounted and all servers except the management server and metadata server that owns the root inode must be stopped. The beegfs mirror init command will refuse to execute if any clients or servers are still running.

The full series of steps on an already running system is:

  1. Check what metadata server owns the root inode by running beegfs entry info --mount=none / and noting which node is listed for META_NODE.

    • Run beegfs mirror list and verify this node is already in a metadata buddy group and is listed as the primary target. If the node is not already in a buddy group, first create buddy groups and set this node as the primary target in its group. If this node is in a group but listed as the secondary target, do not proceed and contact ThinkParQ support for assistance.

  2. Stop all clients.

  3. Stop all metadata and storage servers except the metadata server that owns the root inode.

    • If Remote Storage Targets are in use, stop all remote and sync servers.

  4. Activate metadata mirroring by running beegfs mirror init.

  5. Restart all metadata and storage servers.

    • If Remote Storage Targets are in use, also restart the remote and all sync servers.

  6. Start all clients.

  7. Run beegfs entry info --mount=none / and verify the META_MIRROR column lists the correct buddy group. Run beegfs health check and verify no issues are reported.

    • Depending on the number of files and directories in the root directory, the health check’s CONSISTENCY column for the secondary target in the root buddy group may show Needs Resync with a SYNC_STATE of Running for some time. If the sync does not complete successfully, open a ticket with ThinkParQ support.

At this point metadata mirroring is enabled for the root directory of the BeeGFS, as well as all the files contained in it. Note that existing directories except for the root directory will not be mirrored automatically. Please see Migrating Existing Metadata below to find out how to mirror existing directories.

New files and directories created inside a directory with active metadata mirroring will also have metadata mirroring activated. There might be situations where it is desirable to deactivate metadata mirroring for part of the file systems. This gives a slight performance boost, but the data will not be preserved in the event of a server failure. A directory without metadata mirroring can be created using beegfs:

$ beegfs entry create directory --no-mirror <name>

For information about additional command line parameters refer to the built-in --help.

Directories and files created inside a directory without metadata mirroring will have metadata mirroring disabled.

Migrating Existing Metadata

If a file is moved into a directory with active metadata mirroring, it will have its metadata mirrored. On the other hand, directories will not automatically be mirrored when moved. For a directory to be mirrored, it is therefore necessary to freshly create a directory inside a mirrored directory. The easiest way to enable mirroring for a whole directory tree is to do a recursive copy:

$ cp -a <directory> <mirrored-dir>

This will also copy the file contents, therefore it is possible to use it for enabling metadata and storage mirroring at the same time.