Data Migration¶
Warning
Make sure that the files being migrated are not modified during the process. Otherwise, changes to the files while migration is in progress can get lost. Stop all clients or mount them read-only.
The beegfs-ctl subcommand --migrate allows you to move all files from a certain storage
target, all targets of a node, or a storage pool to other storage targets. The file metadata on the
metadata servers is not transferred.
Migration is directory-based and recursively moves everything below the specified path to other storage targets. To achieve this, each matching file will be copied to a temporary file on a different set of storage targets and then be renamed to the original file name. Thus, migration should not be done while applications are modifying files in the migration directory tree.
If new files are created by applications while the migration is in progress, it is possible that these files are created on the target which you want to free up. Thus, be careful regarding how you access the file system during migration.
When migrating between storage pools, if any of the source files have Buddy Mirroring as the pattern type, make sure the destination pool contains at least one buddy group. Otherwise migrating these files will not be possible.
Data migration has to run on a mounted BeeGFS client.
See beegfs-ctl --migrate --help for more information.
Examples¶
Freeing up a target¶
Migrate can be used to free up storage targets, for example, to allow their removal. Moving all chunks off a certain storage target:
$ beegfs-ctl --migrate --targetid=<targetID> /mnt/beegfs/
This will crawl the whole filesystem and rewrite all files which have chunks on storage target
5. The targets for the new file are randomly selected from the remaining targets in the storage
pool.
Move files between storage pools¶
Moving files between storage pools can for example, be used to archive old or infrequently used project files. Assume there is a pool of fast targets (ID2), and a pool of bigger but slower targets (ID3). Then this looks like this:
$ beegfs-ctl --migrate --storagepoolid=2 --destinationpoolid=3 /mnt/beegfs/<old-project>
