Data Migration¶
Warning
Make sure that the files being migrated are not modified during the process. Otherwise, changes to the files while migration is in progress can get lost. Stop all clients or mount them read-only.
The beegfs subcommand entry migrate allows you to move all files from a certain storage
target, all targets of a node, or a storage pool to other storage targets. The file metadata on the
metadata servers is not transferred.
Migration is directory-based and recursively moves everything below the specified path to other storage targets. To achieve this, each matching file will be copied to a temporary file on a different set of storage targets and then be renamed to the original file name. Thus, migration should not be done while applications are modifying files in the migration directory tree.
If new files are created by applications while the migration is in progress, it is possible that these files are created on the target which you want to free up. Thus, be careful regarding how you access the file system during migration.
When migrating between storage pools, if any of the source files have Buddy Mirroring as the pattern type, make sure the destination pool contains at least one buddy group. Otherwise migrating these files will not be possible.
Data migration has to run on a mounted BeeGFS client.
See beegfs entry migrate --help for more information.
Examples¶
Freeing up a target¶
Migrate can be used to free up storage targets, for example, to allow their removal. For example to move all chunks off certain storage targets:
$ beegfs entry migrate --from-targets=<targetID>,[targetID] --pool=<destinationPoolID> /mnt/beegfs/
This will crawl the whole filesystem and rewrite all files which have chunks on storage target
5 and randomly select new targets from the provided destination pool ID.
Moving files between storage pools¶
Files can be moved between storage pools. For example this is useful to archive old or infrequently used files. Given a pool of fast targets with ID 2, and a pool of bigger but slower targets with ID 3, you could perform the following migration:
$ beegfs entry migrate --from-pools=2 --pool=3 /mnt/beegfs/<old-project>
