Striping

Striping in BeeGFS can be configured on a per-directory and per-file basis. Each directory has a specific stripe pattern configuration, which will be derived to new subdirectories and applied to any file created inside a directory. There are currently two basic parameters that can be configured for stripe patterns: the desired number of storage targets for each file and the chunk size (or block size) for each file stripe.

The stripe pattern parameters can be configured the command-line control tool beegfs. This tool allows you to inspect or change the stripe pattern of files and directories in the file system at runtime. The following command will show you the current stripe settings of your BeeGFS mount root directory (in this case /mnt/beegfs):

$ beegfs entry info /mnt/beegfs/
PATH  ENTRY ID       TYPE  META NODE        META MIRROR  STORAGE POOL  STRIPE PATTERN  STORAGE TARGETS  BUDDY GROUPS  REMOTE TARGETS  COOL DOWN
/     root      directory  node_meta_2 (2)            1  default (1)   RAID0 (4x1M)    (directory)      (directory)   (none)          (n/a)

The entry info can also be printed in the horizontal format from BeeGFS 7 using the --retro flag and additional details can be printed with --verbose:

$ beegfs entry info /mnt/beegfs/ --retro --verbose
Entry type: directory
EntryID: root
ParentID:
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 1M
+ Number of storage targets: desired: 4
+ Storage Pool: 1  (default)
Inlined inode: no
Inode info:
+ Path: 38/51/root
+ Metadata buddy group: 1
+ Current primary metadata node: node_meta_2 [ID: 2]

Use the command entry set to apply new striping settings to a directory. For example to stripe files across 4 storage targets with a chunk size of 1 MiB, run:

$ beegfs entry set --num-targets=4 --chunk-size=4MiB /mnt/beegfs/
Processed 1 entries.
Configuration Updates: Chunksize (4194304), DefaultNumTargets (4)

Stripe settings will be applied to new files, not to existing files in the directory. To apply the pattern to existing files, recreate them by performing a deep copy or using the entry migrate command.

See also

Data Migration

Buddy Mirroring

If you have buddy mirror groups (see Mirroring) defined in your system, you can set the stripe pattern to use buddy groups as stripe targets, instead of individual storage targets. In order to do that, add the option --pattern=mirrored to the command, as follows. In this particular example, the data will be striped across 4 buddy groups with a chunk size of 1 MB:

$ beegfs entry set --num-targets=4 --chunk-size=1MiB --pattern=mirrored /mnt/beegfs/
Processed 1 entries.
Configuration Updates: Chunksize (1048576), DefaultNumTargets (4), StripePattern (Buddy Mirror)

To switch back to non-mirrored mode, set the pattern to raid0.

Impact on network communication

The data chunk size has an impact on the communication between client and storage servers in several ways.

When a process writes data on a file located on BeeGFS, the client identifies the storage targets that contain the data chunks that will be modified (by querying the metadata servers) and send modification messages to the storage servers containing the modified data. The maximum size of such messages is determined by the data chunk size of the file.

If you define --chunk-size=1MiB, 1 MiB will be the maximum size of each message. If the amount of data written to the file is larger than the maximum message size, more messages will have to be sent to the servers and this may cause performance loss. So, slightly increasing the chunk size to a few MB has the effect of reducing the number of messages, and this can have a positive performance impact, even in a system with a single target.

On the other hand, it is important to make sure that a data chunk fits the RDMA buffers available on the client (see Client Node Tuning), in order to prevent the messages from being split, and again increasing the number of messages. See

You also have to consider the file cache settings. When the client is using the buffered cache (tuneFileCacheType = buffered), it uses a file cache buffer of 512 kiB to accumulate changes on the same data. This data is sent to the servers only when data from outside the boundaries of that buffer is needed by the client. So, the larger this buffer, the less communication will be needed between the client and the servers. You should set this buffer size to a multiple of the data chunk size. For example, adding tuneFileCacheBufSize = 2097152 to the BeeGFS client configuration file will raise the file cache buffer size to 2 MiB.