Client Side Caching Modes

BeeGFS clients provide two different file cache modes, settable using the client configuration option tuneFileCacheType:

• buffered: uses a pool of small static buffers for write-back and read-ahead

• native: uses the Linux kernel page-cache

The buffered mode typically provides higher streaming throughput compared to the native cache mode. The buffered mode will only cache a few hundred kilobytes of a file, while the native mode is able to cache multiple gigabytes, dynamically adapting to the amount of available client RAM. The native mode is suitable for workloads that fit entirely in the available Linux client cache.

Note that while native mode is stable, it relies on the Linux kernel mechanisms, which might deviate between versions and architectures. As such, there may be cases where performance and/or client behavior may be different than expected.

The buffered mode is the default cache type, and it is also the only valid cache type for clients that are running on the same machine as a BeeGFS server daemon.

The cache type can be set in the client configuration file (/etc/beegfs/beegfs-client.conf).

Note that BeeGFS servers also dynamically use available RAM for caching, independent of the client cache type.

Is it possible to use different caching types in different directories of the same file system?

Yes, it is. Usually, this is a common solution for the problem when clients are configured to use the buffered caching mode by default, and special applications that benefit from the more aggressive client-side caching of different cache types. You could improve the startup time or runtime of such applications by setting tuneFileCacheType = native on the client configuration file /etc/beegfs/beegfs-client.conf.

Consider that native cache relies on the underlying kernel and architecture, and as such, the workload behavior can vary greatly on circumstances. If you are considering native caching, it is recommended to quantify workloads via experimentation to ensure the end behavior is as expected.

When tests reveal that this setting decreases the application runtime, it is possible to take the following steps to activate that cache type only on the directory from where the application is loaded, while keeping the rest of the system using the default buffered cache type.

• Create a client config file (as a copy of the existing file beegfs-config.conf) to contain the configuration of the second mount point (e.g.: beegfs-config-native.conf). In this second configuration file, change the option tuneFileCacheType to native.

• Create a directory to be the second mountpoint of BeeGFS (e.g.: /mnt/private/beegfs_native).

• Add a new line to file beegfs-mounts.conf associating the path of the newly created client configuration file to the second mountpoint path and configuring it to be readonly (ro), as seen in the example below. Add rw to the first line, to make explicit that that is a writable mountpoint, as follows.

  /mnt/beegfs /etc/beegfs/beegfs-client.conf beegfs rw
  /mnt/private/beegfs_native /etc/beegfs/beegfs-client-native.conf beegfs ro
  

• Restart the BeeGFS client service. At this point, you should have a second mountpoint (e.g.: /mnt/private/beegfs_native) configured to use the native cache type. You could stop this procedure here and load the application from the second mountpoint. However, if you want to continue to use the same paths you used before, under the first mountpoint, take the next step.

• Create a bind mount of the application directory on the second mountpoint to the directory on the first mountpoint. For example, lets consider you have the following paths: - /mnt/beegfs - The main BeeGFS mountpoint. - /mnt/private/beegfs_native - The readonly mountpoint using native cache. - /mnt/private/beegfs_native/app - The directory that you want to mount at /mnt/beegfs/app Then, you should allow only the root user to access the directory /mnt/private and execute the following command to create the bind mount:

  $ mount -o bind,ro /mnt/private/beegfs_native/app /mnt/beegfs/app