Client Side Caching Modes

BeeGFS clients provide two different file cache modes, configurable using the client configuration option tuneFileCacheType:

• buffered: uses a pool of small static buffers for write-back and read-ahead

• native: uses the Linux kernel page-cache

The buffered mode typically provides higher streaming throughput compared to the native cache mode. The buffered mode will only cache a few hundred kilobytes of a file, while the native mode is able to cache multiple gigabytes, dynamically adapting to the amount of available client RAM. The native mode is suitable for workloads that fit entirely in the available Linux client cache.

Note that while native mode is stable, it relies on the Linux kernel mechanisms, which might deviate between versions and architectures. As such, there may be cases where performance and/or client behavior may be different than expected.

The buffered mode is the default cache type, and it is also the only valid cache type for clients that are running on the same machine as a BeeGFS server daemon.

The cache type can be set in the client configuration file (/etc/beegfs/beegfs-client.conf).

Note that BeeGFS servers also dynamically use available RAM for caching, independent of the client cache type.

Is it possible to use different caching types in different directories of the same file system?

Yes, it is. Usually, this is a common solution for the problem when clients are configured to use the buffered caching mode by default, and special applications that benefit from the more aggressive client-side caching of different cache types. You could improve the startup time or runtime of such applications by setting tuneFileCacheType = native on the client configuration file /etc/beegfs/beegfs-client.conf.

Consider that native cache relies on the underlying kernel and architecture, and as such, the workload behavior can vary greatly on circumstances. If you are considering native caching, it is recommended to quantify workloads via experimentation to ensure the end behavior is as expected.

When tests reveal that this setting decreases the application runtime, it is possible to take the following steps to activate that cache type only on the directory from where the application is loaded, while keeping the rest of the system using the default buffered cache type.

• Create a client config file (as a copy of the existing file beegfs-config.conf) to contain the configuration of the second mount point (e.g.: beegfs-config-native.conf). In this second configuration file, change the option tuneFileCacheType to native and set connClientPort to a unique port not used by any other client mount or application on this machine.

• Create a directory to be the second mount point of BeeGFS (e.g.: /mnt/private/beegfs_native).

• Add a new line to file beegfs-mounts.conf associating the path of the newly created client configuration file to the second mount point path and configuring it to be readonly (ro), as seen in the example below. Add rw to the first line, to make explicit that that is a writable mount point, as follows.

  /mnt/beegfs /etc/beegfs/beegfs-client.conf beegfs rw
  /mnt/private/beegfs_native /etc/beegfs/beegfs-client-native.conf beegfs ro
  

• Restart the BeeGFS client service. At this point, you should have a second mount point (e.g.: /mnt/private/beegfs_native) configured to use the native cache type. You could stop this procedure here and load the application from the second mount point. However, if you want to continue to use the same paths you used before, under the first mount point, take the next step.

• Create a bind mount of the application directory on the second mount point to the directory on the first mount point. For example, lets consider you have the following paths: - /mnt/beegfs - The main BeeGFS mount point. - /mnt/private/beegfs_native - The readonly mount point using native cache. - /mnt/private/beegfs_native/app - The directory that you want to mount at /mnt/beegfs/app Then, you should allow only the root user to access the directory /mnt/private and execute the following command to create the bind mount:

  $ mount -o bind,ro /mnt/private/beegfs_native/app /mnt/beegfs/app