BeeGFS DKMS Client Setup

The beegfs-client-dkms package allows building the client kernel module using the DKMS infrastructure.

To avoid breaking existing workflows and deployment tools that may be in use on some sites, both beegfs-client and beegfs-client-dkms packages are provided. They are mutually exclusive and only one of them can be installed on a system at any given time.

Note

All the examples below were tested on a CentOS 7 machine. They may need to be adapted to other versions of CentOS or other distributions.

Changes Introduced in the DKMS Client Package

The DKMS variant of the BeeGFS client introduces some important changes:

  • the previous startup scripts have been removed as building and installing the kernel module is now done by DKMS;

  • as a consequence, module load and filesystem mount are now done via standard OS mechanisms.

DKMS Client Installation

Installing the Package

The EPEL repository needs to be activated on RHEL/CentOS to be able to install dkms.

Installing the package is done easily:

sudo yum install beegfs-client-dkms

And so is replacing the traditional client package with the DKMS one:

sudo yum swap -- remove beegfs-client -- install beegfs-client-dkms

DKMS will build the kernel module automatically during installation. The log file for the build is available there:

/var/lib/dkms/beegfs/<BeeGFS version>/$(uname -r)/x86_64/log/make.log

Note

The module is not loaded automatically by DKMS after build.

Loading the Kernel Module

Once built with DKMS, the kernel module can be loaded like any other:

sudo modprobe beegfs

This can also be done automatically at boot via an entry in /etc/modules-load.d/. For example:

$ cat /etc/modules-load.d/beegfs-client-dkms.conf
# Load the BeeGFS client module at boot
beegfs

Thirdparty OFED installations

To use thirdparty RDMA drivers, for example Mellanox OFED, specify the path to the driver’s include directory in /etc/beegfs/beegfs-client-build.mk:

OFED_INCLUDE_PATH=/usr/src/ofa_kernel/default/include

and then install the beegfs-client-dkms package.

Note that if the BeeGFS client is already installed, for example during upgrades if the OFED driver, a rebuild of the BeeGFS kernel module via DKMS is required. This might not happen automatically, but can be achieved by simply reinstalling the beegfs-client-dkms package.

NB: If a third party RDMA driver is installed and OFED_INCLUDE_PATH is not properly defined to point at the third party kernel headers, it is likely that the beegfs kernel module will fail to load. To build the client without RDMA capabilities, define:

BEEGFS_NO_RDMA=1

Mounting the Filesystem

Mounting a BeeGFS filesystem is now done via an entry in /etc/fstab.

A typical entry would look like this:

beegfs_nodev  /mnt/beegfs  beegfs  rw,relatime,cfgFile=/etc/beegfs/beegfs-client.conf,_netdev,x-systemd.after=beegfs-helperd  0 0

In those entries:

  • the first and third fields must always be beegfs_nodev and beegfs;

  • the second field contains the filesystem mount path, here /mnt/beegfs;

  • the mount option cfgFile= specifies the path of the BeeGFS client configuration file for that mount point.

  • the option x-systemd.after=beegfs-helperd ensures the mounting happens after the helper daemon has been started. This can be removed if running without helper daemon.

In case the client is running on the same machine as any of the BeeGFS daemons consider adding x-systemd.after=beegfs-mgmtd, x-systemd.after=beegfs-storage, or x-systemd.after=beegfs-meta as appropriate to ensure correct startup order.

This example will mount the filesystem automatically at boot, after the network devices have come up. To disable automatic mounting add noauto.

Finally mount the BeeGFS filesystem like any other filesystem:

sudo mount /mnt/beegfs

Known Limitations

BeeOND is currently incompatible with the BeeGFS DKMS client package, as it calls the traditional startup scripts.