Mounting multiple versions of BeeGFS at the same time

If you have multiple instances of BeeGFS file systems with different versions and need to access them from one client machine at the same time (for example during upgrades), you can use the beegfs-client-compat package. This provides a separate kernel module, allowing to mount the file system of the other version at the same time.

Client

Suppose a client machine is running version A and now shall also access a file system running version B. Install the beegfs-client-compat package from the BeeGFS repository of version B. For example, a BeeGFS version 6 compatibility client:

wget https://www.beegfs.io/release/beegfs_6/dists/deb9/x86_64/beegfs-client-compat_6.18-debian9_all.deb
dpkg -i beegfs-client-compat_6.18-debian9_all.deb

Then the kernel module needs to be build:

$ /etc/init.d/beegfs-client rebuild

This will rebuild both kernel modules.

Node: The init.d script currently does not load the legacy version of the module automatically. We need to do this manually:

$ insmod $(find /lib/modules/$(uname -r) -name "beegfs-*.ko")

To make this persistent, add the module to /etc/modules.

Then we need a configuration file for the compatibility client. The best is to copy an existing one from somewhere, since the options might have changed between versions. Place it here: /etc/beegfs/beegfs-client-compat.conf Edit it, and change the connClientUDPPort to avoid conflict with the other client instance.

Then create an additional mount point, for example /mnt/beegfs_compat and add it to /etc/beegfs/beegfs-mounts.conf (replace versB with the other version number, for example v6):

/mnt/beegfs /etc/beegfs/beegfs-client.conf

# The mountpoint for the old version (versB)
/mnt/beegfs_compat /etc/beegfs/beegfs-client-compat.conf beegfs-versB

Note that the file system type of the compatibility system must be explicitly specified as the third argument in the line.

Additionally, a helperd must be configured for the legacy client, as described in the following section.

Finally, restarting the BeeGFS client service will build and load the additional compatibility kernel module and mount both file systems.

Helperd

../_images/helperd_setups.svg

The client kernel module needs the user space helper daemon to be able to do DNS resolutions and to write log messages to a different location than the kernel log.

Since there is no helperd compatibility package, the parallel installation of multiple versions is not easily possible. There are three workarounds:

  • Configure the compatibility client module to run without helpderd. Log messages will go to syslog and the IP address of the management daemon must be used instead of the host name.

  • Using a helperd instance of the “compat” version on a different host.

  • Running the helperd in a virtualised or container environment, like for example docker, libvirt, etc.

Running without helperd

The beegfs-helperd is used by the client kernel module for logging to a file instead of the general kernel log. This allows BeeGFS related messages to be collected in a separate place (for example /var/log/beegfs-client.log). The helperd is also used to resolve host names to IP addresses and communicate them back to the kernel module. Usually, only the name of the management host needs to be resolved.

It is possible to configure the BeeGFS client without a helperd. The following settings need to be changed in the /etc/beegfs/beegfs-client.conf file:

  • Switch log output to the kernel log: logType=syslog. You can retrieve BeeGFS related messages like this (for example): grep beegfs /var/log/syslog

  • Remove the need of DNS resolutions by specifying the IP addresses of the management daemon directly, instead of a host name, sysMgmtdHost = ...

  • Setting sysMountSanityCheckMS=0. This disables a check which normally ensures a helperd can be reached.

Then restart the beegfs-client service.

Note: You might see messages like this one in the log file:

Aug  7 09:34:31 node-002 kernel: [ 3510.794297] beegfs-6: beegfs-6_XNodeS(14218): Messaging (RPC): Unable to connect to: beegfs-helperd helperd [ID: 0]

These can be ignored and are not relevant.

Running helperd on a different host

The package conflict of the two helperd versions can be avoided by installing one of them on a different machine and point the BeeGFS clients to it. Logging and DNS resolutions will be performed on that machine for all clients that use it, resulting in a combined log file.

Setup steps:

  • Install beegfs-helperd-versB on a machine that is not currently running another helperd instance and start the service.

  • Adjust the logHelperdIP-setting in the compatibility client configuration file (enter the IP address of the helperd host).

  • Restart the beegfs-client services.

Using a virtualized environment

Instead of a central helperd instance, as described above, it is possible to use a virtualised environment, which can then run on the same host together with the clients. A local port can be forwarded into the container and the log file can be written to a shared folder.

The necessary configuration steps are the same as above, with logHelperdIP being the address of the virtual machine or the container interface.

Some commonly used virtualisation software:

  • Lightweight container engines like Docker, rkt, LXD: Easy to set up and use, some dependencies and system requirements. (recommended)

  • Simple chroot environments (schroot, systemd-nspawn): Less requirements, but more work.

  • Fully virtualised systems (for example qemu/libvirt): Larger resource overhead.