Release Notes v7.4.5

The BeeGFS 7.4.5 release adds support for Ubuntu 24.04 (Linux 6.8) and fixes a number of bugs.

General Changes/Improvements

  • Added support for lseek() with SEEK_CUR and offset=0 on directories, improving compatibility with NFS clients and applications that use this method to determine the current offset in a directory.

  • Updated RDMA tuning documentation for clarity and to include new connection parameters.

  • Added sysNoEnterpriseFeatureMsg state to client configuration in procfs for better integration with beegfs-ctl --mounted.

  • The client now supports Ubuntu 24.04 (Linux 6.8). This release also includes server packages for Ubuntu 24.04.

Fixes

  • Resolved a SIGBUS error in BeeGFS client caused by receiving stale file size information from the metadata server. This issue was particularly evident when files were accessed simultaneously by multiple applications, leading to incorrect file size updates and SIGBUS errors during memory-mapped operations.

  • Addressed a potential NULL pointer dereference in IBVSocket.c when protection domain (pd) allocation fails.

  • Fixed an issue in the client where creating and closing a lot of server connections could lead to general protection faults resulting in kernel crashes. The underlying issue was the RDMA connection event handler accessing data structures that were already in the process of being destroyed. This issue was only observed on RHEL 8.10.

  • Fixed multiple smaller issues with the CMake build.

Known Issues and Limitations

  • The Metadata daemon does not work reliably on RHEL/CentOS 8 and SLES 15.1 and 15.2 due to a problem in the versions of glibc. The problem was fixed in RHEL/CentOS 8.1 and SLES 15.3.

  • The client module does not compile on SLES 15.2 with Mellanox OFED 5.2

  • We observed some issues with RoCE in the client module on newer kernels that currently affect RHEL 9.4 and Ubuntu 24.04 where the kernel locks up on RDMA connection attempts to meta and storage servers. Infiniband based RDMA connections work as expected.

Supported Linux Distributions and Kernels

Supported distributions

Packages are provided for the x86_64 and aarch64 architecture and the following distributions:

  • RHEL 8 and 9 (packages can also be used on RockyLinux and Alma Linux)

  • SLES 15

  • Debian 10, 11 and 12

  • Ubuntu 18.04, 20.04 and 22.04, 24.04

The distributions we provide packages for are fully supported by the BeeGFS server services (mgmtd, meta, storage, mon). To keep the test matrix manageable and because kernel APIs tend to change frequently, even in distribution minor releases, the BeeGFS client generally only supports the default slow moving distribution kernels (not hwe, backports and similar). Custom kernel builds might or might not work and are generally not supported. There can also be incompatibilities between the client and very recent distribution kernel relases. We advise to have a look at the Client Build Testing section below for more information on what has been tested.

The same is true for RDMA driver versions that are generally not problematic on the servers, but can cause issues on the client. The following Mellanox OFED driver versions are supported: 24.07, 23.10, 5.8, 5.7, 5.6, 5.5, 5.4, 5.3, 5.2, 5.1, 5.0, 4.9

The full integration test suite was run on Debian 10, Debian 11, Debian 12, OpenSUSE 15.4, RHEL 8.5, Rocky 8.7, Rocky 9.4, Ubuntu 20.04, Ubuntu 22.04 and Ubuntu 24.04.

Client build testing

Client build testing was done on the following OS and RDMA driver combinations. Other kernel versions might or might not work. If a version of MOFED was successfully tested on one OS, it is very likely that it will also work in combination with other kernels.

Tested distribution and driver version combinations (click to expand)

  • RHEL 8.3: Kernel RDMA drivers, MOFED 4.9, 5.0, 5.1, 5.2, 5.3, 5.4

  • AlmaLinux 8.4: Kernel RDMA drivers, MOFED 5.3, 5.4

  • AlmaLinux 8.5: Kernel RDMA drivers, MOFED 5.3, 5.4, 5.5

  • AlmaLinux 8.7: Kernel RDMA drivers, MOFED 5.8

  • AlmaLinux 9.0: Kernel RDMA drivers, MOFED 5.6, 5.7

  • Rocky Linux 8.4: Kernel RDMA drivers, MOFED 5.3, 5.4

  • Rocky Linux 8.5: Kernel RDMA drivers, MOFED 5.5

  • Rocky Linux 8.6: Kernel RDMA drivers, MOFED 5.6

  • Rocky Linux 8.8: Kernel RDMA drivers, MOFED 5.8

  • Rocky Linux 8.9: Kernel RDMA drivers, MOFED 23.10

  • Rocky Linux 9.1: Kernel RDMA drivers, MOFED 5.8

  • Rocky Linux 9.2: Kernel RDMA drivers

  • Rocky Linux 9.3: Kernel RDMA drivers, MOFED 23.10, 24.1

  • Rocky Linux 9.4: Kernel RDMA drivers, MOFED 24.07

  • SLES 15.1: Kernel RDMA drivers, MOFED 5.0

  • SLES 15.2: Kernel RDMA drivers, MOFED 5.1, 5.4

  • SLES 15.3: Kernel RDMA drivers, MOFED 5.4, 5.5, 5.6, 5.8

  • SLES 15.4: Kernel RDMA drivers, MOFED 5.8

  • SLES 15.5: Kernel RDMA drivers, MOFED 5.8

  • Debian 10: Kernel RDMA drivers, MOFED 5.2, 5.3, 5.4

  • Debian 11: Kernel RDMA drivers, MOFED 5.6

  • Debian 12: Kernel RDMA drivers

  • Ubuntu 18.04: Kernel RDMA drivers

  • Ubuntu 20.04: Kernel RDMA drivers, MOFED 5.4

  • Ubuntu 22.04.1, 22.04.2, 22.04.3: Kernel RDMA drivers, MOFED 5.6, 5.7, 5.8

  • Ubuntu 24.04.0: Kernel RDMA drivers, MOFED 24.04

Version Interoperability

BeeGFS 7.4.5 is fully compatible with 7.4.4 and 7.4.3, mostly (see warning below) compatible with 7.4.x < 7.4.3 but not fully compatible with older versions of BeeGFS.

Warning

Necessary changes to a message used in metadata hard link creation cause an incompatibility between metadata services >= 7.4.3 and metadata services running versions < 7.4.3. It is therefore necessary to upgrade all metadata nodes in a BeeGFS system to versions >= 7.4.3 at once. Rolling upgrades of metadata services from older versions are not supported. All services other than metadata are not affected by this and can be upgraded in a rolling fashion or used together with other 7.4.x versions.

The incompatibilities between 7.4.5 and 7.3.x are limited to hard link functionality and a change in the monitoring message format. Please refer to the Upgrade Guide for more information.

Upgrading from Older Versions

To upgrade from an older version, please refer to the Upgrade Guide.