Release Notes v7.4.6

The BeeGFS 7.4.6 release adds support for kernels up to Linux 6.11 and fixes a number of bugs.

General Changes/Improvements

  • The client now supports kernel versions up to Linux 6.11 (for example Ubuntu 24.10).

  • The client will no longer invalidate file data caches when a file handle gets closed, which should lead to improved and more consistent caching behavior. It also fixes an issue where the native cache mode would no longer cache on systems that were up longer than tunePageCacheValidityMS milliseconds.

Fixes

  • Fixed an issue where RDMA connection initiation between servers or beegfs-ctl and servers would regularly take around 500ms because of a timeout after applying tight timing constraints while polling for an initial response. By using an exponential backoff strategy, connection setup should now take between one and eight milliseconds in most cases.

  • Fixed an issue with the detection of underlying device and file system UUIDs for devices that use large numbers for device major and/or minor IDs.

  • Fixed an issue with event logging where RmDir events would be logged even if the directory removal failed.

  • Fixed an issue where the client postrm script would fail to remove the client module directories on some operating system versions.

Known Issues and Limitations

  • The Metadata daemon does not work reliably on RHEL/CentOS 8 and SLES 15.1 and 15.2 due to a problem in the versions of glibc. The problem was fixed in RHEL/CentOS 8.1 and SLES 15.3.

  • The client module does not compile on SLES 15.2 with Mellanox OFED 5.2

  • We observed some issues with RoCE in the client module on newer kernels that currently affect RHEL 9.4 and Ubuntu 24.04 where the kernel locks up on RDMA connection attempts to meta and storage servers. Infiniband based RDMA connections work as expected.

Supported Linux Distributions and Kernels

Supported distributions

Packages are provided for the x86_64 and aarch64 architecture and the following distributions:

  • RHEL 8 and 9 (packages can also be used on RockyLinux and Alma Linux)

  • SLES 15

  • Debian 10, 11 and 12

  • Ubuntu 18.04, 20.04 and 22.04, 24.04

The distributions we provide packages for are fully supported by the BeeGFS server services (mgmtd, meta, storage, mon). To keep the test matrix manageable and because kernel APIs tend to change frequently, even in distribution minor releases, the BeeGFS client generally only supports the default slow moving distribution kernels (not hwe, backports and similar). Custom kernel builds might or might not work and are generally not supported. There can also be incompatibilities between the client and very recent distribution kernel relases. We advise to have a look at the Client Build Testing section below for more information on what has been tested.

The same is true for RDMA driver versions that are generally not problematic on the servers, but can cause issues on the client. The following Mellanox OFED driver versions are supported: 25.01, 24.10, 24.07, 23.10, 5.8, 5.7, 5.6, 5.5, 5.4, 5.3, 5.2, 5.1, 5.0, 4.9

With the introduction of the DOCA repositories, we now support Mellanox OFED versions included in the following DOCA releases: 1.5-LTS, 2.5-LTS, and 2.9-LTS. Note that Mellanox OFED versions and DOCA versions follow different versioning schemes, so the supported Mellanox versions in each DOCA release may differ. Always refer to the specific DOCA release notes for detailed compatibility information.

The full integration test suite was run on Debian 10, Debian 11, Debian 12, OpenSUSE 15.4, RHEL 8.5, Rocky 8.7, Rocky 9.4, Ubuntu 20.04, Ubuntu 22.04 and Ubuntu 24.04.

Client build testing

Client build testing was done on the following OS and RDMA driver combinations. Other kernel versions might or might not work. If a version of MOFED was successfully tested on one OS, it is very likely that it will also work in combination with other kernels.

Tested distribution and driver version combinations (click to expand)

  • RHEL 8.3: Kernel RDMA drivers, MOFED 4.9, 5.0, 5.1, 5.2, 5.3, 5.4

  • AlmaLinux 8.4: Kernel RDMA drivers, MOFED 5.3, 5.4

  • AlmaLinux 8.5: Kernel RDMA drivers, MOFED 5.3, 5.4, 5.5

  • AlmaLinux 8.7: Kernel RDMA drivers, MOFED 5.8

  • AlmaLinux 8.10: MOFED 24.10

  • AlmaLinux 9.0: Kernel RDMA drivers, MOFED 5.6, 5.7

  • Rocky Linux 8.4: Kernel RDMA drivers, MOFED 5.3, 5.4

  • Rocky Linux 8.5: Kernel RDMA drivers, MOFED 5.5

  • Rocky Linux 8.6: Kernel RDMA drivers, MOFED 5.6

  • Rocky Linux 8.8: Kernel RDMA drivers, MOFED 5.8

  • Rocky Linux 8.9: Kernel RDMA drivers, MOFED 23.10

  • Rocky Linux 9.1: Kernel RDMA drivers, MOFED 5.8

  • Rocky Linux 9.2: Kernel RDMA drivers

  • Rocky Linux 9.3: Kernel RDMA drivers, MOFED 23.10, 24.1

  • Rocky Linux 9.4: Kernel RDMA drivers, MOFED 24.07

  • Rocky Linux 9.5: MOFED 24.10

  • SLES 15.1: Kernel RDMA drivers, MOFED 5.0

  • SLES 15.2: Kernel RDMA drivers, MOFED 5.1, 5.4

  • SLES 15.3: Kernel RDMA drivers, MOFED 5.4, 5.5, 5.6, 5.8

  • SLES 15.4: Kernel RDMA drivers, MOFED 5.8

  • SLES 15.5: Kernel RDMA drivers, MOFED 5.8

  • Debian 10: Kernel RDMA drivers, MOFED 5.2, 5.3, 5.4

  • Debian 11: Kernel RDMA drivers, MOFED 5.6

  • Debian 12: Kernel RDMA drivers, MOFED 24.10

  • Ubuntu 18.04: Kernel RDMA drivers

  • Ubuntu 20.04: Kernel RDMA drivers, MOFED 5.4, 24.10

  • Ubuntu 22.04.1, 22.04.2, 22.04.3: Kernel RDMA drivers, MOFED 5.6, 5.7, 5.8, 24.10, 25.01

  • Ubuntu 24.04.0: Kernel RDMA drivers, MOFED 24.04, 24.10, 25.01

  • Ubuntu 24.10.0: Kernel RDMA drivers, MOFED 24.10

Version Interoperability

BeeGFS 7.4.6 is fully compatible with 7.4.5, 7.4.4 and 7.4.3, mostly (see warning below) compatible with 7.4.x < 7.4.3 but not fully compatible with older versions of BeeGFS.

Warning

Necessary changes to a message used in metadata hard link creation cause an incompatibility between metadata services >= 7.4.3 and metadata services running versions < 7.4.3. It is therefore necessary to upgrade all metadata nodes in a BeeGFS system to versions >= 7.4.3 at once. Rolling upgrades of metadata services from older versions are not supported. All services other than metadata are not affected by this and can be upgraded in a rolling fashion or used together with other 7.4.x versions.

The incompatibilities between 7.4.6 and 7.3.x are limited to hard link functionality and a change in the monitoring message format. Please refer to the Upgrade Guide for more information.

Upgrading from Older Versions

To upgrade from an older version, please refer to the Upgrade Guide.