Release Notes v7.2.14

The BeeGFS 7.2.14 release adds new configuration to mitigate a performance issue with the native cache mode, stabilizes metadata Buddy Mirroring and includes multiple bug fixes.

General Changes/Improvements

  • Added a new configuration option sysXAttrsCheckCapabilities that introduces more fine grained control over how often the kernel will check for a security.capabilities extended attribute and setuid/setgid bits when writing to files. This resolves a performance issue with the native cache mode where every write to a file would trigger a metadata getxattr request, leading to significantly higher load and lower write throughput when extended attributes were enabled on the client and the meta. For more detailed information about configuration options and security and performance implications, please see our documentation on the topic and the configuration option documentation in the client configuration file.

Note

The new configuration default (“never”) favors performance and potentially reduces security in systems that use the native cache mode and have extended attribute support enabled, while preserving legacy behavior in the buffered cache mode. Also note that by default, the BeeGFS client is configured to mount with the option nosuid, which disables file system based privilege elevation entirely and therefore removes the risk for privilege escalations based on capabilities or setuid bits. If used together with nosuid, we consider sysXAttrsCheckCapabilities = never a safe default.

  • Buddy Mirroring stabilization: Converted some meta service network messages to mirrored messages to leverage the locking mechanism in the mirrored message framework and prevent race conditions between filesystem operations. These race conditions could previously trigger unnecessary resync events due to differing return codes from primary and secondary meta nodes.

  • Buddy Mirroring stabilization: Updated code related to locking in the legacy hardlink message to prevent a very rare race condition with the unlink operation, which could also trigger unwanted resyncs due to differing return codes.

  • beegfs-ctl --migrate will now print a warning when it encounters an unsupported file type.

  • Added log file messages and ctl output to indicate when and which BeeGFS Enterprise Features as defined in the EULA are used or being configured.

  • Tweaked package dependencies and build dependencies, so the client package automatically pulls in all required kernel header packages. This includes some documentation changes that remove obsolete dependencies.

Fixes

  • Fixed an issue on systems that have a PAGE_SIZE != 4096 where the new configuration options connRDMAFragmentSize and connRDMAMetaFragmentSize defaulted to 4096 instead of the page size, leading to issues during RDMA buffer allocation. connRDMAFragmentSize now defaults to a new special value page and all connRDMAMeta* configuration options now default to the new value default that indicates that their connRDMA* counterparts should be used. For more information, please see the client configuration file.

  • Fixed a regression introduced by the previous release where the example code in beegfs-client-devel would not compile because of missing constants in beegfs_client.h.

  • Backported a bugfix for beegfs-fsck from b7.4 that resolves an issue where orphaned inodes were not being removed during filesystem checks in meta-mirrored configurations.

  • Fixed a (purely cosmetical) bug that caused port numbers to be displayed incorrectly in utilities like beegfs-ctl --listnodes or beegfs-check-servers due to an endianness conversion issue.

  • Fixed an issue that caused an incorrect device to be logged during file system UUID check.

Known Issues and Limitations

  • The Metadata daemon does not work reliably on RHEL/CentOS 8 and SLES 15.1 and 15.2 due to a problem in the versions of glibc. The problem was fixed in RHEL/CentOS 8.1 and SLES 15.3.

  • The client module does not compile on SLES 15.2 with Mellanox OFED 5.2

Supported Linux Distributions and Kernels

Supported distributions

Packages are provided for the x86_64 and aarch64 architecture and the following distributions:

  • RHEL 7 and 8

  • SLES 15

  • Debian 9 and 10

  • Ubuntu 18

  • Ubuntu 20

  • Rocky Linux and AlmaLinux should now be configured to use the RHEL 8 repository (note the known issue with RoCE and RHEL 8.5 described above)

The distributions we provide packages for are fully supported by the BeeGFS server services (mgmtd, meta, storage, mon). To keep the test matrix manageable and because kernel APIs tend to change frequently, even in distribution minor releases, the BeeGFS client generally only supports the default slow moving distribution kernels (not hwe, backports and similar). Custom kernel builds might or might not work and are generally not supported. There can also be incompatibilities between the client and very recent distribution kernel relases. We advise to have a look at the Client Build Testing section below for more information on what has been tested.

The same is true for RDMA driver versions that are generally not problematic on the servers, but can cause issues on the client. The following Mellanox OFED driver versions are supported: 5.8, 5.7, 5.6, 5.5, 5.4, 5.3, 5.2, 5.1, 5.0, 4.9

The full integration test suite was run on CentOS 7.9, RHEL 8.5, Rocky 8.6, Rocky 8.7, Alma 8.5, and Debian 10.

Client build testing

Client build testing was done on the following OS and RDMA driver combinations (click to expand). Other kernel versions might or might not work. If a version of MOFED was successfully tested on one OS, it is very likely that it will also work in combination with other kernels.

Tested distribution and driver version combinations

  • RHEL 7.9: Kernel RDMA drivers, MOFED 4.9, 5.0, 5.2, 5.3, 5.4

  • RHEL 8.3: Kernel RDMA drivers, MOFED 4.9, 5.0, 5.1, 5.2, 5.3, 5.4

  • AlmaLinux 8.4: Kernel RDMA drivers, MOFED 5.3, 5.4

  • AlmaLinux 8.5: Kernel RDMA drivers, MOFED 5.3, 5.4, 5.5

  • AlmaLinux 8.7: Kernel RDMA drivers, MOFED 5.8

  • Rocky Linux 8.4: Kernel RDMA drivers, MOFED 5.3, 5.4

  • Rocky Linux 8.5: Kernel RDMA drivers, MOFED 5.5

  • Rocky Linux 8.6: Kernel RDMA drivers, MOFED 5.6

  • Rocky Linux 8.7: MOFED 5.8

  • Rocky Linux 8.8: Kernel RDMA drivers, MOFED 5.8

  • SLES 15.1: Kernel RDMA drivers, MOFED 5.0

  • SLES 15.2: Kernel RDMA drivers, MOFED 5.1, 5.4

  • SLES 15.3: Kernel RDMA drivers, MOFED 5.4, 5,5, 5.6

  • Debian 9: Kernel RDMA drivers, 5.4

  • Debian 10: Kernel RDMA drivers, MOFED 5.2, 5.3, 5.4

  • Ubuntu 18: Kernel RDMA drivers

  • Ubuntu 20: Kernel RDMA drivers, MOFED 5.4

Version Interoperability

BeeGFS v7.2.14 is compatible with BeeGFS 7.2.x.

Upgrading from Older Versions

To upgrade from an older version, please refer to the Upgrade Guide.