Release Notes v7.4.3

The BeeGFS 7.4.3 release adds a new client to server ping, merges client makefiles and fixes remaining issues with the hard link code.

General Changes/Improvements

• Added a new ping mode to beegfs-ctl --ioctl to measure client to server latency. This mode requires both beegfs-client and beegfs-ctl to be updated to 7.4.3. See beegfs-ctl --ioctl --help for more information.

• Added a new client configuration option connRDMAKeyType which can be used to configure the type of rkey is used for RDMA registered memory regions. See the client configuration file for more information about supported key types and their properties.

• Converged the two client makefiles Makefile and Release.mk into one.

• Moved ioctl definitions into one common header file that is used by client and user space code.

• Enhanced the output of beegfs-ctl --getentryinfo --verbose to show more information about deinlined inodes. The output now includes separate sections for dentry and inode information and meta file path and node information. Note: If you have scripts that depend on parsing the verbose output of beegfs-ctl --getentryinfo, you might need to make changes to correctly parse the new format. Output of the non-verbose mode has not changed.

• Modified the log message that used to indicate that a service was built with NVFS support to also indicate if a service was built without support for NVFS.

Fixes

• Removed some code in the client that would cause kernel warnings like memcpy: detected field-spanning write

• Fixed a bug in beegfs-fsck which prevented deletion of orphaned inodes in systems with mirrored metadata servers.

• Made StatMsg a mirrored message to address a class of issues where unwanted resync could be triggered due to different return codes from primary and secondary buddies

• Fixed a bug that caused GDS I/O to fail with large chunk sizes (> 4MB).

Fixes related to changes around hard links

As detailed in the improvements and fixes sections above, version 7.4.3 fixes a lot of the known issues with changes around hard links. Updates to the issues listed in the 7.4.2 release notes:

• Fixed the race condition that could lead to metadata service crashes by improving the locking for the metadata service’s internal data structures.

• Further reduced the likelihood of races leading to dangling dentries, orphaned inodes and orphaned chunks.

• Mitigated the root cause of diverging inode states and the resulting different return codes between primary and secondary metadata nodes by the aforementioned change to the StatMsg.

Known Issues and Limitations

• The Metadata daemon does not work reliably on RHEL/CentOS 8 and SLES 15.1 and 15.2 due to a problem in the versions of glibc. The problem was fixed in RHEL/CentOS 8.1 and SLES 15.3.

• The client module does not compile on SLES 15.2 with Mellanox OFED 5.2

Supported Linux Distributions and Kernels

Packages are provided for the x86_64 and aarch64 architecture and the following distributions:

• RHEL 8 and 9 (packages can also be used on RockyLinux and Alma Linux)

• SLES 15

• Debian 10, 11 and 12

• Ubuntu 18.04, 20.04 and 22.04

The following Mellanox OFED driver versions are supported: 5.8, 5.7, 5.6, 5.5, 5.4, 5.3, 5.2, 5.1, 5.0, 4.9

The full integration test suite was run on Debian 10, Debian 11, Debian 12, OpenSUSE 15.4, RHEL 8.5, Rocky 8.7, Rocky 9.2, Ubuntu 20.04, Ubuntu 22.04 and Ubuntu 23.04.

Client build testing was done on the following OS and MOFED combinations. If a version of MOFED was successfully tested on one OS, it is very likely that it will also work in combination with other kernels.

• RHEL 8.3: Kernel RDMA drivers, MOFED 4.9, 5.0, 5.1, 5.2, 5.3, 5.4

• AlmaLinux 8.4: Kernel RDMA drivers, MOFED 5.3, 5.4

• AlmaLinux 8.5: Kernel RDMA drivers, MOFED 5.3, 5.4, 5.5

• AlmaLinux 8.7: Kernel RDMA drivers, MOFED 5.8

• AlmaLinux 9.0: Kernel RDMA drivers, MOFED 5.6, 5.7

• Rocky Linux 8.4: Kernel RDMA drivers, MOFED 5.3, 5.4

• Rocky Linux 8.5: Kernel RDMA drivers, MOFED 5.5

• Rocky Linux 8.6: Kernel RDMA drivers, MOFED 5.6

• Rocky Linux 8.8: Kernel RDMA drivers, MOFED 5.8

• Rocky Linux 8.9: Kernel RDMA drivers, MOFED 23.10

• Rocky Linux 9.1: Kernel RDMA drivers, MOFED 5.8

• Rocky Linux 9.2: Kernel RDMA drivers

• Rocky Linux 9.3: Kernel RDMA drivers, MOFED 23.10, 24.1

• SLES 15.1: Kernel RDMA drivers, MOFED 5.0

• SLES 15.2: Kernel RDMA drivers, MOFED 5.1, 5.4

• SLES 15.3: Kernel RDMA drivers, MOFED 5.4, 5.5, 5.6, 5.8

• SLES 15.4: Kernel RDMA drivers, MOFED 5.8

• SLES 15.5: Kernel RDMA drivers, MOFED 5.8

• Debian 10: Kernel RDMA drivers, MOFED 5.2, 5.3, 5.4

• Debian 11: Kernel RDMA drivers, MOFED 5.6

• Debian 12: Kernel RDMA drivers

• Ubuntu 18.04: Kernel RDMA drivers

• Ubuntu 20.04: Kernel RDMA drivers, MOFED 5.4

• Ubuntu 22.04.1, 22.04.2, 22.04.3: Kernel RDMA drivers, MOFED 5.6, 5.7, 5.8

Version Interoperability

BeeGFS 7.4.3 is mostly (see warning below) compatible with 7.4.x but not fully compatible with older versions of BeeGFS.

Warning

Necessary changes to a message used in metadata hard link creation cause an incompatibility between 7.4.3 metadata services and metadata services running other 7.4.x versions. It is therefore necessary to upgrade all metadata nodes in a BeeGFS system to 7.4.3 at once. Rolling upgrades of metadata services from older versions are not supported. All services other than metadata are not affected by this and can be upgraded in a rolling fashion or used together with other 7.4.x versions.

The incompatibilities between 7.4.3 and 7.3.x are limited to hard link functionality and a change in the monitoring message format. Please refer to the Upgrade Guide for more information.

Upgrading from Older Versions

To upgrade from an older version, please refer to the Upgrade Guide.