Release Notes v7.3.3

The BeeGFS 7.3.3 release includes some improvements to network interface handling and BeeOND, adds support for InfluxDB v2 to the monitoring service and includes multiple other fixes and QoL improvements. It also adds support for new operating system and OFED versions.

Warning

Before upgrading to this version from any version <7.3.1, please read the section about mandatory Authentication carefully. If you have not configured a connAuthFile, services will no longer start without configuration changes. For more background on why we make this change, please refer to our blog post on the topic.

General Changes/Improvements

  • Server nodes can now to detect when local network interface availabilty changes and send updates to the mgmtd which will relay the information to the clients. This improves responsiveness to device failures, which is especially important in HA setups, by enabling clients to invalidate connections to failed devices without having to wait for timeouts. This also has the side effect of enabling server nodes to detect and use network devices that become available after the BeeGFS service has already started.

  • Targets specified in the target file for BeeOND (beeond start -t <file>) can now be explicitly assigned to the default pool. Before this change, specifying a pool called “Default” or “default” in that file would create a new pool with that name instead of assigning the targets to pool ID 1.

  • Added support for sending file event updates from the event listener to the BeeGFS Hive Index update utility. See beegfs-event-listener for more information.

  • Added support for InfluxDB v2 to the BeeGFS monitoring service and updated the Grafana dashboards to work with InfluxDB v2. Also add a new general overview panel to the Grafana dashboards. See Monitoring service for more information on how to setup the monitoring service and integrate the new dashboards.

  • Added the new parameter connInterfacesList to the client, that can be used to supply a list of interfaces on the command line instead of reading it from a file.

  • Added the option BEEGFS_NO_RDMA to disable RDMA support in the BeeGFS client. This can be useful if no RDMA support is needed and the client build fails because of RDMA driver conflicts. Please see beegfs-client-autobuild.conf for more information.

  • Changed the way to specify the path to nvfs headers in the client build configuration. This is relevant when building the client module with GDS support. Please see beegfs-client-autobuild.conf for more information.

  • Added a clearer error message when beegfs-ctl uses a configuration file for a file system that doesn’t serve the path that is supplied. This is mostly relevant when beegfs-ctl --getentryinfo is used without specifying a configuration file (--cfgFile) or mount point (--mount) on machines that mount multiple different BeeGFS instances.

  • Implemented a cooling mechanism to avoid sending UDP data too quickly which could lead to socket disconnects with messages like Hard Disconnect from Listen(Port: 8008): Operation not permitted.

  • Some improvements to RDMA error logging.

Fixes

  • Replaced the mechanism that the mgmtd uses to store its status files on shutdown with a more robust one. This fixes an issue where system crashes could lead to empty status files in certain scenarios.

  • Fixed multiple issues with network device priority calculation in the client module, both for GDS enabled systems and systems that don’t use GDS. This leads to more optimal data paths regarding NUMA nodes and PCIe topology.

  • Fixed an issue where fsck would recreate a missing inode on a wrong node. Also added functionality that ensures that recreated content directories contain the special #fSiDs# subdirectory to avoid issues with buddy mirroring resyncs.

  • Fixed an issue where the meta service would crash on startup when it received quota information from the mgmtd that contained nodes that it didn’t expect.

  • Fixed an issue where beegfs-ctl --removenode --nodetype=client would crash when called with the flag --unreachable.

  • Minor packaging and QoL fixes.

Known Issues and Limitations

  • The Metadata daemon does not work reliably on RHEL/CentOS 8 and SLES 15.1 and 15.2 due to a problem in the versions of glibc. The problem was fixed in RHEL/CentOS 8.1 and SLES 15.3.

  • The client module might hang if the IBV device(s) used are taken offline during operation.

  • The client module does not compile on SLES 15.2 with Mellanox OFED 5.2

Supported Linux Distributions and Kernels

Packages are provided for the x86_64 and aarch64 architecture and the following distributions:

  • RHEL 8 and 9 (packages can also be used on RockyLinux and Alma Linux)

  • SLES 15

  • Debian 10 and 11

  • Ubuntu 18.04, 20.04 and 22.04

The following Mellanox OFED driver versions are supported: 5.8, 5.7, 5.6, 5.5, 5.4, 5.3, 5.2, 5.1, 5.0, 4.9

The full integration test suite was run on Alma 9.0, Rocky 8.5, Rocky 8.7, Rocky 9.1, RHEL 8.5, Rocky 8.6, Alma 8.5, Debian 10, 11 and Ubuntu 20.04 and 22.04.

Client build testing:

  • RHEL 8.3: no OFED, OFED 4.9, 5.0, 5.1, 5.2, 5.3, 5.4

  • AlmaLinux 8.4: no OFED, OFED 5.3, 5.4

  • AlmaLinux 8.5: no OFED, OFED 5.3, 5.4, 5.5

  • AlmaLinux 8.7: no OFED, OFED 5.8

  • AlmaLinux 9.0: no OFED, OFED 5.6, 5.7

  • Rocky Linux 8.4: no OFED, OFED 5.3, 5.4

  • Rocky Linux 8.5: no OFED, OFED 5.5

  • Rocky Linux 8.6: no OFED, OFED 5.6

  • Rocky Linux 9.1: no OFED, OFED 5.8

  • SLES 15.1: no OFED, OFED 5.0

  • SLES 15.2: no OFED, OFED 5.1, 5.4

  • SLES 15.3: no OFED, OFED 5.4, 5.5, 5.6

  • SLES 15.4: OFED 5.5

  • Debian 10: no OFED, OFED 5.2, 5.3, 5.4

  • Debian 11: no OFED, OFED 5.6

  • Ubuntu 18.04: no OFED

  • Ubuntu 20.04: no OFED, OFED 5.4

  • Ubuntu 22.04: no OFED, OFED 5.6, 5.7

Version Interoperability

BeeGFS v7.3.3 is compatible with BeeGFS v7.3.x.

Upgrading from Older Versions

To upgrade from an older version, please refer to the Upgrade Guide.