Release Notes 8.3

BeeGFS 8.3 expands Remote Storage Targets to allow restoring objects from archive storage classes such as AWS Glacier, adds support for ID-mapping, and introduces community licensing.

Note

Please refer to GitHub for patch release notes.

General Changes/Improvements

  • Automatic Management Database Migration: When the database schema changes in a new BeeGFS version, the on-disk database will be upgraded when the service first starts. As before, a full database backup will be taken before upgrading to allow for manual downgrades. The manual --upgrade flag is now deprecated and effectively a no-op.

  • Background Rebalancing: Metadata and storage nodes now track statistics related to rebalancing progress that can be view using a new beegfs stats rebalance command.

  • Client Support for Linux 6.12 and 6.14: See Client Build Testing for details on specific Linux distributions that were tested.

  • Directory Listing Performance: Implements buffer-size aware directory listing that significantly improves performance for large directories. The metadata server now fills responses based on serialized entry size rather than a fixed 100-entry limit, maximizing network buffer utilization while preventing buffer overflows.

    • Configuration change: The client tuneMsgBufSize default has been increased from 64 KB to 1 MB to match the metadata server’s tuneWorkerBufSize (which was already 1 MB). The effective buffer size for directory listings is min(tuneMsgBufSize, tuneWorkerBufSize).

    • Performance tuning: For extremely large directories with very long filenames, these buffer sizes can be increased further, though 1 MB should be sufficient for most use cases.

    • Backward compatibility: Adds a handshake mechanism between client and server using network message flags to enable seamless upgrades in any order. When both client and metadata server are running 8.3 or later, the new buffer-size aware mode is automatically used. If either component is running an older version, the system automatically falls back to legacy behavior which uses fixed-size directory listing responses with a maximum of 100 entries per response.

  • Improved Target Registration: Metadata and storage nodes now provide a unique registration token when registering new targets with the management. This is used to detect and prevent misconfigurations where the same numerical ID is accidentally reused for multiple targets.

  • Support for ID-Mapped Mounts: BeeGFS can now be shared across different user namespaces / containers while presenting correct ownership for each mount. See the section on ID-Mapped Mounts for how to get started (requires kernel 5.12+).

Data Management API

  • Data management applications can set specific file data states using the data management API for more fine grained control over client behavior when access to a file is blocked by access flags. This notably allows offloaded/stubbed files to be automatically restored when they are opened, instead of always returning an error and requiring the user to try again later.

  • Data management applications can now use an ioctl to update file access flags and data states in addition to beegfs entry set.

Refer to the Data Management API section for more details.

Licensing (libbeegfs-license)

  • This package is now a required dependency of the BeeGFS management service and all file systems must have a valid community, enterprise, or temporary license to comply with the BeeGFS License Agreement.

    • Systems that currently do not have any license installed will begin prompting that a license is required after upgrading. More strict enforcement may be introduced in the future.

  • Community licenses can now be obtained by running beegfs license which will generate a system specific URL that automatically provides limited information about the deployment including the storage capacity, number of metadata and storage services, and the network protocol.

    • Note obtaining a community license is not possible until after systems are upgraded to 8.3 because obtaining a license is only possible using the URL generated by beegfs license. At this time a license can be obtained any time after upgrading. It is highly recommended to obtain a community license as soon as possible.

Refer to the Licensing page for more details.

Command Line Interface (CTL)

General:

  • Added a file type field to the --filter-files flag to simplify filtering by file types. Example: beegfs entry migrate --filter-files='type=file,symlink and glob(name, '*.txt')'. Previously filtering by file type was only possible by specifying the mode field.

  • A warning is now printed when running any command if the system does not have a valid license installed or if the installed license is nearing the expiration date.

New Commands:

  • A new stats rebalance command can be used to monitor the progress of background data rebalancing started by entry migrate.

  • A new remote job cleanup-orphaned command cleans up Remote database entries for paths that no longer exist in BeeGFS. This command reduces the size of the database and increases performance of commands like remote status and remote job list that otherwise always have to iterate over deleted entries. Note this command is optional as some users may wish to keep these entries around as a record of what data was synced even if the associated entries in BeeGFS no longer exist.

Command Updates:

  • The copy command now optionally applies original ownership and permissions from the source tree to the destination using --preserve-ownership and --preserve-permissions flags.

  • The entry info mode now displays the file access flags and data state columns by default. Files are unlocked and available unless the contents are being managed by an a data management application that uses the BeeGFS data management API (such as BeeGFS Remote and Sync).

  • The health check mode now warns if storage targets are not mapped to a node. Unmapped targets are also now displayed under target list (before they were omitted) and can now be removed using target remove. Most systems should not have unmapped targets, unless there are problems when a storage node initially registers its targets with the management (such as duplicate IDs).

  • The license command now includes information on how to obtain an enterprise or community license when the system is unlicensed, close to license expiration, or when a temporary license is installed. This information can also be viewed at any time using the new --get flag.

  • When executing remote push or remote pull a --priority (1-5) can now be specified to determine if certain jobs should take precedence over others.

  • The remote push now supports the --filter-files flag which offers a simple query language to select files to sync/offload based on POSIX metadata such as file access time, modification time, ownership, size, and more.

    Example: Offload all files anywhere in BeeGFS larger than 1GiB that have not been accessed in over 30 days to remote target 1.

    $ beegfs remote push /mnt/beegfs/ \
  --filter-files='size >= 1GiB and atime > 30d' \
  --stub-local \
  --remote-target=1

Remote Storage Targets

  • The Remote service now advertises what capabilities it supports so clients like BeeGFS CTL can identify if a feature requested by the user (such as remote push --filter-files) is actually supported by the Remote server. In turn the Remote service now requires all Sync services to support at least the same set of capabilities as itself, ensuring a consistent set of capabilities can be advertised to clients.

    • Rolling upgrades are still supported. Remote will wait and automatically bring Sync services online once they have been upgraded and support all required capabilities. Mixed versions are also still supported, however Sync services may need to be equal or newer versions than Remote.

  • Builder jobs responsible for walking directory trees in BeeGFS or S3 buckets and triggering Sync jobs resume more efficiently when Sync nodes are restarted by now walking entries in pages and resuming after a restart from the most recent page instead of re-walking all entries.

  • Overall job throughput has been improved by updating builder jobs to automatically throttle job creation and allow existing Sync jobs to complete before adding new work.

  • Remote Storage Targets can now be associated with archival storage classes to determine when an object is archived and how it should be restored. Refer to the beegfs-remote.toml configuration file for how to associate storage classes with a particular remote target.

  • Reworked the scheduler to add support for specifying a priority level and rescheduling work requests that are waiting on some external task to complete (such as restoring archived objects).

File System Checker (fsck)

  • The ability to run fsck on an online system has been improved by ignoring ENOENT errors that could happen when the storage service’s chunk fetcher attempted to read recently deleted chunk files or directories. Note that it is still recommended to execute fsck in read-only mode on online systems.

Fixes

  • Client: Improved cache invalidation mechanism for inodes that were re-striped via background data rebalancing to resolve an issue with hardlinks holding inode references during cache invalidation. This also eliminates a class of theoretical crashes due to access of an invalid stripe pattern.

  • Client: Add missing tunePathBufSize, tunePathBufNum, tuneMsgBufSize, and tuneMsgBufNum parameters to /proc/fs/beegfs/*/config.

  • Client: Renamed connTCPRecvBufSize and connUDPRecvBufSize to connTCPRcvBufSize and connUDPRcvBufSize under /proc/fs/beegfs/*/config to exactly match the client configuration parameter keys they correspond to.

  • Client: The file system UUID string and three strings associated with the root inode’s entry info were not freed before unmounting resulting in a small memory leak. This was inconsequential outside systems that remount BeeGFS many times without rebooting.

  • Client: When GPUDirect Storage (GDS) was enabled the client kernel module would not build on Debian 12.5 and Ubuntu 24.

  • CTL: The top level beegfs help command unnecessarily required root permissions to execute.

  • Mgmtd: Issuing systemctl stop beegfs-mgmtd would result in the management being killed instead of gracefully shutdown. This was because the management was not handling SIGTERM correctly.

  • Mgmtd: Events that indirectly changed storage pool target assignments did not trigger metadata and storage nodes to immediately pull updated pool configuration. These events included mapping new storage targets, creating or deleting buddy groups, and deleting storage targets. Updated configuration could take up to 300 seconds to propagate to all nodes.

  • Mgmtd: When user/groups were removed, quota entries associated with the removed IDs were still kept in the management database with no way to remove them. The management now automatically clears the target specific quota records for IDs no longer reported by that target.

  • Mgmtd: Reachability states are now more immediately updated globally (similar to the existing consistency state behavior). The previous behavior could notably cause a delay at startup before metadata nodes recognized storage targets were online. This issue was only ever observed in automated test environments that deployed then immediately started using the file system.

  • Meta/Storage/CTL: Improved logging and error messages to properly format IPv6 addresses by surrounding them with brackets, avoiding ambiguity with ports as defined in RFC 3986.

  • Meta/Storage: Previously warnings logged during early startup before full logging was initialized were silently dropped. Now these warnings are logged to standard error.

  • Meta/Storage: When the list of local network interfaces changed, if the new list was a prefix of the old one (or if all NICs were gone), the updated list was not propagated to other nodes.

  • Meta: Fixed an issue where the completion of a chunk balancer job could incorrectly release inode locks held by concurrent data management API file state update operations. These updates rely on the locking to atomically verify that no conflicting client read/write sessions are active before applying requested access flags. If the lock was released prematurely, a client could open the file in the short window between the session check and access flag update, leading to sessions that violate the intended access restrictions. This issue could only happen if file state updates happened at the exact moment a rebalancer job completes and was unlikely in real world scenarios.

  • Remote: When specifying --update to configure remote targets as part of a remote push/pull remote targets were only updated on file entries. They are now updated on directories as well.

  • Remote: Cleaning up jobs for files that no longer existed in BeeGFS would return an unnecessary error message unable to clear lock.

  • Storage: When refreshing local interfaces, if no usable NICs were found, the service could panic. Typically this happened due to hardware failures causing all NICs to become unavailable.

Known Issues and Limitations

  • Index cannot be installed on RHEL/Rocky/CentOS 10 due to missing dependencies (pcre).

  • IPv6 temporary, deprecated, and optimistic addresses are not automatically ignored by servers. These addresses may be advertised to other servers and clients as available for inbound connections if servers are not configured to listen or advertise specific interfaces, or if they use a wildcard or unspecified address notation. Refer to the 8.2 upgrade guide for more details.

  • The 8.0 and 8.1 CTL beegfs stats client command and Monitoring service are not forward compatible with 8.2 servers due to a breaking change required to support IPv6. When used with 8.2 the command will return Error: TCP request to meta:2 failed: error reading BeeMsg: EOF and Mon will log Received a message with incompatible feature flags. Message type: RequestMetaDataResp (6005); Flags (hex): 1. Upgrade CTL and Mon to 8.2 before upgrading BeeGFS servers to 8.2. The 8.2 CTL and Mon are also backwards compatible with 8.0 and 8.1.

  • Issues have been observed with Soft-RoCE and the client module on newer kernels that currently affect RHEL 9.4 and Ubuntu 24.04 where the kernel locks up on RDMA connection attempts to meta and storage servers. This issue has never been observed on RoCE implementations other than the kernel RXE driver.

Supported Linux Distributions and Kernels

Supported distributions

The full integration test suite was run on Debian 12, Debian 13, Rocky Linux 8.10, Rocky Linux 9.4, CentOS Stream 10, Ubuntu 22.04 and Ubuntu 24.04.

Packages are provided for the x86_64 and aarch64 architecture and the following distributions:

  • RHEL 8, 9, and 10 (packages can also be used on Rocky Linux and AlmaLinux).

  • SLES 15

  • Debian 11, 12, and 13

  • Ubuntu 20.04, 22.04, and 24.04

The distributions we provide packages for are fully supported by the BeeGFS server services (mgmtd, meta, storage, mon, remote, and sync). Generally specific kernel and RDMA driver versions are not problematic for the servers, but can cause issues building the client module. Refer to the Client Build Testing section below what combinations have been tested.

Client build testing

Client build testing was done with the following combinations of operating systems and RDMA drivers. To keep the test matrix manageable and because kernel APIs tend to change frequently, even in distribution minor releases, the BeeGFS client generally only prioritizes support for the default slow moving distribution kernels (not hwe, backports and similar). Custom kernel builds might or might not work and are generally not supported. There can also be incompatibilities between the client and very recent distribution kernel releases. The same is true for RDMA driver versions that can sometimes cause issues building the client for a specific kernel version.

Starting with BeeGFS 8.2, known combinations that currently do not work are published, so there are no surprises during planning and upgrades. In many cases these reflect breaking changes in upstream kernel/driver versions rather than issues in BeeGFS itself. We expect to support additional combinations over time, potentially in patch releases between minor versions. For the very latest updates check GitHub for available patch releases.

Additional notes:

  • Combinations not explicitly listed below might or might not work.

  • Results from Rocky are expected to apply to other RedHat derivatives including AlmaLinux and RHEL.

  • With the introduction of the DOCA repositories, BeeGFS supports OFED versions included in the following DOCA releases: 1.5-LTS, 2.5-LTS, and 2.9-LTS.

    • Note that OFED versions and DOCA versions follow different versioning schemes, so the supported OFED versions in each DOCA release may differ. Always refer to the specific DOCA release notes for detailed compatibility information.

Click to expand the test matrixes for different distributions:

Tested Rocky Linux versions (with kernel RDMA drivers)
RHEL/Rocky/Alma Kernel version Pass/Fail
10.1 6.12.0-124.31.1.el10_1.x86_64 ✅
10.0 6.12.0-55.41.1.el10_0.x86_64 ✅
10 6.12.0-124.31.1.el10_1.x86_64 ✅
9.7 5.14.0-611.27.1.el9_7.x86_64 ✅
9.6 5.14.0-570.58.1.el9_6.x86_64 ✅
9.5 5.14.0-503.40.1.el9_5.x86_64 ✅
9.4 5.14.0-427.42.1.el9_4.x86_64 ✅
9.3 5.14.0-362.24.1.el9_3.0.1.x86_64 ✅
9.2 5.14.0-284.30.1.el9_2.x86_64 ✅
9.1 5.14.0-162.23.1.el9_1.x86_64 ✅
9.0 5.14.0-70.30.1.el9_0.x86_64 ✅
9 5.14.0-611.27.1.el9_7.x86_64 ✅
8.10 4.18.0-553.97.1.el8_10.x86_64 ✅
8.9 4.18.0-513.24.1.el8_9.x86_64 ✅
8.8 4.18.0-477.27.1.el8_8.x86_64 ✅
8.7 4.18.0-425.19.2.el8_7.x86_64 ✅
8.6 4.18.0-372.32.1.el8_6.x86_64 ✅
8.5 4.18.0-348.23.1.el8_5.x86_64 ✅
8.4 4.18.0-305.25.1.el8_4.x86_64 ✅
8 4.18.0-553.97.1.el8_10.x86_64 ✅
Tested Rocky Linux versions (with DOCA/OFED RDMA drivers)
RHEL/Rocky/Alma Version DOCA Version OFED Version Kernel Version Pass/Fail
10.0 3.2.1-014000 OFED-internal-25.10-1.3.9: 6.12.0-55.41.1.el10_0.x86_64 ❌
10 3.2.1-044000 OFED-internal-25.10-1.7.1: 6.12.0-124.31.1.el10_1.x86_64 ❌
9.6 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 5.14.0-570.58.1.el9_6.x86_64 ✅
9.5 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 5.14.0-503.40.1.el9_5.x86_64 ✅
9.4 3.2.1-014000 OFED-internal-25.10-1.3.9: 5.14.0-427.42.1.el9_4.x86_64 ❌
9.4 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 5.14.0-427.42.1.el9_4.x86_64 ✅
9.4 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: 5.14.0-427.42.1.el9_4.x86_64 ✅
9.3 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 5.14.0-362.24.1.el9_3.0.1.x86_64 ✅
9.3 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: 5.14.0-362.24.1.el9_3.0.1.x86_64 ✅
9.1 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 5.14.0-162.23.1.el9_1.x86_64 ✅
9.1 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: 5.14.0-162.23.1.el9_1.x86_64 ✅
9 3.2.1-044000 OFED-internal-25.10-1.7.1: 5.14.0-611.27.1.el9_7.x86_64 ❌
8.10 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 4.18.0-553.97.1.el8_10.x86_64 ❌
8.10 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: 4.18.0-553.97.1.el8_10.x86_64 ✅
8.9 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 4.18.0-513.24.1.el8_9.x86_64 ✅
8.9 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: 4.18.0-513.24.1.el8_9.x86_64 ✅
8.8 3.2.1-014000 OFED-internal-25.10-1.3.9: 4.18.0-477.27.1.el8_8.x86_64 ❌
8.8 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 4.18.0-477.27.1.el8_8.x86_64 ✅
8.8 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: 4.18.0-477.27.1.el8_8.x86_64 ✅
8.7 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 4.18.0-425.19.2.el8_7.x86_64 ✅
8.6 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 4.18.0-372.32.1.el8_6.x86_64 ✅
8.6 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: 4.18.0-372.32.1.el8_6.x86_64 ✅
8.5 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 4.18.0-348.23.1.el8_5.x86_64 ✅
8.5 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: 4.18.0-348.23.1.el8_5.x86_64 ✅
8.4 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 4.18.0-305.25.1.el8_4.x86_64 ❌
8.4 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: 4.18.0-305.25.1.el8_4.x86_64 ❌
8 3.2.1-044000 OFED-internal-25.10-1.7.1: 4.18.0-553.97.1.el8_10.x86_64 ❌

Note

Rocky Linux 10.0 with kernel 6.12.0-55.41.1.el10_0 and DOCA OFED 3.2.1 (MLNX OFED 25.10), the BeeGFS client module compilation failed because the OFED DKMS sources were not configured against the installed kernel, leaving the required compat/config.h header ungenerated.

Tested Ubuntu versions (with kernel RDMA drivers)
Ubuntu Version Kernel version Pass/Fail
25.10 linux-headers-6.17.0-5 ❌
25.04 linux-headers-6.14.0-15 ✅
24.04.3 linux-headers-6.14.0-27-generic ✅
24.04.2 linux-headers-6.11.0-17-generic ✅
24.04.1 linux-headers-6.8.0-41 ✅
24.04 linux-headers-6.8.0-31 ✅
22.04.5 linux-headers-6.8.0-40-generic ✅
22.04.4 linux-headers-6.5.0-18-generic ✅
22.04.3 linux-headers-6.2.0-26-generic ✅
22.04 linux-headers-5.15.0-25 ✅
20.04.6 linux-headers-5.15.0-67-generic ✅
20.04.5 linux-headers-5.15.0-46-generic ✅
20.04.4 linux-headers-5.13.0-30-generic ✅
20.04.3 linux-headers-5.11.0-27-generic ❌
20.04.2 linux-headers-5.8.0-41-generic ✅
20.04.1 linux-headers-5.4.0-42 ✅
20.04 linux-headers-5.4.0-26 ✅
Tested Ubuntu versions (with DOCA/OFED RDMA drivers)
Ubuntu Version OFED Version DOCA Version Kernel Version Pass/Fail
25.04 linux-headers-6.14.0-15-generic ❌
25.04 linux-headers-6.14.0-15-generic ❌
24.04.3 3.2.1-044000 OFED-internal-25.10-1.7.1: linux-headers-6.14.0-27-generic ✅
24.04.3 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-6.14.0-27-generic ✅
24.04.3 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-6.14.0-27-generic ✅
24.04.2 3.2.1-044000 OFED-internal-25.10-1.7.1: linux-headers-6.11.0-17-generic ✅
24.04.2 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-6.11.0-17-generic ✅
24.04.2 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-6.11.0-17-generic ✅
24.04.1 3.2.1-044000 OFED-internal-25.10-1.7.1: linux-headers-6.8.0-41-generic ✅
24.04.1 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-6.8.0-41-generic ✅
24.04.1 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-6.8.0-41-generic ✅
24.04 3.2.1-044000 OFED-internal-25.10-1.7.1: linux-headers-6.8.0-31-generic ✅
24.04 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-6.8.0-31-generic ✅
24.04 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-6.8.0-31-generic ✅
22.04.5 3.2.1-044000 OFED-internal-25.10-1.7.1: linux-headers-6.8.0-40-generic ✅
22.04.5 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-6.8.0-40-generic ✅
22.04.5 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-6.8.0-40-generic ✅
22.04.5 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-6.8.0-40-generic ❌
22.04.4 3.2.1-044000 OFED-internal-25.10-1.7.1: linux-headers-6.5.0-18-generic ✅
22.04.4 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-6.5.0-18-generic ✅
22.04.4 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-6.5.0-18-generic ✅
22.04.4 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-6.5.0-18-generic ✅
22.04.3 3.2.1-044000 OFED-internal-25.10-1.7.1: linux-headers-6.2.0-26-generic ✅
22.04.3 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-6.2.0-26-generic ✅
22.04.3 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-6.2.0-26-generic ✅
22.04.3 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-6.2.0-26-generic ✅
22.04.2 not_installed linux-headers-5.19.0-32-generic ❌
22.04.2 not_installed linux-headers-5.19.0-32-generic ❌
22.04.2 not_installed linux-headers-5.19.0-32-generic ❌
22.04.2 not_installed linux-headers-5.19.0-32-generic ❌
22.04.1 not_installed linux-headers-5.15.0-43-generic ❌
22.04.1 not_installed linux-headers-5.15.0-43-generic ❌
22.04.1 not_installed linux-headers-5.15.0-43-generic ❌
22.04.1 not_installed linux-headers-5.15.0-43-generic ❌
22.04 3.2.1-044000 OFED-internal-25.10-1.7.1: linux-headers-5.15.0-25-generic ✅
22.04 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-5.15.0-25-generic ✅
22.04 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-5.15.0-25-generic ✅
22.04 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-5.15.0-25-generic ✅
20.04.6 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-5.15.0-67-generic ✅
20.04.6 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-5.15.0-67-generic ✅
20.04.6 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-5.15.0-67-generic ✅
20.04.6 linux-headers-5.15.0-67-generic ❌
20.04.5 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-5.15.0-46-generic ✅
20.04.5 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-5.15.0-46-generic ✅
20.04.5 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-5.15.0-46-generic ✅
20.04.5 linux-headers-5.15.0-46-generic ❌
20.04.4 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-5.13.0-30-generic ✅
20.04.4 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-5.13.0-30-generic ✅
20.04.4 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-5.13.0-30-generic ✅
20.04.4 linux-headers-5.13.0-30-generic ❌
20.04.3 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-5.11.0-27-generic ❌
20.04.3 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-5.11.0-27-generic ❌
20.04.3 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-5.11.0-27-generic ❌
20.04.3 linux-headers-5.11.0-27-generic ❌
20.04.2 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-5.8.0-41-generic ❌
20.04.2 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-5.8.0-41-generic ❌
20.04.2 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-5.8.0-41-generic ❌
20.04.2 linux-headers-5.8.0-41-generic ❌
20.04.1 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-5.4.0-42-generic ❌
20.04.1 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-5.4.0-42-generic ❌
20.04.1 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-5.4.0-42-generic ❌
20.04.1 linux-headers-5.4.0-42-generic ❌
20.04 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-5.4.0-26-generic ❌
20.04 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-5.4.0-26-generic ❌
20.04 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-5.4.0-26-generic ❌
20.04 linux-headers-5.4.0-26-generic ❌

Note

The some Ubuntu DOCA installation for latest-3.2-LTS failed because the NVIDIA DOCA APT repo is out of sync (package index size/hash mismatch), preventing installation of the DOCA OFED package.

Tested Debian versions (with kernel RDMA drivers)
Debian Version Kernel version Pass/Fail
13.3 linux-headers-6.12.63+deb13-amd64 ✅
13.2 linux-headers-6.12.57+deb13-amd64 ✅
13.1 linux-headers-6.12.48+deb13-amd64 ✅
13.0 linux-headers-6.12.38+deb13-amd64 ✅
12.13 linux-headers-6.1.0-42-amd64 ✅
12.12 linux-headers-6.1.0-41-amd64 ✅
12.11 linux-headers-6.1.0-37-amd64 ✅
12.10 linux-headers-6.1.0-34-amd64 ✅
12.9 linux-headers-6.1.0-31-amd64 ✅
12.8 linux-headers-6.1.0-28-amd64 ✅
12.7 linux-headers-6.1.0-26-amd64 ✅
12.6 linux-headers-6.1.0-23-amd64 ✅
12.5 linux-headers-6.1.0-21-amd64 ✅
12.4 linux-headers-6.1.0-17-amd64 ✅
12.2 linux-headers-6.1.0-13-amd64 ✅
12.1 linux-headers-6.1.0-12-amd64 ✅
12.0 linux-headers-6.1.0-9-amd64 ✅
11.11 linux-headers-5.10.0-37-amd64 ✅
11.10 linux-headers-5.10.0-31-amd64 ✅
11.9 linux-headers-5.10.0-30-amd64 ✅
11.8 linux-headers-5.10.0-27-amd64 ✅
11.7 linux-headers-5.10.0-25-amd64 ✅
11.6 linux-headers-5.10.0-21-amd64 ✅
11.5 linux-headers-5.10.0-19-amd64 ✅
11.4 linux-headers-5.10.0-17-amd64 ✅
11.3 linux-headers-5.10.0-37-amd64 ✅
11.2 linux-headers-5.10.0-37-amd64 ✅
11.1 linux-headers-5.10.0-37-amd64 ✅
11.0 linux-headers-5.10.0-37-amd64 ✅
Tested Debian versions (with DOCA/OFED RDMA drivers)
Debian Version OFED Version DOCA Version Kernel Version Pass/Fail
13.0 ❌
12.5 3.2.1-014000 OFED-internal-25.10-1.3.9: linux-headers-6.1.0-21-amd64 ✅
12.5 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-6.1.0-21-amd64 ✅
12.5 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-6.1.0-21-amd64 ✅
12.1 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-6.1.0-12-amd64 ✅
12 3.2.1-044000 OFED-internal-25.10-1.7.1: linux-headers-6.1.0-42-amd64 ✅
12 3.2.0-125000 OFED-internal-25.10-1.2.8: linux-headers-6.1.0-42-amd64 ✅
12 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-6.1.0-42-amd64 ✅
12 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: linux-headers-6.1.0-42-amd64 ✅
11.3 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-5.10.0-37-amd64 ✅
11 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: linux-headers-5.10.0-37-amd64 ✅

Note

The Debian 13.0 DOCA installation for latest-3.2-LTS failed because the NVIDIA DOCA APT repo for debian13.0 drops the TLS connection (OpenSSL unexpected EOF), so apt-get cannot fetch the package index.

Tested OpenSUSE versions (with kernel RDMA drivers)
OpenSUSE Version Kernel version Pass/Fail
16.0 6.12.0-160000.9-default ✅
15.6 6.4.0-150600.23.84-default ✅
15.5 5.14.21-150500.55.88-default ✅
15.4 5.14.21-150400.24.100-default ✅
Tested OpenSUSE versions (with DOCA/OFED RDMA drivers)
OpenSUSE Version OFED Version DOCA Version Kernel Version Pass/Fail
15.6 package doca-ofed is not installed 6.4.0-150600.23.84-default ❌
15.6 package doca-ofed is not installed 6.4.0-150600.23.84-default ❌
15.5 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 5.14.21-150500.53-default ✅
15.5 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: 5.14.21-150500.53-default ✅
15.4 3.2.1-014000 OFED-internal-25.10-1.3.9: 5.14.21-150400.24.100-default ❌
15.4 2.9.4-0.1.2 OFED-internal-24.10-4.1.4: 5.14.21-150400.22-default ❌
15.4 2.5.5-0.0.5 OFED-internal-23.10-6.1.6: 5.14.21-150400.22-default ✅

Hint

An green ✅ indicates the client builds with that combination. A red ❌ indicates the client does not currently build with that combination.

Version Interoperability

Starting with version 8, BeeGFS more strictly adheres to semantic versioning guarantees. This means BeeGFS 8.3 components are compatible with components running any BeeGFS 8.y.z release, however the use of new functionality introduced in 8.3 requires the involved components to be running 8.3.

Note that BeeGFS 8 is not compatible with older major versions of BeeGFS due to changes in network and on-disk formats.

Upgrading from Older Versions

To upgrade from an older version, please refer to the Upgrade Guide.