Release Notes 8.4¶
BeeGFS 8.4 introduces the ability to automatically sync file changes and restore stubbed files in BeeGFS using Remote Storage Targets, experimental support for NFSv4-style ACLs and experimental (opt-in) long-term client-side metadata caching.
Note
Please refer to GitHub for patch release notes.
General Changes/Improvements¶
Client Support for Linux 6.19 and 7.0: See Client Build Testing for details on specific Linux distributions that were tested.
NFSv4 ACL Support (enterprise feature / experimental): In addition to POSIX ACLs, NFSv4-style ACLs can now be used, providing a richer feature set and more fine grained permission controls. See the Access Control Lists section for how to get started.
Long-Term Client-Side Metadata Caching (experimental): This provides an alternative to the existing time-based caching mechanism, allowing clients to indefinitely cache certain metadata (i.e., entry attributes and lookup results) until it is explicitly invalidated by the owning metadata service. This provides a significant performance improvement for workloads that frequently re-read metadata that is infrequently modified. This feature is disabled by default, for more details see the section on Client Metadata Caching.
Command Line Interface (CTL)¶
General:
Added alerts after regular command output (printed to stderr) to surface critical issues, for example if targets are running low on space or unreachable. These health checks run with every command but can be skipped by setting the global
--disable-alertsflag.Renamed the “ethernet” NIC type “TCP” for consistency with other BeeGFS components.
Information about BeeGFS entries is now collected using the new
BEEGFS_IOC_GET_ENTRYINFO_V2ioctl, improving performance across all commands that rely on entry info.
New Commands:
A new
index migratecommand to migrate legacy Hive Index databases to the new format.
Command Updates:
Added a hint to
quota list-usagewhen no information is returned that either the IDs don’t exist or are not configured in the quota system.The
health checkmode now checks the license status and warns when licenses are nearing expiration or expired.The
health checkmode now checks if the mgmtd has TLS enabled and warns when the mgmtd’s TLS certificate is nearing expiration or expired.“Temporary” licenses were renamed “trial” licenses in the
licenseoutput.The
indexcommands now support working with a remote Index using the--index-addrflag.Verbose output in the
copy --list-diffmode has been improved so-v 1lists different/missing files as they are checked/compared and-v 2lists each checked file and whether it is missing or differs.
Data Management API¶
Added a new
BEEGFS_IOC_GET_ENTRYINFO_V2ioctl that allow callers to fetch all BeeGFS metadata including the stripe pattern, RST configuration, session counts, file data state, and more.
Refer to the Data Management API section for more details.
Remote Storage Targets¶
Support for automatically triggering data movement based on file system modifications. See Configuring Automatic Sync and Restore to get started.
Files and directories can be configured to be kept automatically in sync with one or more remote targets by setting a
--remote-cooldownthat indicates when to start syncing after a file is closed. Cooldowns are set on a per file basis, and inherited from parent directories by default.Set a
--restore-policyof auto or delayed when offloading a file withbeegfs remote push --stub-localto optionally allow the file’s contents to be automatically restored if a user opens the file. By default the restore policy defaults to manual to preserve pre-8.4 behavior.
Support for exporting metrics and structured logs via the OpenTelemetry Protocol to backends such as Prometheus and Telegraf/InfluxDB. See RST Monitoring for more details.
Remote and Sync services now explicitly require being run as root to avoid job failures due to insufficient permissions and to ensure consistent behavior setting configuration. This also provides a performance optimization by dropping repeated euid checks after startup.
Index¶
BeeGFS Index (formerly known as BeeGFS Hive Index) is a downstream redistribution of the Grand Unified File Index (GUFI) that includes support for BeeGFS specific metadata, and integrates tightly with BeeGFS tooling such as CTL.
This release updates the BeeGFS Index distribution to the latest version of GUFI, and moves the support for BeeGFS specific metadata into a new file system plugin. This will require a one-time migration from the old database schema, but simplifies keeping the Index distribution in sync with the upstream GUFI releases going forward. See the BeeGFS Index section for more details and how to migrate.
Note
As part of this change the beegfs-hive-index package has been retired and replaced by the
beegfs-index package. This was done to prevent accidental upgrades since the new version of
Index is not directly compatible with the legacy database schema used by the Hive Index package.
Licensing (libbeegfs-license)¶
When no license is installed on a system, the number of clients is now limited to five. Install any license on the system to remove this restriction.
Each system is now limited to a single trial license.
BeeOND now requires a valid community or enterprise license to start.
Refer to the Licensing page for more details.
Management¶
The version string is now logged at startup.
Monitoring (Mon)¶
Metadata target capacity and health metrics are now available. These mirror the existing metrics available for storage targets.
Watch¶
By default the metadata node ID is now communicated to subscribers as part of the gRPC stream context. This requires Watch to now wait for the metadata service it is listening to connect before it can connect to subscribers. If the metadata node ID is not required and you wish Watch to always immediately connect to subscribers set
skip-node-id-detection = true.Setting
max-wait-for-response-after-connect = 0now tells Watch to wait indefinitely for subscribers to acknowledge the last event received when they are connecting. This is the recommended configuration and new default shipped in the default configuration file going forward.Subscribers can set an event filter to limit what event types they will receive from Watch.
Improved performance of event streaming, especially when multiple subscribers are in use.
For developers:
Added subscriber and dispatch libraries written in Go to simplify integrating with Watch.
Fixes¶
Client: Improve directory listing performance by allowing parallel directory listings on kernels that support it, instead of defaulting to the legacy sequential only handler.
Client: On clients configured with
tuneFileCacheType=native, metadata servers could log errors likeFailed to decode message. Message type: GetFileVersion (3035)andReceived an invalid messageif the file name was changed while fetching the file version.Client: Startup warning when SELinux was enabled incorrectly pointed users towards
sysSelinuxEnabledinstead ofsysSELinuxEnabled.CTL: Errors determining the path to a BeeGFS mount point could be masked and produce a misleading unsupported-filesystem error. For example if there was a communication error to a metadata node.
CTL: Auto-detecting the management address would always use the IP address from procfs, however the mgmtd’s TLS certificate might only be valid for the mgmtd’s hostname. Now CTL first attempts to extract the user configured
sysMgmtdAddr(which can be a hostname or IP) from thebeegfs-client.confused for the client mount, falling back on the procfs IP to maintain legacy behavior if the client config file is not readable by CTL or malformed.CTL: The
beegfs licensecommand did not warn if the system was over the licensed capacity.CTL: When an entry was temporarily unavailable (i.e., because it was in the inode lock store), instead of clearly indicating some fields were unavailable, commands like
beegfs entry infowould display potentially misleading/confusing default or empty values (e.g., “0”). The affected fields included pattern details, remote target configuration, client sessions, and file states.Copy: The command parser incorrectly treated
-Pas requiring an argument. As a result combinations of flags like-P -p 1024resulted in the-pflag being consumed as an argument to-Pand1024being treated as positional argument leading to confusing warnings an unexpected behavior.Copy: Improved memory safety and correctness when collecting the list of paths to be updated.
Mgmtd: The linked version of libc is now listed as a package dependency.
Mgmtd: When importing quota limits from a BeeGFS 7 system warnings were always logged about very large limits being treated as unlimited. These are now only printed if the limit exceeds 2^63.
Mgmtd: Log messages printed when fetching quota information contained an unintended newline and did not pretty print errors related to users and groups.
Mgmtd: Outgoing communication log messages now clarify they reference node UIDs (vs other ID types).
Mgmtd: When creating the database file failed, a generic SQLite error was returned. This now returns a more meaningful OS error (e.g., permission denied).
Mgmtd: When quotas were enabled but not licensed (e.g., due to license expiry), nodes would continue to enforce limits based on the last known state. Now when quotas are enabled but not licensed, the mgmtd will clear the list of exceeded IDs on all nodes so the system can function as if quotas were disabled.
Mgmtd: The
--initflag could not be used on systems where systemd was not available (e.g., containers) unless--log-target=stderr.Mgmtd: When accepting consistency state updates, the states known to the node making the update were not checked to verify the node had the same current states as the management. This was a regression from the BeeGFS 7 mgmtd and is intended to prevent theoretical state update races.
Meta: When downloading target states from the mgmtd failed, the target consistency state could remain uninitialized, which could later result in an indeterminate value being sent to the mgmtd. When this happened the mgmtd would log errors like
BeeMsg body deserialization failed: Invalid value 8 for conversion into enum TargetConsistencyState. The fix ensures a valid state is always set even when the mgmtd is unavailable, and the mgmtd was hardened to prevent state update races.Meta: In a corner case it was possible to be unable to seek to the oldest available file system modification event. This would typically manifest by Watch entering an indefinite reconnect loop.
Meta: When an error occurred fetching entry info it was possible to receive a hardcoded
PATHNOTEXISTSerror instead of the actual underlying error making troubleshooting confusing.Meta: Resyncing a buddy mirror could fail with vague network errors like
Communication error: Soft disconnect from x.x.x.x:8005; Peer: beegfs-meta meta01 [ID: 1]. (Message type: ResyncRawInodes (2115))if the inode contained an empty user xattr value. A guard was added to distinguish zero-length reads from orderly peer shutdowns.Storage: Resyncing a buddy mirror could hang if the number of sync candidates was equal to or greater than the max queue size of 50,000 and there were chunk files deleted while the mirror was degraded. To mitigate this, a new
tuneResyncQueueLimitparameter was introduced that can be increased to accommodate scenarios where a large number of changes need to be synced to the secondary. Note in some scenarios it may still be necessary wipe the secondary and resync.Storage: When used without a config file
tuneFileWriteSizewould default to64kinstead of the updated128kdefault set bybeegfs-storage.conf.Fsck: Modification logging is started on metadata nodes sequentially, however if logging failed to start on some nodes, logging would not be stopped on nodes as part of aborting the fsck run. This could lead to subsequent fsck runs failing with a
SIGSEGV.Mon: Grafana dashboards recent memory utilization spikes could be masked by the preceding idle baseline causing the gauge to remain green even when the host was experiencing memory pressure.
Mon: Load gauges could display “No data” due to even a single null sample. Now the gauges display based on the last not null data points.
Mon: Telegraf dashboards could go blank during brief
beegfs-monoutages.Watch: When the event ring buffer overflowed, a misleading dropped sequence ID would be logged.
Known Issues and Limitations¶
Index databases created prior to 8.4 need to be migrated to the new database format.
IPv6 temporary, deprecated, and optimistic addresses are not automatically ignored by servers. These addresses may be advertised to other servers and clients as available for inbound connections if servers are not configured to listen or advertise specific interfaces, or if they use a wildcard or unspecified address notation. Refer to the 8.2 upgrade guide for more details.
The 8.0 and 8.1 CTL
beegfs stats clientcommand and Monitoring service are not forward compatible with 8.2 servers due to a breaking change required to support IPv6. When used with 8.2 the command will returnError: TCP request to meta:2 failed: error reading BeeMsg: EOFand Mon will logReceived a message with incompatible feature flags. Message type: RequestMetaDataResp (6005); Flags (hex): 1. Upgrade CTL and Mon to 8.2 before upgrading BeeGFS servers to 8.2. The 8.2 CTL and Mon are also backwards compatible with 8.0 and 8.1.Issues have been observed with Soft-RoCE and the client module on newer kernels that currently affect RHEL 9.4 and Ubuntu 24.04 where the kernel locks up on RDMA connection attempts to meta and storage servers. This issue has never been observed on RoCE implementations other than the kernel RXE driver.
Supported Linux Distributions and Kernels¶
Supported distributions¶
The full integration test suite was run on Debian 12, Debian 13, Rocky Linux 8.10, Rocky Linux 9.4, CentOS Stream 10, Ubuntu 22.04, Ubuntu 24.04, Ubuntu 26.04
Packages are provided for the x86_64 and aarch64 architecture and the following distributions:
RHEL 8, 9, and 10 (packages can also be used on Rocky Linux and AlmaLinux).
SLES 15
Debian 12, and 13
Ubuntu 22.04, 24.04 and 26.04.
The distributions we provide packages for are fully supported by the BeeGFS server services (mgmtd, meta, storage, mon, remote, and sync). Generally specific kernel and RDMA driver versions are not problematic for the servers, but can cause issues building the client module. Refer to the Client Build Testing section below what combinations have been tested.
Client build testing¶
Client build testing was done with the following combinations of operating systems and RDMA drivers. To keep the test matrix manageable and because kernel APIs tend to change frequently, even in distribution minor releases, the BeeGFS client generally only prioritizes support for the default slow moving distribution kernels (not hwe, backports and similar). Custom kernel builds might or might not work and are generally not supported. There can also be incompatibilities between the client and very recent distribution kernel releases. The same is true for RDMA driver versions that can sometimes cause issues building the client for a specific kernel version.
Starting with BeeGFS 8.2, known combinations that currently do not work are published, so there are no surprises during planning and upgrades. In many cases these reflect breaking changes in upstream kernel/driver versions rather than issues in BeeGFS itself. We expect to support additional combinations over time, potentially in patch releases between minor versions. For the very latest updates check GitHub for available patch releases.
Additional notes:
Combinations not explicitly listed below might or might not work.
Results from Rocky are expected to apply to other RedHat derivatives including AlmaLinux and RHEL.
To keep the test matrix manageable, we currently test the latest two minor releases of every available major version for each distribution. For example, on Rocky Linux this covers 8.9, 8.10, 9.7, 9.8, 10.1, and 10.2.
In addition to the kernel provided RDMA drivers, BeeGFS client compilation has also been tested with the DOCA/OFED RDMA drivers and with NVIDIA GPUDirect Storage (GDS) rebuilds. Refer to the respective test matrixes below for the specific combinations that were tested.
With the introduction of the DOCA repositories, BeeGFS supports OFED versions included in the following DOCA releases: 2.5-LTS, 2.9-LTS, and 3.2-LTS.
Note that OFED versions and DOCA versions follow different versioning schemes, so the supported OFED versions in each DOCA release may differ. Always refer to the specific DOCA release notes for detailed compatibility information.
Click to expand the test matrixes for different distributions:
Tested Rocky Linux versions (with kernel RDMA drivers)
| RHEL/Rocky/Alma | Kernel version | Pass/Fail | Failure reason |
|---|---|---|---|
| 10.2 | 6.12.0-211.22.1.el10_2.x86_64 | ✅ | - |
| 10.1 | 6.12.0-124.56.1.el10_1.x86_64 | ✅ | - |
| 9.8 | 5.14.0-687.15.1.el9_8.x86_64 | ✅ | - |
| 9.7 | 5.14.0-611.55.1.el9_7.x86_64 | ✅ | - |
| 8.10 | 4.18.0-553.134.1.el8_10.x86_64 | ✅ | - |
| 8.9 | 4.18.0-513.24.1.el8_9.x86_64 | ✅ | - |
Tested Rocky Linux versions (with DOCA/OFED RDMA drivers)
| RHEL/Rocky/Alma Version | DOCA Version | OFED Version | Kernel Version | Pass/Fail | Failure reason |
|---|---|---|---|---|---|
| 10.2 | not_installed | OFED-internal-26.04-0.8.6: | 6.12.0-211.22.1.el10_2.x86_64 | ❌ | DOCA package install failed |
| 10.2 | not_installed | OFED-internal-25.10-2.4.1: | 6.12.0-211.22.1.el10_2.x86_64 | ❌ | DOCA package install failed |
| 10.1 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | 6.12.0-124.56.1.el10_1.x86_64 | ✅ | - |
| 10.1 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | 6.12.0-124.56.1.el10_1.x86_64 | ✅ | - |
| 9.8 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | 5.14.0-687.15.1.el9_8.x86_64 | ✅ | - |
| 9.8 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | 5.14.0-687.15.1.el9_8.x86_64 | ✅ | - |
| 9.7 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | 5.14.0-611.55.1.el9_7.x86_64 | ✅ | - |
| 9.7 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | 5.14.0-611.55.1.el9_7.x86_64 | ✅ | - |
| 8.10 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | 4.18.0-553.134.1.el8_10.x86_64 | ✅ | - |
| 8.10 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | 4.18.0-553.134.1.el8_10.x86_64 | ✅ | - |
| 8.10 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | 4.18.0-553.134.1.el8_10.x86_64 | ❌ | BeeGFS client rebuild failed |
| 8.10 | 2.5.6-0.0.7 | OFED-internal-23.10-7.1.8: | 4.18.0-553.134.1.el8_10.x86_64 | ✅ | - |
| 8.9 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | 4.18.0-513.24.1.el8_9.x86_64 | ✅ | - |
| 8.9 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | 4.18.0-513.24.1.el8_9.x86_64 | ✅ | - |
| 8.9 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | 4.18.0-513.24.1.el8_9.x86_64 | ✅ | - |
| 8.9 | 2.5.6-0.0.7 | OFED-internal-23.10-7.1.8: | 4.18.0-513.24.1.el8_9.x86_64 | ✅ | - |
Tested Rocky Linux versions (with DOCA/OFED RDMA drivers and NVIDIA GDS rebuilds)
| RHEL/Rocky/Alma Version | DOCA Version | OFED Version | Kernel Version | Pass/Fail | GDS Rebuild | NVIDIA Driver Version | NVIDIA Driver Flavor | NVIDIA-FS Version | GDS Release Version | CUDA Toolkit Version | Failure reason |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 10.2 | not_installed | OFED-internal-26.04-0.8.6: | 6.12.0-211.22.1.el10_2.x86_64 | ❌ | not-run | not-found | not-found | not-found | not-found | not-found | DOCA package install failed |
| 10.2 | not_installed | OFED-internal-25.10-2.4.1: | 6.12.0-211.22.1.el10_2.x86_64 | ❌ | not-run | not-found | not-found | not-found | not-found | not-found | DOCA package install failed |
| 10.1 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | 6.12.0-124.56.1.el10_1.x86_64 | ✅ | ✅ | 610.43.02-1.el10 | open | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 10.1 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | 6.12.0-124.56.1.el10_1.x86_64 | ✅ | ✅ | 610.43.02-1.el10 | open | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 9.8 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | 5.14.0-687.15.1.el9_8.x86_64 | ✅ | ✅ | 610.43.02-1.el9 | proprietary | 2.25.7 | 12.9.2-1 | 13.3.0-1 | - |
| 9.8 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | 5.14.0-687.15.1.el9_8.x86_64 | ✅ | ✅ | 610.43.02-1.el9 | proprietary | 2.25.7 | 12.9.2-1 | 13.3.0-1 | - |
| 9.7 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | 5.14.0-611.55.1.el9_7.x86_64 | ✅ | ✅ | 610.43.02-1.el9 | proprietary | 2.25.7 | 12.9.2-1 | 13.3.0-1 | - |
| 9.7 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | 5.14.0-611.55.1.el9_7.x86_64 | ✅ | ✅ | 610.43.02-1.el9 | proprietary | 2.25.7 | 12.9.2-1 | 13.3.0-1 | - |
| 8.10 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | 4.18.0-553.134.1.el8_10.x86_64 | ✅ | ✅ | 610.43.02-1.el8 | proprietary | 2.25.7 | 12.9.2-1 | 13.3.0-1 | - |
| 8.10 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | 4.18.0-553.134.1.el8_10.x86_64 | ✅ | ✅ | 610.43.02-1.el8 | proprietary | 2.25.7 | 12.9.2-1 | 13.3.0-1 | - |
| 8.10 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | 4.18.0-553.134.1.el8_10.x86_64 | ❌ | not-run | not-found | not-found | not-found | not-found | not-found | BeeGFS client rebuild failed |
| 8.10 | 2.5.6-0.0.7 | OFED-internal-23.10-7.1.8: | 4.18.0-553.134.1.el8_10.x86_64 | ✅ | ✅ | 610.43.02-1.el8 | proprietary | 2.25.7 | 12.9.2-1 | 13.3.0-1 | - |
| 8.9 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | 4.18.0-513.24.1.el8_9.x86_64 | ✅ | ✅ | 610.43.02-1.el8 | proprietary | 2.25.7 | 12.9.2-1 | 13.3.0-1 | - |
| 8.9 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | 4.18.0-513.24.1.el8_9.x86_64 | ✅ | ✅ | 610.43.02-1.el8 | proprietary | 2.25.7 | 12.9.2-1 | 13.3.0-1 | - |
| 8.9 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | 4.18.0-513.24.1.el8_9.x86_64 | ✅ | ✅ | 610.43.02-1.el8 | proprietary | 2.25.7 | 12.9.2-1 | 13.3.0-1 | - |
| 8.9 | 2.5.6-0.0.7 | OFED-internal-23.10-7.1.8: | 4.18.0-513.24.1.el8_9.x86_64 | ✅ | ✅ | 610.43.02-1.el8 | proprietary | 2.25.7 | 12.9.2-1 | 13.3.0-1 | - |
Tested Ubuntu versions (with kernel RDMA drivers)
| Ubuntu Version | Kernel version | Pass/Fail | Failure reason |
|---|---|---|---|
| 26.04 | linux-headers-7.0.0-14-generic | ✅ | - |
| 25.10 | linux-headers-6.17.0-5-generic | ✅ | - |
| 24.04.4 | linux-headers-6.8.0-100-generic | ✅ | - |
| 24.04.3 | linux-headers-6.8.0-71-generic | ✅ | - |
| 22.04.5 | linux-headers-5.15.0-119-generic | ✅ | - |
| 22.04.4 | linux-headers-5.15.0-94-generic | ✅ | - |
Tested Ubuntu versions (with DOCA/OFED RDMA drivers)
| Ubuntu Version | DOCA Version | OFED Version | Kernel Version | Pass/Fail | Failure reason |
|---|---|---|---|---|---|
| 26.04 | 26.01.1.0.0.0.609.1-0ubuntu2 | 26.01.1.0.0.0.609.1-0ubuntu2 | linux-headers-7.0.0-14-generic | ✅ | - |
| 24.04.4 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | linux-headers-6.8.0-100-generic | ✅ | - |
| 24.04.4 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | linux-headers-6.8.0-100-generic | ✅ | - |
| 24.04.4 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | linux-headers-6.8.0-100-generic | ✅ | - |
| 24.04.3 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | linux-headers-6.8.0-71-generic | ✅ | - |
| 24.04.3 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | linux-headers-6.8.0-71-generic | ✅ | - |
| 24.04.3 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | linux-headers-6.8.0-71-generic | ✅ | - |
| 22.04.5 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | linux-headers-5.15.0-119-generic | ✅ | - |
| 22.04.5 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | linux-headers-5.15.0-119-generic | ✅ | - |
| 22.04.5 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | linux-headers-5.15.0-119-generic | ✅ | - |
| 22.04.5 | 2.5.6-0.0.7 | OFED-internal-23.10-7.1.8: | linux-headers-5.15.0-119-generic | ✅ | - |
| 22.04.4 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | linux-headers-5.15.0-94-generic | ✅ | - |
| 22.04.4 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | linux-headers-5.15.0-94-generic | ✅ | - |
| 22.04.4 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | linux-headers-5.15.0-94-generic | ✅ | - |
| 22.04.4 | 2.5.6-0.0.7 | OFED-internal-23.10-7.1.8: | linux-headers-5.15.0-94-generic | ✅ | - |
Tested Ubuntu versions (with DOCA/OFED RDMA drivers and NVIDIA GDS rebuilds)
| Ubuntu Version | DOCA Version | OFED Version | Kernel Version | Pass/Fail | GDS Rebuild | NVIDIA Driver Version | NVIDIA Driver Flavor | NVIDIA-FS Version | GDS Release Version | CUDA Toolkit Version | Failure reason |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 26.04 | 26.01.1.0.0.0.609.1-0ubuntu2 | 26.01.1.0.0.0.609.1-0ubuntu2 | linux-headers-7.0.0-14-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 24.04.4 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | linux-headers-6.8.0-100-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 24.04.4 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | linux-headers-6.8.0-100-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 24.04.4 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | linux-headers-6.8.0-100-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 24.04.3 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | linux-headers-6.8.0-71-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 24.04.3 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | linux-headers-6.8.0-71-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 24.04.3 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | linux-headers-6.8.0-71-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 22.04.5 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | linux-headers-5.15.0-119-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 22.04.5 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | linux-headers-5.15.0-119-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 22.04.5 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | linux-headers-5.15.0-119-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 22.04.5 | 2.5.6-0.0.7 | OFED-internal-23.10-7.1.8: | linux-headers-5.15.0-119-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 22.04.4 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | linux-headers-5.15.0-94-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 22.04.4 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | linux-headers-5.15.0-94-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 22.04.4 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | linux-headers-5.15.0-94-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 22.04.4 | 2.5.6-0.0.7 | OFED-internal-23.10-7.1.8: | linux-headers-5.15.0-94-generic | ✅ | ✅ | 610.43.02-1ubuntu1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
Tested Debian versions (with kernel RDMA drivers)
| Debian Version | Kernel version | Pass/Fail | Failure reason |
|---|---|---|---|
| 13.5 | linux-headers-6.12.90+deb13.1-amd64 | ✅ | - |
| 13.4 | linux-headers-6.12.85+deb13-amd64 | ✅ | - |
| 12.14 | linux-headers-6.1.0-49-amd64 | ✅ | - |
| 12.13 | linux-headers-6.1.0-45-amd64 | ✅ | - |
| 11.11 | linux-headers-5.10.0-44-amd64 | ✅ | - |
| 11.10 | linux-headers-5.10.0-31-amd64 | ✅ | - |
Tested Debian versions (with DOCA/OFED RDMA drivers)
| Debian Version | DOCA Version | OFED Version | Kernel Version | Pass/Fail | Failure reason |
|---|---|---|---|---|---|
| 13 | not_installed | not-found | not-found | ❌ | apt update repository failed for DOCA |
| 13 | not_installed | not-found | not-found | ❌ | apt update repository failed for DOCA |
| 12 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | linux-headers-6.1.0-49-amd64 | ✅ | - |
| 12 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | linux-headers-6.1.0-49-amd64 | ✅ | - |
| 12 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | linux-headers-6.1.0-49-amd64 | ✅ | - |
| 12 | 2.5.6-0.0.7 | OFED-internal-23.10-7.1.8: | linux-headers-6.1.0-49-amd64 | ✅ | - |
Tested Debian versions (with DOCA/OFED RDMA drivers and NVIDIA GDS rebuilds)
| Debian Version | DOCA Version | OFED Version | Kernel Version | Pass/Fail | GDS Rebuild | NVIDIA Driver Version | NVIDIA Driver Flavor | NVIDIA-FS Version | GDS Release Version | CUDA Toolkit Version | Failure reason |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 13 | not_installed | not-found | not-found | ❌ | not-run | not-found | not-found | not-found | not-found | not-found | apt update repository failed for DOCA |
| 13 | not_installed | not-found | not-found | ❌ | not-run | not-found | not-found | not-found | not-found | not-found | apt update repository failed for DOCA |
| 12 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | linux-headers-6.1.0-49-amd64 | ✅ | ✅ | 610.43.02-1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 12 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | linux-headers-6.1.0-49-amd64 | ✅ | ✅ | 610.43.02-1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 12 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | linux-headers-6.1.0-49-amd64 | ✅ | ✅ | 610.43.02-1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
| 12 | 2.5.6-0.0.7 | OFED-internal-23.10-7.1.8: | linux-headers-6.1.0-49-amd64 | ✅ | ✅ | 610.43.02-1 | proprietary | 2.29.4 | 13.3.0-1 | 13.3.0-1 | - |
Tested OpenSUSE versions (with kernel RDMA drivers)
| OpenSUSE Version | Kernel version | Pass/Fail | Failure reason |
|---|---|---|---|
| 16.1 | not-found | ❌ | Job did not start (container image likely unavailable) |
| 16.0 | 6.12.0-160000.34-default | ✅ | - |
| 15.6 | 6.4.0-150600.21-default | ✅ | - |
Tested OpenSUSE versions (with DOCA/OFED RDMA drivers)
| OpenSUSE Version | DOCA Version | OFED Version | Kernel Version | Pass/Fail | Failure reason |
|---|---|---|---|---|---|
| 15.6 | 3.4.0-085000 | OFED-internal-26.04-0.8.6: | 6.4.0-150600.21-default | ✅ | - |
| 15.6 | 3.2.2-035000 | OFED-internal-25.10-2.4.1: | 6.4.0-150600.21-default | ✅ | - |
| 15.6 | 2.9.5-0.1.8 | OFED-internal-24.10-5.1.6: | 6.4.0-150600.21-default | ✅ | - |
| 15.6 | 2.5.6-0.0.7 | OFED-internal-23.10-7.1.8: | 6.4.0-150600.21-default | ✅ | - |
Hint
An green ✅ indicates the client builds with that combination. A red ❌ indicates the client does not currently build with that combination.
Version Interoperability¶
Starting with version 8, BeeGFS more strictly adheres to semantic versioning guarantees. This means BeeGFS 8.4 components are compatible with components running any BeeGFS 8.y.z release, however the use of new functionality introduced in 8.4 requires the involved components to be running 8.4.
Note that BeeGFS 8 is not compatible with older major versions of BeeGFS due to changes in network and on-disk formats.
Upgrading from Older Versions¶
To upgrade from an older version, please refer to the Upgrade Guide.
