BeeGFS 8.2 Upgrade Guide

Please read and understand the information below completely before attempting the upgrade.

Possible upgrade paths

The following upgrade paths from older versions are supported:

  • v6.y ⟶ v7.4.z ⟶ v8.2.z

  • v7.y ⟶ v7.4.z ⟶ v8.2.z

  • v8.y ⟶ v8.2.z

The required steps depend on your starting version.

General requirements

Always check the release notes and upgrade guides between the source and destination releases for any new features or configuration options you may wish to update. Most configuration changes are optional, with the following exception:

  • If you are using Remote Storage Targets a new required sysBypassFileAccessCheckOnMeta client parameter was introduced in 8.1 that must be set in the client used for all Remote and Sync nodes when upgrading to or past 8.1.

Upgrade from 6 or ≤7.3

If you are still on BeeGFS 7.3 or earlier, first upgrade to 7.4 following the BeeGFS 7.4 upgrade guide. Once complete, continue with with the path to upgrade from 7.4.

Upgrade from 7.4

It is possible to upgrade directly from BeeGFS 7.4 to BeeGFS 8.2. First check all servers and clients are on supported Linux distributions and kernel versions for BeeGFS 8.2. Then follow the steps in the BeeGFS 8 upgrade guide.

Upgrade from any 8.x release

It is possible to upgrade from any BeeGFS 8 release to BeeGFS 8.2. Follow the steps in this guide.

Before Upgrading

  1. Check all servers and clients are on supported Linux distributions and kernel versions.

Upgrade to 8.2

  1. If you use buddy mirroring, make sure that all buddies are in GOOD state in the previous output before proceeding.

  2. Stop the BeeGFS system:

    • stop/unmount all clients

    • stop all storage services

    • stop all meta services including any event listeners (if in use)

    • stop the management service

  3. Refer to the backup guide and take a full system backup if possible. If a full backup is not possible ensure you at least take a minimal backup of important system configuration.

    Note

    A minimal backup of important system configuration after stopping all services and before upgrading packages is strongly recommended. Proceed without any backups at your own risk!

  4. Download the BeeGFS package repository file for your distribution to all nodes. Visit the BeeGFS download page and follow the step-by-step directions to add the BeeGFS package repositories to your Linux distribution’s package manager.

  5. Upgrade all BeeGFS packages.

  6. Update optional configuration:

    1. To opt out of the new client-side ACL caching and restore the pre-8.2 behavior, set sysACLsRevalidate=always in beegfs-client.conf on all clients.

    2. Support for IPv6 was introduced in this release. If IPv6 addresses are configured on the interfaces used by BeeGFS they will be automatically detected and made available for incoming/outgoing connections to other nodes.

      1. By default core file system services will prefer IPv4 over IPv6 and the existing preferred interface configuration will be respected. Optionally in addition to preferred interfaces, you can now provide specific addresses, and/or the protocol (IPv4 or IPv6) using the interfaces parameter in beegfs-mgmtd.toml, or the connInterfacesFile, connInterfacesList and connNetFilterFile in beegfs-{meta,storage,client}.conf. Refer to the documentation in each configuration file for details about the new syntax.

        • If you want/need to disable use of IPv6 entirely and always use IPv4 sockets, you can set ipv6-disable=true in beegfs-mgmtd.toml and connDisableIPv6=true in beegfs-{meta,mon,storage,client}.conf.

      2. By default, the beegfs tool will auto-configure its management address when the client is mounted and will use IPv4 or IPv6 depending on the client configuration. When manually specifying an IPv6 address use the standard [address]:port IPv6 notation, for example --mgmtd-addr=[2001:db8::1]:8010.

      3. By default the Remote, Sync, and Watch services listen on all available IPv4 addresses (0.0.0.0:<port>). As with the beegfs tool, if you specify an IPv6 address, it must use the standard [address]:port IPv6 notation. If dual-stack sockets are enabled (net.ipv6.bindv6only=0), to listen on all IPv4 and IPv6 addresses specify the server address as [::]:<port>. If dual-stack sockets are disabled, [::]:<port> will listen only on all available IPv6 addresses.

      Note

      IPv6 temporary, deprecated, and optimistic addresses are not automatically ignored by servers. These addresses may be advertised to other servers and clients as available for inbound connections if servers are not configured to listen or advertise specific interfaces, or if they use a wildcard or unspecified address notation.

      This will not immediately cause problems on upgrades to 8.2 as IPv4 addresses are still preferred. If you have ephemeral IPv6 addresses on your system and want to prefer IPv6 after upgrading, configure specific stable IPv6 addresses using the management’s interfaces and metadata + storage server connInterfaces[File|List] parameters. For Remote/Sync/Watch, listen on a specific IPv6 server.address instead of an unspecified address ([::]).

    3. Enable full support for SELinux if desired.

  7. Start all services and clients and verify that the system works as expected.

  8. After finishing the upgrade run the following commands to verify the upgrade:

    1. Verify all nodes and targets are online and in a good state: beegfs health check

      beegfs health check

      Tip

      If the “Available Capacity” check warns that targets are low on free space, you can adjust the “capacity pool” thresholds defined in the beegfs-mgmtd.toml so they are better suited to your environment.

FAQs

The following section addresses common questions when upgrading BeeGFS.

Why does the management log the database schema is outdated?

After upgrading a system already running BeeGFS 8 to a newer version, the management service refuses to start and logs an error like:

Database schema version 1 is outdated. Please upgrade to latest version 2

Starting in BeeGFS 8, management data is stored in a SQL database. New BeeGFS versions may introduce changes to the database schema, for example to add new fields. The management includes a mechanism for upgrading (i.e., “migrating”) the database schema after an upgrade by manually running:

/opt/beegfs/sbin/beegfs-mgmtd --upgrade

Note

If your database is located at a non-standard path, ensure to also specify --db-file.

The upgrade process will first backup the current database in the same directory as the existing one, with the old version as the suffix. For example by default backups will exist at a path like:

/var/lib/beegfs/mgmtd.sqlite.v1

The backup is mainly needed if you need to downgrade BeeGFS. Because the database size is typically small, it is recommended to keep around at least one previous version indefinitely.

Provided the backup succeeds the database upgrade will be attempted. If for some reason the upgrade fails an error will be logged, for example if you tried to upgrade between unsupported versions. If the resolution is not clear please open a support ticket. If you choose to downgrade in the meantime, please perform the following steps:

  • Record the error(s) logged from the --upgrade.

  • Downgrade all BeeGFS packages.

Note if the upgrade fails the original database file will be left untouched, so there should be no reason to overwrite the current database file with the backup after a failed database upgrade. The backup is intended if you need to downgrade BeeGFS for some other reason.