Running BeeGFS in Containers¶

Overview¶

Traditionally, BeeGFS server services are installed using a particular Linux distribution’s package manager (yum, apt, zypper) and managed (start/stop) by systemd. An alternative approach is to package applications into a container image that contains all of the dependencies needed by that application. This can drastically simplify installing and managing applications, providing a consistent runtime environment for the application that can be isolated from other processes running on the system.

Definitions¶

Base OS: The operating system running on a physical or virtual machine that is used to run containers.

Container Image: Lightweight, standalone, executable package of software that includes everything needed to run an application.

Container: Running instance of a container image.

Prerequisites¶

BeeGFS container images are available through Docker Hub and are distributed using the Docker Image Manifest Version 2, Schema 2 format, which means they should generally work with any container engine or container runtime that supports either the Docker or Open Container image (OCI) formats. Depending on your use case you will typically either use a high-level container engine like Docker or Podman, a high-level runtime like Containerd, or a low level runtime like runc to actually run BeeGFS images.

For simplicity this guide will focus on how to get up and running using Docker. To follow along you will need to install the correct server version of Docker Engine based on your Linux distribution and architecture. Note there is a distinction between the open source Docker Engine available for various Linux platforms and Docker Desktop which requires a subscription to use on Linux, MacOS, and Windows. This guide only requires use of the open source Docker Engine and the following steps were specifically validated using Ubuntu 20.04 and Docker version 20.10.19.

This guide expects network interface(s) including IP addresses are already configured on the host used to run BeeGFS containers. These can be externally facing interfaces (optionally RDMA enabled), or the loopback device if you just want to test on an isolated system. If you want to test RDMA the appropriate packages/modules must already be installed/configured.

It may also be helpful to be familiar with the standard BeeGFS quick start guide as the documentation will occasionally refer back to standard installation steps to help the reader understand how those translate to running BeeGFS in containers.

Downloading/Pulling BeeGFS Images¶

Instead of configuring your package manager (apt, dnf, yum, etc) to download/install BeeGFS packages, provided the servers you are using have internet access simply run:

docker pull beegfs/beegfs-mgmtd:latest
docker pull beegfs/beegfs-meta:latest
docker pull beegfs/beegfs-storage:latest

Optionally to pull images for a specific version, specify it like:

docker pull beegfs/beegfs-mgmtd:7.3.1
docker pull beegfs/beegfs-meta:7.3.1
docker pull beegfs/beegfs-storage:7.3.1

If the servers you are using do not have internet access first pull the image on a machine with internet access, then save them as tar files using docker save so they can be copied to air-gapped servers and loaded with docker load.

Note

The earliest available BeeGFS images are for BeeGFS 7.3.1.

Image Verification¶

All BeeGFS server Docker images with tags latest and version numbers are signed with DCT (Docker Content Trust) and Cosign.

Docker Content Trust¶

Content trust is disabled by default in the Docker client. To enable it, set the DOCKER_CONTENT_TRUST environment variable to 1. This will prevent users from working with Docker images without signatures. DCT digital signatures data can be manually verified for a Docker image by comparing the Signer Key fingerprint, i.e. 898988ce0d2f, for all BeeGFS Docker images:

$ docker trust inspect --pretty docker.io/beegfs/beegfs-mgmtd

Signatures for docker.io/beegfs/beegfs-mgmtd

SIGNED TAG   DIGEST                                                             SIGNERS
7.3.1        9df48c96066c5e754fbc2b8baeff3bee62bd666c83508acb47a11650c9774d6a   gitlab-ci

List of signers and their keys for docker.io/beegfs/beegfs-mgmtd

SIGNER      KEYS
gitlab-ci   898988ce0d2f

Administrative keys for docker.io/beegfs/beegfs-mgmtd

    Repository Key:   5836ea8276e6d98cfd6d671276d61b45b9c5cfaefeb85a86257c7b9e28c0c973
    Root Key: 3d0fe14af5ea08210e29351dbd7ef05ef1041a1c5dd687f2413cd4ec89bf92f3

Cosign¶

Cosign signatures can be verified with the Cosign application. The Cosign verify command is used to verify the Docker images’ signatures $ cosign verify --key <pub key path> <image uri>. All BeeGFS server Docker images are signed with the following public key:

-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEAR5OydewbwaPKjd/5SqQ4CbGOH68
E5RSXAH6n+WRe3NL0UN7aeI2kAoqyI1wOG0TdCx0RSb4q0KrKSSjVWsVXg==
-----END PUBLIC KEY-----

For example, to verify the beegfs-mgmtd image with tag 7.3.1:

$ cosign verify --key cosign.pub docker.io/beegfs/beegfs-mgmtd:7.3.1
Verification for index.docker.io/beegfs/beegfs-mgmtd:7.3.1 --
The following checks were performed on each of these signatures:
    - The cosign claims were validated
    - The signatures were verified against the specified public key

[{"critical":{"identity":{"docker-reference":"index.docker.io/beegfs/beegfs-mgmtd"},"image":{"docker-manifest-digest":"sha256:c0b876a27c764ad23fdf200496b51fa99266b4bbb42e76ae0d7e91dcc561a2e5"},"type":"cosign container image signature"},"optional":null}]

Quick Start: BeeGFS Containers¶

This section provides an example to allow you to quickly get started with BeeGFS containers. More detail on what these commands do and alternative/more secure options will be provided in the following sections.

Note

The quick start example stores data outside the container under /tmp. Note this may be wiped after a reboot on some Linux distributions.

Steps¶

Step 1: Define shell variables with the interface(s) BeeGFS should use for communication, and the IP that should be used for the BeeGFS management service (typically the IP of the first interface).

export beegfs_connInterfacesList=<INTERFACES> # Example: ib0,ib1
export beegfs_sysMgmtdHost=<IPv4 ADDRESS> # Example: 100.127.1.39

Step 2: Start the BeeGFS management service. The --volume argument accepts the absolute path and name of a directory where the file system can store its management data on the host machine.

docker run --name="beegfs-mgmtd" --privileged --network="host" \
  --volume=/tmp/mgmt_tgt_mgmt01:/mnt/mgmt_tgt_mgmt01 \
  --env beegfs_setup_1="beegfs-setup-mgmtd -p /mnt/mgmt_tgt_mgmt01 -C -S mgmt_tgt_mgmt01" \
  -d beegfs/beegfs-mgmtd:latest \
  storeAllowFirstRunInit=false connDisableAuthentication=true \
  connInterfacesList=$beegfs_connInterfacesList storeMgmtdDirectory=/mnt/mgmt_tgt_mgmt01

Step 3: Start the BeeGFS metadata service. The --volume argument accepts the absolute path and name of a directory where the file system can store its metadata on the host machine.

docker run --name="beegfs-meta" --privileged --network="host" \
  --volume=/tmp/meta_01_tgt_0101:/mnt/meta_01_tgt_0101 \
  --env beegfs_setup_1="beegfs-setup-meta -C -p /mnt/meta_01_tgt_0101 -s 1 -S meta_01" \
  -d beegfs/beegfs-meta:latest \
  sysMgmtdHost=$beegfs_sysMgmtdHost storeAllowFirstRunInit=false connDisableAuthentication=true \
  connInterfacesList=$beegfs_connInterfacesList storeMetaDirectory=/mnt/meta_01_tgt_0101

Step 4: Start the BeeGFS storage service. The --volume argument accepts the absoute path to a storage target. A storage target is a directory where the file system can store raw user file contents on the host machine.

docker run --name="beegfs-storage" --privileged --network="host" \
  --volume=/tmp/stor_01_tgt_101:/mnt/stor_01_tgt_101 --volume=/tmp/stor_01_tgt_102:/mnt/stor_01_tgt_102 \
  --env beegfs_setup_1="beegfs-setup-storage -C -p /mnt/stor_01_tgt_101 -s 1 -S stor_01_tgt_101 -i 101" \
  --env beegfs_setup_2="beegfs-setup-storage -C -p /mnt/stor_01_tgt_102 -s 1 -S stor_01_tgt_101 -i 102" \
  -d beegfs/beegfs-storage:latest \
  sysMgmtdHost=$beegfs_sysMgmtdHost storeAllowFirstRunInit=false connDisableAuthentication=true \
  connInterfacesList=$beegfs_connInterfacesList storeStorageDirectory=/mnt/stor_01_tgt_101,/mnt/stor_01_tgt_102

Step 5: Verify the BeeGFS metadata and storage service were able to register with the management service.

# docker logs beegfs-mgmtd
[...]
(1) 21:35:05 DGramLis [HBeatMgr] >> New root directory metadata node: beegfs-meta meta_01 [ID: 1]
(2) 21:35:05 DGramLis [Node registration] >> New node: beegfs-meta meta_01 [ID: 1]; RDMA; Source: 100.128.1.39
(2) 21:35:08 DirectWorker1 [Change consistency states] >> Metadata node is coming online. ID: 1
(2) 21:35:08 XNodeSync [Assign node to capacity pool] >> Metadata node capacity pool assignment updated. NodeID: 1; Pool: Normal; Reason: Free capacity threshold
(3) 21:35:23 Worker1 [RegisterTargetMsgEx.cpp:54] >> New target registered. targetID: 0-634DCA9B-1; newTargetNumID: 101
(3) 21:35:23 Worker2 [RegisterTargetMsgEx.cpp:54] >> New target registered. targetID: 1-634DCA9B-1; newTargetNumID: 102
(2) 21:35:23 DGramLis [Node registration] >> New node: beegfs-storage stor_01_tgt_101 [ID: 1]; RDMA; Source: 100.128.1.39
(2) 21:35:26 DirectWorker1 [Change consistency states] >> Storage target is coming online. ID: 101
(2) 21:35:26 DirectWorker1 [Change consistency states] >> Storage target is coming online. ID: 102

Step 6 (optional): Mount the BeeGFS file system on one or more clients.

Step 7 (optional): Stop the BeeGFS containers and clean up.

docker stop beegfs-mgmtd beegfs-meta beegfs-storage; \
docker rm beegfs-mgmtd beegfs-meta beegfs-storage; \
rm -rf /tmp/mgmt_tgt_mgmt01/ && rm -rf /tmp/meta_01_tgt_0101/ && rm -rf
/tmp/stor_01_tgt_101 && rm -rf /tmp/stor_01_tgt_102/; \
unset beegfs_connInterfacesList beegfs_sysMgmtdHost

Advanced Topics¶

This section goes in-depth on how to execute BeeGFS Docker images using Docker and Docker Compose.

Running BeeGFS in Containers using Docker Run¶

The docker run command allows you to create and execute containers based on existing Docker images. Refer to the full Docker run reference for additional details on the flags described in this section.

To setup BeeGFS you first need to initialize the directories used to store data (management, metadata, and storage targets). This is typically done by executing the beegfs-setup-<SERVICE> command, and for the storage service multiple commands may be needed if there are multiple targets. When running BeeGFS in containers the setup commands are instead specified as environmental variable(s), then startup and initialization scripts inside the container handle setup if the provided targets have not been initialized. This is also how we’ll specify a shared secret so we can enable connection authentication. The first part of the command to run the BeeGFS management service looks like:

docker run --name="beegfs-mgmtd" \
  --env beegfs_setup_1="beegfs-setup-mgmtd -p /data/mgmt_tgt_mgmt01 -C -S mgmt_tgt_mgmt01" \
  --env CONN_AUTH_FILE_DATA="shared_secret_string" \

Note

For containers it is important to always use -S and specify the string ID. By default BeeGFS uses the hostname, which for containers can change when they are recreated, or if the network mode is set to “host” will match the base OS hostname which would break things if multiple containers of the same type are running on the same server.

By default anything written to the /mnt/mgmt_tgt_mgmt01 directory won’t persist if the container is removed (for example if the server reboots). To avoid this we can provide one or more locations outside the container (already mounted to the base OS) that will persist beyond the life of the container. When running BeeGFS in production this will likely represent some faster storage device, for example an LUN from an internal RAID adapter or presented from an external storage system over a storage area network (SAN). Regardless of the backing device, after it is formatted/mounted in the base OS, you can bind mount it into the container by specifying:

--volume=/mnt/mgmt_tgt_mgmt01:/data/mgmt_tgt_mgmt01 \

Typically we want to expose each BeeGFS service to the outside world so it can be accessed by clients and services running on multiple servers. While there are a number of ways to configure networking in Docker, the simplest and most performant is to simply use the host’s network stack inside the container:

--network="host" \

Note

The use of the host network mode gives the container full access to local system services such as D-bus, and is consider insecure if hard separation between a BeeGFS container and other containers or processes running in the base OS is required. However for many common BeeGFS use cases this approach may be acceptable. For example if a particular server is only serving one BeeGFS file system and not hosting other applications (or BeeGFS file systems) in a multi-tenant environment. If separation is required it is possible to use other network modes to allow external access in a more secure manner, for example using the default Docker bridge network mode. If RDMA is in use the performance impact of alternate approaches should be minimal, but have not been extensively tested or evaluated with BeeGFS.

Next we’ll specify the name of the container image we want to execute using the -d flag to start in detached mode:

-d beegfs/beegfs-mgmtd:latest \

Typically any “runtime” arguments would be set at /etc/beegfs/beegfs-<SERVICE>.conf, but when we run BeeGFS in containers we just specify them as arguments to the container:

storeAllowFirstRunInit=false connDisableAuthentication=true \
connInterfacesList=<COMMA_SEPARATED_LIST> storeMgmtdDirectory=/data/mgmt_tgt_mgmt01

Note

The value for storeMgmtdDirectory must match what was specified in the -p section of the setup command (/mnt/mgmt_tgt_mgmt01).

If we put it all together the docker run command for the management service looks like:

docker run --name="beegfs-mgmtd" \
  --env beegfs_setup_1="beegfs-setup-mgmtd -p /data/mgmt_tgt_mgmt01 -C -S mgmt_tgt_mgmt01" \
  --env CONN_AUTH_FILE_DATA="shared_secret_string" \
  --volume=/mnt/mgmt_tgt_mgmt01:/data/mgmt_tgt_mgmt01 \
  --network="host" \
  -d beegfs/beegfs-mgmtd:latest \
  storeAllowFirstRunInit=false connDisableAuthentication=true \
  connInterfacesList=<COMMA_SEPARATED_LIST> storeMgmtdDirectory=/data/mgmt_tgt_mgmt01

The command for the metadata service will look similar to the management service:

docker run --name="beegfs-meta" \
  --env beegfs_setup_1="beegfs-setup-meta -C -p /data/meta_01_tgt_0101 -s 1 -S meta_01" \
  --env CONN_AUTH_FILE_DATA="shared_secret_string" \
  --volume=/mnt/meta_01_tgt_0101:/data/meta_01_tgt_0101 \
  --network="host" \
  -d beegfs/beegfs-meta:latest \
  sysMgmtdHost=<MGMT_IP> storeAllowFirstRunInit=false connDisableAuthentication=true \
  connInterfacesList=<COMMA_SEPARATED_LIST> storeMetaDirectory=/data/meta_01_tgt_0101

In the command for the storage service we can specify multiple targets including additional setup commands, volumes, and extending the storeStorageDirectory list:

docker run --name="beegfs-storage" \
  --env beegfs_setup_1="beegfs-setup-storage -C -p /data/stor_01_tgt_101 -s 1 -S stor_01_tgt_101 -i 101" \
  --env beegfs_setup_2="beegfs-setup-storage -C -p /data/stor_01_tgt_102 -s 1 -S stor_01_tgt_101 -i 102" \
  --env CONN_AUTH_FILE_DATA="shared_secret_string" \
  --volume=/mnt/stor_01_tgt_101:/data/stor_01_tgt_101 --volume=/mnt/stor_01_tgt_102:/data/stor_01_tgt_102 \
  --network="host" \
  -d beegfs/beegfs-storage:latest \
  sysMgmtdHost=<MGMT_IP> storeAllowFirstRunInit=false connDisableAuthentication=true \
  connInterfacesList=<COMMA_SEPARATED_LIST> storeStorageDirectory=/data/stor_01_tgt_101,/data/stor_01_tgt_102

Note

Before running the above commands ensure to adjust the sysMgmtdHost and connInterfacesList for each service to match your system.

Using RDMA with BeeGFS Containers¶

By default Docker containers are “unprivileged” and cannot perform functionality necessary to take advantage of RDMA-capable hardware like NVIDIA InfiniBand adapters. For simplicity the quick start section used Docker’s --privileged flag, which gives all Linux capabilities to the container and allows RDMA to work. A more secure approach is to add select capabilities to the container and provide access to specific devices which will be demonstrated in this section. These additional docker run flags will need to be specified before the Docker image name.

First we need to add the IPC_LOCK Linux capability to the container, this is required so BeeGFS is able to lock memory used for RDMA connections:

--cap-add=IPC_LOCK \

Next we need to give the container access to the InfiniBand devices we want BeeGFS to use. By default the container will be able to read, write, and mknod these devices. BeeGFS will need access to the RDMA communication manager (rdma_cm) and each of the InfiniBand interfaces (devices nodes indicated by uverbsX) you want the container to use:

--device=/dev/infiniband/rdma_cm --device=/dev/infiniband/uverbs0 --device=/dev/infiniband/uverbs1 \

If we put it all together for the management service, the command looks like:

docker run --name="beegfs-mgmtd" \
  --env beegfs_setup_1="beegfs-setup-mgmtd -p /data/mgmt_tgt_mgmt01 -C -S mgmt_tgt_mgmt01" \
  --env CONN_AUTH_FILE_DATA="shared_secret_string" \
  --volume=/mnt/mgmt_tgt_mgmt01:/data/mgmt_tgt_mgmt01 \
  --network="host" \
  --cap-add=IPC_LOCK \
  --device=/dev/infiniband/rdma_cm --device=/dev/infiniband/uverbs0 --device=/dev/infiniband/uverbs1 \
  -d beegfs/beegfs-mgmtd:latest \
  storeAllowFirstRunInit=false \
  connInterfacesList=<COMMA_SEPARATED_LIST> storeMgmtdDirectory=/data/mgmt_tgt_mgmt01

For the metadata service the command looks like:

docker run --name="beegfs-meta" \
  --env beegfs_setup_1="beegfs-setup-meta -C -p /data/meta_01_tgt_0101 -s 1 -S meta_01" \
  --env CONN_AUTH_FILE_DATA="shared_secret_string" \
  --volume=/mnt/meta_01_tgt_0101:/data/meta_01_tgt_0101 \
  --network="host" \
  --cap-add=IPC_LOCK \
  --device=/dev/infiniband/rdma_cm --device=/dev/infiniband/uverbs0 --device=/dev/infiniband/uverbs1 \
  -d beegfs/beegfs-meta:latest \
  sysMgmtdHost=<MGMT_IP> storeAllowFirstRunInit=false \
  connInterfacesList=<COMMA_SEPARATED_LIST> storeMetaDirectory=/data/meta_01_tgt_0101

For the storage service the command looks like:

docker run --name="beegfs-storage" \
  --env beegfs_setup_1="beegfs-setup-storage -C -p /data/stor_01_tgt_101 -s 1 -S stor_01_tgt_101 -i 101" \
  --env beegfs_setup_2="beegfs-setup-storage -C -p /data/stor_01_tgt_102 -s 1 -S stor_01_tgt_101 -i 102" \
  --env CONN_AUTH_FILE_DATA="shared_secret_string" \
  --volume=/mnt/stor_01_tgt_101:/data/stor_01_tgt_101 --volume=/mnt/stor_01_tgt_102:/data/stor_01_tgt_102 \
  --network="host" \
  --cap-add=IPC_LOCK \
  --device=/dev/infiniband/rdma_cm --device=/dev/infiniband/uverbs0 --device=/dev/infiniband/uverbs1 \
  -d beegfs/beegfs-storage:latest \
  sysMgmtdHost=<MGMT_IP> storeAllowFirstRunInit=false \
  connInterfacesList=<COMMA_SEPARATED_LIST> storeStorageDirectory=/data/stor_01_tgt_101,/data/stor_01_tgt_102

Note

Before running the above commands adjust the specified –devices, sysMgmtdHost, and connInterfacesList for each service to match your system.

Automatically Starting Containers after a Reboot¶

In a traditional BeeGFS deployment services are automatically configured to start using systemd. By default containers will not automatically restart if they exit (or the system reboots) unless you specify a restart policy as part of the Docker run command.

Additional Docker Commands¶

Viewing a Container’s Logs¶

BeeGFS containers will not log to /var/log either inside the container or in the base OS. Instead use docker logs <CONTAINER_NAME> to view the logs for a particular service.

Executing Commands in a Container¶

If you need to execute a command in the container do not use “docker attach” since we didn’t allocate a pseudo-tty (-t) or tell the container to listen on stdin (-i) when we started it. Instead use “docker exec” to run additional commands inside the container.

For example if you want a bash shell inside the container run docker exec -it <container-name> /bin/bash`. From here you can run any available commands inside the container and use ``exit when you want to return to the base OS leaving the container running in the background.

Note

The shell prompt (root@<HOSTNAME>:/#) won’t change if you’re using the -network="host" option. You can verify you’re attached to the container by running “cat /proc/1/cgroup” and verifying the paths container /docker/.

Stopping a Container¶

To stop a container run docker stop <CONTAINER_NAME>

Removing a Container¶

When you stop a container it will not automatically be removed. This can cause issues if say you need to update configuration. To remove a container run docker rm <CONTAINER_NAME>.

Running BeeGFS containers using Docker Compose¶

Docker Compose is a tool designed to simplify defining how multiple containers should run. With Compose you store the configuration representing how you want your containers to run in a YAML file that can manage multiple containers with a single command. This is drastically simpler than having to write/remember long Docker run commands, and also allows you to version control your container configuration. Each of the flags to Docker run has a corresponding configuration option in Docker Compose, see the Compose file reference for full details.

Defining a Docker Compose File¶

The above RDMA enabled BeeGFS containers example can be converted to a Docker Compose file as follows:

version: '3'
services:

  beegfs-mgmtd:
    container_name: beegfs-mgmtd
    image: beegfs/beegfs-mgmtd:latest
    environment:
      - beegfs_setup_1="beegfs-setup-mgmtd -p /data/mgmt_tgt_mgmt01 -C -S mgmt_tgt_mgmt01"
      - CONN_AUTH_FILE_DATA=shared_secret_string
    volumes:
      - /mnt/mgmt_tgt_mgmt01:/data/mgmt_tgt_mgmt01
    network_mode: "host"
    cap_add:
      - IPC_LOCK
    devices:
      - /dev/infiniband/rdma_cm
      - /dev/infiniband/uverbs0
      - /dev/infiniband/uverbs1
    command: "storeAllowFirstRunInit=false connInterfacesList=<COMMA_SEPARATED_LIST> storeMgmtdDirectory=/data/mgmt_tgt_mgmt01"
    restart: unless-stopped

  beegfs-meta:
    container_name: beegfs-meta
    image: beegfs/beegfs-meta:latest
    environment:
      - beegfs_setup_1=beegfs-setup-meta -C -p /data/meta_01_tgt_0101 -s 1 -S meta_01
      - CONN_AUTH_FILE_DATA=shared_secret_string
    volumes:
      - /mnt/meta_01_tgt_0101:/data/meta_01_tgt_0101
    network_mode: "host"
    cap_add:
      - IPC_LOCK
    devices:
      - /dev/infiniband/rdma_cm
      - /dev/infiniband/uverbs0
      - /dev/infiniband/uverbs1
    command: "sysMgmtdHost=<MGMT_IP> storeAllowFirstRunInit=false connInterfacesList=<COMMA_SEPARATED_LIST> storeMetaDirectory=/data/meta_01_tgt_0101"
    restart: unless-stopped
    depends_on:
      - beegfs-mgmtd

  beegfs-storage:
    container_name: beegfs-storage
    image: beegfs/beegfs-storage:latest
    environment:
      - beegfs_setup_1=beegfs-setup-storage -C -p /data/stor_01_tgt_101 -s 1 -S stor_01_tgt_101 -i 101
      - beegfs_setup_2=beegfs-setup-storage -C -p /data/stor_01_tgt_102 -s 1 -S stor_01_tgt_101 -i 102
      - CONN_AUTH_FILE_DATA=shared_secret_string
    volumes:
      - /mnt/stor_01_tgt_101:/data/stor_01_tgt_101
      - /mnt/stor_01_tgt_102:/data/stor_01_tgt_102
    network_mode: "host"
    cap_add:
      - IPC_LOCK
    devices:
      - /dev/infiniband/rdma_cm
      - /dev/infiniband/uverbs0
      - /dev/infiniband/uverbs1
    command: "sysMgmtdHost=<MGMT_IP> storeAllowFirstRunInit=false connInterfacesList=<COMMA_SEPARATED_LIST> storeStorageDirectory=/data/stor_01_tgt_101,/data/stor_01_tgt_102"
    restart: unless-stopped
    depends_on:
      - beegfs-mgmtd
      - beegfs-meta

Note

Note the use of the restart unless-stopped option, and depends_on option to control how containers start and stop to ensure the management service starts first, followed by the metadata then storage service.

Interacting with the Docker Compose File¶

Save the above file as docker-compose.yml, ensure the current working directory is set to the same directory, then use the following commands to interact with it.

Starting All Containers¶

Start all containers with docker-compose up. By default this command aggregates the output of each container, and when you exit (ctrl+c) the containers will stop. Alternatively use docker-compose up --detach to start containers in the background and leave them running.

Viewing All Container Logs¶

If you started Docker Compose in detached mode you can use docker-compose logs to view the aggregated output of each container.

Stopping All Containers¶

Stop all containers with docker-compose stop. As with docker stop you will need to remove the containers if you want to reconfigure anything.

Removing All Containers¶

Remove all containers with docker-compose rm

Stopping and Removing All Containers¶

To stop and remove containers with one command use docker-compose down.

Running BeeGFS in Containers¶

Overview¶

Definitions¶

Prerequisites¶

Downloading/Pulling BeeGFS Images¶

Image Verification¶

Docker Content Trust¶

Cosign¶

Quick Start: BeeGFS Containers¶

Steps¶

Advanced Topics¶

Running BeeGFS in Containers using Docker Run¶

Using RDMA with BeeGFS Containers¶

Automatically Starting Containers after a Reboot¶

Additional Docker Commands¶

Viewing a Container’s Logs¶

Executing Commands in a Container¶

Stopping a Container¶

Removing a Container¶

Running BeeGFS containers using Docker Compose¶

Defining a Docker Compose File¶

Interacting with the Docker Compose File¶

Starting All Containers¶

Viewing All Container Logs¶

Stopping All Containers¶

Removing All Containers¶

Stopping and Removing All Containers¶

Documentation

Table of Contents

Previous topic

Next topic