Access Control Lists

BeeGFS stores Access Control Lists (ACLs) as Extended Attributes of metadata files on the metadata server.

The client nodes need to run the Linux kernel version 3.1 or above to support ACLs. The ACL enforcement facilities needed by BeeGFS are not supported by older kernel versions.

To enable ACLs, edit the metadata configuration file (/etc/beegfs/beegfs-meta.conf) and check if the following options are set to true.

storeClientXAttrs       = true
storeClientACLs         = true

Then, edit the client configuration file (/etc/beegfs/beegfs-client.conf) and check if the following options are set to true.

sysXAttrsEnabled = true
sysACLsEnabled   = true

If you change the configuration files, remember to restart their respective services afterward.

Note that enabling ACLs comes at some cost of reduced metadata performance, because additional data needs to be transferred and checked.

ACL caching and revalidation

Until BeeGFS 8.2, the client module took a very conservative approach to caching and revalidating ACLs. Locally cached ACLs were invalidated on every access, leading to increased latency and metadata load because of large numbers of RPCs to the meta servers needed to revalidate and fetch them every time they were accessed on any client in the system.

BeeGFS 8.2 addresses this by changing the default behavior and introducing a new configuration option sysACLsRevalidate that can be set to either cache or always. The new default cache option configures the client to cache ACLs for the same time other file or directory attributes including access permissions are cached. That time can be adjusted by changing the values of tuneDirSubentryCacheValidityMS (for directories) and tuneFileSubentryCacheValidityMS (for files). Even with the default time settings of 1s for directories and no cache (0s) for files, the cache option for sysACLsRevalidate has a significant positive effect on performance because ACLs for path components will be cached during path resolution and repeated access to the same directory. The effects can be improved by setting longer cache times, which comes at the expense of using a slightly increased propagation time for ACLs between clients. In the vast majority of cases, the performance benefits should outweigh that cost, but in case strict guarantees for ACL propagations are required, sysACLsRevalidate can be set to always to revert back to legacy behavior and revalidate ACLs on every access.

Additional performance considerations with Extended Attributes

The Linux kernel uses a security mechanism that automatically removes setuid/setgid bits and capabilities from files when they are changed. This is done to prevent users from executing binaries with elevated privileges that were changed after the privileges were originally set. That mechanism requires that, by default, the kernel has to check each file for existing capabilities on every write which leads to a large overhead in metadata RPCs to fetch the “security.capability” extended attribute. To optimize this, Linux allows file systems to set a flag (S_NOSEC) on the file, which short-circuits these checks.

The sysXAttrsCheckCapabilities client configuration option configures the file system mount to allow the client to set that flag on new inodes either immediately and without any checks at all (sysXAttrsCheckCapabilities = never), after a first lookup of the extended attribute returns an empty result (sysXAttrsCheckCapabilities = cache) or not at all to always check (sysXAttrsCheckCapabilities = always). The flag will automatically be cleared when capabilities are modified on this client. It will, however, currently not be cleared if a different client modifies capabilities or sets setuid/setgid bits, which can lead to capabilities not being cleared, even after the file is written to. If this is a concern, the option should be set to always.

As long as BeeGFS is mounted using the “nosuid” mount option (which is recommended and the default setting), elevating privileges via setuid/setgid bits and capabilities is disabled and it is safe to set sysXAttrsCheckCapabilities to never.