BeeGFS

BeeGFS is a parallel filesystem for HPC clusters. Files are striped as chunks across multiple storage servers, so many clients can read and write in parallel rather than contending for one server.

The architecture separates four roles. Management is a rendezvous point where servers and clients find each other — it holds cluster metadata, not user data, and is not on the performance path. Storage servers hold the chunk files, and can use multiple targets on different media. Metadata servers hold the directory tree and the map of which chunks belong to which file; each metadata server has exactly one target, and metadata capacity is scaled by adding servers. Clients mount the filesystem through a kernel module, conventionally at /mnt/beegfs.

Why metadata separation matters

Splitting metadata from data is the design decision that determines behaviour under load.

Metadata operations — stat, open, directory listing — are small, numerous, and latency-sensitive. Data operations are large and bandwidth-sensitive. Serving both from the same servers means a job doing many small file operations degrades throughput for everyone else.

The practical consequence is a workload pattern worth knowing: many small files is the pathological case. A climate model writing one file per process per timestep generates enormous metadata load and poor throughput. Fewer, larger files with parallel writes is dramatically better, and this is usually a model configuration decision rather than a storage one.

Its actual advantage

The reason to choose BeeGFS over Lustre is operational rather than technical. It installs from packages, runs in userspace for the server components, and can be set up in an afternoon by someone who is not a storage specialist. Lustre is more capable at the very largest scale and considerably more demanding to run.

For a mid-sized institutional cluster where nobody’s job title is “filesystem engineer”, that difference decides it. Graphical tooling exists and is mostly for monitoring; day-to-day work is command-line.

It is not a backup. Striping distributes data, it does not replicate it by default, and a failed storage target without mirroring means data loss. This is worth being explicit about because “distributed” reads as “redundant” to people who have not run one.

See also: Lustre, GlusterFS, Ceph, HPC, and the cluster context.