AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Scaling Federated Learning Across Docker, Kubernetes, and Slurm with NVIDIA FLARE

Collected Sep 30, 2026

NVIDIA FLARE separates persistent federation services from the processes that execute each job, allowing sites within one federation to use different execution backends. According to NVIDIA, one site can run Docker, another Kubernetes, and a third Slurm, with each site keeping local control over compute allocation and the datasets, images, secrets, and scheduling policies used by each study.

A FLARE deployment has two operational layers. Long-running server and client parent processes maintain the federation, authenticate connections, and coordinate work. Separate server and client job workers execute a submitted federated learning job. The parent processes can remain available without occupying the GPUs required for training; when a job is submitted, each parent launches a worker through its configured execution platform. The worker receives the job, uses the requested resources, returns results, and exits.

Jobs describe resource intent separately from platform details. A job can request GPUs, whole schedulable CPU units, and host memory. Each site's launcher combines those requirements with the local study configuration and translates them into a Docker container, Kubernetes pod, or Slurm allocation.

Docker and Kubernetes deployment support is available in NVIDIA FLARE 2.8. FLARE 2.9 adds Slurm support. In Docker, a persistent parent container runs the FLARE server or client, and FLARE launches a separate job container per submitted job. In Kubernetes, a Helm chart installs each persistent server or client as a parent pod, and the parent creates a separate job pod placed by the Kubernetes scheduler. The Slurm launcher submits each job worker as a batch job, with Slurm selecting compute nodes and enforcing GPU, CPU, memory, partition, account, quality of service, and time limits. It supports bare execution, Pyxis/Enroot containers, and Apptainer containers.

FLARE studies provide a logical multi-tenant boundary within one deployment. Each study defines participating client sites and admin users who can open a session for that study. A site-owned local/study_runtime.yaml file maps a study to local resources including dataset mounts, environment variables, secret-backed settings, an approved default job image, Kubernetes pod templates, Docker runtime settings, and Slurm sandbox, partition, account, and QoS policy. NVIDIA states studies are intended for organizations sharing the same FLARE server and public key infrastructure but needing logical separation between experiments; organizations requiring a separate PKI, infrastructure administrator, or failure blast radius should use separate FLARE deployments.

FLARE 2.9 supports portable resource requirements in resource_spec within a job's meta.json, using the keys num_of_gpus, num_of_cpus, and memory. CPU values represent whole schedulable CPU units, and memory uses positive integer quantities with Mi, Gi, or Ti units. A @default entry applies one resource profile to every targeted site, including the server, while named client or server entries provide only values that differ. Each launcher translates resolved values into native settings.

Read at NVIDIA Developer Blog

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Federated learning (FL) projects often begin with a straightforward setup: one server, a few clients, and one dataset at each site. As those projects grow, the...