How to Read Data into GPU Memory in Python with NVIDIA GPUDirect Storage

NVIDIA published a developer tutorial showing how to read files into GPU memory from Python using KvikIO, its Python and C++ library that sits above the cuFile C API for GPUDirect Storage (GDS). The walkthrough, tested on an x86_64 system with an H100 GPU, CUDA 13.3 and cuFile 1.18 against a local ext4-formatted NVMe device, covers installing KvikIO, inspecting configuration, performing a 1 GiB read, and confirming which data path the transfer actually took.
The motivation is data growth. NVIDIA cites the 2025 Stanford AI Index finding that compute for notable AI models has been roughly doubling every five months, while LLM training dataset sizes have been doubling roughly every eight months. GDS addresses the storage side of that gap by creating a direct path between supported storage and GPU memory, skipping the staging buffer in CPU memory and reducing CPU load; NVIDIA says a growing set of storage vendors supports it. The company also notes that as of August 2025 nearly half of new AI projects on GitHub were built primarily in Python, while the main application-facing interface to GDS has been the cuFile C API.
The architecture matters more than the code. KvikIO and cuFile make two independent decisions. KvikIO first decides whether to call cuFile at all or fall back to its own POSIX compatibility path, which stages data through host memory. If cuFile is called, cuFile then independently decides whether the transfer uses the accelerated nvidia-fs kernel module or its own CPU-staged compatibility path, based on system support, the file system and its configuration. The practical consequence, which the tutorial stresses, is that a successful cuFile call confirms only the first decision. Reaching the cuFile API does not by itself prove the accelerated path was active.
On the KvikIO side, two runtime settings control behavior. compat_mode can be AUTO, ON or OFF and is exposed in Python through kvikio.defaults; AUTO tries cuFile and falls back to POSIX if cuFile is unavailable. gds_threshold, which defaulted to 16 KiB on the test system, routes reads smaller than that size straight to the KvikIO POSIX path. Both can be set with the environment variables KVIKIO_COMPAT_MODE and KVIKIO_GDS_THRESHOLD, or changed at runtime through defaults.set, which accepts either a dictionary of settings or a single name and value.
Setup is a pip install of the CUDA 13 build, kvikio-cu13, which pulls in CuPy as a dependency. The tutorial's example creates a 1 GiB file of random data with direct I/O and fsync, allocates a CuPy array of matching size, and reads the file into it inside a kvikio.CuFile context manager, checking that the returned byte count matches the buffer size. That check verifies completeness, not the transfer path.
To see the path, NVIDIA uses cuFile's per-process runtime statistics. Setting CUFILE_STATS_LEVEL=3 and running Python interactively keeps the process alive so gds_stats can inspect its counters. The per-GPU output separates nvfs, which counts operations that went through the nvidia-fs kernel module, from posix, which counts operations handled by the cuFile compatibility path. With KvikIO's default 4 MiB task size, the 1 GiB read is split into 256 operations; the tutorial reports n=256, nvfs=256 and posix=0, meaning every operation took the accelerated path.
The tutorial then isolates each layer. Forcing KVIKIO_COMPAT_MODE=off keeps operations in nvfs. Adding CUFILE_FORCE_COMPAT_MODE=true leaves KvikIO still calling cuFile but moves the operations into the posix counters. Setting KVIKIO_COMPAT_MODE=on bypasses cuFile entirely, so no cuFile statistics are recorded at all. Anyone benchmarking GDS performance should note the implication: an nvfs count is evidence of the direct path, while a posix count is evidence of staging through host memory, and the two layers must be checked separately because either can silently degrade throughput.
A second configuration layer lives in cufile.json, independent of KvikIO settings. On the test configuration, cuFile had four worker threads, max_request_parallelism of four and min_io_threshold_size_kb of 8192 (8 MiB). Because KvikIO's task size is 4 MiB by default, each of the 256 requests reaches cuFile as a single piece, producing the 256 nvfs reads. Raising KVIKIO_TASK_SIZE to 1 GiB instead sends one large request, which cuFile splits at most four ways, yielding four 256 MiB pieces; three go to worker threads and the fourth runs on the calling thread. Those pieces are units of scheduled work, not individual transfers. Since the CuPy destination buffer is not registered with cuFile for storage DMA, cuFile reads into a registered GPU bounce buffer and copies into the destination. Since GDS 1.17 those bounce buffers can be configured as slabs, and on this setup the nvidia-fs path uses slabs no larger than 16 MiB, so the 8 MiB threshold does not govern the actual transfer size. It would only matter for requests of 16 MiB or so, where an even four-way split would fall below the target.
KvikIO is part of NVIDIA CUDA-X for data science, formerly RAPIDS, and grew out of GDS integration built for the cuDF GPU DataFrame library, where parallel cuFile reads sped up Parquet and ORC ingestion. NVIDIA says KvikIO has also been used with GDS in cuCIM digital pathology workflows and in the cuPhoton xDataReader for scientific image analysis, alongside GPU-accelerated Presto analytics work. That history explains why the tutorial closes by noting that libraries such as cuDF depend on KvikIO for many I/O operations, so misconfigured paths can affect workloads well beyond hand-written scripts.
Why it matters: developers building GPU data pipelines in Python can now reach GDS without writing C, but the two-layer design means the default settings may quietly fall back to CPU-staged transfers. Teams should validate path selection with cuFile statistics rather than assume that a successful read used the accelerated route; this is my inference from the tutorial's emphasis, but it follows directly from how the counters are reported. Frameworks such as cuDF inherit whichever path their KvikIO task size and compatibility mode produce, so tuning those settings is likely to matter for ingestion-heavy workloads.
Based on reporting from the original publisher. Visit the source for full context and later updates.
Publisher excerpt
GPU applications are processing ever-growing datasets with increasingly powerful accelerators. According to the 2025 Stanford AI Index, compute for notable AI...