AivexaNewsSearch
AI news for builders and product teamsChecked every hour

Announcing SOCI-indexed images for faster SageMaker AI jobs

Collected Oct 27, 2026

AWS announced Seekable OCI (SOCI) indexing for Amazon SageMaker AI Training and Amazon SageMaker AI processing jobs. The feature lets SageMaker AI lazy-load container images from Amazon Elastic Container Registry (ECR) instead of downloading every layer before the job starts. Adopting it requires no changes to a job definition, entry point or data channels; teams either point at a pre-indexed AWS Deep Learning Container or add an index to their own image.

SOCI is an open source technology for lazy-loading container images. The index is metadata that maps every file inside the image, stored in ECR as an OCI-compliant artifact linked to the original image. Because it is a separate artifact, the image digest and any existing signatures stay valid. When a job is created, SageMaker AI checks for an index: if one exists it starts the container before the full download finishes and streams remaining files as code touches them; if not, behavior is unchanged. If the index cannot be read or lazy loading fails, the job falls back to a standard download and keeps running.

AWS benchmarked eight images from 1.8 GB to 35.4 GB compressed across GPU and CPU instances (ml.g4dn, ml.g5, ml.g6, ml.g6e, ml.g7, ml.g7e, ml.p4d, ml.p5 and ml.m7i, xlarge to 48xlarge) in us-west-2 and us-east-1, measuring median download time from pull start until the container was ready. A 1.8 GB image dropped from 48 seconds to 3–11 seconds. A 13.2 GB image fell from 3 minutes 42 seconds to about 6 seconds. A 35.4 GB image went from 7 minutes 17 seconds to 3–11 seconds. Across the eight images the improvement ranged from roughly 8x to 46x, and lazy loading stayed in a 3–11 second band regardless of size.

On a 13.2 GB image run across 13 instance type and size combinations, standard download ranged from 3 minutes 16 seconds to 4 minutes 20 seconds while lazy loading held at 6–8 seconds, about 34x faster on every configuration.

Training speed was then checked with full jobs: ResNet-50 image classification on Imagenette (9,469 images) using PyTorch (13.2 GB) and TensorFlow (9.9 GB) containers on ml.p5.48xlarge, ml.p4d.24xlarge, ml.g7e.48xlarge, ml.g7.48xlarge, ml.g6e.12xlarge, ml.g6e.24xlarge and ml.g6e.48xlarge; and full-parameter supervised fine-tuning of Qwen2.5-0.5B on alpaca-cleaned (51,760 examples) with the Hugging Face Trainer in a 9.4 GB container on ml.p5.48xlarge, ml.p4d.24xlarge and ml.g6.48xlarge. From the second epoch on, time per epoch matched within run-to-run variation on every instance. The first epoch took 3–16 seconds longer with lazy loading, a fixed cost across instance sizes, and each job finished 1–6 minutes sooner overall.

Startup patterns that front-load reads still came out ahead. torch.compile plus a just-in-time compiled CUDA extension added 7–15 seconds to the first epoch, leaving a net saving of 3 minutes 25 seconds to 3 minutes 56 seconds. A 6 GB model baked into the image streamed its weights at load time, adding 2–3.5 minutes, yet the job still finished 2–2.5 minutes sooner. When an entry point reads most of the image before training (scanning, checksumming or preloading), the first epoch waits for roughly the bytes a full pull would have fetched, so the net gain is small.

For adoption, AWS publishes SOCI-indexed variants of popular Deep Learning Containers with a -soci suffix on the tag, for example pytorch-training:2.9-gpu-py312-cu130-ubuntu22.04-sagemaker-v1-soci in the 763104351884 registry. Pre-indexed variants cover PyTorch Training, TensorFlow Training, vLLM and SGLang, with arm64 variants for some frameworks. Images built FROM a DLC base need their own index, because the base index does not carry over to derived images.

To index a custom image, AWS describes using a Linux host such as an Amazon Linux 2023 EC2 instance with containerd, the SOCI snapshotter and Finch or nerdctl. The steps install the tools, authenticate to ECR, pull the original image, run nerdctl image convert --soci to produce a new tag, and push it for linux/amd64. That produces the index and an OCI image index manifest, leaving the original image digest untouched. For CI/CD, a CFN ECR SOCI Index Builder uses an EventBridge rule on ECR push events to build indexes automatically without pipeline changes. Verification uses aws ecr batch-get-image with the OCI image index media type and a check for a child manifest whose artifactType begins with application/vnd.amazon.soci.index. Using the image needs only the tag change, as in a ModelTrainer training_image value ending in -soci.

Limits worth noting: lazy loading applies only to ECR private repositories, not other registries or images pulled through a private Docker registry with TrainingRepositoryAccessMode set to Vpc. Images below roughly 4 GB see small improvements. Indexed images run on a containerd-based runtime, so entry points and setup scripts that call the Docker CLI, read /var/run/docker.sock or derive identifiers from cgroup paths should be reviewed. AWS mounts a /.dockerenv marker for backward compatibility but advises against new dependencies on it. The feature is available in all AWS Regions.

In practice, the gain is proportional to image size and to how little of the image a workload touches before training begins. The fixed per-job saving makes the feature most valuable for hyperparameter sweeps, quick experiments and short processing jobs, where a few minutes of pull time can dominate total runtime.

Why it matters: Teams running many short SageMaker AI jobs, especially tuning sweeps, pay for image pulls as billable job time, and the benchmarks suggest most of that overhead disappears with a tag change. The main work for custom images is producing and maintaining an index, which AWS offers to automate via its ECR builder; per the reasoning above, the payoff shrinks for small images or entry points that read most of the image up front.

Read at AWS Machine Learning Blog

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

Seekable OCI (SOCI) indexing for Amazon SageMaker AI training and processing jobs takes full container image download off the critical path. SageMaker AI lazy-loads your image to start containers in seconds instead of minutes, with no changes to your job definition. See the benchmarks and how to adopt it.