Streamline your your AI/ML data transfers with new GKE Volume Populator

As an AI/ML developer, you have a lot of decisions to make when it comes to choosing your infrastructure — even if you’re running on top of a fully managed Google Kubernetes Engine (GKE) environment. While GKE acts as the central orchestrator for your AI/ML workloads — managing compute resources, scaling your workloads, and simplifying complex workflows — you still need to choose an ML framework, your preferred compute (TPU or GPUs), a scheduler (Ray, Kueue, Slurm) and how you want to scale your workloads. By the time you have to configure storage, you’re facing decision fatigue! 

You could simply choose Google’s Cloud Storage for its size, scale and cost efficiency. However, Cloud Storage may not be a good fit for all use cases. For instance, you might benefit from a storage accelerator in front of Cloud Storage like Hyperdisk ML for better model weights load times. But in order to benefit from the acceleration these bring, you would need to develop custom workflows to orchestrate data transfer across storage systems. 

Introducing GKE Volume Populator 

GKE Volume Populator is targeted at organizations that want to store their data in one data source and let GKE orchestrate the data transfers. To achieve this, GKE leverages the Kubernetes Volume Populator feature through the same PersistentVolumeClaim API that customers use today. 

GKE Volume Populator along with the relevant CSI drivers dynamically provision a new destination storage volume and transfer data from your Cloud Storage bucket to the destination storage volume. Your workload pods then wait to be scheduled until the data transfer is complete.

Using GKE Volume Populator provides a number of benefits:

• Low management overhead: As part of a managed solution that’s enabled by default, GKE Volume Populator handles the data transfer, so you don’t need to build a bespoke solution for data hydration but leave it to GKE.

• Fine-grained access control: GKE Volume Populator supports namespace-level Cloud Storage bucket access authentication.

• Optimized resource utilization: Your workload pods are blocked for scheduling until the data transfer completes. You can use your GPUs/TPUs for other tasks while data is being transferred.

• Easy progress tracking: Monitor the data transfer progress by checking the event message on your PVC object. 

Customers like Abridge AI report that GKE Volume Populator is helping them streamline their AI development processes.

“Abridge AI is revolutionizing clinical documentation by leveraging generative AI to summarize patient-clinician conversations in real time. By adopting Google Cloud Hyperdisk ML, we’ve accelerated model loading speeds by up to 76% and reduced pod initialization times. Additionally, the new GKE Volume Populator feature has significantly streamlined access to large models and LoRA adapters stored in Cloud Storage buckets. These performance improvements enable us to process and generate clinical notes with unprecedented efficiency — especially during periods of high clinician demand.” - Taruj Goyal, Software Engineer, Abridge

Accelerate your data via Hyperdisk ML

Let’s say you have an AI/ML inference workload, and your data is stored in a Cloud Storage bucket, you want to move your data from the Cloud Storage bucket to a Hyperdisk ML instance to accelerate the loading of model weights, scale up to 2,500 concurrent nodes and reduce the pod over-provisioning. Here's how to do this with GKE Volume Populator:

1. Prepare your GKE Cluster: Create a GKE cluster with the corresponding CSI driver, and enable Workload Identity Federation for GKE.

2. Set up necessary permissions: Configure permissions so that GKE Volume Populator has read access to your Cloud Storage bucket. 

3. Define Your data source: Create a GCPDataSource This specifies:

• The URL of the Cloud Storage bucket that contains your data
• The Kubernetes Service Account you created with read access to the bucket

4. Create your PersistentVolumeClaim: Create a PVC that refers to the GCPDataSource you created in step 3 and the corresponding StorageClass for the destination storage.

5. Deploy Your AI/ML workload: Create your inference workload with the PVC. Configure this workload to use the PVC you created in step 4. 

GKE Volume Populator is generally available, and support for Hyperdisk ML is in preview. To enable it in your console, reach out to your account team.