Introducing Batch on GKE—modernizing HPC with Kubernetes in the cloud

One of our most important goals at Google Cloud is to make cloud computing easier, so that you can focus on answering questions that matter the most to you, your business and your users—not on managing infrastructure. 

Today, we are excited to announce the preview of Batch on Google Kubernetes Engine (GKE), a cloud-native solution for running batch workloads at scale in an optimized manner. Batch on GKE brings the functionality and familiarity of a traditional batch job scheduler into a cloud-first world. It frees your applications from the limitations of fixed-sized compute clusters by dynamically allocating resources to meet the needs of your application.

Google Cloud is the home of Kubernetes—originally developed here and released as open source in 2014. GKE is a managed, production-ready environment for deploying containerized applications that relieves your teams from the operational toil of Kubernetes cluster management, allowing you to focus on your business needs. We heard you wanted to bring the benefits of GKE to batch workloads such as media rendering, genomics sequencing, silicon design verification and financial portfolio risk analysis, so we built Batch on GKE to bring what you love about GKE to these real-world batch workloads.

The preview release of Batch on GKE comes with the following capabilities:

• Autoscaling and just-in-time provisioning to ensure you pay for just what you need 
• Rightsizing of virtual machines to tailor fit CPU and memory for the job at hand 
• Smart reuse of virtual machines and smart packing of jobs to reduce waste and the time jobs spend waiting in a queue
• Resource budgets to allocate the maximum spend per team
• Graphics Processing Unit (GPU) support
• Job submission tool that high performance computing practitioners will find familiar

Batch on GKE’s easy-to-use, familiar interface enables you to deliver business results for your batch computing use cases. The following diagram shows how users can leverage Batch on GKE and other Google Cloud services to build a genomics processing and analysis system.

To install and get started with Batch on GKE, you can view the product documentation and visit the Batch on GKE page. As you explore Batch on GKE, we’d love to hear your feedback and feature requests – email us.

Meeting you where you are with partners
Batch on GKE is a great solution for modernizing your batch workloads. We’re also committed to helping you migrate your existing systems as-is to Google Cloud or augment your on-premise setup by connecting to Google Cloud. We partner with SchedMD, Altair and Univa to integrate their market-leading schedulers with our platform and meet you where you are.  

Engineering simulation made easy
We’re also working closely with Rescale to enable their full-service HPC platform to run engineering simulations on Google Cloud and leverage on-demand GCP clusters and virtually unlimited cores. With this integration, you can run and manage simulations from a vast application library, including ANSYS Fluent, LS-DYNA, Star-CCM+ and more. Click here for a list of all the software that Rescale supports on Google Cloud. 

If you’re attending SuperComputing 2019, be sure to visit the Google Cloud booth (#1363). You can find more information about our SC19 presence here. You can also learn more about how Google Cloud’s flexible infrastructure can accelerate your HPC workloads.

Special thanks to Senanu Aggor, Product Marketing, and Annie Ma-Weaver, Partner Manager, for making this blog post possible.