GCP Instance Groups: A Different Way To Autoscale Part 1

Author: Gaurav Jain

Introduction

At Attentive, a major chunk of our tech resources are spent towards providing our customers with highly accurate map features in the shortest amount of time. These features could be roads, building footprint, trees, parking lots, swimming pools, etc. This list is inexhaustive, as we are constantly adding more features to our catalog.

To ensure our goal is achieved seamlessly, we have designed our architecture in a manner that it can handle all the large Machine learning computation in a highly efficient manner; while scaling horizontally.

This article is the first part in the series where we describe how we run, manage and monitor a large set of celery workers to execute our custom deep learning pipelines at scale

Goal

To introduce how Google Cloud Platform (GCP) instance groups can be used to easily spin up multiple stateless servers of the same type.

What is an Instance group?

Google Cloud Platform defines an instance group as a collection of virtual machine (VM) instances that you can manage as a single entity.

Instance groups allow you to scale your VMs automatically or manually.

Manually it gives you the ease to just change a number to change the number of instances.

Automatically it can take your CPU, Memory, or custom metric usage.

Why should we use instance groups?

Instance groups are great for applications that require swap memory.

If your application is dockerized and the resource consumption is not high, then the advice would be to use Kubernetes for a long-term solution.

However, for high resource consuming applications, instance groups could be used till the resource requirements are reduced.

What is an Instance group?

Instance groups is a managed service by GCP, which allows you to spin up multiple instances that follow an exact template.

How to setup an instance group?

Navigate to instance groups

Create a new instance group

1. Go to Compute Engine -> Instance groups in your GCP console

2. Give your instance group a name

3. Select your region and zone

4. Configure the auto scaling specifications you require or you can also turn it off

Now let’s talk about instance templates

Create or select an existing template

What is an Instance template?

According to Google Cloud, an instance template is an immutable resource that you can use to create virtual machine (VM) instances and managed instance groups (MIGs).

How to make an Instance template:

1. We can make instance groups or instances from these templates
2. Mention the specifications you require for these machines.
3. For boot drive, you can select a custom image or use a public image such as Ubuntu
4. Click on management, security, disks, networking and sole tenancy.
5. In management, either give a start-up script(which we can not edit later) or give metadata with key startup-script-url of the script stored in a GCP bucket of the same project

Our Use Case

We’re identifying map features out of high resolution aerial imagery. This computation needs to be scaled by the number of jobs we get on our platforms, and due to the large size of the data we work on, our memory and compute requirements are quite high. We have a swarm of celery workers where each worker runs ML prediction over one parcel boundary. We have currently more than 200vCPUs, 1000GBs RAM and 70 GPUs in production using Instance Groups.

In the next part, we will talk about how to configure autoscaling using custom metric that is other than CPU or Memory utilisation

Some Extra Tips

Applying changes to the instance group

1. Create a new instance template.
2. Go to the edit instance group tab and change the template and then reboot your instances.
3. You can do a rolling restart so that even while restarting, you don’t require downtime.

Multi-Region Set Up

If you want to scale to multiple locations, then you will need to have a different Instance group for each location using the same instance template