India’s Leading Domain & Web Hosting Provider
Support
Domain & Login
Blog
Google Compute Engine Services: Machine Families, MIGs & Spot VMs
Google Compute Engine is Google Cloud's infrastructure-as-a-service for running virtual machines on machine families such as C4, N4, Tau and GPU-backed A3. NTSPL selects the right series per workload, builds autoscaled Managed Instance Groups, moves interruptible jobs to Spot VMs and Batch, and sizes committed use discounts for Indian teams running in Mumbai and Delhi regions.
Google Cloud services in scope
The Problem Compute Engine machine families (C4, N4, Tau T2D/T2A, A3) and Managed Instance Groups (MIGs) Solve
Most Compute Engine estates we inherit were built by lifting VMware templates one-for-one: n1-standard-8 everywhere, pd-standard disks, no instance groups, and a monthly bill that never drops at night. The real engineering problem is matching shape to work. A latency-sensitive API wants C4 with Hyperdisk Balanced; a CI fleet idles 70% of the day and wants N4 custom shapes in an autoscaled MIG; a genome alignment run can tolerate preemption and belongs on Spot VMs at a fraction of on-demand price. Choosing wrongly costs either money or tail latency, and usually both.
NTSPL starts with 14 days of Cloud Monitoring data per VM: CPU steal, memory working set, disk IOPS and egress. We map each workload to a machine series, decide which tiers move into regional MIGs with health-checked autohealing, and separate interruptible work into Batch jobs on Spot capacity. Commitments come last, sized only against the baseline that remains after rightsizing, so you never commit to waste.
When Compute Engine VMs Are the Right Home (and When GKE or Cloud Run Win)
Signals that point to compute on Google Cloud
- Monthly Compute Engine spend is flat across day and night even though traffic drops by more than half after 10 pm IST.
- Average CPU across the fleet sits below 20% while VMs still run first-generation N1 shapes with no committed use discounts.
- Batch jobs such as sequencing, rendering or simulations queue for hours behind a fixed on-premises cluster at quarter-end or project deadlines.
- A VM failure today means a human SSHing in at 3 am, because no instance group or autohealing health check exists.
- Your data team is renting GPUs elsewhere because nobody has validated A3 or G2 capacity and quota for your projects.
When a different approach is better
- Containerized microservices with many small, independently deployed services usually run more cheaply and with less patching on GKE Autopilot than on hand-managed MIGs.
- Request-driven HTTP APIs that sit idle for long periods are better on Cloud Run, which scales to zero and bills per request or instance time.
- Licensed Windows or SAP estates with strict socket-based licensing may belong on Sole-tenant nodes or Google Cloud VMware Engine rather than standard multi-tenant VMs.
Compute Engine Building Blocks: C4, N4, Tau, A3, MIGs, Spot VMs and Batch
| Service | Role in the architecture | Choose it when |
|---|---|---|
| Compute Engine machine families (C4, N4, Tau T2D/T2A, A3) | Provides the VM shapes: C4 for consistently high per-core performance, N4 for flexible general-purpose shapes, Tau for scale-out price-performance, and A3 with NVIDIA H100 GPUs for training and inference. | You can describe the workload's bottleneck (single-thread speed, memory ratio, core count or GPU) and want the cheapest shape that removes it. |
| Managed Instance Groups (MIGs) | Runs identical VMs from an instance template with autoscaling on CPU, load-balancer utilization or Cloud Monitoring metrics, plus autohealing and rolling or canary updates. | A stateless tier needs to scale horizontally, recover from VM failure without paging anyone, or receive new images without downtime. |
| Spot VMs | Supplies spare Compute Engine capacity at deep discounts with a 30-second preemption notice and no maximum runtime. | Work is checkpointed, idempotent or queue-driven, such as rendering, genomics or model batch scoring. |
| Committed use discounts (resource-based and spend-based) | Lowers the price of steady vCPU, memory and GPU consumption in exchange for one-year or three-year commitments. | After rightsizing, a predictable floor of usage runs 24x7 for at least a year. |
| Batch | Fully managed job scheduler that provisions VMs, including Spot and GPU shapes, runs containerized or script tasks in parallel, and tears capacity down when jobs finish. | Teams currently maintain a Slurm head node or cron-driven VM fleet only to run finite, parallel jobs. |
| Hyperdisk | Block storage that decouples provisioned IOPS and throughput from capacity, and the required disk type for newer series such as C4 and N4. | Databases or build agents need guaranteed IOPS that Persistent Disk would only deliver by over-provisioning capacity. |
Mixed on-demand and Spot regional compute fleet
Mixed on-demand and Spot regional compute fleet
User traffic enters through the external Application Load Balancer to a regional MIG of C4 or N4 VMs spread across three zones in asia-south1, which autoscales on load-balancer utilization. Asynchronous work lands on Pub/Sub and is processed by Batch jobs on Spot VMs that read and write Cloud Storage and checkpoint progress so preemptions cost minutes, not hours.
Select any component
Mixed on-demand and Spot regional compute fleet: click any of its 9 components to see its role and the design choice NTSPL makes for it, or trace the 6-step primary path.
Compute Engine Engineering Decisions: Machine Series, Autoscaler Signals and Spot Checkpointing
Machine series selection: C4, N4, Tau or older N2
We place latency-critical and per-core-licensed workloads on C4, whose higher clock and newer Intel silicon reduce core counts for databases priced by vCPU. N4 suits general web and application tiers because custom shapes let you buy, say, 6 vCPU and 20 GB instead of rounding up. Tau T2D fits scale-out stateless services where cost per request matters more than per-core speed. N2 remains useful only where sustained use discounts outweigh newer-generation gains.
MIG autoscaling signals and stabilization
CPU utilization is a poor signal for I/O-bound tiers. For web tiers behind a load balancer we scale on serving capacity at 60-70% of measured requests per second per VM. For queue workers we scale on Pub/Sub undelivered messages per VM. We set initialization periods from measured boot-to-ready times, typically 90-180 seconds, enable predictive autoscaling only where at least three days of cyclical history exist, and cap scale-in to 20% of the group per 10 minutes.
Spot VM eligibility and checkpointing
A job is Spot-eligible when a restart loses less than 15 minutes of work and the deadline tolerates one or two preemptions. We add checkpointing to Cloud Storage every 5-10 minutes, handle the 30-second ACPI shutdown signal in a shutdown script, and spread Batch task groups across several machine types and zones so capacity in any one shape running dry does not stall the pipeline.
Commitment sizing after rightsizing
We commit against the P10 of hourly vCPU and memory usage over 60 days, not the average, so commitments stay fully used through seasonal dips. Resource-based CUDs suit stable fleets pinned to one series in one region; spend-based flexible commitments suit estates still migrating between N2, N4 and C4. We review coverage monthly and never commit capacity that Spot or scale-to-zero designs are about to remove.
Disk type and image strategy
Newer series such as C4 and N4 attach Hyperdisk rather than Persistent Disk, so we size IOPS and throughput explicitly instead of inflating capacity. Boot disks use Hyperdisk Balanced at the minimum provisioned performance; data disks receive measured IOPS plus 30% headroom. Golden images are built with Packer, hardened against CIS benchmarks, stored in a shared image family, and rolled out through MIG canary updates at 10%, then 50%, then 100%.
How NTSPL Moves a VM Fleet onto Autoscaled MIGs and Batch
Utilization capture with Ops Agent
Install the Ops Agent on every VM and collect 14 days of CPU, memory working set, Hyperdisk or Persistent Disk IOPS and egress, so shape choices rest on measured data rather than template sizes.
Machine series mapping and rightsizing
Assign each workload to C4, N4, Tau T2D or A3 and a custom or predefined shape, then apply Recommender rightsizing before anything is migrated or committed.
Golden image and instance template pipeline
Build CIS-hardened images with Packer into an image family and publish versioned instance templates with Shielded VM, OS Login and a dedicated service account.
Regional MIG rollout with autohealing
Deploy stateless tiers as regional MIGs across three zones, tune autoscaler signal, initialization period and scale-in controls, then shift load-balancer traffic in 10% steps.
Batch and Spot conversion for finite jobs
Move checkpointable jobs to Batch job templates with Spot task groups, retry-on-preemption and a standard-VM fallback for deadline-critical runs.
Commitment purchase and coverage review
After 30-60 days on the new shapes, buy resource-based or spend-based CUDs against the P10 hourly floor and review coverage monthly in Cloud Billing.
What you receive
Hardening Compute Engine: OS Login, Shielded VM and Per-MIG Service Accounts
Compute Engine's attack surface is the VM itself: SSH keys sprayed into project metadata, external IPs on every instance, unpatched images and default service accounts with Editor rights. NTSPL removes project-wide SSH keys in favour of OS Login, blocks external IPs by organization policy, gives each MIG a dedicated service account scoped to the buckets and topics it uses, and enables Shielded VM so boot-level tampering is detected.
- OS Login with two-step verification, so SSH access follows IAM and is revoked with the user account
- Organization policy constraints compute.vmExternalIpAccess and compute.requireShieldedVm to block exposed or unverified VMs
- Per-workload service accounts with no default Compute Engine service account and no Editor role
- Confidential VM on supported series for regulated genomic or financial data processed in memory
- Identity-Aware Proxy TCP forwarding for administrative access instead of bastion hosts with public IPs
Compute Engine Pricing Levers: Rightsizing, CUDs, Spot VMs and Sustained Use Discounts
Compute Engine bills per second after the first minute for vCPU, memory, GPUs and attached Hyperdisk or Persistent Disk, with separate charges for network egress and premium OS licences. Sustained use discounts of up to 30% apply automatically only to older series such as N1, N2, N2D and C2, so fleets on N4 or C4 need commitments or Spot to cut cost. Idle disks and stopped GPU reservations keep billing.
- Rightsize first using Recommender and Ops Agent memory data; typical estates drop 25-35% of vCPU before any discount
- Resource-based or spend-based CUDs on the remaining 24x7 baseline, sized to the P10 hourly floor
- Spot VMs through Batch for checkpointed jobs, priced at 60-91% below on-demand
- Autoscaler scale-in schedules and instance schedules that stop dev and test VMs outside 8 am to 9 pm IST
- Hyperdisk performance tuning so IOPS are bought explicitly instead of through oversized disk capacity
Compute Engine vs GKE vs Cloud Run: where should my workload run on Google Cloud?
Choose by operational unit and scaling pattern. Compute Engine wins when you need OS-level control, licensed software tied to vCPUs, GPUs for long training runs, or legacy applications that cannot be containerized within the project timeline; you manage images and patching. GKE wins when you already ship containers, run more than a handful of services and want bin-packing, rolling deployments and policy across a fleet; Autopilot removes node management while keeping Kubernetes APIs. Cloud Run wins for stateless HTTP or event-driven services with spiky or idle traffic, because it scales to zero and needs no cluster. Many NTSPL designs mix all three: Cloud Run for APIs, GKE for platform services, and Compute Engine with Batch on Spot VMs for heavy, finite compute.
Key facts about compute on Google Cloud
Compute on Google Cloud in Practice
3 compute scenarios from Genomics and bioinformatics, Animation and VFX, Agritech: the starting position, the services chosen, the architecture, the rollout and the measured outcome.
Genomics lab cuts whole-genome turnaround from 6 days to 30 hours with Batch on Spot
Animation studio bursts to 4,000 render cores for deadlines without buying hardware
Agritech firm delivers 6 a.m. crop advisories to 2.1 million farmers on time
Tyre manufacturer links four plants to cloud MES and IIoT analytics at 99.99%
Compute on Google Cloud: Frequently Asked Questions
How much can Spot VMs actually save compared with on-demand Compute Engine?
Spot VMs are typically priced 60-91% below on-demand, depending on machine type and region, and prices can change up to once a month. The real saving depends on preemption rate: a rendering or genomics job that checkpoints every 10 minutes usually realises 55-75% net savings after retries. We blend Spot task groups with a standard-VM fallback so deadlines are met even when spare capacity tightens.
Do sustained use discounts apply to C4 and N4 machines?
No. Sustained use discounts apply only to specific series including N1, N2, N2D, C2, M1 and M2, and they reach up to 30% for VMs running the whole month. Newer series such as N4 and C4 do not receive them, so the cost case for moving relies on better price-performance per core, rightsizing, committed use discounts and Spot capacity rather than automatic monthly discounts.
How do you migrate our VMware or Hyper-V VMs to Compute Engine?
We use Migrate to Virtual Machines to replicate disks continuously from vSphere or other sources, run test clones in an isolated VPC, then cut over in a short window with final sync. Before replication we rightsize each VM using its on-premises utilization so you do not carry 16-vCPU templates that use two cores. Stateless tiers are rebuilt as MIGs rather than migrated one-to-one.
Can we run Compute Engine workloads entirely within India for DPDP Act compliance?
Yes. Google Cloud operates the Mumbai (asia-south1) and Delhi (asia-south2) regions, each with three zones. We pin VMs, disks, snapshots and images to these regions using the resource locations organization policy, so personal data processed under the DPDP Act 2023 is not stored elsewhere by accident. Snapshot and image storage locations are set explicitly, because multi-regional defaults can otherwise place copies outside India.
When should we use Batch instead of our own Slurm cluster on Compute Engine?
Use Batch when your jobs are independent or embarrassingly parallel tasks and nobody wants to maintain a scheduler head node. Batch provisions VMs per job, supports Spot, GPUs and container or script runnables, and deletes capacity afterwards. Keep Slurm, deployed with Google's Cluster Toolkit, when researchers depend on Slurm semantics, MPI jobs needing tight placement, or existing submission scripts that are expensive to rewrite.
How do you prevent a misconfigured autoscaler from creating a surprise bill?
Every MIG gets an explicit maximum replica count tied to a budget, Cloud Billing budget alerts at 50%, 90% and 100% of forecast, and a Monitoring alert when the group sits at its maximum size for more than 15 minutes. Project-level CPU and GPU quotas act as a final ceiling. We also review autoscaler logs weekly during the first month to catch scaling oscillation.