Modernizing 60+ GCP Cloud Functions: Gen1 to Gen2, builds halved
Every Cloud Functions pipeline was a little different and most were slow. I standardized all of them, cut checkout time by about 80% and build time in half, and laid down a safe path from Gen1 to Gen2.
A fleet of Cloud Functions tends to grow one function at a time, and the pipelines grow the same way: copied, tweaked, and never unified. The result was dozens of functions across several repositories, each deploying through a slightly different Jenkins pipeline, most of them slow, and none of them ready for Gen2. I standardized the whole set and modernized the runtimes, sanitized here to the technique.
Where the time was going
Two things dominated every build. First, the checkout: each build did a full clone of the deploy repository, 30 to 60 seconds of pure waiting before any real work. Second, dependencies: nothing was cached, so every build reinstalled everything from scratch. Multiply that across dozens of functions deploying many times a day and it adds up to real engineer-hours and real cloud build minutes.
What I changed
- Clone-or-update checkout. Instead of a full clone every build, the pipeline fetches and resets when the repository is already present. Checkout dropped from 30 to 60 seconds down to 5 to 10.
- Dependency caching. Builds now reuse cached dependencies instead of reinstalling, taking cache reuse from zero to 80 to 90%.
- Smaller deploy packages. Trimmed what gets shipped, roughly halving the deployment package size.
- Runtime updates. Brought Go, Node.js and Python to current, supported runtimes on Gen1, and added the Gen2 equivalents.
- Gen2 pipelines and examples. New Gen2 deploy pipelines plus working examples in all three languages, so adding a Gen2 function is copy-and-go, not research.
Along the way I fixed a batch of long-standing papercuts: an incorrect workspace path, a vendoring inconsistency from blank imports, an entry-point mismatch between Gen1 and Gen2, and oversized deployment packages.
Before and after
| Metric | Before | After | Change |
|---|---|---|---|
| Checkout time | 30 to 60 s | 5 to 10 s | ~80% faster |
| Total build time | 3 to 5 min | 1.5 to 2.5 min | ~50% faster |
| Deployment size | 100% | 40 to 50% | ~50% smaller |
| Cache reuse | 0% | 80 to 90% | new |
Why Gen2, and how to get there safely
Gen2 is not just a version bump. It runs on Cloud Run under the hood, which brings concurrency (up to 1000 simultaneous requests per instance instead of one), native Eventarc integration, faster cold starts, and more granular pay-per-use. The migration is where people get nervous, so I kept it boring:
- Start in dev with HTTP functions on the updated runtimes, and validate each one on its own.
- Move to Pub/Sub-triggered functions only after the HTTP ones are proven.
- Use blue-green cutovers so traffic shifts only once the new version is healthy.
- Replicate through staging, then production, starting with non-critical functions before the critical ones.
What I would tell someone starting this
- Standardize the pipeline before you optimize it. You cannot speed up twenty different pipelines; you can speed up one and apply it twenty times.
- The cheapest build win is usually the checkout. Stop full-cloning on every run.
- Cache dependencies deliberately. Zero to 80% cache reuse is often the single biggest drop in build time.
- Migrate runtimes in a fixed order (dev, staging, production; HTTP before Pub/Sub; non-critical before critical) and the scary migration becomes routine.