Korvus Analytics · 2024
A platform that stopped depending on one person
A hand-built environment replaced with infrastructure as code, taking new engineer onboarding from five weeks to three days and spreading deploy access from one person to five.
- Industry
- B2B analytics software
- Location
- Kraków, Poland
- Engagement
- 13 weeks
- Team
- 2 NordsCode engineers, 5 client engineers
Outcomes
- 3 daysNew engineer to first production deploy, down from 5 weeks
- Measured across the four engineers who joined in the year after handover, against an average of five weeks for the three who joined before it.
- 5 of 5Engineers able to deploy independently, up from 1
- Every engineer on the team has since shipped to production without assistance. Previously one person held the only working set of deployment credentials.
- 63%Less sprint time spent on infrastructure work
- From an average of 31 hours per two week sprint across the team to 11 hours, tracked in their own issue system over the six months following handover.
The challenge
What we found when we arrived.
Korvus Analytics builds reporting software for mid-market retailers, with roughly 90 customers on a multi-tenant platform. The infrastructure had been assembled over four years by their first engineering hire, entirely through cloud console clicks, and existed nowhere as code. He was the only person with working deployment credentials and the only person who understood the network layout.
The immediate trigger was that he was moving to a different role inside the company. Beyond the obvious risk, the practical cost was already high: new engineers took around five weeks to reach a first production deploy, staging had drifted far enough from production that passing there meant very little, and roughly 31 hours of every two week sprint went into infrastructure work spread across the team.
Our approach
What we did, in the order we did it.
Sequence matters more than tooling on engagements like this one. Each step below existed because the previous one produced something the next one needed.
- 01
Document reality before changing it
Two weeks capturing the environment exactly as it existed, including the parts that looked wrong. Three undocumented security group rules turned out to be load-bearing for a customer integration, and finding that during discovery rather than during a migration mattered.
- 02
Import rather than rebuild
Existing resources brought under Terraform through state import where possible, so the platform was described in code without a risky recreation of live infrastructure. Rebuilds were reserved for the components that genuinely needed to change.
- 03
Rebuild staging from the same modules
Staging recreated from the production modules with a sanitised data set, which restored the meaning of a passing staging deployment and immediately surfaced two configuration differences that had been causing intermittent production-only bugs.
- 04
Transfer ownership deliberately
Four pairing weeks in which Korvus engineers made the changes and we reviewed them, followed by written runbooks for the eleven operational procedures the team actually performs. The original engineer moved roles in week eleven, with two weeks of the engagement remaining as a safety margin.
The results
What changed, and how we know.
All five engineers now deploy to production independently and make infrastructure changes through reviewed pull requests. The original owner moved into his new role on schedule, and in the eighteen months since has not been called back for a platform question, which was the outcome the company cared about most.
Onboarding for the four engineers who joined afterwards averaged three days from first commit to first production deploy, against roughly five weeks previously. Infrastructure work fell from around 31 hours per two week sprint to 11, and the two configuration differences uncovered when staging was rebuilt resolved a class of intermittent bug the team had been chasing for over a year.
What it was built on
- Google Cloud
- Terraform
- Cloud Run
- Cloud SQL
- GitLab CI
- Cloud Monitoring
Engagement type
DevOps EngineeringRecognise any of this?
Most engagements start with a call describing a situation that sounds a lot like one of these. Tell us yours and we will say plainly whether we can help.