Building an Automated Shared Services Capability for a Central Government Department
A fledgling project within UK government was created to build a platform to support the transfer of data. The platform hosted multiple applications managed by six business units.
One of the core challenges on this project was to allow the platform team as well as business units to deploy their infrastructure and applications in a scalable and repeatable manner while using the best security practices. Version 1 tackled these challenges by delivering a robust self-hosted CI/CD solution.
Client Profile
CI/CD Pipeline
Typical components of a CI/CD pipeline include checking in code, testing it, and deploying it.
Version 1 automated the installation of all components using Packer images that would then get provisioned onto EC2 instances using Terraform. These instances would be part of an autoscaling group, placed behind an Application Load Balancer to minimise downtime during deployments and to take advantage of self-healing.
Jenkins was then utilised to build, test, and deploy commits. Scripts were written to fully automate the installation and configuration of Jenkins as well as all of its plugins. All deployment jobs used Pipeline as Code (PaC) written in Groovy. To mitigate long deployment queues, Jenkins was built using distributed build architecture which consisted of one Jenkins controller spinning up agents on Amazon EKS when required.
The project adopted a trunk-based development workflow (i.e. regularly merging short-lived feature branches to the main branch). To achieve this, Gitlab was deployed as the continuous integration server. It was configured to prevent direct commits to the main branch and to only allow feature branches to be merged on the approval of a minimum of two reviewers and after the pipeline for that job had succeeded.
These feature branches would provision real infrastructure to simulate the live environment as closely as possible without interfering with it. Once merge requirements were met, a merge could be triggered which would output the intended changes to the Jenkins console, which displayed one final prompt to officially abort or apply the changes.
To further lessen the risk of deploying bad code, code inspection and vulnerability tools were also arranged and integrated into the Jenkins pipelines. Pipeline stages were dedicated to scanning infrastructure code with tfsec, application code with SonarQube, and vulnerability scanning with Nexus IQ. Any issues discovered by these tools would automatically stop the pipeline from deploying code, thereby mitigating the risk of insecure, poor-quality code.
Other useful stages of the pipeline were dedicated to snapshotting the current instances in case they needed to be recovered later and also sending notifications of the build status to Slack.
Technology Used
-
Jenkins
-
GitLab
-
Nexus
-
Packer
-
Terraform
-
Vault
-
SonarQube
-
EKS
-
Vault
-
Nessus
Security
Security was treated as paramount and ingrained in all aspects of the platform. Bastions were deployed in the production environment which was only accessible via government-issued laptops. OpenVPN was set up for users to access the non-production environment. AWS VPC endpoints and proxies were deployed to control traffic in and out of the environment, with Nexus Repository deployed to hold officially approved artifacts.
Certbot was set up to provide certificates for applications. Packer was used to build hardened golden images, with ClamAV and Nessus deployed for virus and vulnerability scanning. Vault was deployed to securely manage secrets. Access to modify resources via the AWS console was restricted to prevent unauthorised modification and configuration drift.
Benefits
Implementation of CI/CD
The implementation of a robust CI/CD process allowed teams to rapidly deploy applications and code in an automated, repeatable manner. In addition, the integrated testing and review processes decreased the risk of deploying improper code.
About Version 1
Version 1 have a strong history of DevOps advisory, project delivery and support. With a range of specialists, from hands-on engineers, architects, and organisation and enterprise change advisors, our experienced team is well equipped to support your organisation’s DevOps transformation.
With customers across both the public and private sector, we bring our customers along the journey from legacy to modern development and operational practices and culture. Our team of DevOps specialists can deliver a bespoke implementation suited for your organisational needs.
