Kubernetes Cluster & CI/CD in 5 minutes
ESM creates and automates all Infrastructure and DevOps requirements of a modern IT department.
Carrying Evalest ‘s extensive experience in IT companies, ESM creates server clusters with best practices and cutting-edge tools that otherwise would take many years to even grasp, in a matter of minutes.
More specifically it:
- Creates disctributed web server systems
- Streamlines automatic deployments. From simple git commits to your projects all the way to their server deployment and scaling, using CI/CD best practises
- Tracks all server changes in Git utilizing the GitOps methodology. That way you can re-create your whole cluster again at any point in time.
- Uses cutting-edge open source tools, so you are left with a sophisticated system that every developer would love to work with.
- Works with any kind of machines ( Cloud, VPS, even plain laptops!)
The single dependency of this project is Docker. Cloning and running it, creates a VM which acts as the portal to your new server.
This VM allows you to automatically:
1 Create multiple Kubernetes master & worker nodes on bare metal machines by providing their IPs and some default settings.
2 Configures the clusters with:
- Flux for Continuous Deployement (CD) and GitOps
- MetalLB load balancer
- Nginx Ingress to expose services publicly
- Cert Manager to automate TLS issuance
- Sealed Secrets for secret declaration in GitOps
- Private Gitlab Runner for CI / CD jobs
- Prometheus monitoring with Grafana
- KubeApps for one click deploy any popular app (WordPress, Databases etc)
How it works
To keep your computer clean and dependency free, the first step ESM takes is to create and configure a Docker container (VM) which can be reused multiple times.
We have injected the container with Ansible playbooks to perform its tasks. To enable ESM to perform all configuration and deployment tasks, you’ll need to:
- Populate the “hosts” file with the desired IP addresses you want to configure
- Edit the default variables, such as Gitlab registry credentials, GitOps repository URL etc with your own.
The ansible playbooks perform tasks in bundles that make more sense. From simple ones like granting you kubectl access to your cluster, to complex ones like creating all Kubernetes node machines and configuring them with all the features.
In addition to these, one can run ad-hoc tasks like creating a GitLab runner or adding monitoring to the cluster by running individual tasks using our Ansible.
CI / CD
Continuous Integration and Continuous Deployment in our system is achieved with 2 key elements. GitLab CI and Flux.
To do that, Flux is paired with a git repository (aka GitOps) as well as with your docker registries too. When kubernetes deployments are being added or changed in the GitOps repo, Flux reflects the change in the cluster. Flux is also configured to update your cluster with any image updates pushed to your docker registries and update GitOps in the process.
More details about this CI /CD process can be found in our article:
ESM uses Nginx Ingress to expose projects and services to the internet under specific domain names. To do that efficiently it installs and configures a bare matal load balancer MetalLb.
For in-cluster communication whilst not exposing services to the public, ‘ClusterIp’ is our preferred configuration.
To secure our projects with TLS certificates and expose them via https, we use Cert Manager. ESM installs and configures it automatically from the Ansible playbooks. In Cert Manager, a certificate controller is deployed to the cluster that automatically issues TLS certificates for any project in a matter of seconds. Cert Manager will go through the issuing process with Let’s Encrypt through ACME. All one needs to do is add a few lines to an Ingress.
Code language: YAML (yaml)
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: example-com-ingress annotations: kubernetes.io/ingress.class: nginx cert-manager.io/cluster-issuer: "letsencrypt-prod" spec: tls: - hosts: -example.com secretName: example-com-tls rules: - host: example.com http: paths: - path: / backend: serviceName: example-com-service servicePort: 80
A dedicated sealed secrets controller is deployed in the cluster to carry out encrypting and decrypting kubernetes secrets. In order to safely keep secrets alongside the GitOps deployments and services they have to be encrypted. The Sealed secrets controller identifies such secrets and automatically decrypts them to be ready for use.
Since the ESM project is capable of providing a server management working environment out of the box, complete cluster disaster recovery and cluster upgrades are additional significant benefits.
The overall process can be summarized in the following illustration: