Following a git commit to runtime environments #
In this 5 minute tutorial you'll learn how Kosli tracks "life after git" and shows you events from:
- CI-pipelines (eg, building the docker image, running the unit tests, deploying, etc)
- runtime environments (eg, the blue-green rollover, instance scaling, etc)
You'll follow an actual git commit to an open-source project called cyber-dojo.
In our example cyber-dojo’s
runner service should run with three replicas. However, due to an oversight while switching
from Google Kubernetes Engine (GKE) to AWS Elastic Container Service (ECS), it was running with just one replica.
You will follow the commit that fixed this.
Getting ready #
You need to:
- Install the
- Get your Kosli API token.
- Set the KOSLI_API_TOKEN environment variable.
- Set the KOSLI_ORG environment variable to
cyber-dojoorganization is public so any authenticated user can read its data):
Pipeline events #
Listing pipelines #
Find out which
cyber-dojo repositories have a CI pipeline reporting to Kosli:
kosli ls flows
You will see:
NAME DESCRIPTION VISIBILITY creator UX for Group/Kata creation public custom-start-points Custom exercises choices public dashboard UX for a group practice dashboard public differ Diff files from two traffic-lights public exercises-start-points Exercises choices public languages-start-points Language+TestFramework choices public nginx Reverse proxy public repler REPL for Python images public runner Test runner public saver Group/Kata model+persistence public shas UX for git+image shas public web UX for practicing TDD public
cyber-dojo overview #
- cyber-dojo is a web platform where teams practice TDD without any installation.
- cyber-dojo has a microservice architecture with a dozen git repositories.
- Each git repository has its own Github Actions CI pipeline producing a docker image.
- These docker images run in two AWS environments named aws-beta and aws-prod.
Following the artifact #
The runner service had one instance running instead of three.
The commit which fixed the problem was
runner repository. Follow this commit using the
kosli get artifact runner:16d9990
You will see:
Name: cyberdojo/runner:16d9990 Flow: runner Fingerprint: 9af401c4350b21e3f1df17d6ad808da43d9646e75b6da902cc7c492bcfb9c625 Created on: Mon, 22 Aug 2022 11:35:00 CEST • 15 days ago Git commit: 16d9990ad23a40eecaf087abac2a58a2d2a4b3f4 Commit URL: https://github.com/cyber-dojo/runner/commit/16d9990ad23a40eecaf087abac2a58a2d2a4b3f4 Build URL: https://github.com/cyber-dojo/runner/actions/runs/2902808452 State: COMPLIANT History: Artifact created Mon, 22 Aug 2022 11:35:00 CEST branch-coverage evidence received Mon, 22 Aug 2022 11:36:02 CEST Deployment #18 to aws-beta environment Mon, 22 Aug 2022 11:37:17 CEST Deployment #19 to aws-prod environment Mon, 22 Aug 2022 11:38:21 CEST Started running in aws-beta#84 environment Mon, 22 Aug 2022 11:38:28 CEST Started running in aws-prod#65 environment Mon, 22 Aug 2022 11:39:22 CEST Scaled down from 3 to 2 in aws-beta#117 environment Wed, 24 Aug 2022 18:03:42 CEST No longer running in aws-beta#119 environment Wed, 24 Aug 2022 18:05:42 CEST Scaled down from 3 to 1 in aws-prod#94 environment Wed, 24 Aug 2022 18:10:28 CEST No longer running in aws-prod#96 environment Wed, 24 Aug 2022 18:12:28 CEST
Let's look at this output in detail:
- Name: The name of the docker image is
cyberdojo/runner:16d9990. Its image registry is defaulted to
dockerhub. Its :tag is the short-sha of the git commit.
- Flow: The name of the Kosli flow.
- Fingerprint: The unique immutable SHA256 fingerprint of the artifact.
- Created on: The artifact was created on 22nd August 2022, at 11:35 CEST.
- Commit URL: You can follow the commit URL to the actual commit on Github since cyber-dojo's git repositories are public.
- Build URL: You can follow the build URL to the actual Github Action for this commit.
- State: COMPLIANT means that all the promised evidence for the artifact (in this case
branch-coverage) was provided before deployment.
- CI pipeline events
- Runtime environment events
- The artifact was reported running in both environments.
- The artifact's number of running instances scaled down.
- The artifact was reported exited.
The information about this artifact is also available through the web interface.
runnerservice uses Continuous Deployment; if the tests pass the artifact is blue-green deployed to both its runtime environments without any manual approval steps. Some cyber-dojo services (eg web) have a manual approval step, and Kosli supports this.
Environment Snapshots #
Kosli environments store information about what is running in your actual runtime environments (eg server, Kubernetes cluster, AWS, ...). We use one Kosli environment per runtime environment.
The Kosli CLI periodically fingerprints all the running artifacts in a runtime environment and reports them to Kosli. Whenever a change is detected, a snapshot of the environment is saved.
Cyber-dojo runs the
kosliCLI from inside its AWS runtime environments using a lambda function to report the running services to Kosli.
The History of the artifact tells you your artifact started running in snapshot #65 of
You query Kosli to see what was running in
aws-prod snapshot #65:
kosli get snapshot aws-prod#65
The output will be:
COMMIT ARTIFACT PIPELINE RUNNING_SINCE REPLICAS 16d9990 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/runner:16d9990 runner 11 days ago 3 Fingerprint: 9af401c4350b21e3f1df17d6ad808da43d9646e75b6da902cc7c492bcfb9c625 7c45272 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/shas:7c45272 shas 11 days ago 1 Fingerprint: 76c442c04283c4ca1af22d882750eb960cf53c0aa041bbdb2db9df2f2c1282be ...some output elided... 85d83c6 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/runner:85d83c6 runner 13 days ago 1 Fingerprint: eeb0cfc9ee7f69fbd9531d5b8c1e8d22a8de119e2a422344a714a868e9a8bfec 1a2b170 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/differ:1a2b170 differ 13 days ago 1 Fingerprint: d8440b94f7f9174c180324ceafd4148360d9d7c916be2b910f132c58b8a943ae
You see in this snapshot, the
runner:16d9990 artifact is indeed running with 3 replicas.
You have proof the git commit has worked.
Blue-green deployment #
There were two versions of
runnerat this point in time! The first had three replicas (to fix the problem), but there was also a second (from commit
85d83c6) with only one replica.
You are seeing a blue-green deployment happening;
runner:85d83c6is about to be stopped and will not be reported in snapshot
Diffing snapshots #
env diff command allows you to see differences between two versions of your
Let's find out what's different between the
kosli diff snapshots aws-prod#64 aws-prod#65
The response will be:
Only present in aws-prod#65 Name: 274425519734.dkr.ecr.eu-central-1.amazonaws.com/runner:16d9990 Fingerprint: 9af401c4350b21e3f1df17d6ad808da43d9646e75b6da902cc7c492bcfb9c625 Pipeline: runner Commit URL: https://github.com/cyber-dojo/runner/commit/16d9990ad23a40eecaf087abac2a58a2d2a4b3f4 Started: Mon, 22 Aug 2022 11:39:17 CEST • 15 days ago
The output above shows that
runner:16d9990 started running in snapshot 65 of
You have seen how Kosli can follow a git commit on its way into production,
and provide information about the artifacts history, without any access to cyber-dojo's