I have used an old project, without updating for a long time so that the automated pipeline together with snyk will suggest over time at least updates of dependencies and containers, as well as at the code level.
The idea is that once the code arrives on github through a push, GH detects it, notifies Snyk of the changes, and both Github and Snyk start executing a series of actions that include the following tasks:
- Download dependencies, compile the project, create the package, run the tests, check for critical CVEs in the code, notify the Discord channel about the result.
- With the package created above, create a Docker container, upload it to a public or private hub, and notify the Discord channel.
- With the previously created container, allow Snyk or similar to scan the content of the container, that is, monitor it and notify the Discord channel.
- Create releases in GH and notify it on Discord.
- K8s grabs the Docker container release and creates the pods to put them into production. In this publication I will not show this, I will leave it for another publication.
Once you have a project with its tests and you want it to go through a CI/CD process with security in mind, the first thing you think of is that you have to upload it to github, save the code, for this we would use something like:
git add .
git commit -m "first commit."
git tag -a -m "first commit."
git push --follow-tags
Once the code is in GH, we would like GH to do builds and test runs of the project every time someone uploads something, run scans on the code and dependencies looking for known CVEs, major bugs in the code or just suggestions, all to try to keep a project as clean and safe as possible, right?
For this, GH provides us with actions, basically they are .yml files that are placed in a certain directory of your project so that once they are uploaded, they are detected and
start an asynchronous execution.
To create one, in the Actions tab of your project, you will see New Workflow, everything starts like this. When you click, you’ll see a bunch of actions that you’ll want to hit configure and very little else.
If you give Java-with-Maven, GH will create a file called maven.yml by default in a new directory that it will create in your repository called .github/workflows.
The content will look like this:
# This workflow will build a Java project with Maven, and cache/restore any dependencies to improve the workflow execution time
# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven
name: Java CI with Maven
on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up JDK 11
uses: actions/setup-java@v3
with:
java-version: '11'
distribution: 'temurin'
cache: maven
- name: Build with Maven
run: mvn -B package --file pom.xml
What does this file do? sends a checkout of the project, and executes the corresponding phases of package, namely:
1.validate.
2.compile.
- test.
- package.
If in that file, we add the following at the previous -name level:
- name: Discord notification
env:
DISCORD_WEBHOOK: ${{ secrets.DISCORD_WEBHOOK }}
uses: Ilshidur/action-discord@master
with:
args: 'The project {{ EVENT_PAYLOAD.repository.full_name }} has been build.'
will notify a Discord channel of the build result. To do this, we first have to create a channel in Discord and a webhook to communicate with Github and Discord, once we have that webhook, we will create a secret in our repository so that Github can manage them. Those webhooks can be revocable in time, so if you configure them like that, you will have to change them in GH.
Once you have created your discord channel and you are inside, you as an administrator can go to Settings/Integrations and you will see Webhooks, click on Create Webhook, copy the url somewhere temporary, and we go to Github.
In your repository, you go to Settings, Security, Secrets, Actions. You click on New repository secrets, in NAME you put DISCORD_WEBHOOK, in Secret, the url you copied before. click on Add Secret. From now on, all the secrets that I will mention in this writing will be the same.
Once your first action is ready, you commit and push it to the Github website. You do a git pull on your computer to bring you the changes. Almost immediately, GH will execute the action, you will be able to see the result in the actions tab. Obviously, the reverse way can also be done, you create the .github/workflows directories in the root of your project and put the .yml files there. I personally find it better to use the web configurator because of the tabulation errors you may introduce.
There is a way to check the validity of these yml files locally, using act. https://github.com/nektos/act
It’s what you would use if you have to debug these yml files, but in this post, I won’t mention act anymore, I’ll leave it for another post.
This is the url with the Discord information:
https://discord.com/developers/docs/resources/webhook
https://discord.com/developers/docs/getting-started
Then we would think about creating a Dockerfile to start having a continuous deployment. There are many ways to write a Dockerfile, so many that today it is especially recommended that the Dockerfile be built in multistage mode, that is, that the layers that are going to change the least are placed in the first lines and at the end of the Put on yourself the ones that can potentially change the most over time. The explanation is the caching time necessary to create the image, we want it to take as little time as possible.
We will also want the image to be as lightweight as possible and even when put into production, offer potential attackers the fewest points of attack and have a least privileged user running the packaged application.
An example, it is not the best possible but it serves to get an idea:
The official documentation is at: https://docs.docker.com/engine/security/
This project is hosted at:github.com/alonsoir/demo-jdbc
The files that we are going to analyze are:
1) Dockerfile
# build stage build the jar with all our resources
FROM openjdk:8-jdk as build
ARG PROFILE
VOLUME /tmp
WORKDIR /
ADD . .
RUN ./mvnw clean install
RUN mv /$JAR_PATH/target/demo-jdbc-0.0.1-SNAPSHOT.jar /app.jar
# package stage
FROM openjdk:8-jdk-alpine
WORKDIR /
# copy only the built jar and nothing else
COPY --from=build /app.jar /
COPY entrypoint.sh /entrypoint.sh
EXPOSE 8080
ENTRYPOINT ["/entrypoint.sh"]
Let’s analyze them a bit, the multistage Dockerfile file starts with FROM openjdk:8-jdk as build, which indicates that it is going to use a version (deprecated today) to compile the executable, with maven.
At this point we should use a newer version of the JDK, for example:FROM openjdk:11-jdk-slim-bullseye AS build-env
A good reason to choose a modern JDK is that they tend to have very narrow security issues and are tried to fix right away.
See what Snyk has to say about that jdk11 image:
https://snyk.io/test/docker/openjdk%3Ajdk-slim-bullseye
And now, see what Snyk has to say about that jdk8 image:
https://security.snyk.io/package/linux/debian:9/openjdk-8
Openjdk8 has critical, high, medium, and low vulnerabilities, while 11 has low vulnerabilities.
FIX, over this weekend, critical vulnerabilities have surfaced.
https://snyk.io/test/docker/openjdk%3A11-jdk-slim
There is no 100% secure software, it is normal for these vulnerabilities to appear, but we will agree that it is better to have low vulnerabilities than critical ones.
The rest of the operations that appear in the construction phase are the typical ones that appear in a Dockerfile, indicate in which volume of the host machine you want to work (/tmp), in which directory you want to store the result of the compilation operation (/ del container) and we command to execute mvnw that is saved in the same project repository.
This can be controversial because there will be people who say the following, why don’t you download maven through wget or through the OS tool of the build phase, Alonso? Offering wget in the production container is something we should never do, because if you allow that tool, an attacker could and will use it to download their own toolbox of malicious tools.
It’s about making it as difficult as possible for criminals.
Now, the phase of copying the jar so that a virtual machine runs it. Please, I am going to use the term copy the jar as a simplification of install the application in the container.
What is recommended here in my opinion is to use an operating system with a light, updated JDK, without open critical CVEs and most importantly, without bash or zsh. Distroless image. Following this policy, all we have to do next is copy the *-RELEASE.jar file into the jar.
I know a lot of people will use alpine or ubuntu, because it’s so lightweight, but in my opinion it’s a mistake to offer an attack vector like providing bash and the package manager so someone can interact with that container. They must look like safes, once they’re deployed to the k8s on duty or whatever.
If we do not want to use distroless images because we want to package everything using rpm, dpkg or apk, in a release creation phase, we will have these operations in which we will also take advantage of uploading them to private repositories and signing them.
It is true that professional criminals, I refuse to call them hackers, usually bring their own toolbox with them and even create their own bash in memory, do their job and leave.
You should try to mitigate these operations, for example, deny in the firewall of the security group that manages these pods any incoming connection from the internet to these containers, it can only be entered through the IPs managed by the web firewall and we also need a mechanism in the kernel that prevents some kind of shell from being built in memory.
It is about closing all possible attack vectors by adapting to what the criminals are doing.
Already put, ideally we would have the same version of the kernel, the same user space, the same glibc libraries necessary to create the encryption processes and the connection sockets, etc. for all the managed container infrastructure in the pods.
If the entropy of a container is given by the number of components it contains, that is, the number of permutations that exist when these components interact with each other, the kernel, the user space where the kernel system libraries and libraries live of the same application, all this composes an entropy that should tend to be the smallest possible, having the least possible number of permutations.
The whole distroless concept tries to minimize said number of permutations. It also tries to homogenize said number of components in an infrastructure.
Basically, what we want is that our containers offer zero entry points to exploit critical or high vulnerabilities, because even if we create these «safe» images, at some point these vulnerabilities appear.
In those moments in which we detect these vulnerabilities with our pipeline scanner, we must use a container that has already fixed said vulnerability as soon as possible or more carefully monitor the production systems with the SNORT type technology that we have.
https://github.com/GoogleContainerTools/distroless
After giving the tab on the execution container, we see that the Dockerfile indicates that the working directory is the / of the container, it copies only the jar to the root, the entrypoint.sh file and exposes port 8080.
While writing the text, snyk warned me that a solution for the container that hosts my application demo-jdbc was to move from openjdk:8-jdk-alpine with 87 known vulnerabilities, 3 criticisms of openjdk:19-jdk-alpine with zero known vulnerabilities, although you have to take into account that it can be unstable. The app must be thoroughly tested and for this it is always recommended to have our integration, unit and stress tests to see how it behaves in situations similar to reality. It is better to know than to assume.
https://packages.debian.org/search?keywords=openjdk-19-jre-headless
2) entrypoint.sh
#!/bin/sh -l
time=$(date)
echo "::set-output name=time::$time"
java -jar app.jar $1 $2
This file is simply going to use the jar’s java -jar command to run the application.
3) .github/workflows/docker-image.yml
name: Docker Image CI
on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKER_HUB_USER }}
password: ${{ secrets.DOCKER_HUB_PASS }}
- name: Build and push
uses: docker/build-push-action@v2
with:
context: .
file: ./Dockerfile
pull: true
push: true
cache-from: type=registry,ref=aironman/euromillions-test-java8:latest
cache-to: type=inline
tags: aironman/euromillions-test-java8:latest
build-args: PROFILE=nectar,ARG2=test
- name: Discord notification
env:
DISCORD_WEBHOOK: ${{ secrets.DISCORD_WEBHOOK }}
uses: Ilshidur/action-discord@master
with:
args: 'The project {{ EVENT_PAYLOAD.repository.full_name }} has been deployed in {{ EVENT_PAYLOAD.repository.html_url }}. '
This file is practically generated by GH, I have only added the part of the secrets and that of the Discord, to notify that the download, construction and upload to the Docker registry that you have has been executed correctly. Above I have described how to use that of the secrets in GH.
4) .github/workflows/maven.yml
# This workflow will build a Java project with Maven, and cache/restore any dependencies to improve the workflow execution time
# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven
name: Java CI with Maven
on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up JDK 11
uses: actions/setup-java@v3
with:
java-version: '11'
distribution: 'temurin'
cache: maven
- name: Build with Maven
run: mvn -B package --file pom.xml
- name: Discord notification
env:
DISCORD_WEBHOOK: ${{ secrets.DISCORD_WEBHOOK }}
uses: Ilshidur/action-discord@master
with:
args: 'The project {{ EVENT_PAYLOAD.repository.full_name }} has been build.'
Described above.
5) .github/workflows/release.yml
# This workflow will build a Java project with Maven, and cache/restore any dependencies to improve the workflow execution time
# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven
name: Java release with Maven
on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
jobs:
opensource-security:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@master
- name: Run Snyk to check for vulnerabilities
uses: snyk/actions/maven@master
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
command: monitor
args: --severity-threshold=critical
code-security:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@master
- name: Run Snyk to check for vulnerabilities
uses: snyk/actions/maven@master
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
command: code test
args: --severity-threshold=critical
release:
needs: [opensource-security, code-security]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up JDK 11
uses: actions/setup-java@v1
with:
java-version: 11
distribution: 'temurin'
cache: maven
- name: Set Git user
run: |
git config user.email ${{ secrets.EMAIL }}
git config user.name "GitHub Actions"
- name: Publish JAR
run: mvn -B release:prepare release:perform -DskipTests
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Discord notification
env:
DISCORD_WEBHOOK: ${{ secrets.DISCORD_WEBHOOK }}
uses: Ilshidur/action-discord@master
with:
args: 'The project {{ EVENT_PAYLOAD.repository.full_name }} has been released.'
This file tries to create a RELEASE in Github, for this it will first execute the scanning phases described here:
https://github.com/snyk/actions/tree/master/maven
Note that snyk provides more actions than snyk/actions/maven, check which one suits your needs best.
You may need to create an rpm, or a .deb
When these scan operations are fulfilled, the mvn release command is executed.
6) .github/workflows/snyk-container.yml
# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
# A sample workflow which checks out the code, builds a container
# image using Docker and scans that image for vulnerabilities using
# Snyk. The results are then uploaded to GitHub Security Code Scanning
#
# For more examples, including how to limit scans to only high-severity
# issues, monitor images for newly disclosed vulnerabilities in Snyk and
# fail PR checks for new vulnerabilities, see https://github.com/snyk/actions/
name: Snyk Container
on:
push:
branches: [ "master" ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ "master" ]
schedule:
- cron: '32 8 * * 1'
permissions:
contents: read
jobs:
snyk:
permissions:
contents: read # for actions/checkout to fetch code
security-events: write # for github/codeql-action/upload-sarif to upload SARIF results
actions: read # only required for a private repository by github/codeql-action/upload-sarif to get the Action run status
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Run Snyk to check for vulnerabilities
uses: snyk/actions/maven@master
env:
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
command: code monitor
- name: Build a Docker image
run: docker build -t aironman/euromillions-test-java8 .
- name: Run Snyk to check Docker image for vulnerabilities
# Snyk can be used to break the build when it detects vulnerabilities.
# In this case we want to upload the issues to GitHub Code Scanning
# Si fueses super celoso de la seguridad, dejarías ésto a false para que no construya el contenedor
# Éste es un proyecto para aprender, por lo que lo dejo a true
continue-on-error: true
uses: snyk/actions/docker@14818c4695ecc4045f33c9cee9e795a788711ca4
env:
# In order to use the Snyk Action you will need to have a Snyk API token.
# More details in https://github.com/snyk/actions#getting-your-snyk-token
# or you can signup for free at https://snyk.io/login
SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
with:
image: aironman/euromillions-test-java8
args: --file=Dockerfile
- name: Upload result to GitHub Code Scanning
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: snyk.sarif
- name: Discord notification
env:
DISCORD_WEBHOOK: ${{ secrets.DISCORD_WEBHOOK }}
uses: Ilshidur/action-discord@master
with:
args: 'The project {{ EVENT_PAYLOAD.repository.full_name }} has passed Snyk. Please go to Security link.'
This operation will be executed by cron at a certain time. If you don’t remember how to use cron, you can use this website to find out when it will run.
https://crontab.guru/#32_8___1
It is basically saying that at 08:32 every Monday, the task will be executed.
Thank you very much for read it.
Aironman techblog 