Add custom Root CA into KRE docker container

We are running KRE offline and have our own PKI infrastructure. If I run a test on a internal website Chrome says “NET::ERR_CERT_AUTHORRITY_INVALID” and that makes sense because it doesn’t know our root ca’s.

How can I slipstream our ca’s into a test when I start it with “docker run” without modifying the image?
Because the container is not running constantly and is only started when you execute a test I don’t know how to get this working.

Hi @Arjan.Malestein,
Do you use Docker image built by Katalon?

Hello @huynguyen,

Yes we are using the Docker image made by Katalon.

@anon46315158 : That wil not work because the docker container is started to run one test. After that the container will be removed again. As far as my knowlegde goes should everything be in one command line.

I tryed to map the certificates into the container via “-v /opt/katalon/ca-certificates:/usr/local/share/ca-certificates:Z -v /opt/katalon/ca-certificates/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:Z” but still chrome is saying the the certificate is not trusted.

/opt/katalon/ca-certificates contains al the necessary ca’s for the website and ca-certificates.crt is from a different server that we made a while ago and is successful in accessing the website.

yeah right.
you can simply do it in few steps:

  • start the container in background (daemon) mode (use -d option). you can use also --name to easily identify it.

  • do the docker cp

  • execute the tests with docker exec -it katalonc blah_blah

  • stop and remove the container docker stop {container_name} && docker rm {container_name}

Thank You @anon46315158. Sounds like this may work. I will consider this as a last resort. My first choose would be via a one liner as described in the katalon manuals. So if that is possible I would be very happy.

@Arjan.Malestein i don’t think you can solve it via ‘one liner’ … if by one liner you mean just a ‘docker run’ line.

But i can make a bash one liner for you if desired, sort of docker run blah && docker cp && docker exec blah … and so on. :smiley:

I tried that, but that didn’t went well for me. But feel free to put anything together. My Linux/Bash/Docker knowledge is very low and it’s all new to me.

But why is the mapping “ -v /opt/katalon/ca-certificates:/usr/local/share/ca-certificates:Z -v /opt/katalon/ca-certificates/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:Z ” not working. Are that the wrong locations I’m using?

@Arjan.Malestein could be. hidden may be the path of volumes.

Kindly post your one liner command in use for actually running the tests and i will split it into the right script steps.
And please, kindly use code formatting when you post code snipets.
e.g.

-v /opt/katalon/ca-certificates:/usr/local/share/ca-certificates:Z -v /opt/katalon/ca-certificates/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:Z ” not working. Are that the wrong locations I’m using?

^^^ the above is not readable for me.

but if you surround top/bottom the code with triple single quotes became:

-v /opt/katalon/ca-certificates:/usr/local/share/ca-certificates:Z -v /opt/katalon/ca-certificates/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:Z

I know young people are all the time in rush and have good eyes, but I am not that young and i need glasses for reading.

As i managed to find time to learn about programming by myself, keeping up with new technologies and other voodoo, I think you as an AQA wannabe can spend some time to read a bit how to properly format your requests when asking for help. Is not that hard!

thank you!

Hi @Arjan.Malestein,
If you have read the Katalon manuals, you might have noticed the file katalonc.sh available in the docker run command. This file contains the script that will be executed inside the container, and it can be modified so that before running katalonc, we will add the custom root CA for Chrome.
Given the container environment is Ubuntu and your preferred browser is Chrome, you can refer to https://chromium.googlesource.com/chromium/src/+/refs/heads/lkgr/docs/linux/cert_management.md for how to add a root CA for Chrome to recognize.
Just a few thoughts on this, and I know you still have a little bit more to elaborate, but this doesn’t require much effort and I think you can do this.
Please consider this as another option for you. As I know, @anon46315158 is always the guy with excellent solutions. Maybe you should follow his advice before continuing with anything else.

Helllo @anon46315158 and @huynguyen,
Thank you for your time and patience with me. It is appreciated.

It looks like the certificate works on OS level. When I run docker run -it --rm ${MAPPINGS} ${CERTS} ${CONTAINER} bash -c "curl -iiV https://runa.something.domain" it says the certificate and encryption is correct. When I run the command without the variable ${CERTS} I reveive the message that the connection is insecure. It looks like Chrome is having the problem with reading the correct certificate and needs to be updated.

When I run

docker run -t --rm ${MAPPINGS} ${CONTAINER} bash -c "certutil -d sql:$HOME/.pki/nssdb -L"

to list all certificates, as mentioned in the artricle @huynguyen gave, it gives the message bash: certutil: command not found. How do we update the root certificates in chrome?

This is my code I’m using to run the test. I have split it up in sections with variables to keep it readable.

PROJECTNAME=Runa
TESTSUITE=SimpleRunaLogin
MAPPINGS='-v /opt/katalon:/katalon/katalon/project:Z -v /opt/katalon/license:/root/.katalon/license:Z'
CERTS='-v /opt/katalon/ca-certificates:/usr/local/share/ca-certificates:Z -v /opt/katalon/ca-certificates/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:Z'
CONTAINER='docker.io/katalonstudio/katalon:latest'
PROJECT="-projectPath=/katalon/katalon/project/${TESTSUITE}/${PROJECTNAME}.prj -testSuitePath=TestSuites/${TESTSUITE}"
RUNSPECS='-browserType=Chrome -retry=0 -statusDelay=15'
docker run -t --rm ${MAPPINGS} ${CERTS} ${CONTAINER} katalonc.sh ${PROJECT} ${RUNSPECS}

Hi @Arjan.Malestein,
In the article, there is a section to get the necessary tools. Please invoke those commands before using certutil.

Not sure if this is a god idea, you are maping a certain folder from host inside the container, potentialy overiding the content inside the container.
I will try to reproduce/debug your approach, but still you have to answer why you don’t simply copy the needed certs inside the container instead of bindmounting?

However, thank you for following the advice for code formatting, my eyes are happy :slight_smile:

I’m openening the Katalon Docker Image via :

docker run -it --rm docker.io/katalonstudio/katalon:latest bash

and when I try to install the necessary tools via :

apt-get install libnss3-tools

I get the error " Unable to locate package libnss3-tools". If it tries to download it from internet, this is not possible because we are air-gapped.
Isn’t it possible to make this a standard installed package in the Katalon Images?

What is the next option to get the certificates updated?

The docker image stays read-only and wil not be modified in this way. The only thing you do with the -v option is creating a link to a folder outside your container. That folder is writeable because it is on your local/host system.

I’m convinced to move to your solution and let go of my own idea. But as you see above, that doesn’t work yet.

you are mounting a certain folder from outside the container into /usr/local/share/ca-certificates folder inside the container (and same for few others). So all existing certificates in /usr/local/share/ca-certificates are ‘bye-bye’ because the content of the folder inside the image is overrided by the content of the folder outside.

and again, you are confusing the image with the container. the image is read-only but you runt the test in the container which is an instance of the image

If it tries to download it from internet, this is not possible because we are air-gapped.

so, how do you retrieve the docker image?
you can prepare a custom image on a separate machine to fit your needs. there are few options to do this: with a dockerfile, from a running container …

I know we override all other public certificates, but we don’t need them because we only use our private pki certificates. But if copying in works I will use that.
The container is not running, it is started every time you start a test. The container is started based on the image. So if you don’t change the image the container is also not changed. But I’m no docker expert so I could be wrong there.

I download the Katalon Image on a separate machine from the internet and export/import it than to the test environment. I know you can modify images but my preferred choose is not to make to much customizations to standard components. My experience is that this can give you a hard time in the feature and you need to do the customization every time there is an update.
But if there is no other way to get it working, than that is the way to go.

Can you at least think about adding “certutil” to the standard Katalon docker image

i will post later the needed steps to prepare a custom image based on a running container, a bit busy now

@Arjan.Malestein sorry for late.
Here the needed steps to manually create a custom katalon image.
On the machine you use for grabbing the image, do as bellow:

  1. Pull the latest katalon image (or whatever version you desire to use as a base) and check for it’s presence:
docker pull katalonstudio/katalon:latest
docker images

REPOSITORY                       TAG     IMAGE ID      CREATED     SIZE
docker.io/katalonstudio/katalon  latest  dfa806514683  2 days ago  1.69 G
  1. Fire up a container in background (detached mode).
    You don’t need to do any bind mount (-v) at this point since however will be ignored during the commit stage.
    Use whatever you like for the container name:
docker run -itd --name my_container katalonstudio/katalon:latest bash
docker ps
CONTAINER ID  IMAGE                                   COMMAND  CREATED        STATUS            PORTS   NAMES
330d4b0a8d86  docker.io/katalonstudio/katalon:latest  bash     5 seconds ago  Up 5 seconds ago          my_container

you should see your container with status up.

  1. Copy the certs inside the container using docker cp command
    (i will just create a test file and copy into the / folder)
touch test.file
docker cp ./test.file my_container:/
  1. Jump inside the container (note the prompt has changed):
docker exec -it my_container bash
root@330d4b0a8d86:/#

check if the files you need are in the right place:

root@330d4b0a8d86:/# ls -al
total 72
dr-xr-xr-x.  22 root   root    4096 Dec 12 09:20 .
-rw-rw-r--.   1 root   root       0 Dec 12 09:19 test.file
-----
  1. Run the commands to install the tools needed and update the certs.
    (i will just install htop)
root@330d4b0a8d86:/# htop
bash: htop: command not found
root@330d4b0a8d86:/# apt update && apt install htop

Once done, log-out from the container (just type exit). Your container should still run at this moment.

  1. Commit your changes using docker commit to a new image (use whatever you like for repo_name, image_name and version)
docker commit my_container my_repo/my_image:some_version
Getting image source signatures 
Copying blob bba47f1b6b65 skipped: already exists  
Copying blob c98acc4f5f2e done  
Copying config c9720cde91 done  
Writing manifest to image destination
Storing signatures
---

check the images, note the new one:

docker images
REPOSITORY                       TAG           IMAGE ID      CREATED         SIZE
localhost/my_repo/my_image       some_version  c9720cde919f  23 seconds ago  1.72 GB
docker.io/katalonstudio/katalon  latest        dfa806514683  2 days ago      1.69 GB

at this point you can stop and remove the container:

docker stop my_container && docker rm my_container
  1. Now copy your image on the air-gapped machine and run it as you usually use the katalon one.
    (i will just create a test container and check if the files and tools are in place)
docker run -itd --name test_container my_repo/my_image:some_version bash
3b6f27c336ea20c52a14d4dd9adbd095e62b09d85c438ec02c2f11dab5318d14

docker ps
CONTAINER ID  IMAGE                                    COMMAND  CREATED        STATUS            PORTS   NAMES
3b6f27c336ea  localhost/my_repo/my_image:some_version  bash     4 seconds ago  Up 3 seconds ago          test_container

docker exec -it test_container bash
root@3b6f27c336ea:/# ls -al test.file
-rw-rw-r--. 1 root root 0 Dec 12 09:19 test.file

Of-course, all from above can be done also by using a Dockerfile and docker build command, see for reference:

1 Like

Hello @anon46315158,

Thanks for the clear explanation. I hoped that it would be possible without a custom image, but I see now that it will not work.
I don’t know if i have time to do this before the Christmas holidays, but I’ll let you know if it worked.

Thank you very much.

well … with your air-gaped setup i think is the only solution at this moment.
if the executor will be connected to the internet the above can be done straight in the running container, but since it is needed to grab some additional packages … this is it.

on the other side, provided the above workaround will work well, you can open a feature request to have libnss3-tools included in the official image. some other users may face the need to add custom certs for testing.
cc @huynguyen