Profiles and Console Mode Bug

[Kelly_Ferrone1]

OK I have definitely found a bug. Basically when I run my project from the studio everything works and then from the console it only sometimes works. I at least figured out how to reliably reproduce the bug. Here goes . . .

---

Steps to reproduce console bug:

Step 1: Get local docker test set up

Add this file to top level dir for easy local testing:

docker-compose.yml

version: "3.7"
services:
  katalon:
    image: katalonstudio/katalon:latest
    container_name: katalon
    hostname: katalon
    volumes: 
      - .:/katalon/katalon/source
    entrypoint: katalon-execute.sh
    command: -propertiesFile=/katalon/katalon/source/Properties/console.properties

Step 2: Here is what causes the bug

Ignore the Profiles/ directory in the project, we have secrets in ours so it does not make sense to have in git. However deleting this directory will most definitely cause issues when you want to run in console mode.

Here is our .gitignore for following along:

.gitiginore

bin/
.settings/
/.classpath
.svn/
Libs/
Reports/
report/
Drivers/
Properties/
Profiles/
.DS_Store
~$*.xlsx
.idea
console.properties
.project

Step 3: Add the Profile back

In this step, I actually commited and pushed the repo and cloned into another folder so I could best simulate what our CI server will do. Then from our Git repo I snagged the snippet for running the tests onto my clipboard. Then went and created the new default.glbl file in the new Profiles/ dir I just created and pasted the snippet.

Step 4: Console Mode Properties

Now I create a dir called Properties/ and paste our properties snippet into the file below.

console.properties

browserType=Chrome (headless)
retry=0
testSuitePath=Test Suites/Tester
executionProfile=default

Step 5:

To truly follow along, I have a Test Listener with a Before Test Case hook. In this function I implement/call a CustomKeyword function I made. I don’t think it matters what the CustomKeyword actually does, as long as it calls in the before test case listener. So maybe just print, “Hello World!”

Step 6: See the bug

Now we have the project set up and all we need to do is run:

docker-compose up

And you will (hopefully) get a the same or similar problem. For us, it is this error on the first time a Keyword is used in our Test Listeners.

katalon    | 2019-02-26 13:58:22.994 ERROR k.k.c.m.CustomKeywordDelegatingMetaClass - ? java.lang.ExceptionInInitializerError
katalon    | 2019-02-26 13:58:22.996 ERROR c.k.k.core.context.internal.TestHooker   - ? com.kms.katalon.core.exception.StepErrorException: org.codehaus.groovy.runtime.InvokerInvocationException: java.lang.ExceptionInInitializerError

---

A Possible Solution

Cool, now we can see the bug. Here is what I found to get the bug to go away.

1. Basically I went to my original project where it was created by Katalon studio and copied the Profiles/ dir.
2. In the tester project I cloned on the side, I deleted the Profiles/ dir I manually created.
3. Next I pasted the original directory with the original *.glbl files which were created by Katalon into the cloned project.
4. Run docker-compose up and voila! Everything works!!!
5. From original project run docker-compose up and voila! Everything works!!!

So it seems like only the Katalon studio GUI can properly create a Profile and it is expected to be committed into your repo.

I looked all over the project and scanned for any and every change after any action in an attempt to discover what the difference is between the Profiles created in Katalon versus the same files created through the terminal. I came across nothing, the files were exactly the same, permissions and everything???

---

Possibly Related Issue

Originally I thought my error had something to do with the unresolvable global variables error like so:

groovy: 21 Groovy:unable to resolve class internal.GlobalVariable

Here is a whole discussion about it.

I thought this because I was indeed having issues with global variables and my error was happening right after the unable to resolve class internal.GlobalVariable error, which only seems to be a warning.

---

Final Thoughts

I only have one idea to get around this for now. Guess the *.glbl files should not have any secrets in them and the secrets should just come from environment variables. Then I can commit the Profiles/ dir with all the *.glbl files.

By the way, for us, this issue is not stopping any of the actual Test Cases from running. Our Gherkin is werkin and our test cases run like a charm. It’s just all the code in the Test Listeners fail because of this issue and none of our before and after code will run. This means our tests preconditions are not set up and therefore the test will fail because of this. Strangely enough the listener code only silently fails and then continues on.

---

Wow, what a battle . . .

Anyone else come across this issue?

Anyone have any solutions?

[jay3]

Nice questions