KRE 9.3.1 fails to load Keywords when executing the first TCs, but eventually gets them later in the run

[michel.g]

Hello,

Weird surprise after upgrading to KRE 9.3.1.

Most of our test cases start by calling a custom keyword:
CustomKeywords.'testdata.someModule.Function'('someParam')

When running test suites using such test cases, the first test cases fail, with KRE logging:

Reason:
java.lang.ClassNotFoundException: testdata.someModule
(...) at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194) 
etc etc

(see stack trace below)

Eventually, after n TCs left KO, at some point, it goes back to normal, it stops complaining about “ClassNotFoundException”, and it works. In other words, KRE starts working correctly only at the (n+1)-th TC.

With KS 9.3.1, running any test suite with such test cases just works. KS starts working correctly at the first TC.

Any idea what’s wrong? Is this a bug of the upgrade? Is there some part of the Java environment that is not loader early enough? Do we need to change some Java setup?

Thanks in advance for any insight.
–Michel

Test Cases/someFolder/someTC FAILED.
Reason:
java.lang.ClassNotFoundException: testdata.someModule
	at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.getCustomKeywordClassAndSetMetaClass(CustomKeywordDelegatingMetaClass.java:105)
	at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.invokeStaticMethod(CustomKeywordDelegatingMetaClass.java:50)
	at someTC.run(mysource:9)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:448)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:439)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:418)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:410)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:285)
	at com.kms.katalon.core.common.CommonExecutor.accessTestCaseMainPhase(CommonExecutor.java:65)
	at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:148)
	at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:106)
	at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:180)
	at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
	at TempTestSuite1709455911422.run(TempTestSuite1709455911422.groovy:35)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

[michel.g]

Another fact: This issue does not exist with KRE 9.3.0.

But with KRE 9.3.0, as with KRE 9.1.0, we face another issue in the same area: your very first Test Suite may not be executed. See bug description under https://forum.katalon.com/t/kre-9-1-0-does-not-run-the-first-testsuite-of-a-testsuitecollection-the-next-are-fine/110556 .

In both cases, it seems KRE may have an issue with in the first minutes of a run. It feels like it starts too quickly, while its contexts are not fully loaded.

In both cases, it should be fixed.

Regards,
–Michel

[Elly_Tran]

Hi,

Thank you for letting us know. We will investigate this and let you know soon

[abraham.vargas]

This is still happening in 9.4.0 - Happens randomly. On one run, it finds the classes, on the next one it doesn’t. Any workaround that can be applied while the bug gets fixed?

[system]

Hi there,

Thank you very much for your topic. Please note that it may take a little while before a member of our community or from Katalon team responds to you.

Thanks!

[albert.vu]

Hi @xuan.tran ơi,

Could you take a look at this bug?

Thanks,
Albert

[Elly_Tran]

Hi,

I have created a ticket for this. We will investigate on it and let you know soon

[kazurayam]

To me this looks similar to the following case

Change in Keyword is not reflecting automatically in testcases

In every Katalon Project you can find a folder named .cache where KS/KRE caches the compiled classes of Keywords. [スクリーンショット 2024-05-14 7.40.15] The .cache folder is automatically created by KS/KRE so it is safe to delete. Please try deleting the .cache folder. I guess in the KRE project you have a corrupted binary of your keyword so that you should clean the cache. I believe KS/KRE has a subtle bug about the .cache; but it is easy to fix. Just remove it.

[drai1]

Hi,

We are experiencing same issue, when we try to run on test suites on CICD pipelines with latest 9.4.0 KRE. It happens for some test suite collections, mostly start of test suite collections and some test suites after some time run without this error, but sometime whole test suite collection fails.
ERROR: Test Cases/Login/TC_Login FAILED.
Reason:
java.lang.ClassNotFoundException: base.LOGIN
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.getCustomKeywordClassAndSetMetaClass(CustomKeywordDelegatingMetaClass.java:105)
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.invokeStaticMethod(CustomKeywordDelegatingMetaClass.java:50)
at TC_Login.run(TC_Login:18)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:448)

The only workaround we did for now is, as we had the older KRE 8.6.6 saved too, we switched to that KRE and everything is working without this error. But, please have a look at this bug.

[Elly_Tran]

Hi,

Can you please try again with v9.5.0? This issue has been fixed in that version

[kazurayam]

@drai1

We are experiencing same issue, when we try to run on test suites on CICD pipelines with latest 9.4.0 KRE. I

@drai1

Please check the Git repository of your project. Does the repository contains <projectdir>/.cache folder?

If yes, I think that the .cache folder in the repository contains the binary class files which were compiled by Groovy 2.x using Katalon v8.6.6. When the CI/CD pipeline clones the repository, the old class files will be checked out. When KRE9.4.0 find the old class files in the .cache folder, you will surely encounter problems.

You MUST remove the .cache folder out of the repository completely for yourself.

How?

On your machine, you want to do this

$ cd $PROJECTDIR
$ git pull
$ rm -rf .cache
$ git -r --cached .cache
$ git add .
$ git commit -m "removed the .cache folder out of the repository"
$ git push

“git -r -cached .cache” is significant.

What it does? ---- Please read any Git tutorials documents and understand.

[kazurayam]

@nam.nguyen

I think that one more person was annoyed by the problem of .cache folder.

I know you were assinged responsible for this bug.

Are you still busy in developping AI-something so that you can not fix this long outstanding & really annoying bug?

[shie]

wen 9.5 in testops scheduler? we’re also encountering issues with CustomKeywords

[Elly_Tran]

Hi,

It has been released.

[shie]

In testops selecting "latest"would still DL v9.4 based on the logs

[Elly_Tran]

Please try to download v9.5.0 over this: Releases · katalon-studio/katalon-studio · GitHub