Groovy error for custom code while running from command line

Feedback & Reviews Bugs Report
,
#1

While running Katalon test cases from the command line, I’m getting random (very random) failures that return exit code 2 which I believe is “execution has erroneous test cases”. The test collection/suite executed has a single test case which runs successfully 99.999% of the time. Stack trace of the failure indicated there is an issue with custom groovy code in the project that has several public utility methods:

Test Cases/Form - Apply Now Application - Online - Happy Path FAILED.
Reason:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
file:/D:/Katalon%20Test/Monitoring/Scripts/Form%20-%20Apply%20Now%20Application%20-%20Online%20-%20Happy%20Path/Script1554324347883.groovy: 16: unable to resolve class global_package.UtilityMethods
@ line 16, column 1.
import global_package.UtilityMethods as GlobalUtilityMethods
^

1 error

at com.kms.katalon.core.main.ScriptEngine.getScript(ScriptEngine.java:199)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
at com.kms.katalon.core.main.TestSuiteExecutor.accessTestCaseMainPhase(TestSuiteExecutor.java:133)
at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:116)
at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:83)
at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
at TempTestSuite1562848994406.run(TempTestSuite1562848994406.groovy:35)

Running version 6.1.5 build 3 - test case is running against a headless Chrome instance and has been very reliable to date.

Interested to hear if there are suggestions to understand and possibly resolve these very rare errors.

Thank you.

#2

I have not seen this specific problem before. However, I have seen something similar. But first, a question:

Had you just prior to seeing the issue modified that class?

My workflow goes like this:

  1. On a Win7 box I develop test/suites. The code is then checked in.
  2. On two different Win10 boxes, the code is pulled and the suite(s) are executed “live” in Katalon
    2.1 That is, I don’t typically use command line.
  3. If my core classes have changed, some things break and it’s always the new code in the keyword classes.
    3.1 If I remember to right-click “Refresh” on my keywords folder first and then run the suites, I get no problems.

And yes, that’s a Katalon bug (previously reported way back in 5.3 I think).

Maybe that might help you?

#3

No, the code class in question has not changed recently. In fact, the server running the Katalon project from command line has code refreshed (from source repo) once daily in the evening. This error occurred 3 times and then ran without error and has run without error since. Same batch files, same collections/suites, same test case, same code… just random failures.

#4

That’s the red flag, in my view. Same symptoms.

I’ll bet there is something that’s not compiled properly that later gets compiled and thereafter never reappears.

#5

Suggestions?

Only other insight I can provide is analytics shows the tests cases running 50% slower during this time. Other test cases executed by command line during this time ran slower but were successful.

Is it possible that server resources (cpu/memory/etc) could cause Katalon to throw this error?

#6

None. Hopefully @devalex88 / @ThanhTo will pickup on it.

#7
#8