I can't run unit tests with JUnit in Katalon Studio

[jhilario]

When I want to run a test class I get the following error:

[grylion54]

Have a read of the below link:

Running JUnit4 in Katalon Studio

I have made a demo project and published it on GitHub: GitHub - kazurayam/junit4ks: JUnit for Katalon Studio. A custom keyword enables you run JUnit4 to test your custom keywords in Katalon Studio. Background I am going to develop a custom Keyword in a Katalon Studio project. See a post in the Katalon forum for the background story. My custom keyword is expected to be fairly large, complicated, and therefore bug-prone. So I want to execute thorough unit-testing on my Groovy classes and metho…

[jhilario]

I saw the project, and it’s very interesting, but Is there an easier way?

[grylion54]

Here is something that Bing’s AI has to “say” (note that it could be lying through its teeth) .
: I’m going to suggest you do more reading of Katalon’s documentation.

Certainly! Katalon Studio is a powerful tool for automated testing, and it supports running JUnit tests. Let me guide you on how to set up and execute JUnit tests within Katalon Studio.

1. JUnit in Katalon Studio:

• Katalon Studio allows you to integrate JUnit tests into your test automation projects. You can use JUnit to perform unit testing on your custom keywords, Groovy classes, and methods.
• To get started, create a custom keyword that enables you to run JUnit4 tests within Katalon Studio. You can find a demo project on GitHub that demonstrates this: JUnit for Katalon Studio.
• The custom keyword, JUnitCustomKeywords.runWithJUnitRunner, executes your JUnit-based tests. These tests interact with external websites via WebDriver and are backed by the Katalon Studio runtime environment.
• The test results are visible in the Log Viewer and Console within the Katalon Studio GUI1.

2. Steps to Set Up JUnit Tests in Katalon Studio:

• Create a custom keyword that calls the JUnit API to execute your unit tests.
• Place your application classes (to be tested) in the Keywords directory.
• Store your test classes in the Include/scripts/groovy directory.
• Write a simple test runner in a Katalon Studio test case. For example:

import static com.kazurayam.junit4ks.JUnitCustomKeywords.runWithJUnitRunner
import junittutorial.CalculatorTest

runWithJUnitRunner(CalculatorTest.class)

• The test runner will report logs in the Log Viewer and Console as usual1.

3. Reporting and Integration:

• Your JUnit-based tests can be seamlessly integrated into your Katalon Studio projects.
• You can also upload JUnit reports to Katalon TestOps manually for further analysis2.

Remember, this approach allows you to leverage the power of JUnit while benefiting from Katalon Studio’s features and runtime environment. Happy testing!

[kazurayam]

I am the author of com.kazurayam.junitks.JUnitCustomKeywords class.

@grylion54

3. Reporting and Integration:

• Your JUnit-based tests can be seamlessly integrated into your Katalon Studio projects.
• You can also upload JUnit reports to Katalon TestOps manually for further analysis 2 .

These sentences do not apply to the com.kazurayam.junitks.JUnitCustomKeywords class. Yes, the runWithJUnitRunner method will report logs in the Log Viewer and Console. But it does NOT compile any report as a file. It does not compile so-called JUnitReports in XML.

[kazurayam]

In another recent post by @lfilipovi, I reported a bug of Katalon Studio’s capability to run JUnit tests.

Not running selenium JUnit tests after migration from selenium to katalon studio

@Elly_Tran I found a bug. At the Line#152 of Wrong if (runSuccess) { KeywordMain.stepFailed( Right if ( ! runSuccess) { KeywordMain.stepFailed( Too silly mistake, isn’t it? I am afraid that the com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords is not tested enough. I suppose that nobody has ever used it. @lfilipovi seems to be the first penguin who tried it and reported a problem here in the user forum.

Please have a look.

@jhilario

I am rather afraid that you would have hard time in developing JUnit tests in Katalon Studio.

[kazurayam]

@jhilario

I DO NOT RECOMMEND you to develop & run JUnit4 tests in Katalon studio.

But I (@kazurayam) developed the com.kazurayam.junit4ks.JUnitCustomKeywords class 5 years ago? Why?

— I did not aim to write any Web UI automated tests with JUnit in Katalon Studio. I wanted to write some unit tests for my own Custom Keywords in Katalon Studio with JUnit.

You may or may not be aware, Katalon Studio provides no support for unit-testing your Custom Keywords. In token of that, Katalon’s documentation has no section titled “How to test your Custom Keywords”. Katalon assumes that your Custom Keywords are born bug-free automagically.

I can not develop my custom classes bug-free automagically. Therefore I had to invent a way for myself. So I developed my library.

Let me introduce my Custom Keywords ExecutionProfilesLoader to you as an example, which is runnable in Katalon Studio and which I unit-tested with JUnit.

GitHub - kazurayam/ExecutionProfilesLoader: Katalon Studio plugin. Groovy...

I developed com.kazurayam.ks.globalvariable.ExecutionProfilesLoader class in Katalon Studio because this class had to be executable in Katalon Studio. At the same time, the ExecutionProfilesLoader was complicated enough so that it deserved a set of thorough unit-tests. Therefore I wrote a set of unit-tests for the ExecutionProfilesLoader keyword with com.kazurayam.junit4ks.JUnitCustomKeywords.

I never use JUnit in Katalon Studio for automated UI testing.

If you are going to develop WebUI automated test with JUnit, I think it is a poor idea to do it in Katalon Studio. Other ordinary IDEs (JetBrain IDEA, Eclipse, etc) on top of Gradle or Maven with pure Selenium WebDriver + JUnit4 or JUnit5 — these environments are far more comfortable, well-tested, robust and bug-free to work on JUnit tests than in Katalon Studio. The following book gives you a thorough tutorials.

Hands-On Selenium WebDriver with Java

[anon46315158]

@jhilario

I saw the project, and it’s very interesting, but Is there an easier way?

Yes.
Stick to Junit!
Why am I saying this?
For few reasons …

Katalon use own implementations and concept regarding what a test-case, a test-suite means.
If the user need is migrate from Junit, TestNG or whatever other framework then some manual work is needed.
TestCases, Suites and so on must be re-written, simply importing the code and expect magic won’t work.
You have to ‘import’ the code by manually copy paste and addapt it as katalon testcases.
There was a brief video-tutorial somewhwere on how this may be achieved and steps involved, I cannot find it anymore.
Feel free to search for it.

The reason for using Junit, TestNG or whatever other framework with Katalon may have the only sense if you would like to do some unittests for your own develloped classes / keywords and eventually validate those without using KRE.

This has been briefly debated in:

Alternatives to getting a Katalon Runtime license, to run a test suite

I have this Test Suite, that is supposed to serve as a guardrail for my project. I have long since created a GitHub Action that runs that Guardrail Test Suite. However, as I quickly found out, that GitHub Action is useless, because the API Key that I have is of the free version and my freelance client doesn’t have a paid version that I could use. They also won’t get one either… What are some ways that I can run this guardrail test suite from GitHub without having to (shell out thousands of doll…

… and the efforts made by @kazurayam are, for my eyes, more or less for same reasons.

Hope this helps!

[kazurayam]

@jhilario

Is there an easier way?

Do not try to run JUnit tests inside Katalon Studio.

Do it in IntelliJ IDEA, Eclipse or whatever IDE you are currently working on. That is the easiest way. See the following tutorials for example

Testing | IntelliJ IDEA

[jhilario]

Yes, I have been use JUnit inside IntelliJ and It’s very helpful, but In my job I have to use Katalon, I wanted to know how implement unit testing in my custom keywords

[grylion54]

How about then continue to use JUnit inside IntelliJ but with Katalon, then use it as it can be used…

Not running selenium JUnit tests after migration from selenium to katalon studio

It wouldn’t take much to change it to run in Katalon Studio though as a Test Case. gSiteURL= "https://opensource-demo.orangehrmlive.com/web/index.php/auth/login"; WebUI.openBrowser(gSiteURL) WebUI.waitForPageLoad(10) WebUI.maximizeWindow() def driver = DriverFactory.getWebDriver(); driver.findElement(By.name("username")).isDisplayed(); driver.findElement(By.name("username")).click(); driver.findElement(By.name("username")).sendKeys("Admin"); driver.findElement(By.name("password")).isDisplay…

Is this a JUnit course for school?

[kazurayam]

@jhilario

Yes, I have been use JUnit inside IntelliJ and It’s very helpful, but In my job I have to use Katalon, I wanted to know how implement unit testing in my custom keywords

Your new “custom keywords” — there could be 2 types.

1. classes that are dependent on the Katalon API com.kms.katalon.core.*
2. classes that are independent of the Katalon API

You desperately need to work inside Katalon Studio to develop the type 1 classes because Katalon’s jar files are bundled inside the Katalon’s products (Katalon Studio, Katalon Runtime Engine) only; the jar files are not downloadable from any public maven repositories. KS is not open-sourced.

However you do not need Katalon Studio to develop the type 2 classes. You can develop your classes in IntelliJ IDEA while testing them with JUnit, build a jar using Gradle. Then you can import the jar into the Drivers folder of your target Katalon projects as documented at the doc.

I have developed a lot of Katalon-independent libraries in IntelliJ IDEA and import them into my katalon projects. For example, I use the following:

GitHub - kazurayam/timekeeper: Timekeeper is a Java/Groovy library that helps...

and

GitHub - kazurayam/subprocessj: A Java library which enables you to easily...

These projects are independent of the Katalon API, so I developed them as an isolated project in IntelliJ IDEA with Gradle where I performed unit-tests with JUnit5 Jupyter.

Based on my previous experience, I think that the significant portions of my “custom classes” are Katalon-API-independent so that I can develop them outside KS. I always try to minimize the size of the com.kms.katalon.* dependent classes.

You can reuse your jar file(s) in your multiple Katalon projects easily. Therefore it is a good idea to seperate projects outside of Katalon Studio. You may or may not be aware, when you have a set of custom Groovy classes contained inside a Katalon project A, it is very difficult to reuse the code set in another Katalon project B and C. Though Katalon representatives may proudly say “we offer the Import and export keywords in Katalon Studio is available”. Well, I see it a fake.

I would highly recommend you to seperate your codes into 2 portions: com.kms.katalon-dependent portions and independent portions. I think that the KS-dependent portions can be very thin, which only does calling Katalon API to pass information to/from your externally developed library. Most of the business logic could be isolated from the KS-API. In short, I never use KS to implement the codes for complexed logics at all. I always do it in IDEA.

You would rarely need JUnit for the thin portions which are left inside KS. You can write some simple Test Cases that call your KS-API-dependent Keywords to check if the keywords run or not. You won’t need JUnit for it.

[anon46315158]

@jhilario

I wanted to know how implement unit testing in my custom keywords

It those are indeed unittests (written to test some custom clases you wrote) and your custom classes do not depend on Katalon specific imports than you can store them in your Katalon project but do not run them with katalon.
Just run them as a maven goal or with gradle.
See for example an introductory tutorial for junit5 and gradle:

JUnit 5 Tutorial: Running Unit Tests With Gradle - Petri Kainulainen

This is why command line tools exist. To decouple your work from specific IDE’s features.
You can store also gradlewrapper in your project to decouple it even more from specific environments (different maven or gradle version may lead to different results)

[kazurayam]

@anon46315158

See for example an introductory tutorial for junit5 and gradle:

Unfortunately, this tutorial would not apply to a Katalon Studio project straight forward. I would explain why.

---

Gradle/Maven has its standard directroy structure for the Groovy program source codes, which looks

src/main/groovy/yourpackages
src/main/java/yourpackages
src/test/groovy/yourpackages
src/test/java/yourpackages

Gradle build system assumes that this starndard directory structure is applied to your project as default. If your project has different directory structure, you need to explicitly declare the source code directory structure.

As you already know, Katalon Studio has its unique directory structure, like

Keywords/yourpackages
Include/scripts/groovy/yourpackages

This structure is completely different from the Gradle/Maven’s standard directory strucuture. This difference makes your life as a Gradle/Maven user very challenging. You have to wrote code in the build.gradle file to declare how your custom directory structure looks like so that the Groovy/Java compilers can accept your source codes. Where to locate the .class files? The .class files will be generated by your unique build system, but the .class files must be accessible for Katalon Studio runtime. This is another difficult issue. How to resolve external dependencies? This is another issue to be resolved.

The build tools Gradle/Maven are really large and complicated software. The documation has hundreds of pages. It is difficult to learn enough and find some solutions for your non-typical usages. I am sure that this lesson would be much harder than using my com.kazurayam.junit4ks.JUnitCustomKeywords class.

---

Why Katalon project is designed to have unique directory structure which is different from the Gradle/Maven’s standard one? — I guess that the developers of the original Katalon Studio software a decade ago did not use Gradle/Maven at all. That’s all.

[anon46315158]

@kazurayam

Unfortunately, this tutorial would not apply to a Katalon Studio project straight forward. I would explain why.

That’s a very basic tutorial. Anyway, with gradle or maven one can specify the src and test paths easily.
there are plenty examples on stackoverflow

[kazurayam]

@anon46315158

one can specify the src and test paths easily.

Not easy at all. If you try to, you will encounter a lot of myterious error messages …

Yes, there are plenty of Gradle/Maven build examples that conform to the starndard directory structure. But I have never seen any examples of Gradle build that uses non-standard source code directories.

[anon46315158]

@kazurayam

Not easy, I think.

Well, the ‘easiness’ level is dependant on the user determination
And, as i mentioned, one of the conditions is to not use Katalon specific imports in such custom classes, so the deps can be solved from maven central.
For the rest, will be the same as using IDEA from Intelij, or Eclipse or whatever, same steps needed.
e.g:

How to run JUnit tests with Gradle?

or

How do I replace the default source folders for gradle?

[jhilario]

I am trying to use the plug in you developed, I downloaded the .jar and put it in the project libraries, but when I try to run the test case I get an error.

Test case:

scripts/groovy:

Error:

Caught: java.lang.NoSuchMethodError: 'java.lang.Object org.codehaus.groovy.runtime.memoize.EvictableCache.getAndPut(java.lang.Object, org.codehaus.groovy.runtime.memoize.MemoizeCache$ValueProvider)'
java.lang.NoSuchMethodError: 'java.lang.Object org.codehaus.groovy.runtime.memoize.EvictableCache.getAndPut(java.lang.Object, org.codehaus.groovy.runtime.memoize.MemoizeCache$ValueProvider)'
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

[kazurayam]

Please show the FULL stacktrace message; Don’t trim. I need to see all lines.

[kazurayam]

@jhilario

Caught: java.lang.NoSuchMethodError: 'java.lang.Object org.codehaus.groovy.runtime.memoize.EvictableCache.getAndPut(java.lang.Object,

I have never seen such Exception.

org.codehaus.groovy.runtime.memoize.EvictableCache ? I have never seen this class name.

I guess that your Katalon project is running in an environment where 2 or more different versions of Groovy accessible for your code. Version conflicts could cause anything mysterious.

You should try the same in a clean environment.