Not running selenium JUnit tests after migration from selenium to katalon studio

Hello,
I am trying to migrate my selenium JUnit test to Katalon Studio. I have put Java class with my JUnit test into the Include/scripts/groovy/com/lfilipovic/demo package and made Katalon test case which is supposed to run Java JUnit test.

This test case looks like this:

import com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords as TestNGKW
import com.lfilipovic.demo.SearchForEmployeeTest

TestNGKW.runJUnitTestClasses([SearchForEmployeeTest.class])

My JUnit test in Java looks like this:

package com.lfilipovic.demo;

import java.util.HashMap;
import java.util.Map;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.Dimension;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.interactions.Actions;

public class SearchForEmployeeTest {
  private WebDriver driver;
  private Map<String, Object> vars;
  JavascriptExecutor js;
  
  @Before
  public void setUp() {
    driver = new ChromeDriver();
    js = (JavascriptExecutor) driver;
    vars = new HashMap<String, Object>();
  }
  
  @After
  public void tearDown() {
    driver.quit();
  }
  
  @Test
  public void searchForEmployee() {
    driver.get("https://opensource-demo.orangehrmlive.com/web/index.php/auth/login");
    driver.manage().window().setSize(new Dimension(1394, 1104));
    driver.findElement(By.name("username")).click();
    driver.findElement(By.name("username")).sendKeys("Admin");
    driver.findElement(By.name("password")).click();
    driver.findElement(By.name("password")).sendKeys("admin123");
    driver.findElement(By.cssSelector(".oxd-button")).click();
    driver.findElement(By.cssSelector(".oxd-main-menu-item-wrapper:nth-child(9) .oxd-text")).click();
    {
      WebElement element = driver.findElement(By.cssSelector(".active > .oxd-text"));
      Actions builder = new Actions(driver);
      builder.moveToElement(element).perform();
    }
    {
      WebElement element = driver.findElement(By.tagName("body"));
      Actions builder = new Actions(driver);
      builder.moveToElement(element, 0, 0).perform();
    }
    driver.findElement(By.cssSelector(".oxd-select-text--focus > .oxd-select-text-input")).click();
    {
      WebElement element = driver.findElement(By.cssSelector(".orangehrm-horizontal-padding"));
      Actions builder = new Actions(driver);
      builder.moveToElement(element).release().perform();
    }
    driver.close();
  }
}

PROBLEM: Katalon runs this class successfully but it does not actually runs a test. That is to say that code under Before and After annotation is running successfully but code under Test annotation does not run.

Does anyone have the same issue or does anyone knows how to fix this?

1 Like

I looked at the API javadoc of com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords#runJUnitTestCasses(List)

The documentation is questionable:

@Keyword static JUnitRunnerResult >runJUnitTestClasses(List testClasses)

Runs the given list of TestNG test classes by invoking TestNG#run()#run().

It says it expects a list of TestNG test classes and return JUnitRunnerResult — this doesn’t make sense to me. TestNG and JUnit are 2 different beasts, they are not interchangeable.

So I doubt the quaility of the TestNGBuiltinKeywords class.

You can read the source code of the keyword at

Please study it yourself.

You disclosed the source of your JUnit4 test case and Katalon Test case that executes the JUnit4 test case. With these source codes, I tried to reproduce your case on my machine. I slightly modified your JUnit4 Test case so that each method emits debug-prints to the console.

package com.lfilipovic.demo;

import java.util.HashMap;
import java.util.Map;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;

import org.openqa.selenium.By;
import org.openqa.selenium.Dimension;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.interactions.Actions;

@RunWith(JUnit4.class)
public class SearchForEmployeeTest {
	
	private WebDriver driver;
	private Map<String, Object> vars;
	JavascriptExecutor js;
  
	@Before
	public void setUp() {
		System.out.println("setUp() was called");
		/*
		driver = new ChromeDriver();
		js = (JavascriptExecutor) driver;
		vars = new HashMap<String, Object>();
		*/
	}
  
	@After
	public void tearDown() {
		System.out.println("tearDown() was called");
		/*
		driver.quit();
		*/
	}
  
	@Test
	public void searchForEmployee() {
		System.out.println("searchForEmployee() was called");
		/*
		driver.get("https://opensource-demo.orangehrmlive.com/web/index.php/auth/login");
		driver.manage().window().setSize(new Dimension(1394, 1104));
		driver.findElement(By.name("username")).click();
		driver.findElement(By.name("username")).sendKeys("Admin");
		driver.findElement(By.name("password")).click();
		driver.findElement(By.name("password")).sendKeys("admin123");
		driver.findElement(By.cssSelector(".oxd-button")).click();
		driver.findElement(By.cssSelector(".oxd-main-menu-item-wrapper:nth-child(9) .oxd-text")).click();
		{
			WebElement element = driver.findElement(By.cssSelector(".active > .oxd-text"));
			Actions builder = new Actions(driver);
			builder.moveToElement(element).perform();
		}
		{
			WebElement element = driver.findElement(By.tagName("body"));
			Actions builder = new Actions(driver);
			builder.moveToElement(element, 0, 0).perform();
		}
		driver.findElement(By.cssSelector(".oxd-select-text--focus > .oxd-select-text-input")).click();
		{
			WebElement element = driver.findElement(By.cssSelector(".orangehrm-horizontal-padding"));
			Actions builder = new Actions(driver);
			builder.moveToElement(element).release().perform();
		}
		driver.close();
		*/
	}
}

As you can see, I commented out the lines that work with the WebDriver API. I wanted to see if JUnit4 calls @Before, @Test, and @After-annotated methods.

When I ran this, I got the following output in the console:

2024-02-15 18:11:14.924 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2024-02-15 18:11:14.927 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/TC3
setUp() was called
searchForEmployee() was called
tearDown() was called
2024-02-15 18:11:15.594 ERROR c.k.k.core.keyword.internal.KeywordMain  - ❌ Running TestNG test classes: '[class com.lfilipovic.demo.SearchForEmployeeTest]' failed
2024-02-15 18:11:15.628 ERROR c.k.k.core.keyword.internal.KeywordMain  - ❌ Keyword runJUnitTestClasses was failed (Root cause: com.kms.katalon.core.exception.StepFailedException: Running TestNG test classes: '[class com.lfilipovic.demo.SearchForEmployeeTest]' failed
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:51)
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:23)
	at com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords$_runJUnitTestClasses_closure3.doCall(TestNGBuiltinKeywords.groovy:157)
	at com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords$_runJUnitTestClasses_closure3.call(TestNGBuiltinKeywords.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:75)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:69)
	at com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords.runJUnitTestClasses(TestNGBuiltinKeywords.groovy:150)
	at TC3.run(TC3:6)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:448)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:439)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:418)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:410)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:285)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:144)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:135)
	at TempTestCase1707988269519.run(TempTestCase1707988269519.groovy:25)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
)
2024-02-15 18:11:15.641 ERROR c.k.katalon.core.main.TestCaseExecutor   - ❌ Test Cases/TC3 FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Running TestNG test classes: '[class com.lfilipovic.demo.SearchForEmployeeTest]' failed
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:51)
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:23)
	at com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords$_runJUnitTestClasses_closure3.doCall(TestNGBuiltinKeywords.groovy:157)
	at com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords$_runJUnitTestClasses_closure3.call(TestNGBuiltinKeywords.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:75)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:69)
	at com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords.runJUnitTestClasses(TestNGBuiltinKeywords.groovy:150)
	at TC3.run(TC3:6)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:448)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:439)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:418)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:410)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:285)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:144)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:135)
	at TempTestCase1707988269519.run(TempTestCase1707988269519.groovy:25)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

2024-02-15 18:11:15.695 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/TC3

This message proves that JUnit called @Before, @Test and @After-annotated methods. This is normal. I believe that JUnit is not guilty.

However, as you see in the message, an Exception was thrown.

On the other hand, I created a Gradle project with JUnit4 and executed just the same JUnit4 test case. It ran OK. I got no Exception. Therefore I guess, the com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords has some problem which caused the Exception.

I found that the message emitted by Katalon Studio is too poor; it does not tell me anything informative.

I am afraid you are going to have a hard time. I would recommend you to keep your selenium JUnit test as is.

@duyluong

any comment?

@Elly_Tran

I found a bug.

At the Line#152 of

Wrong

            if (runSuccess) {
                KeywordMain.stepFailed(

Right

            if ( ! runSuccess) {
                KeywordMain.stepFailed(

Too silly mistake, isn’t it?

I am afraid that the com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords is not tested enough. I suppose that nobody has ever used it. @lfilipovi seems to be the first penguin who tried it and reported a problem here in the user forum.

1 Like

It wouldn’t take much to change it to run in Katalon Studio though as a Test Case.

gSiteURL= "https://opensource-demo.orangehrmlive.com/web/index.php/auth/login";
WebUI.openBrowser(gSiteURL)
WebUI.waitForPageLoad(10)

WebUI.maximizeWindow()

def driver = DriverFactory.getWebDriver();

driver.findElement(By.name("username")).isDisplayed();
driver.findElement(By.name("username")).click();
driver.findElement(By.name("username")).sendKeys("Admin");

driver.findElement(By.name("password")).isDisplayed();
driver.findElement(By.name("password")).click();
driver.findElement(By.name("password")).sendKeys("admin123");

WebUI.waitForPageLoad(10)

driver.findElement(By.cssSelector(".oxd-button")).isDisplayed();
driver.findElement(By.cssSelector(".oxd-button")).click();
WebUI.waitForPageLoad(10)

driver.findElement(By.cssSelector(".oxd-main-menu-item-wrapper:nth-child(9) .oxd-text")).isDisplayed();
driver.findElement(By.cssSelector(".oxd-main-menu-item-wrapper:nth-child(9) .oxd-text")).click();
WebUI.waitForPageLoad(10)

I guess, @lfilipovi would have many other Selenium test cases in Java. @lfilipovi would not like the burden of rewriting them into Katalon Test Cases.

Another idea:

@lfilipovi wants to create a new Keyword class com.lfilipovi.keyword.JUnitBuiltingKeyword.

@lfilipovi want to copy the source of com.kms.katalon.core.testng.keyword.TestNGBuiltingKeywords class into his new keyword class while changing the package and class name appropriately,

and @lfilipovi want to fix the problem in his new keyword class. I mean, change
if (runSuccess) {

if ( ! runSuccess) {

@lfilipovi wants to write Katalon Test cases like

import com.lfilipovic.keyword.JUnitBuiltinKeywords as JUnitKW
import com.lfilipovic.demo.SearchForEmployeeTest

JUnitKW.runJUnitTestClasses([SearchForEmployeeTest.class])

I tried this idea myself, and I encountered another problem.

I created a Keyword class com.lfilipovic.keyword.JUnitBuiltinKeywords as follows:

package com.lfilipovic.keyword

import java.text.MessageFormat

import org.junit.runner.JUnitCore
import org.junit.runner.Result

import com.kms.katalon.core.annotation.Keyword
import com.kms.katalon.core.keyword.BuiltinKeywords
import com.kms.katalon.core.keyword.internal.KeywordMain
import com.kms.katalon.core.logging.KeywordLogger
import com.kms.katalon.core.model.FailureHandling
import com.kms.katalon.core.testng.keyword.JUnitRunnerResult
import com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords
import com.kms.katalon.core.testng.keyword.internal.JUnitRunnerResultImpl


public class JUnitBuiltinKeywords extends BuiltinKeywords {

	private static final KeywordLogger logger = KeywordLogger.getInstance(TestNGBuiltinKeywords.class);

	@Keyword
	public static JUnitRunnerResult runJUnitTestClasses(List testClasses, FailureHandling flowControl) {
		println "JUnitBuiltinKeywords#runJUnitTestClasses() was called"
		return (JUnitRunnerResult) KeywordMain.runKeyword({
			Result result = JUnitCore.runClasses(testClasses as Class[])
			println "result.wasSuccessful()=" + result.wasSuccessful();
			boolean runSuccess = result.wasSuccessful();
			JUnitRunnerResultImpl junitRunnerResult = new JUnitRunnerResultImpl(runSuccess ? 'passed' : 'failed', result)
			
			if ( ! runSuccess) {
			
				KeywordMain.stepFailed(
						MessageFormat.format("Running TestNG test classes: ''{0}'' failed", testClasses), flowControl)
			} else {
				logger.logPassed(MessageFormat.format("Running JUnit test classes: ''{0}'' passed", testClasses));
			}
			return junitRunnerResult
		}, flowControl, "Keyword runJUnitTestClasses was failed");
	}
}

I changed the JUnit4 Test case class as follows:

package com.lfilipovic.demo;

import java.util.HashMap;
import java.util.Map;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import static org.junit.Assert.*;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;

import org.openqa.selenium.By;
import org.openqa.selenium.Dimension;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.interactions.Actions;

@RunWith(JUnit4.class)
public class SearchForEmployeeTest {
	
	private WebDriver driver;
	private Map<String, Object> vars;
	JavascriptExecutor js;
  
	@Before
	public void setUp() {
		System.out.println("setUp() was called");
		
		driver = new ChromeDriver();
		js = (JavascriptExecutor) driver;
		vars = new HashMap<String, Object>();
		
	}
  
	@After
	public void tearDown() {
		System.out.println("tearDown() was called");
		
		driver.quit();
		
	}
  
	@Test
	public void searchForEmployee() {
		System.out.println("searchForEmployee() was called");
		
		driver.get("https://opensource-demo.orangehrmlive.com/web/index.php/auth/login");
		driver.manage().window().setSize(new Dimension(1394, 1104));
		driver.findElement(By.name("username")).click();
		driver.findElement(By.name("username")).sendKeys("Admin");
		driver.findElement(By.name("password")).click();
		driver.findElement(By.name("password")).sendKeys("admin123");
		driver.findElement(By.cssSelector(".oxd-button")).click();
		driver.findElement(By.cssSelector(".oxd-main-menu-item-wrapper:nth-child(9) .oxd-text")).click();
		{
			WebElement element = driver.findElement(By.cssSelector(".active > .oxd-text"));
			Actions builder = new Actions(driver);
			builder.moveToElement(element).perform();
		}
		{
			WebElement element = driver.findElement(By.tagName("body"));
			Actions builder = new Actions(driver);
			builder.moveToElement(element, 0, 0).perform();
		}
		driver.findElement(By.cssSelector(".oxd-select-text--focus > .oxd-select-text-input")).click();
		{
			WebElement element = driver.findElement(By.cssSelector(".orangehrm-horizontal-padding"));
			Actions builder = new Actions(driver);
			builder.moveToElement(element).release().perform();
		}
		driver.close();
		
	}
}

I created a Test Case as test runner as follows:

import com.kms.katalon.core.model.FailureHandling
import com.lfilipovic.keyword.JUnitBuiltinKeywords as JUnitKW
import com.lfilipovic.demo.SearchForEmployeeTest

JUnitKW.runJUnitTestClasses([SearchForEmployeeTest.class], FailureHandling.STOP_ON_FAILURE)

When I executed the test runner, it failed. It emitted the following messages in the console:

2024-02-16 17:09:57.260 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2024-02-16 17:09:57.265 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/TC3
JUnitBuiltinKeywords#runJUnitTestClasses() was called
setUp() was called
tearDown() was called
result.wasSuccessful()=false
2024-02-16 17:09:58.120 ERROR c.k.k.core.keyword.internal.KeywordMain  - ❌ Running TestNG test classes: '[class com.lfilipovic.demo.SearchForEmployeeTest]' failed
2024-02-16 17:09:58.130 ERROR c.k.k.core.keyword.internal.KeywordMain  - ❌ Keyword runJUnitTestClasses was failed (Root cause: com.kms.katalon.core.exception.StepFailedException: Running TestNG test classes: '[class com.lfilipovic.demo.SearchForEmployeeTest]' failed
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:51)
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:23)
	at com.lfilipovic.keyword.JUnitBuiltinKeywords$_runJUnitTestClasses_closure1.doCall(JUnitBuiltinKeywords.groovy:32)
	at com.lfilipovic.keyword.JUnitBuiltinKeywords$_runJUnitTestClasses_closure1.doCall(JUnitBuiltinKeywords.groovy)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:75)
	at com.lfilipovic.keyword.JUnitBuiltinKeywords.runJUnitTestClasses(JUnitBuiltinKeywords.groovy:25)
	at TC3.run(TC3:8)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:448)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:439)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:418)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:410)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:285)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:144)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:135)
	at TempTestCase1708070989140.run(TempTestCase1708070989140.groovy:25)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
)
2024-02-16 17:09:58.137 ERROR c.k.katalon.core.main.TestCaseExecutor   - ❌ Test Cases/TC3 FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Running TestNG test classes: '[class com.lfilipovic.demo.SearchForEmployeeTest]' failed
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:51)
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy)
	at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:23)
	at com.lfilipovic.keyword.JUnitBuiltinKeywords$_runJUnitTestClasses_closure1.doCall(JUnitBuiltinKeywords.groovy:32)
	at com.lfilipovic.keyword.JUnitBuiltinKeywords$_runJUnitTestClasses_closure1.doCall(JUnitBuiltinKeywords.groovy)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:75)
	at com.lfilipovic.keyword.JUnitBuiltinKeywords.runJUnitTestClasses(JUnitBuiltinKeywords.groovy:25)
	at TC3.run(TC3:8)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:448)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:439)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:418)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:410)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:285)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:144)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:135)
	at TempTestCase1708070989140.run(TempTestCase1708070989140.groovy:25)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

2024-02-16 17:09:58.155 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/TC3

The messages contain no information. It just says “I failed”. The messages do not tell me the reason why the JUnit4 test script failed.

The Keyword has a design shortage at :

			if ( ! runSuccess) {
				KeywordMain.stepFailed(MessageFormat.format("Running TestNG test classes: ''{0}'' failed", testClasses), flowControl)
			} else {

The message text is poor because as designed. I copy & pasted this code fragment from the com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords class.
The Katalon’s built-in keyword has the same design fault. I believe that Katalon developers have never used the TestNGBuiltinKeywords class for their own sake.

1 Like

Several years ago, I developed a Java library named junit4ks. The junit4ks library provides a java class that you can use in your Katalon Test Case to execute any JUnit4-based test classes (in java, in Groovy). It plays just the same role as com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords class. I think that the junit4ks performs better.

Let me explain how to use junit4ks.

It is published in a GitHub repository:

You can download the jar from the Maven Central at:

https://mvnrepository.com/artifact/com.kazurayam/junit4ks

You want to download the jar and locate it into the Drivers folder of your Katalon Project, like this:

And I make a Katalon Test Case as JUnit test runner:

import com.kms.katalon.core.model.FailureHandling
import com.lfilipovic.demo.SearchForEmployeeTest
import com.kazurayam.junit4ks.JUnitCustomKeywords

JUnitCustomKeywords.runWithJUnitRunner(SearchForEmployeeTest.class);

The source of com.kazurayam.junit4ks.JUnitCustomKeywords is available here.

When I ran it, it failed as expected. But it emitted a bunch of informative messages that tells what’s wrong in the JUnit4 test case:

2024-02-16 17:41:31.214 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2024-02-16 17:41:31.218 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/TC4
setUp() was called
tearDown() was called
2024-02-16 17:41:32.341 ERROR c.k.junit4ks.JUnitCustomKeywords         - ❌ searchForEmployee(com.lfilipovic.demo.SearchForEmployeeTest) FAILED.
Reason:
java.lang.IllegalStateException: The path to the driver executable must be set by the webdriver.chrome.driver system property; for more information, see https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver. The latest version can be downloaded from http://chromedriver.storage.googleapis.com/index.html
	at com.google.common.base.Preconditions.checkState(Preconditions.java:847)
	at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:134)
	at org.openqa.selenium.chrome.ChromeDriverService.access$000(ChromeDriverService.java:35)
	at org.openqa.selenium.chrome.ChromeDriverService$Builder.findDefaultExecutable(ChromeDriverService.java:159)
	at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:355)
	at org.openqa.selenium.chrome.ChromeDriverService.createDefaultService(ChromeDriverService.java:94)
	at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:123)
	at com.lfilipovic.demo.SearchForEmployeeTest.setUp(SearchForEmployeeTest.java:32)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:105)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at com.kazurayam.junit4ks.JUnitCustomKeywords$_runWithJUnitRunner_closure1.doCall(JUnitCustomKeywords.groovy:174)
	at com.kazurayam.junit4ks.JUnitCustomKeywords$_runWithJUnitRunner_closure1.doCall(JUnitCustomKeywords.groovy)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at com.kms.katalon.core.keyword.internal.KeywordMain.runKeyword(KeywordMain.groovy:75)
	at com.kms.katalon.core.keyword.internal.KeywordMain$runKeyword.call(Unknown Source)
	at com.kazurayam.junit4ks.JUnitCustomKeywords.runWithJUnitRunner(JUnitCustomKeywords.groovy:162)
	at com.kazurayam.junit4ks.JUnitCustomKeywords$runWithJUnitRunner.callStatic(Unknown Source)
	at com.kazurayam.junit4ks.JUnitCustomKeywords.runWithJUnitRunner(JUnitCustomKeywords.groovy:60)
	at TC4.run(TC4:5)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:448)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:439)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:418)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:410)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:285)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:144)
	at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:135)
	at TempTestCase1708072884739.run(TempTestCase1708072884739.groovy:25)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

Now I have got a clue to fix the problem in the JUnit4 test case developed by @lfilipovi

How to fix it? — the sample code by @grylion54 tells @lfilipovi how to.

@lfilipovi

You still have a long way to go. Are you going to migrate your Selenium+JUnit4 test to Katalon Studio?

Hi,

I have let my team know and will fix this soon. Thank you for raising this bug to us. :bowing_woman: