Method for retrieving an array-based list of all testIDs within a given folder structure in Katalon Studio?

So given an example Tests Explorer folder structure:

  • Test Cases
    • MyTestsFolder
      -TestCase1
      -TestCase2
      -TestCase3
      -TestCase4

Is there a method to programmatically retrieve the test IDs for TestCase1-4 into an array for parsing into a WebUI.callTestCase loop?

I can find the API calls for getting this info for TestData, but not having any luck for test cases

Perhaps instead of retrieving the test IDs into an array, you create a switch/case to run the specific TestCase for the loop counter.

Loop:
for (int cnt = 0; cnt < 4; cnt++)
{
	switch(cnt) {
	   case 0:
		   WebUI.callTestCase(findTestCase('MyTestsFolder/TestCase1'), [:], FailureHandling.OPTIONAL)
		   break;
	   case 1:
		   WebUI.callTestCase(findTestCase('MyTestsFolder/TestCase2'), [:], FailureHandling.OPTIONAL)
		   break;
	   case 2:
		   WebUI.callTestCase(findTestCase('MyTestsFolder/TestCase3'), [:], FailureHandling.OPTIONAL)
		   break;
	   case 3:
		   WebUI.callTestCase(findTestCase('MyTestsFolder/TestCase4'), [:], FailureHandling.OPTIONAL)
		   break;
	   default:
		   WebUI.comment('Nope, missed it by that much')
		   break;
	}
    // blah blah blah  for the rest of what you have to do
}

Aye, it’s more trying to get around having to manually put in the IDs of a large number of test cases and then have to update each suite every time the folder changes, was hoping just to be able to pull whatever is in a given folder and go with it :blush:

Like:

Thanks! Will give that a try and see how it goes!

I have made a project:

In this project’s root directory, I have a tree as follows:

$ cd test
:~/katalon-workspace/test (master *)
$ tree Scripts
Scripts
├── CallerA
│   └── Script1684964166177.groovy
├── CallerB
│   └── Script1684964147338.groovy
├── CallerC
│   └── Script1684964464063.groovy
├── MyTestsFolder
│   ├── TestCase1
│   │   └── Script1684961601822.groovy
│   ├── TestCase2
│   │   └── Script1684961680635.groovy
│   └── TestCase3
│       └── Script1684961612155.groovy
└── TestCase4
    └── Script1684964397879.groovy

9 directories, 7 files

Test Cases/TC1:

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

WebUI.comment("message1=" + message1)

Test Cases/TestCase2

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

WebUI.comment("message2=" + message2)

Test Cases/TC3

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

WebUI.comment("message3=" + message3)

I made a Test Case named “CallerA”

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

WebUI.callTestCase(findTestCase("MyTestsFolder/TestCase1"), ["message1": "foo"])

WebUI.callTestCase(findTestCase("MyTestsFolder/TestCase2"), ["message2": "bar"])

WebUI.callTestCase(findTestCase("MyTestsFolder/TestCase3"), ["message3": "buz"])

The CallerA worked just as expected:

I made another Test Case CallerB

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths
import java.util.stream.Collectors

import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

List<Path> listChildDirectoriesOf(Path dir) throws IOException {
	return Files.list(dir)
		.filter { Files.isDirectory(it) }
		.collect(Collectors.toList());
}

Path projectDir = Paths.get(RunConfiguration.getProjectDir())
Path scriptsDir = projectDir.resolve("Scripts")
Path myTestsFolder = scriptsDir.resolve("MyTestsFolder")
assert Files.exists(myTestsFolder)
List<Path> testCases = listChildDirectoriesOf(myTestsFolder).sort()
assert testCases.size() > 0

testCases.forEach({ p ->
	String n = scriptsDir.relativize(p).toString()
	println n
	WebUI.callTestCase(findTestCase(n), ["message1": "foo", "message2": "bar", "message3": "buz"])
})

The CallerB works as well

I think that the CallerB is what @kevin.jackey wanted.

I added another TestCase CallerC

import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import static com.kms.katalon.core.testobject.ObjectRepository.findWindowsObject
import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testng.keyword.TestNGBuiltinKeywords as TestNGKW
import com.kms.katalon.core.testobject.TestObject as TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import com.kms.katalon.core.windows.keyword.WindowsBuiltinKeywords as Windows
import internal.GlobalVariable as GlobalVariable
import org.openqa.selenium.Keys as Keys

WebUI.callTestCase(findTestCase("TestCase4"), ["message1": "foo", "message2": "bar", "message3": "buz"])

The TestCase4 is as this:

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

WebUI.comment("message1=" + message1)
WebUI.comment("message2=" + message2)
WebUI.comment("message3=" + message3)

The CallerC worked fine

My opinion

The CallerA is usual. Most of Katalon users live this way.

The CallerB is messy. Possibly he, @kevin, has too many Test Cases. Un-enumerable number of test cases — it causes this mess. Perhaps he had better reduce the number of Test Cases so that he can enumerate them by writing their names explicitly.

The CallerC is the simplest to achieve the same result, but it has no room of extensibility.


I made one more Test Case CallerD

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

WebUI.callTestCase(findTestCase("TestCase5"), ["message1": "foo", "message2": "bar", "message3": "buz"])

The “TestCase5” looks as follows:

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

// here I can do anyting over the variables message1, message2 and message3

WebUI.callTestCase(findTestCase("MyTestsFolder/TestCase1"), ["message1": message1, "message2": message2, "message3": message3])
WebUI.callTestCase(findTestCase("MyTestsFolder/TestCase2"), ["message1": message1, "message2": message2, "message3": message3])
WebUI.callTestCase(findTestCase("MyTestsFolder/TestCase3"), ["message1": message1, "message2": message2, "message3": message3])

The CallerD worked as well:

The keyword callTestCases() enable you to introduce more layers of abstraction where you can implement any data manipulation. For example, you can load values from JSON in the caller test case.

I made a JSON file <projectDir>/config.json

{
	"message1": "Wingardium Leviosa",
	"message2": "Expelliarmus",
	"message3": "Expecto Patronum"
}

I made a Test Case “TestCase6”

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

import groovy.json.JsonSlurper
import internal.GlobalVariable

// load the config file and put it into a variable of type Map
JsonSlurper slurper = new JsonSlurper()
def config = slurper.parse(new File('./config.json'))

WebUI.callTestCase(findTestCase("MyTestsFolder/TestCase1"), ["message1": config.message1, "message2": config.message2, "message3": config.message3])
WebUI.callTestCase(findTestCase("MyTestsFolder/TestCase2"), ["message1": config.message1, "message2": config.message2, "message3": config.message3])
WebUI.callTestCase(findTestCase("MyTestsFolder/TestCase3"), ["message1": config.message1, "message2": config.message2, "message3": config.message3])

I ran the TestCase6, it worked as follows

Please note, the message texts are loaded from the json file, are different from what CallerA and CallerB passed.

The callee Test Cases (TestCase1, TestCase2, TestCase3) are not modified at all, but displayed different messages: a sort of data-bining was performed here.

1 Like

@kazurayam - Thank you for going through all this!!

Based on the above I believe that Item B, though messy, may resolve my issue; however I’m definitely bookmarking this because I’m sure I’ll be using the majority of these examples as things progress!

UPDATE: Only thing I’m finding with Item B is that it doesn’t appear to be pulling in the scripts within subfolders in a path, only the subfolder headers themselves, but this is much easier!!

Example: When running the below using “/Folder A/” as the path I only get back the string “Subfolder A”, “Subfolder B”, but this is already a massive improvement over what I had before!!

Test Suites
   - Folder A
      - SubFolder A
         - Test Script 1
         - Test Script 2

       - SubFolder B
           - Test Script 3
           - Test Script 4

The following article gives you a sample code of walking a directory tree recursively

I amended the CallerB so that it can retrieve Test Cases in a directory recursively walking into the subdirectories

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase

import java.nio.file.FileVisitResult
import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths
import java.nio.file.SimpleFileVisitor
import java.nio.file.attribute.BasicFileAttributes
import java.util.stream.Collectors

import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

// https://www.baeldung.com/java-list-directory-files#walking
List<Path> listFilesRecursively(Path dir) throws IOException {
	List<Path> directoryList = new ArrayList<>()
	Files.walkFileTree(dir, new SimpleFileVisitor<Path>() {
		@Override
		public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
			if (!Files.isDirectory(file)) {
				directoryList.add(file.toAbsolutePath())
			}
			return FileVisitResult.CONTINUE
		}	
	})
	directoryList.sort()
	return directoryList
}

Path projectDir = Paths.get(RunConfiguration.getProjectDir())
Path scriptsDir = projectDir.resolve("Scripts")
Path myTestsFolder = scriptsDir.resolve("MyTestsFolder")
assert Files.exists(myTestsFolder)

List<Path> testCases = listFilesRecursively(myTestsFolder).sort()
assert testCases.size() > 0

testCases.forEach({ p ->
	String n = scriptsDir.relativize(p.getParent()).toString()
	println n
	WebUI.callTestCase(findTestCase(n), ["messageX": "xyzzy", "message1": "foo", "message2": "bar", "message3": "baz"])
})


When I ran the CallerB, I got the following message in the console:

MyTestsFolder/Subfolder/TestCaseSubX
MyTestsFolder/TestCase1
MyTestsFolder/TestCase2
MyTestsFolder/TestCase3

This is WONDERFUL, will give it a go as time permits this week and reply back with my findings. Thank you so much!

UPDATE: Hmm… the newest CallerB function seems to be going one layer too deep and causing an error.

For comparison, here is the driver script I’m using showing the calls being made when I call at the Parent Folder level (the original CallerB) and the new CallerB:

import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.checkpoint.CheckpointFactory as CheckpointFactory
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import com.kms.katalon.core.testcase.TestCaseFactory as TestCaseFactory
import com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testdata.TestDataFactory as TestDataFactory
import com.kms.katalon.core.testobject.ObjectRepository as ObjectRepository
import com.kms.katalon.core.testobject.TestObject as TestObject

import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile

import internal.GlobalVariable as GlobalVariable

import com.kms.katalon.core.annotation.SetUp
import com.kms.katalon.core.annotation.SetupTestCase
import com.kms.katalon.core.annotation.TearDown
import com.kms.katalon.core.annotation.TearDownTestCase

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths
import java.util.stream.Collectors

import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

import btt_portal.*

//Variable creation and population
Common_Functions common_functions = new Common_Functions()
String driver_user_name = user_name
String driver_site_name = site_name
int driver_object_timeout = Integer.valueOf(object_timeout)

/*

///Folder Tree Call
def pathname_array = ['01-Rgsn/02 - UI/02-Menu/03-Bus. Analy']

for (int counter; counter < pathname_array.size(); counter = counter + 1) {
common_functions.run_folder_tree_tests(pathname_array[counter],driver_user_name,driver_site_name,driver_object_timeout)
}
*


/*

///Parent Folder Call
def pathname_array = ['01-Rgsn/02 - UI/02-Menu/03-Bus. Analy/01-Slw Pgs'
					 ]
		 
for (int counter; counter < pathname_array.size(); counter = counter + 1) {
	common_functions.run_folder_tests(pathname_array[counter],driver_user_name,driver_site_name,driver_object_timeout)
}
*/

Here’s the original CallerB function as I implemented it:

	/***********************************************************************************************************************
	 * Function Name: run_folder_tests and listChildDirectoriesOf
	 * 
	 * Function Overview: Runs all test scripts found in a given folder path (does not include subfolders)
	 * 
	 * Function Input Variable(s): 
	 * 			@input_folder_path - Path to the desired Test Script folder (NOTE: Do NOT include the high level "Test Suites" layer in the path)
	 * 			@input_user_name - User name to be used for the test script iteration (will override the currently set Test Script variable)
	 * 			@input_site_name - Site name to be used for the test script iteration (will override the currently set Test Script variable)
	 * 			@input_object_timeout - Object Timeout Time name to be used for the test script iteration (will override the currently set Test Script variable)
	 * 			
	 *******************************************************************************************************************************/

	public String run_folder_tests(input_folder_path, String input_user_name, String input_site_name, int input_object_timeout = 20) {
		KeywordUtil log = new KeywordUtil()

		Path projectDir = Paths.get(RunConfiguration.getProjectDir())
		Path scriptsDir = projectDir.resolve("Scripts")
		Path myTestsFolder = scriptsDir.resolve(input_folder_path)
		assert Files.exists(myTestsFolder)
		List<Path> testCases = listChildDirectoriesOf(myTestsFolder).sort()
		assert testCases.size() > 0

		testCases.forEach({test_case ->
			String current_script_path = scriptsDir.relativize(test_case).toString()
			log.markWarning('current script path: ' + current_script_path)
			WebUI.callTestCase(findTestCase(current_script_path), ['user_name' : input_user_name, 'site_name' : input_site_name,'object_timeout' : input_object_timeout], FailureHandling.CONTINUE_ON_FAILURE)
		})
	}

	def List<Path> listChildDirectoriesOf(Path dir) throws IOException {
		return Files.list(dir)
				.filter {Files.isDirectory(it) }
				.collect(Collectors.toList());
	}

and the newer function:

	/***********************************************************************************************************************
	 * Function Name: run_folder_tree_tests and listFilesRecursively
	 *
	 * Function Overview: Runs all test scripts found in a given folder path (including subfolders)
	 *
	 * Function Input Variable(s):
	 * 			@input_folder_path - Path to the desired Test Script folder (NOTE: Do NOT include the high level "Test Suites" layer in the path)
	 * 			@input_user_name - User name to be used for the test script iteration (will override the currently set Test Script variable)
	 * 			@input_site_name - Site name to be used for the test script iteration (will override the currently set Test Script variable)
	 * 			@input_object_timeout - Object Timeout Time name to be used for the test script iteration (will override the currently set Test Script variable)
	 *
	 *******************************************************************************************************************************/

	public String run_folder_tree_tests(input_folder_path, String input_user_name, String input_site_name, int input_object_timeout = 20) {
		KeywordUtil log = new KeywordUtil()

		Path projectDir = Paths.get(RunConfiguration.getProjectDir())
		Path scriptsDir = projectDir.resolve("Scripts")
		Path myTestsFolder = scriptsDir.resolve(input_folder_path)
		assert Files.exists(myTestsFolder)

		List<Path> testCases = listFilesRecursively(myTestsFolder).sort()
		assert testCases.size() > 0

		testCases.forEach({test_case ->
			String current_script_path = scriptsDir.relativize(test_case).toString()
			log.markWarning('current script path: ' + current_script_path)
			WebUI.callTestCase(findTestCase(current_script_path), ['user_name' : input_user_name, 'site_name' : input_site_name,'object_timeout' : input_object_timeout], FailureHandling.CONTINUE_ON_FAILURE)
		})
	}

	// https://www.baeldung.com/java-list-directory-files#walking
	def List<Path> listFilesRecursively(Path dir) throws IOException {
		List<Path> directoryList = new ArrayList<>()
		Files.walkFileTree(dir, new SimpleFileVisitor<Path>() {
					@Override
					public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
						if (!Files.isDirectory(file)) {
							directoryList.add(file.toAbsolutePath())
						}
						return FileVisitResult.CONTINUE
					}
				})
		directoryList.sort()
		return directoryList
	}

Script name output from the original CallerB function:

and Error output from the Tree version of CallerB:

annnd finally I was able to work around the above by stripping out the filename extention part of the path string, possibly not the best solution but it works:

		testCases.forEach({test_case ->
			String current_script_path = scriptsDir.relativize(test_case).toString()
			int lastslashpos = current_script_path.lastIndexOf('\\')
			current_script_path = current_script_path.substring(0,lastslashpos)
			WebUI.callTestCase(findTestCase(current_script_path), ['user_name' : input_user_name, 'site_name' : input_site_name,'object_timeout' : input_object_timeout], FailureHandling.CONTINUE_ON_FAILURE)
		})