Create test suite consisting purely of code?

gengland · June 1, 2021, 9:10pm

Out of interest, is it possible to create a test suite which is script only? For example, let’s say I want to loop x times, calling WebUI.callTestCase(xxxxxx)?
If I don’t have at least one test case in the manual view, when I run it I get the message telling me that the current test suite has no test cases to run.
Furthermore, in the unlikely event that this is possible, where would I place my code? Presumably not in the @SetUp, @TearDown, @SetupTestCase or @TearDownTestCase sections, so is there a @Main or something like that?

Russ_Thomas · June 1, 2021, 10:16pm

No, not really.

You’re better off creating a one-test suite and make the test loop as long as necessary.

I do something similar - but I don’t use a suite. It’s a test that loops for 9 hours.

kazurayam · June 2, 2021, 1:00am

I tried it, it seems possible. Let me tell you what I have done.

I created a new Katalon Studio project.

I created a Test Case named TC1. It source code in Script mode looks like this:

println("Hello, world!")

I created a Test Suite named ‘TS1’. It includes the TC1 only. When I ran TS1, I could see the famous greeting in the Console.

I opened the Katalon projects directory with another editor (I used Emacs). I looked into the <projectdir>/Libs directory where I found 2 *.groovy source files were generated by Katalon Studio:

  /Users/kazurayam/katalon-workspace/CreateATestSuiteByCode/Libs:
  total used in directory 72 available 264.5 GiB
  drwxr-xr-x   7 kazurayam  staff    224  6  2 09:47 .
  drwxr-xr-x  26 kazurayam  staff    832  6  2 09:46 ..
  -rw-r--r--   1 kazurayam  staff  23916  6  2 09:46 CustomKeywords.groovy
  -rw-r--r--   1 kazurayam  staff   1598  6  2 09:42 TempTestCase1622594558717.groovy
  -rw-r--r--   1 kazurayam  staff   2036  6  2 09:42 TempTestSuite1622594560620.groovy
  drwxr-xr-x   3 kazurayam  staff     96  6  2 09:41 internal

I looked at the TempTestSuitexxxxx.groovy code and thought that it must be an implementation script of a Test Suite in Groovy. Do you want to see it? OK, I just copy & paste it here.

import com.kms.katalon.core.logging.KeywordLogger
import com.kms.katalon.core.exception.StepFailedException
import com.kms.katalon.core.reporting.ReportUtil
import com.kms.katalon.core.main.TestCaseMain
import com.kms.katalon.core.testdata.TestDataColumn
import com.kms.katalon.core.testcase.TestCaseBinding
import com.kms.katalon.core.driver.internal.DriverCleanerCollector
import com.kms.katalon.core.model.FailureHandling
import com.kms.katalon.core.configuration.RunConfiguration
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import internal.GlobalVariable as GlobalVariable

Map<String, String> suiteProperties = new HashMap<String, String>();


suiteProperties.put('id', 'Test Suites/TS1')

suiteProperties.put('name', 'TS1')

suiteProperties.put('description', '')

suiteProperties.put('stopImmediately', 'false')
 

DriverCleanerCollector.getInstance().addDriverCleaner(new com.kms.katalon.core.webui.contribution.WebUiDriverCleaner())
DriverCleanerCollector.getInstance().addDriverCleaner(new com.kms.katalon.core.mobile.contribution.MobileDriverCleaner())
DriverCleanerCollector.getInstance().addDriverCleaner(new com.kms.katalon.core.cucumber.keyword.internal.CucumberDriverCleaner())
DriverCleanerCollector.getInstance().addDriverCleaner(new com.kms.katalon.core.windows.keyword.contribution.WindowsDriverCleaner())
DriverCleanerCollector.getInstance().addDriverCleaner(new com.kms.katalon.core.testng.keyword.internal.TestNGDriverCleaner())



RunConfiguration.setExecutionSettingFile("/Users/kazuakiurayama/katalon-workspace/CreateATestSuiteByCode/Reports/20210602_094240/TS1/20210602_094240/execution.properties")

TestCaseMain.beforeStart()

TestCaseMain.startTestSuite('Test Suites/TS1', suiteProperties, new File("/Users/kazuakiurayama/katalon-workspace/CreateATestSuiteByCode/Reports/20210602_094240/TS1/20210602_094240/testCaseBinding"))

This must be what @gengland want to create for himself. Yes, you can write this code manually with any text editors!

kazurayam · June 2, 2021, 1:10am

I made a file <projectdir>/Include/scripts/groovy/MockTestSuite.groovy. Into it I copy&pasted the Libs/TempTestSuitexxxxxxxx.groovy . I changed the code slightly so that it becomes a Groovy class with public static void main() method.

import com.kms.katalon.core.logging.KeywordLogger
import com.kms.katalon.core.exception.StepFailedException
import com.kms.katalon.core.reporting.ReportUtil
import com.kms.katalon.core.main.TestCaseMain
import com.kms.katalon.core.testdata.TestDataColumn
import com.kms.katalon.core.testcase.TestCaseBinding
import com.kms.katalon.core.driver.internal.DriverCleanerCollector
import com.kms.katalon.core.model.FailureHandling
import com.kms.katalon.core.configuration.RunConfiguration
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import internal.GlobalVariable as GlobalVariable

class MockTestSuite {
	
	public static void main(String[] args) {
		Map<String, String> suiteProperties = new HashMap<String, String>();
		suiteProperties.put('id', 'Test Suites/TS1')
		suiteProperties.put('name', 'TS1')
		suiteProperties.put('description', '')
		suiteProperties.put('stopImmediately', 'false')

		DriverCleanerCollector.getInstance().addDriverCleaner(new com.kms.katalon.core.webui.contribution.WebUiDriverCleaner())
		DriverCleanerCollector.getInstance().addDriverCleaner(new com.kms.katalon.core.mobile.contribution.MobileDriverCleaner())
		DriverCleanerCollector.getInstance().addDriverCleaner(new com.kms.katalon.core.cucumber.keyword.internal.CucumberDriverCleaner())
		DriverCleanerCollector.getInstance().addDriverCleaner(new com.kms.katalon.core.windows.keyword.contribution.WindowsDriverCleaner())
		DriverCleanerCollector.getInstance().addDriverCleaner(new com.kms.katalon.core.testng.keyword.internal.TestNGDriverCleaner())
		RunConfiguration.setExecutionSettingFile("/Users/kazuakiurayama/katalon-workspace/CreateATestSuiteByCode/Reports/20210602_094240/TS1/20210602_094240/execution.properties")
		TestCaseMain.beforeStart()
		TestCaseMain.startTestSuite('Test Suites/TS1', suiteProperties, new File("/Users/kazuakiurayama/katalon-workspace/CreateATestSuiteByCode/Reports/20210602_094240/TS1/20210602_094240/testCaseBinding"))
	}
}

Then I made a Test Cases/MockTestSuiteRunner, which looks like:

import MockTestSuite

MockTestSuite.main()

I executed the Test Cases/MockTestSuiteRunner by clicking
GreenArrowButton
button. I found in the Console the famous greeting message again.

2021-06-02 09:47:56.407 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2021-06-02 09:47:56.414 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/MockTestSuiteRunner
2021-06-02 09:47:57.437 DEBUG testcase.MockTestSuiteRunner             - 1: MockTestSuite.main()
2021-06-02 09:47:57.551 INFO  c.k.katalon.core.main.TestSuiteExecutor  - START Test Suites/TS1
2021-06-02 09:47:57.553 INFO  c.k.katalon.core.main.TestSuiteExecutor  - hostName = kazuakiurayama - kazuakinoair.airport
2021-06-02 09:47:57.575 INFO  c.k.katalon.core.main.TestSuiteExecutor  - os = Mac OS X 64bit
2021-06-02 09:47:57.577 INFO  c.k.katalon.core.main.TestSuiteExecutor  - hostAddress = 192.168.0.9
2021-06-02 09:47:57.579 INFO  c.k.katalon.core.main.TestSuiteExecutor  - katalonVersion = 8.0.0.208
2021-06-02 09:47:57.678 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2021-06-02 09:47:57.678 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/TC1
2021-06-02 09:47:57.705 DEBUG testcase.TC1                             - 1: println("Hello, world!")
Hello, world!
2021-06-02 09:47:57.718 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/TC1
2021-06-02 09:47:57.968 INFO  com.kms.katalon.core.util.KeywordUtil    - Start generating HTML report folder at: /Users/kazuakiurayama/katalon-workspace/CreateATestSuiteByCode/Reports/20210602_094240/TS1/20210602_094240...
2021-06-02 09:47:58.015 INFO  com.kms.katalon.core.util.KeywordUtil    - HTML report generated
2021-06-02 09:47:58.016 INFO  com.kms.katalon.core.util.KeywordUtil    - Start generating CSV report folder at: /Users/kazuakiurayama/katalon-workspace/CreateATestSuiteByCode/Reports/20210602_094240/TS1/20210602_094240...
2021-06-02 09:47:58.018 INFO  com.kms.katalon.core.util.KeywordUtil    - CSV report generated
2021-06-02 09:47:58.151 INFO  c.k.katalon.core.main.TestSuiteExecutor  - --------------------
2021-06-02 09:47:58.157 INFO  c.k.katalon.core.main.TestSuiteExecutor  - END Test Suites/TS1
2021-06-02 09:47:58.157 INFO  c.k.katalon.core.main.TestSuiteExecutor  - ====================
2021-06-02 09:47:58.182 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/MockTestSuiteRunner

In the console, I found the Test Suite TS1 was executed successfully without any problem.

A funny thing here is that a Test Case executes a Test Suite, like a Uroboros!
from Wikipedia

Conclusion:

I found that it is possible to write a Groovy code that behaves as a Test Suite using text editor.
But I find it adds no value. I would like KS to generate Test Suites for me as usual.

gengland · June 8, 2021, 2:46pm

Most interesting. I’d not thought to look at the underlying groovy scripts themselves. Thanks for all your investigative work here. I’ll have to consider the pros and cons for how I move forward in what I’m doing.

Topic		Replies	Views
Test Suite creation (.ts) using script Katalon Studio integrations , katalon-studio	5	1066	November 5, 2020
Is there a way to execute one case in command line without creating suite? Archive integrations , katalon-studio	4	395	August 8, 2022
Test Suite Question Katalon Studio katalon-studio , web-testing	4	252	February 13, 2020
Adding test cases through command line Katalon Studio katalon-studio , web-testing	4	750	January 14, 2020
About Test Case and Test Suite Katalon Studio katalon-studio	2	314	April 10, 2020

Create test suite consisting purely of code?

Conclusion:

Related Topics