I have published a demo project in GitHub
I used Mac. This project assumes that OS command
/bin/sh is available, which is usually true on Mac and Linux. You should be able to rewrite this for Windows + PowerShell environment easily.
This project demonstrates how to automate making a zip file of Reports generated by a Test Suite execution.
Please download this project from https://github.com/kazurayam/Post_TestSuite_processing/releases , open it with your local Katalon Studio, run a Test Suite Collection named
TSC . Just run it to see what
- Open a Test Suite Collection named
Test Suites/TCS. It comprises with 2 Test Suites.
- Click the Execute button to run it. It will run for a few seconds.
TS1will run for quckly and pass;
TS_post_TS1_proocessingwill fail. It fails intentionally for demostration purpose. Don’t mind it.
TSCfinished, a folder named
outwill be created under the project directory.
However, Katalon Studio has a bug; the
outfolder, which is newly created by script, will not become visible in the Test Explorer pane. You need to close the project and reopen it. By doing so Katalon Studio will find the new
- In the
outfolder, you will find two files created by the
yyyyMMdd_hhmmssrepresent a varying time stamp.
- Please check the content of the
Reports_yyyyMMdd_hhmmss.zipfile using your favorite archiver software. In the zip file, you will find the test report in HTML/CSV/XML generated by the
TS1. For example, like this:
Test Suite/TS1, which calls a Test Case
TC1is a skeltal Test Case example, like this:
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI WebUI.comment("Here you can do whatever you like")
Problem to solve
Why I developed this demo project? Let me tell it you.
Katalon Studio users often want to make a zip file of the Reports of a Test Suite execution. If you don’t mind doing it manually after a Test Suite finished, you can:
- In Windows Explorer GUI, open the project folder
- Dig the folder tree to reach
- Right-click the folder and choose “archive it” menu.
- You are done. You will get a zip file which contains HTML/CSV/XML reports.
But I don’t like to do it manually. I want to automate zipping the Reports . Not only zipping, I want more. For example, I want to transfer the file to some remote storage for further processing.
How can I make it possible in a Katalon Studio project?
I found 3 technical issues that I had to overcome.
Katalon users naively expects that HTML/XML reports of a Test Suite
TS1 to be present and accessible in the Report folder as soon as finished. They expects the reports should be accessible in the
@AfterTestSuite -annotated method in a TestListener.
Unfortunately, it is not the case. When a
@AfterTestSuite -annotated method invoked, the HTML/XML reports are not present there yet.
I can show you an evidence.
When you run the Test Suite Collection
TSC , you should be able to find the following messages in the Console.
TS1ReportDirLister output ---------------------------------------------- executing: ls -la ./Reports/20211023_172734/TS1/20211023_172736 return code = 0 total 40 drwxr-xr-x 6 kazurayam staff 192 Oct 23 17:27 . drwxr-xr-x 3 kazurayam staff 96 Oct 23 17:27 .. -rw-r--r-- 1 kazurayam staff 2649 Oct 23 17:27 execution.properties -rw-r--r-- 1 kazurayam staff 11108 Oct 23 17:27 execution0.log -rw-r--r-- 1 kazurayam staff 0 Oct 23 17:27 execution0.log.lck -rw-r--r-- 1 kazurayam staff 64 Oct 23 17:27 testCaseBinding
This message is emitted by Test Listeners/TS1ReportDirLister. It executes a Command
ls -la ./Reports/20211023_172734/TS1/20211023_172736 in the
@AfterTestSuite -annotated method. As you can see, there is a bulky
execution0.log file but no HTML/XML reports yet.
The fact is, Katalon Studio will compile HTML/XML report after
@AfterTestSuite -annotated method finished. Therefore you can not make a zip of the report in the
@AfterTestSuite -annotated method after
In order to write a script to zip a folder, the script must know the path of target folder. On the other hand, Katalon Studio assigns the path of a report folder for
As you can see, this path has variable parts (time stamp). How can my script find the concrete value of the variable folder path?
How to run a Command from Groovy
In order to create a zip file of a folder contents, I want to use the good old bash shell commands:
echo etc. Also I want to use
curl command to transfer files over network.
I will create, by Groovy script (TestListener), a shell script file named
consume_TS_report.sh . Then, how can I execute that shell script with
/bin/sh (bash shell interpreter) from my Groovy script (TestCase)?
Resolving timing issue
You can not make a zip of the Test Suite
TS1 by TS1 itself. Possible solution is to create another Test Suite
TS_post_TS1_processing . You want to create a Test Suite Collection
TSC and let it call
TS_post_TS1_processing sequentially in this order. At the timing when
TS_post_TS1_processing is activated, the reports of
TS1 has been already compiled. The following Test Suites should be able to read the reports.
Resolving variable report folder path issue
com.kms.katalon.core.configuration.RunConfiguration class implements String getReportFolder() method. With this method call, we can get to know the report folder path of the current Test Suite.
Resolving how to run command from Groovy
java.lang.ProcessBuilder class enables running arbitrary commands in a new process forked from Groovy script. The following article covers how to use it.
However I found that ProcessBuilder is difficult to use for simple use cases. So I have developed a small wrapper for ProcessBuilder named
The jar is already bundled in the
Drivers folder of this Git project.
You can download the jar of
subprocessj from the Maven Central repository.
The tutorial of
subprocessj is here:
The source code of
subprocessj is hosted here:
When you run the Test Suite Collection
TSC and when the Test Suite
TS1 has finished, the Test Listeners/PostTS1Processor will be invoked. It has
@AfterTestSuite -annotated method. Please read the source.
What does the method do ?
- if the Test Suite
TS1has been finished,
- identify the path of report folder of
- in the
outfolder, create a file named
- write a few lines of bash commands. The commands will do the following:
- make a zip file of the report folder
- move the zip file into the
- try to transfer the zip file by HTTP POST request to a remote URL
out/consume_TS_Report.shis something like the following:
zip Reports_20211023_172736 -r "Reports/20211023_172734/TS1/20211023_172736" mv Reports_20211023_172736.zip "out/" # I know this would fail curl -X POST https://localhost:80 -F 'file=@out/Reports_20211023_172736.zip'
- do "
chmod +x consume_TS_report.sh" to change the file permission to make it executable in the OS command line.
out/consume_TS_Report.shscript file, I wrote a line of
curlcommand to transfer the zip file to a URL (https://localhost:80), which will inevitably fail as the URL is not alive. This is just an example. You can do anything you like using shell commands here.
The Test Suite Collection
TSC will call the Test Suite
TS_post_TS1_processing calls a Test Case
What this test case does? For the detail, read the source code.
- The test case script locates the
- It forks a new OS process to calls
/bin/sh(bash shell interpreter) while specifying the script file. In short, it execute the script file. It waits for the forked process to finish.
- When the forked process finished, the test case checks the return code. If the return code != 0, the test case fails.
- It prints the error messages from the forked process if any.