I have made a demo project on GitHub and published it:
last edited at 30 Apr, 2021
Problem to solve
In the Katalon User Forum, there was a question that asked how to automatically copy the Test Reports in HTML/PDF format to another location once a test suite finished. In response to the question, I replied with a post where I described my previous solution. To be honest, I am not very much happy with my previous solution. Russ Thomas replied a post where he mentioned that, instead of bothering around the built-in Reports, he developed his own reporting functionality.
So I was motivated to seek for what custom report I can develop in Katalon Studio.
Solution
Katalon Studio provides a feature called Test Listeners. If you make full use of the TestListener feature, you can compile your own reports of test execution with full control over contents/location/timing. You can compile report in any format you like. You can save the file wherever you want.
I have made a skeletal implementation of my custom Test Report. The report is in JSON text format. It is plain; I mean the report is not ornamented at all.
I have produced a jar file named kazurayam-ks-plainreport-x.x.x.jar
for general use. You can download and use it in your own Katalon project.
I used the Gson library to jsonize anonymous Groovy objects. Gson is bundled in Katalon Studio.
Description
How to install the plugin
- create your own Katalon Studio project
- visit the Releases page and download the latest version of kazurayam-ks-plainreport-x.x.x.jar`
- save the downloaded jar file into your project’s
Plugins
folder - stop/restart Katalon Studio
Add a Test Listener
- In the
Test Listener
directory, create a new Test Listenner with any name. - Copy the sample code Test Listeners/TLPlainReport and paste it into you new Test Listener, save it to disk.
- The
TLPlainReport
is common to every case. No code editing is required.
This new Test Listener is required.
Create your Test Cases and Test Suite.
You can create your Test Cases and Test Suite as you like.
For demo purpose, you can reuse the following test cases as example.
Here I assume you will create a Test Suite named “TS1” which binds the TC1 and TC2.
Running the demo
Select the Test Suite “TS1” and run it.
Demo output
Output location
Once the Test Suites/TS1
finished, a new folder <projectDir>/PlainReport
will be created. Inside it you will find a 3 files.
execution0.log
messages.<TestSuiteID>.<yyyyMMdd_hhmmss>.txt
report.<TestSuiteID>.<yyyyMMdd_hhmmss>.json
The execution0.log
file is copied from the Reports folder just to for easier reference. In the log file you can find all messages emitted by your tests with a lot of additives including timestamp.
The report.TestSuiteID.yyyyMMdd_hhmmss.json
file contains information from:
- the TestSuiteContext object
- the TestCaseContext objects
- the
execution.properties
file in theReport
folder
The messages.TestSuiteID.yyyyMMdd_hhmmss.txt
file contains the messages emitted by failed Test Cases in Java printStackTrace format.
Console log
If you look at the console log, you can find output like this:
@AfterTestSuite
Reports/20210101_133234/TS1/20210101_133234/execution0.log 30098bytes
Reports/20210101_133234/TS1/20210101_133234/execution0.log.lck 0bytes
Reports/20210101_133234/TS1/20210101_133234/execution.properties 2660bytes
Reports/20210101_133234/TS1/20210101_133234/testCaseBinding 128bytes
This message proves that 2 files in the Report folder ( execution.properites
and execution0.log
) are available at the event of @AfterTestSuite
. In these 2 files you can find almost all information out of Katalon Studio how the test suite was configured and how it ran.
How the demo designed
Please read the source of the project to find how the demo designed.
-
Test Listeners/TLPlainReport.groovy
-
Keywords/com/kazurayam/ks/plainreport/ReportGenerator
-
Keywords/com/kazurayam/ks/plainreport/PlainReport
Desired Reporting frameworks …
In the Katalon Forum, many people have expressed their wishes that they want to view the test reports using their favorites reporting frameworks. For example;
Yes, you can develop your code so that it satisfies your requirements by extracting necessary information and feed it to your favorites reporting framework. That is no different from what I have done here.
However, I would remind you that it would involve a lot of effort for you compiling nicely formatted custom report.