Execution start/end date time and duration of Test Suite Collection

Product Forums Katalon Studio
, ,
1

hi guys,

In the Test Suite Collection Report, I can’t find execution start/end date time and duration of it. There is a section gives the summary information of the test suite, but I want to know the summary information of the test suite collection (sum of execution duration of test suites in collection). Is there any way to get it?
Please give me your advise.

Thank you
BRs/Tomomi

2

I have created a GitHub project

Problem to solve

Katalon Studio provides a feature called “Test Suite Collection”. A Test Suite Collection is a collection of multiple Test Suites. You can execute the Test Suites as one batch by executing the Test Suite Collection.

Let me present an example. I have a Katalon Studio project where I have 2 Test Suites: TS1 and TS2 . And I have a Test Suite Collection named TSC . When I ran TSC , it generated outputs in the Reports folder, for example as follows:

$ cd Reports
:~/tmp/TestSuiteCollectionReportsCollector/Reports (master *)
$ tree .
.
├── 20220129_104341
│   ├── TS1
│   │   └── 20220129_104343
│   │       ├── 20220129_104343.csv
│   │       ├── 20220129_104343.html
│   │       ├── JUnit_Report.xml
│   │       ├── execution.properties
│   │       ├── execution.uuid
│   │       ├── execution0.log
│   │       ├── testCaseBinding
│   │       └── tsc_id.txt
│   ├── TS2
│   │   └── 20220129_104345
│   │       ├── 20220129_104345.csv
│   │       ├── 20220129_104345.html
│   │       ├── JUnit_Report.xml
│   │       ├── execution.properties
│   │       ├── execution.uuid
│   │       ├── execution0.log
│   │       ├── testCaseBinding
│   │       └── tsc_id.txt
│   └── TSC
│       └── 20220129_104341
│           └── 20220129_104341.rp
└── Self-healing
    └── broken-test-objects.json

8 directories, 18 files

In the Reports/20220129_104341/TS1/20220129_104343/JUnit_Report.xml file, I found:

<testsuites name="TS1" time="3.763" tests="1" failures="0" errors="0">

In the Reports/20220129_104341/TS2/20220129_104345/JUnit_Report.xml file, I found:

<testsuites name="TS2" time="3.782" tests="1" failures="0" errors="0">

But I could not find the sum of TS1 and TS2 duration. I had to manually calculate it. It was a poor experience.

Solution

I have developed a Custom Keyword. It looks into the Reports folder, identify the latest Test Suite Execution (TSC for short), find the JUnitReport.xml files of the Test Suites executed in the TSC, calculate the duration of the TSC in seconds, display the duration in the console. For example

image

This would be useful especially when a TSC contains many Test Suites and each Test Suites take long time to finish.

Description

  • It is assumed that you already have a Katalon Studio project with a Test Suite Collection.
  • download the the jar file of TestSuiteCollectionReportsCollector from the Releases page.
  • locate the jar file into the Drivers folder of your project
  • Stop and retart Katalon Studio to let it acknowledge the new jar. Re-open the project.
  • make a Test Case script of any name, that calls the custom keyword. See the sample code.
Double time = CustomKeywords."com.kazurayam.ks.reports.TestSuiteCollectionReportsCollector.execute"()
println time + " seconds"

Path outfile = Paths.get(".").resolve("stats.json")
CustomKeywords."com.kazurayam.ks.reports.TestSuiteCollectionReportsCollector.write"(outfile)
  • Execute your Test Suite Collection.
  • After the TSC, run the test case script, which will display the total duration of the TSC.
  • Similar information will be written into the <projectDir>/stats.json file.
[
{"name":"TS1","time":3.763,"tests":1,"failures":0,"errors":0,"URI":"file:/Users/username/project/Reports/20220129_081007/TS1/20220129_081008/JUnit_Report.xml"},
{"name":"TS2","time":3.782,"tests":1,"failures":0,"errors":0,"URI":"file:/Users/username/project/Reports/20220129_081007/TS2/20220129_081010/JUnit_Report.xml"},
{"name":"sum","time":7.545,"tests":2,"failures":0,"errors":0}
]
3

By the way, my code can NOT modify the Test Suite Collection Report compiled by Katalon Studio. If you want it, you should raise a feature request to Katalon team.

4

Thank you so much for your advice and github project info. It’s very helpful.
I was hoping for a feature like that, but understand that the feature does not exist at this time.