Making the Execution Profile printed for each testsuite in JUnit_Report.xml

Product Forums Katalon Studio
,
1

Hello,

I have Test Suites that may get executed through different execution profiles, as part of a single Test Suite Collection (TSC). But after execution, the global JUnit_Report.xml file does not indicate which execution profiles have been used in each sequence.

In other words, let’s say I have a TSC with 3 instances of the same myTS:
myTS - myProfile1
myTS - myProfile2
myTS - myProfile3

When this TSC is run, I get a global JUnit_Report.xml with all the relevant data on the three execution of myTS:

<testsuites name="myTSC" tests="30" failures="8" etc>
   <testsuite name="myTS" tests="10" failures="3" etc>...</testsuite>
   <testsuite name="myTS" tests="10" failures="1" etc>...</testsuite>
   <testsuite name="myTS" tests="10" failures="4" etc>...</testsuite>
</testsuites>

This is fine, as long as I know in which order my profiles were used.

But I would feel better if each testsuite block is somehow “tagged” with the relevant profile id. That would make my test run data safer for automated reporting.

For now, the very words “myProfile1”, “myProfile2”, “myProfile3” are printed nowhere in the JUnit_Report.xml.

I have explored several ways to achieve this, but I have not found a relevant solution.

The most promising hints I got are:

a) set the profileId as a property inside the test suite execution block, along with:
<testsuite name="myTS" tests="..." failures="..." etc>
<properties>
<property name="deviceName" value=""/>
<property name="devicePlatform" value=""/>
<property name="logFolder" value="..."/>
<property name="logFiles" value="...">
etc etc
</properties>
</testsuite>

(Either add a new property, or hack an unused, existing property, such as “devicePlatform”.)

b) get this done with coding some “BeforeTestSuite” or “AfterTestSuite” TestListener.

Please find below a code sample.

  • Pulling the execution profile works.
  • Yet I cannot access the TestSuiteLogRecord stuff
import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.logging.model.TestSuiteLogRecord

class PullingMyProfileId {
	/**
	 * Executes after every test suite starts.
	 * @param testSuiteContext: related information of the executed test suite.
	 */
	@AfterTestSuite
	def sampleBeforeTestSuite(TestSuiteContext testSuiteContext) {
		println ("The Profile is: " + RunConfiguration.getExecutionProfile())
		// WORKS: this is what I want printed in the XML
				
		println ("The TS is: " + testSuiteContext.getTestSuiteId())
		// WORKS, but irrelevant
		
		println ("The TestSuiteLogRecord's devicePlatform is: " + TestSuiteLogRecord.getDevicePlatform())
		// FAILS: I don't know how to get or set "log records" properties
	}

Has somebody already faced the same issue, and possibly came up with a simpler or more elegant solution?

Or, do you have some experience to share in accessing testsuite runtime log properties from code?

Thanks in advance.
– MichelG

2

Unfortunately, the built-in JUnit_Report.xml is NOT customisable at all. You can not change it.

Up until now, a few people have expressed their wishes for the Profile Name to be included in the built-in reports. For example,

But not supported yet.

3

I have developed my own execution report, which contains the name of the Execution Profile applied and the GlobalVariable=value pairs.

You might be interested in how to find out all GlobalVariables given (as the Katalon Stuio API does not provide that feature). Read the following source.

This helper class iterates over all properties in an instance of internal.GlobalVariables given to a test case run using Java Reflection API.

4

Thanks a lot for your suggestions.
In this one case, I have not been able to use it directly for this very issue.
But please be sure that suggestions you’ve made on other issues have been extremely helpful for me and my team.