[RE-OPENED] Reports are overwritten by retry run

[Said_A]

Hi guys,

i am using Jenkins to run my test cases with the following command:

xvfb-run -a ./katalon -noSplash -runMode=console -consoleLog -projectPath="/tests/katalon/Automation.prj" -reportFolder="Reports/regression" -reportFileName="report_$BUILD_NUMBER" -retry=1 -retryFailedTestCases=true -testSuitePath="Test Suites/Regression" -executionProfile="default" -browserType="Firefox (headless)"

I am writing the reports into the specific subfolder Reports/regression/ and have configured to retry the failed test cases.
My issue is that if a test case failes and will be retried, my report file only contains the retried cases. It looks like the report file from the “initial run” is simply overwritten.

Am i doing something wrong?

Best regards,
Said

[gontar95]

any solution ?

[devalex88]

Thank you for the question. I’ve moved this into Feature / Improvement Suggestions for consideration.

[gontar95]

My workaround for the moment (a custom Listner) :
Under the Folder Test Listeners you can create a new file which contains the Listners you want

import org.apache.commons.io.FileUtils



class Change_report_name_if_exist {
	
	@AfterTestSuite
	def printInfo(TestSuiteContext testSuiteContext) {

		
		

		Date today = new Date()
		String todaysDate = today.format('yyyy-MM-dd HH_mm_ss')

		
		FileUtils.moveFile(
			FileUtils.getFile("/path/JUnit_Report.xml"),
			FileUtils.getFile('/path/'+todaysDate+'.xml'));  
	
		  
}
}

[devalex88]

This suggestion has been implemented in recent versions.

[JFrame]

@devalex88 If I understand correctly, I don’t see this is fixed. I am running a collection with 2 suites in it, each with two testcases. I am running in console mode with retry set to one. If one of the two tests in either suite fails, it reruns that test, but at the end I get 2 report.html and one of them only shows the single test that was rerun, without the one that was passing the first time. So it sounds like Im still having the same issue where the original report is overwritten by the single test retry of that suite.

So is this fixed in some other way? maybe only in when run locally or something? If this was supposed to be resolved we may want to re-open?

I am using Katalon 7.0.9 docker image