How to rename the pdf or html reports

[mqamar]

Hi,
We have a need to rename the pdf reports to more meaningful names. I have the code to rename any file but the problem is I can’t inject that code in
@AfterTestSuite method because reports are generated after @AfterTestSuite method has been executed. After doing some research I’ve created two listeners now. In the second listener I want to add the renamereports code in onFinish() method. My problem is I can never get the second listener to run.
My quesiton is how to trigger a method in second TestListener?
Any advice will be appreciated.

I’m expecting the results to be:

Running BeforeTestSuite
Running AfterTestSuite
Rename reports here

  //  Register the listener directly in the script
 @Listeners(RenameReports.class)

 class SetUpTearDownTestSuite {


		@BeforeTestSuite
		def sampleBeforeTestSuite(TestSuiteContext testSuiteContext) {
			
			System.out.println("Running BeforeTestSuite")
		}

		@AfterTestSuite
		def sampleAfterTestSuite(TestSuiteContext testSuiteContext) {
			
				
			System.out.println("Running AfterTestSuite")
			WebUI.closeBrowser()
		}	
}

**public class RenameReports implements ISuiteListener** {

		@Override
		public void onStart(ISuite suite) {
			// This is called before the suite starts
		}
		
		
		@Override
		public void onFinish(ISuite suite) {
					
			System.out.println("Rename reports here");		
		}
}

[kazurayam]

@mqamar

reports are generated after @AfterTestSuite method has been executed

That’s right.

@mqamar

I’m expecting the results to be:

Running BeforeTestSuite
Running AfterTestSuite
(Reports are generated by Katalon)
Rename reports here

No way. You cannot do it in a single run of a Test Suite.

---

However you can create 2 Test Suites and bind them to form a single Test Suite Collection. For example

TSC1
     +--- originalTS
     +--- followerTS

Here I assume that TSC1 will execute originalTS and followerTS sequentially.

When you execute the TSC1 (Test Suite Collection 1), the originalTS will run and finish with the PDF report is compiled, then followerTS will follow. Therefore the followerTS will have a chance to rename the PDF report that has been produced by the originalTS instantly before.

The only question for you will be how the followerTS can know the file path of PDF produced by the orginalTS. There could be a few choices.

If I were you, I will write a script that scans the <projectDir>/Reports/ folder and look up the sub-folder which was created latest. The folders under the Reports are named in the format of YYYYMMDD_hhmmss, so it is easy to identify the latest one.

---

Do you think that this is too much messy steps for such a small change (rename the pdf file)?

Yes, I think so. In my humble opinion, renaming the report generated by Katalon involves too much boring efforts. I would leave it as is.

[kazurayam]

You do not have to rely on the Katalon’s reporting. You may want to try Extent Report Plugin

Extent Report Integration

With this plugin, your TestListener in a Test Suite have a chance to compile a report for itself. Your code will name the report as it wants to.

Possibly Extent Report plugin would interest you more than wandering around the Katalon built-in reports.

Please note, this plugin requires an KS Enterprise license.

[mqamar]

We’ve lots of test suites and we run them in test suite collection. Let me try this, I’m not sure it’s going to be easy. Why can’t we just call second listener?

[mqamar]

Will this Extent report plugin run in Jenkins pipeline?

[kazurayam]

@mqamar

Why can’t we just call second listener?

I do not understad your question. I believe that Katalon Studio is able to call 2 or more TestListeners.

Let me show you an example.

I made “Test Listeners/TestListener2” and “Test Listeners/TestListener3” as follows

import com.kms.katalon.core.annotation.AfterTestSuite
import com.kms.katalon.core.annotation.BeforeTestSuite
import com.kms.katalon.core.context.TestSuiteContext

class TestListener2 {
	
	@BeforeTestSuite
	def sampleBeforeTestSuite(TestSuiteContext testSuiteContext) {
		println "TestListener2 started"
	}
	
	@AfterTestSuite
	def sampleAfterTestSuite(TestSuiteContext testSuiteContext) {
		println "TestListener2 ran and finished"
	}
}

import com.kms.katalon.core.annotation.AfterTestSuite
import com.kms.katalon.core.annotation.BeforeTestSuite
import com.kms.katalon.core.context.TestSuiteContext

class TestListener3 {
	
	@BeforeTestSuite
	def sampleBeforeTestSuite(TestSuiteContext testSuiteContext) {
		println "TestListener3 started"
	}
	
	@AfterTestSuite
	def sampleAfterTestSuite(TestSuiteContext testSuiteContext) {
		println "TestListener3 ran and finished"
	}
}

I wrote “Test Cases/TC1” as follows:

println "TC1 ran and finished"

I made “Test Suites/TS1” as follows

I ran the “Test Suites/TS1”, then I got the following output in the console:

11月 14, 2024 6:45:08 午前 com.kms.katalon.core.logging.KeywordLogger startSuite
情報: START Test Suites/TS1
11月 14, 2024 6:45:08 午前 com.kms.katalon.core.logging.KeywordLogger logRunData
情報: hostName = kazuakiurayama - localhost
11月 14, 2024 6:45:08 午前 com.kms.katalon.core.logging.KeywordLogger logRunData
情報: os = Mac OS X 64bit
11月 14, 2024 6:45:08 午前 com.kms.katalon.core.logging.KeywordLogger logRunData
情報: hostAddress = 127.0.0.1
11月 14, 2024 6:45:08 午前 com.kms.katalon.core.logging.KeywordLogger logRunData
情報: katalonVersion = 10.0.0.223
TestListener2 started

TestListener3 started
11月 14, 2024 6:45:09 午前 com.kms.katalon.core.logging.KeywordLogger startTest
情報: --------------------
11月 14, 2024 6:45:09 午前 com.kms.katalon.core.logging.KeywordLogger startTest
情報: START Test Cases/TC1
TC1 ran and finished
11月 14, 2024 6:45:09 午前 com.kms.katalon.core.logging.KeywordLogger endTest
情報: END Test Cases/TC1
TestListener2 ran and finished
TestListener3 ran and finished
11月 14, 2024 6:45:09 午前 com.kms.katalon.core.logging.KeywordLogger logInfo
情報: Start generating HTML report folder at: /Users/kazuakiurayama/katalon-workspace/healthcare-tests2/Reports/20241114_064503/TS1/20241114_064503...
...

I could see that the TestListener2 and TestListener3 ran and finished.

I mean, I could see that Katalon Studio called 2 TestListeners successfully.

So I do not understand what you wrote:

@mqamar

Why can’t we just call second listener?

---

By the way, in which order TestListener2 and TestListner3 will be executed? How can I change the sequence so that TestListener3 runs before TestListener2? ---- I don’t think it is possible. Katalon Studio determines the sequence. No way I can specify the sequence.

[kazurayam]

@mqamar

Will this Extent report plugin run in Jenkins pipeline?

I suppose Yes it should. But I haven’t examined it as I do not have an Katalon Runtime Engine license.

Extent Report is a light-weight Java library with a few number of external dependencies. The “Katalon Extent Report Plugin” does install the necessary jar files into the runtime classpath of a Katalon project and provide a set of adaptor classes that let your code (possibly a TestListener) to call the Extent Report classes to generate reports. It should run in Jenkins pipeline without any problem.

[kazurayam]

I copied & pasted the code fragment you provided in the original post.

The code does not compile, gave me a lot of compilation errors. So, no way it will run.

[mqamar]

I tried to go with the first option you suggested earlier (create a second testsuite after the first PDFs are ready) but later I realized it would be very hard to implement because some of the test suites have sub folders and others have sub folders of sub folders. This means each second test suite will be different. Too much work. I would like to get a confirmation from someone whether the Extent Report will work in Jenkins.
Sorry about the compiler errors, I didn’t add the import files or remove the bold **.
As for the multiple listeners, I understand we can have multiple listeners. I was trying to find out how can I run onFinish method of ISuiteListener? (assuming onFinish is executed after @AfterTestSuite)

Thank you for assisting me.

[kazurayam]

@mqamar

I would like to get a confirmation from someone whether the Extent Report will work in Jenkins.

Why not you just try it?
As long as you have an Enterprise license, you can try it with no additinal charge.

@mqamar

of ISuiteListener?

In the Katalon Javadoc, I could not find a name ISuiteListener. Is it your own custom class?

[mqamar]

ISuiteListener is a TESTNG class.

For ‘Extent Report’, you are right I should just try it myself. I’m working on installing the newer version of katalon first because ‘Extent Report’ is not compatible with my current katalon version (8.1.0).

[kazurayam]

@mqamar

TESTNG

Katalon Studio does not bundle the TestNG jar. You need to add it to resolve the compile errors.

But I am afraid it would be perplexing to hook into TestNG in Katalon Studio just to rename the reports compiled by Katalon. Compiling a report for yourself (using Extent Report) would be much intuitive, will result something more productive.

[kazurayam]

I would remind you that the Extent Report plugin requires you to modify all of your existing Test Cases.

See Extent Report Integration

• Customize Test Case: Enhance the test case customization by incorporating custom functions for capturing screenshots and logging.

In Test Case:
Call the custom function in test case to attach and take the screenshot.

CustomKeywords.'com.katalon.extent.report.ExtentReport.attachLog'("Add Test Description Here")
CustomKeywords.'com.katalon.extent.report.ExtentReport.addScreenshot'()

You have to replace
WebUI.comment("Add Test Description Here")
to
CustomKeywords.'com.katalon.extent.report.ExtentReport.attacheLog'("And Test Description Here").

You have to replace
WebUI.takeScreenshot(String)
to
CustomKeywords.'com.katalon.extent.report.ExtentReport.addScreenshot'().

So I think that the Extent Report plugin is applicable to a new project. It is not practically applicable to an existing project, especially the existing project is large.

One more note. Katalon Studio can take a screenshot and attach it to the reports automatically. However this automatism does not appy to the Exetent Report plugin. You have to wrtie your test cases so that it proactively catch any Exceptions and take screenshots, add the screenshots into the report. The required changes could be a lot.

Disappointed? Maybe.

[kazurayam]

Just for your reference, there was another long topic regarding the Extent Report plugin

Katalon and Extent Reports Integration

@mohit.kumar @kazurayam Hi Mohit, Below is the function i can see after placing the latest plugin given by you [image] but again no Extent folder is generated and it’s jumping to else condition i.e “To Generate an Extent Report execute tests from the Test Suite.”)" in your functions “public Object startEReport(TestCaseContext testCaseContext)” and “public void takeScreenshotFailure(TestCaseContext testCaseContext)” [image]

which concerns about how to name and locate the reports using this plugin. You can control it but a lot of works need to be done. Reporing feature is not easy as always.

[mqamar]

We don’t have any comments in the project or screen shots, but we do rely on failed test cases screen shots that are automatically created. Are you saying, once we go for extent report, we will loose the automatic failed test cases screenshots?

[kazurayam]

@mqamar

Are you saying, once we go for extent report, we will loose the automatic failed test cases screenshots?

Yes, I am saying so. Taking screenshots on failure is a feture builtin the Katalon’s Test Case Executor class. The Executor is ignorant of any external reporting frameworks such as Extent Report.

@mohit.kumar
you, the originator of the plugin. Any comment?

[mohit.kumar]

@mqamar

Hi,
We have a need to rename the pdf reports to more meaningful names. I have the code to rename any file but the problem is I can’t inject that code in

Katalon locks the project during test execution, which means you can’t rename the default report format at that time.

To address this, the Extent Report Plugin can be useful—it allows you to rename the report; however, it only generates HTML reports.

Additionally, the Extent Report Plugin automatically captures screenshots for any test failures. It also enables users to take screenshots manually during execution by calling the plugin’s screenshot keyword.

Hope this helps clarify!

[kazurayam]

@mohit.kumar

it only generates HTML reports.

The HTML file generated by Extent Report is a single self-contained *.html file. The html links to no external *.css files, *.js files, no img files to be associated with. An img will be base64-encoded into a long string and pasted in the html file. So the *.html file is portable. You can send the single *.html file wherever you want. You can attach it to email. You can open the received html file with browser. It would look fine. Therefore I do not think we need a file in pdf format.

@mohit.kumar

Am I right?

[mohit.kumar]

Yes, you’re absolutely correct.

[kazurayam]

@mohit.kumar

Additionally, the Extent Report Plugin automatically captures screenshots for any test failures.

Let me ask you, @mohit.kumar

In the doc Extent Reports integration | Katalon Docs, users are supposed to make a TestListener which has a @AfterTestCase-annotaed method, like

@AfterTestCase
def sampleAfterTestCase(TestCaseContext testCaseContext) throws IOException {
    CustomKeywords.'com.katalon.extent.report.ExtentReport.takeScreenshotFailure'(testCaseContext)
}

By this, a screenshot will be taken and attached to the Extent Report output when a Test Case failed and finished.

Do you mean this setup enables “Extent Report takes screenshot automatically on failure” ?

Or, does Extent Report plugin implements anothor way, other than TestListener, that listens to the test failures and react to them?