Can I generate HTML Report in Test Listener? --- Yes, I can

I developed and published a GitHub project:


This project proposes a solution to the problem raised at the topic in the Katalon Community :

The original poster argued:

zedromason
Nov 12
Hello, is there a setting to generate an HTML file so it can be accessed immediately before the test script is finished?
I’m having trouble. I manually integrated Jira with a listener and uploaded supporting documents to Jira. The only files my script can upload are .png and console0, but the HTML file isn’t found because it only appears after the script finishes running.

Problem to solve

Katalon Studio generates a built-in HTML report after the Test Suite is entirely finished; after the @AfterTestSuite-annotated method in Test Listener finished. But I want to generate the built-in HTML report in the @AfterTestSuite-annotated method and post-process the file further (e.g., upload to JIRA).

Solution

A @AfterTestSuite-annotated method in my Test Listener should invoke the processing of generating the built-in HTML report. I would not rely on Katalon Studio to generate it. I will trigger it myself.

Solution Description

I developed a Test Listener Test Listeners/PurgeHTMLReport.groovy, which is as short as this:

import java.nio.file.Path
import java.nio.file.Paths

import com.kms.katalon.core.annotation.AfterTestSuite
import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.context.TestSuiteContext
import com.kms.katalon.core.logging.model.TestSuiteLogRecord
import com.kms.katalon.core.reporting.ReportWriterUtil
import com.kms.katalon.core.reporting.ReportWriterUtil.SuiteReportGenerationOptionsBuilder
import com.kms.katalon.core.setting.ReportBundleSettingStore

class PurgeHTMLReport {

	@AfterTestSuite
	def afterTestSuite(TestSuiteContext testSuiteContext) {
		String projectDir = RunConfiguration.getProjectDir()
		Path reportFolder = Paths.get(RunConfiguration.getReportFolder())
		Path exportLocation = reportFolder.resolve("test-suite-report.html")
		File report = exportTestSuite(exportLocation.toFile(), projectDir, reportFolder.toFile())
		println "HTML report: " + report.toString()
	}

	/**
	 * quoted from com.kms.katalon.core.reporting.KatalonExportReportProvider#exportTestSuite() with slight modifications
	 */
	File exportTestSuite(File exportLocation, String projectDir, File reportFolder) {
		def settings = ReportBundleSettingStore.getStore(projectDir).getSettings()
		assert reportFolder.exists()
		TestSuiteLogRecord testSuiteLogRecord =
			ReportWriterUtil.parseTestSuiteLog(reportFolder.getAbsolutePath())
		ReportWriterUtil.writeHTMLReport(
			SuiteReportGenerationOptionsBuilder.create()
				.suiteLogRecord(testSuiteLogRecord)
				.settings(settings)
				.reportDir(reportFolder)
				.outputFile(exportLocation)
				.build())
		return exportLocation
	}
}

With this Test Listener implemented, I ran a Test Suite. In the Reports folder, I could find a new file named test-suite-report.html was generated by my Test Listener PurgeHTMLReport:

The test-suite-report.html looked just fine. I found nothing wrong in the new HTML report.

I can do any post-processing over the new test-suite-report.html in my @AfterTestSuite-annotated method in the TestListener PurgeTestSuiteReport. I can upload the file to any external locations such as JIRA, GitHub Issues, Slack etc.

How I invented this solution.

Every Katalon studio installation contains the source codes of com.kms.katalon.core.* classes. You can find it in the $KATALON_STUIO_INSTALLATION_FOLDER/configuration/resources/source. I read the source code. of com.kms.katalon.core.reporting.KatalonExportReportProvider class. Especially the exportTestSuite method was most interesting, which looks as follows:

    @Override
    public File exportTestSuite(File exportLocation, String projectDir, String reportId, String formatType)
            throws IOException, URISyntaxException, XMLParserException, XMLStreamException {
        var settings = ReportBundleSettingStore.getStore(projectDir).getSettings();
        File reportFolder = new File(projectDir, reportId);
        TestSuiteLogRecord testSuiteLogRecord = ReportWriterUtil.parseTestSuiteLog(reportFolder.getAbsolutePath());
        switch (formatType) {
            case "HTML":
                ReportWriterUtil.writeHTMLReport(SuiteReportGenerationOptionsBuilder.create()
                        .suiteLogRecord(testSuiteLogRecord)
                        .settings(settings)
                        .reportDir(reportFolder)
                        .outputFile(exportLocation)
                        .build());
                return exportLocation;
            case "CSV (Summary)":
                ReportWriterUtil.writeLogRecordToCSVFile(testSuiteLogRecord, exportLocation,
                        Arrays.asList(testSuiteLogRecord.filterFinalTestCasesResult()), false);
                return exportLocation;
            case "CSV (Details)":
                ReportWriterUtil.writeLogRecordToCSVFile(testSuiteLogRecord, exportLocation,
                        Arrays.asList(testSuiteLogRecord.filterFinalTestCasesResult()), true);
                return exportLocation;
            case "PDF":
                TestSuitePdfGenerator pdfGenerator = new TestSuitePdfGenerator(testSuiteLogRecord);
                try {
                    pdfGenerator.exportToPDF(exportLocation.getAbsolutePath());
                } catch (JasperReportException e) {
                    logger.logWarning(ExceptionsUtil.getStackTraceForThrowable(e));
                    throw new IOException(e);
                }

                return exportLocation;
            default:
                break;
        }

        return null;
    }

I thought that Katalon Studio calls this method to generate the HTML report. The method signature was easy to understand. I would be able to write a @AfterTestSuite-annotated method in my Test Listener that implement the report generation in the similar way.

Required Katalon Version

The PugeHTMLReport class requires Katalon Studio v10.x. It will not work on v9.x and older.

Conclusion

So I tried implementing it. Done it. It looks working fine.

4 Likes

Woow! Congrats @kazurayam !

Looks amazing!

In the Reports folder, you got 2 .html files with just the same content. Duplicating!

If you do not need the one generated by Katalon Studio, by the project setting, you can stop Katalon Studio to generate the HTML report, as follows:

You deserve a medal sir

1 Like

My PurgeHTMLReport class has the following fragment:

The getSettings() method of the ReportBundleSettingStore class is available only since v10.x. The v9.x does not implement that method.

Therefore my PurgeHTMLReport class requires Katalon Studio v10.x and newer versions. It will not work on v9.x and older versions.

By the way, the HTML template was significantly changed (improved) as of v10.x. I think, you should not stick to the v9.x and older versions any longer.

I renamed the PurgeHTMLReport class to GenerateTestSuiteReports. It supports generating PDF report as well as HTML report.

import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths

import com.kms.katalon.core.annotation.AfterTestSuite
import com.kms.katalon.core.configuration.RunConfiguration
import com.kms.katalon.core.context.TestSuiteContext
import com.kms.katalon.core.logging.KeywordLogger
import com.kms.katalon.core.logging.model.TestSuiteLogRecord
import com.kms.katalon.core.reporting.ReportWriterUtil
import com.kms.katalon.core.reporting.ReportWriterUtil.SuiteReportGenerationOptionsBuilder
import com.kms.katalon.core.reporting.pdf.TestSuitePdfGenerator
import com.kms.katalon.core.reporting.pdf.exception.JasperReportException
import com.kms.katalon.core.setting.ReportBundleSettingStore
import com.kms.katalon.core.setting.ReportSettings
import com.kms.katalon.core.util.internal.ExceptionsUtil


class GenerateTestSuiteReports {
	
	@AfterTestSuite
	def afterTestSuite(TestSuiteContext testSuiteContext) {
		// generate test-suite-report.html file
		Path htmlReport = ReportGenerator.generateHTMLReport("custom/test-suite-report.html")
		println "HTML report: " + htmlReport.toString()
		
		// generate test-suite-report.pdf file
		Path pdfReport = ReportGenerator.generatePDFReport("custom/test-suite-report.pdf")
		println "PDF report: " + pdfReport.toString()	
	}
	
	
	
	/**
	 * 
	 */
	private class ReportGenerator {
		
		private static final KeywordLogger logger = KeywordLogger.getInstance(ReportGenerator.class);
		
		private static Path projectDir = Paths.get(RunConfiguration.getProjectDir()).toAbsolutePath()
		private static Path reportFolder = Paths.get(RunConfiguration.getReportFolder()).toAbsolutePath()
		private static ReportSettings settings = ReportBundleSettingStore.getStore(projectDir.toString()).getSettings()
		
		static void validateParams() {
			assert Files.exists(projectDir)
			assert settings != null
			assert Files.exists(reportFolder)
		}
		
		static void ensureParentDir(Path file) {
			Path parentDir = file.getParent()
			if (!Files.exists(parentDir)) {
				Files.createDirectories(parentDir)
			}
		}
		
		static Path generateHTMLReport(String htmlFileName) {
			validateParams()
			Path htmlReport = reportFolder.resolve(htmlFileName).toAbsolutePath()
			ensureParentDir(htmlReport)
			TestSuiteLogRecord testSuiteLogRecord = ReportWriterUtil.parseTestSuiteLog(reportFolder.toString())
			ReportWriterUtil.writeHTMLReport(
				SuiteReportGenerationOptionsBuilder.create()
					.suiteLogRecord(testSuiteLogRecord)
					.settings(settings)
					.reportDir(reportFolder.toFile())
					.outputFile(htmlReport.toFile())
					.build())
			return htmlReport
		}
		
		static Path generatePDFReport(String pdfFileName) {
			validateParams()
			Path pdfReport = reportFolder.resolve(pdfFileName).toAbsolutePath()
			ensureParentDir(pdfReport)
			TestSuiteLogRecord testSuiteLogRecord = ReportWriterUtil.parseTestSuiteLog(reportFolder.toString())
			TestSuitePdfGenerator pdfGenerator = new TestSuitePdfGenerator(testSuiteLogRecord);
			try {
				pdfGenerator.exportToPDF(pdfReport.toString());
			} catch (JasperReportException e) {
				logger.logWarning(ExceptionsUtil.getStackTraceForThrowable(e));
				throw new IOException(e);
			}
			return pdfReport
		}
	}
}