Manual Generate HTML Report

Integrations
1

Hello, is there a setting to generate an HTML file so it can be accessed immediately before the test script is finished?
I’m having trouble. I manually integrated Jira with a listener and uploaded supporting documents to Jira. The only files my script can upload are .png and console0, but the HTML file isn’t found because it only appears after the script finishes running.

1 Like
2

Dear zedromason,

Thank you for your question, here I might have some recommendations which I have found:

Problem Analysis

Based on the documentation, HTML reports in Katalon are generated AFTER test execution completes, not during execution. According to the Katalon Studio Test Reports documentation:

“This feature enables auto-generation of reports in the selected format… generated after each test suite execution”

The HTML report file is not available until the test script finishes running. This is why your listener can only access.png files and console logs during execution, and these are generated in real-time, but the HTML report is compiled at the end.

Root Cause

This is by design in Katalon Studio. The HTML report requires complete test execution data to generate a comprehensive summary, so it cannot be created until all tests finish running.

Solutions

Option 1: Use Real-Time Monitoring with Katalon TestOps (Recommended)

Instead of waiting for the HTML report, integrate with Katalon TestOps for real-time monitoring:

Solution: Enable Katalon TestOps integration to view execution reports in real-time before the test finishes.

Source: Katalon TestOps Real-Time Monitoring

“For real-time monitoring and better reporting capabilities, consider integrating your project with Katalon TestOps.”

Option 2: Upload Available Files During Execution

Since your listener has access to.png and console logs during execution, you can:

  1. Collect intermediate artifacts (screenshots, logs) during test execution
  2. Upload these to Jira immediately via your listener
  3. Upload the final HTML report after execution completes using a post-execution hook

Implementation Example:

groovy

// In your listener - upload during execution
def onTestCaseFinished(TestCaseContext testCaseContext) {
    // Upload screenshots and logs to Jira immediately
    uploadScreenshotsToJira(testCaseContext)
    uploadLogsToJira(testCaseContext)
}

// After test suite completes - upload HTML report
def onTestSuiteFinished(TestSuiteContext testSuiteContext) {
    // Now the HTML report is available
    uploadHTMLReportToJira(testSuiteContext)
}

Option 3: Generate Custom Real-Time Reports

Create your own intermediate report format that updates during execution:

  • Use a JSON or CSV file that your listener updates after each test case
  • Upload this intermediate report to Jira during execution
  • Replace it with the final HTML report when execution completes

Reference: View Katalon Studio Test Results in Jira Tickets

So far, our recommendation will be:

Use Option 1 (Katalon TestOps) for the best real-time visibility, or combine Option 2 with your existing listener to upload intermediate results during execution and the final HTML report afterward.

Please let me know if I can further assist you,

Bella

3

Hi Bella, thanks for your response. Since TestOps uses a license, I’ll use option 2, and I think I’ve implemented it. However, the HTML file isn’t found in the logs I created. Is there something wrong with my script?

@AfterTestCase
    def afterTestCase(TestCaseContext testCaseContext) {
        long caseDuration = System.currentTimeMillis() - caseStartTime
        String testCaseId = testCaseContext.getTestCaseId()
        String status = testCaseContext.getTestCaseStatus()
        String simpleTestCaseId = extractSimpleTestCaseId(testCaseId)
        String message = testCaseContext.getMessage() ?: ""
        
        KeywordUtil.logInfo("Test Case Selesai: ${simpleTestCaseId}")
        KeywordUtil.logInfo("Status: ${status}")
        KeywordUtil.logInfo("Durasi: ${caseDuration / 1000}s")

        testCaseResults[testCaseId] = new TestCaseResult(
            testCaseId, 
            simpleTestCaseId, 
            status, 
            message, 
            caseDuration
        )
    }

    @AfterTestSuite
    def afterTestSuite(TestSuiteContext testSuiteContext) {
        long suiteDuration = System.currentTimeMillis() - suiteStartTime
        KeywordUtil.logInfo("Test Suite Selesai: ${testSuiteContext.getTestSuiteId()}")
        KeywordUtil.logInfo("Total Durasi Suite: ${suiteDuration / 1000}s")
        KeywordUtil.logInfo("Jumlah Test Case: ${testCaseResults.size()}")
        
        try {
            processTestResults()
        } catch (Exception e) {
            KeywordUtil.logInfo("Error dalam penanganan Jira di afterTestSuite: ${e.message}")
            e.printStackTrace()
        }
    }
1 Like
4

Your @AfterTestSuite method is missing the critical step: actually creating and writing the HTML file . You’re logging information but not generating the HTML output.

Hope this can be a Solution checking out this document here: Test Fixtures and Test Listeners (Test Hooks) in Katalon Studio | Katalon Docs.

Please let me know how that goes,

Bella

6

Based on the official Katalon Studio documentation, here’s the answer to your question about generating HTML files before test execution completes:

Problem Analysis

You’re trying to upload test reports to Jira during test execution using a listener, but the HTML report file isn’t available until after the test script finishes running. Currently, you can only upload .png files and console0.log, but not the HTML report.

Root Cause: HTML Reports Are Generated AFTER Execution

According to the official Katalon documentation on test reports:

“The generated reports are automatically created after test execution finishes.”

Key Finding:

  • :white_check_mark: Console logs (console0.log) are generated during test execution
  • :white_check_mark: Screenshots (.png) are captured during test execution
  • :cross_mark: HTML reports are generated AFTER test execution completes

This is by design in Katalon Studio. The HTML report requires the complete execution context (all test results, logs, screenshots) to be finalized before it can be generated.

Why HTML Reports Can’t Be Generated in Real-Time

The HTML report generation process requires:

  1. Complete test execution data - All test steps, results, and logs
  2. All screenshots and artifacts - Collected during execution
  3. Report aggregation - Combining all data into a single HTML file
  4. File finalization - Writing the complete report to disk

This cannot happen until the test suite execution is completely finished.

Solution: Access Console Logs During Execution

Since you need to upload files to Jira during test execution, use the console logs which ARE available in real-time:

Step 1: Get the Report Folder Path at Runtime

Use this code in your test listener to get the current report folder:

import com.kms.katalon.core.configuration.RunConfiguration

// Get the report folder path during execution
String reportFolder = RunConfiguration.getReportFolder()
println("Report folder: " + reportFolder)

Step 2: Access Console Logs During Execution

The console0.log file is written to disk during test execution:

import com.kms.katalon.core.configuration.RunConfiguration
import java.nio.file.Files
import java.nio.file.Paths

// In your test listener (e.g., afterTestCase)
def reportFolder = RunConfiguration.getReportFolder()
def consoleLogPath = Paths.get(reportFolder, "console0.log")

// Check if console log exists and read it
if (Files.exists(consoleLogPath)) {
    String consoleContent = new String(Files.readAllBytes(consoleLogPath))
    println("Console log available during execution: " + consoleLogPath)
    // Upload consoleContent to Jira here
}

Step 3: Upload to Jira After Execution Completes

For the HTML report, use an AfterTestSuite listener to upload it immediately after execution finishes:

import com.kms.katalon.core.configuration.RunConfiguration
import java.nio.file.Files
import java.nio.file.Paths

// In your test listener (e.g., afterTestSuite)
def reportFolder = RunConfiguration.getReportFolder()
def htmlReportPath = Paths.get(reportFolder, "index.html") // or the HTML report filename

// Wait a moment for the report to be fully written
Thread.sleep(2000)

// Check if HTML report exists
if (Files.exists(htmlReportPath)) {
    println("HTML report ready for upload: " + htmlReportPath)
    // Upload htmlReportPath to Jira here
}

Alternative: Generate Custom HTML Report During Execution

If you absolutely need an HTML report during execution, you can create a custom lightweight HTML report in your listener:

import com.kms.katalon.core.configuration.RunConfiguration

// In your listener
def reportFolder = RunConfiguration.getReportFolder()
def customHtmlPath = new File(reportFolder, "interim_report.html")

// Create a simple HTML report with current execution data
def htmlContent = """
<html>
<head><title>Interim Test Report</title></head>
<body>
<h1>Test Execution In Progress</h1>
<p>Report Folder: ${reportFolder}</p>
<p>Generated at: ${new Date()}</p>
</body>
</html>
"""

customHtmlPath.text = htmlContent
println("Interim HTML report created: " + customHtmlPath.absolutePath)
// Upload customHtmlPath to Jira here

Recommended Approach

Scenario Solution
Upload logs during execution Use console0.log in your listener
Upload screenshots during execution Access .png files from the report folder
Upload HTML after execution Use AfterTestSuite listener to upload the final HTML report
Need interim HTML during execution Create a custom lightweight HTML report in your listener

References

Key Takeaway: HTML reports are generated after test execution completes by design. For real-time uploads to Jira, use console logs and screenshots which are available during execution, then upload the final HTML report immediately after execution finishes using an AfterTestSuite listener.

1 Like
7

Please have a look at my essay:

My Test Listener could trigger the process that generate the katalon-built-in HTML report in its @AfterTestSuite-annotated method. With this, @zedromason should be able to post the katalon-built-in HTML report to JIRA.

1 Like
8

I think Katalon didn’t anticipate when they first developed the product that users would want to post-process HTML/CSV/PDF reports in their custom ways. So they designed the product as is.

I doubt it. Report creation is not rocket science. You don’t need to take it too seriously.

10

I read the source of the com.kms.katalon.core.main.TestSuiteExecutor class. Especially its execute method is most interesting, which is as follows:

    public void execute(Map<String, String> suiteProperties, File testCaseBindingFile) {
        this.suiteProperties = suiteProperties;

        eventManger.publicEvent(ExecutionListenerEvent.BEFORE_TEST_EXECUTION, new Object[0]);

        logger.startSuite(testSuiteId, suiteProperties);

        eventManger.publicEvent(ExecutionListenerEvent.BEFORE_TEST_SUITE, new Object[] { testSuiteContext });

        accessTestSuiteMainPhase(suiteProperties, testCaseBindingFile);

        setStatusTestSuiteContext(testSuiteContext);

        executeListenerAfterTestSuite();    // here, the PurgeHTMLReport will be executed

        cleanDriver();

        logger.endSuite(testSuiteId, Collections.emptyMap());

        eventManger.publicEvent(ExecutionListenerEvent.AFTER_TEST_EXECUTION, new Object[0]);

        resourceTrackerService.stop();
    }

I suppose that Katalon Studio GUI has an internal processing sequence as follows:

  1. Katalon Studio IDE calls com.kms.katalon.core.main.TestSuiteExecutor#execute() which does
  • call @BeforeTestSuite-annotated method of Test Listeners
  • execute all the bundled Test Cases by calling accessTestSuiteMainPhase(suiteProperties, testCaseBindingFile);
  • call @AfterTestSuite-annotated method of Test Listeners
  1. once TestSuiteExecutor#execute() finished, then Katalon Studio IDE starts generating the reports by calling com.kms.katalon.core.reporting.KatalonExportReportProvider#exportTestSuite()

This is what is possibly given to us.

If I add the PurgeHTMLReport TestListener in my project, what will happen?

I noticed the following “clearn up” statements:

        cleanDriver();

        logger.endSuite(testSuiteId, Collections.emptyMap());

        eventManger.publicEvent(ExecutionListenerEvent.AFTER_TEST_EXECUTION, new Object[0]);

        resourceTrackerService.stop();

My PurgeHTMLReport will effectively diregard these “clean up” statements and force generating the HTML report before the “clean up” are done. Then I got a question:

How much significant are these clean up statements for generating a healthy HTML report?

I think, these clean up statements are not significant for HTML report at all because

  1. At the statement accessTestSuiteMainPhase(suiteProperties, testCaseBindingFile);, all of Test Cases must have finished all test steps, done serializing the logs, the screenshots and any other artifacts.

  2. The WebDriver instances may be still open, but it would not matter for the reporting

  3. The resourceTrackerService is still running … I don’t know what it is but I guess it does not matter for HTML report.

So I am opportunistic that my PurgeHTMLReport would work without any problem.

I am not entirely sure, though, because the sources of Katalon Studio GUI (except the com.kms.katalon.core.** packages) are not disclosed public so that I can not read them.

11

Hi @zedromason ,

Happy new year. Have you got your issue resolved? If yes, feel free to share your workaround with us. If no, please give more information if possible. Thank you

1 Like