# Custom Test Report compiled by TestListener (kazurayam-ks-plainreport) **URL:** \ **Category:** Tips and Tricks\ **Tags:** tips-tricks, katalon-studio\ **Created:** [December 24, 2020, 4:45am UTC](https://forum.katalon.com/t/custom-test-report-compiled-by-testlistener-kazurayam-ks-plainreport/50537 "2020-12-24T04:45:25Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![kazurayam](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kazurayam/32/333_2.png) [@kazurayam](https://forum.katalon.com/u/kazurayam)\ **Post date:** [December 24, 2020, 4:45am UTC](https://forum.katalon.com/t/custom-test-report-compiled-by-testlistener-kazurayam-ks-plainreport/50537/1 "2020-12-24T04:45:25Z")
I have made a demo project on GitHub and published it: > **[GitHub - kazurayam/kazurayam-ks-plainreport](https://github.com/kazurayam/kazurayam-ks-plainreport)** > > Contribute to kazurayam/kazurayam-ks-plainreport development by creating an account on GitHub. last edited at 30 Apr, 2021 * * * # Problem to solve In the Katalon User Forum, there was a [question](http://forum.katalon.com/t/get-results-html-path-filename-through-code/50526) that asked how to automatically copy the Test Reports in HTML/PDF format to another location once a test suite finished. In response to the question, I replied with a [post](http://forum.katalon.com/t/get-results-html-path-filename-through-code/50526/5) where I described my previous solution. To be honest, I am not very much happy with my previous solution. Russ Thomas replied a [post](http://forum.katalon.com/t/get-results-html-path-filename-through-code/50526/3) where he mentioned that, instead of bothering around the built-in Reports, he developed his own reporting functionality. So I was motivated to seek for what custom report I can develop in Katalon Studio. # Solution Katalon Studio provides a feature called [Test Listeners](https://docs.katalon.com/katalon-studio/docs/fixtures-listeners.html#test-listeners-test-hooks). If you make full use of the TestListener feature, you can compile your own reports of test execution with full control over contents/location/timing. You can compile report in any format you like. You can save the file wherever you want. I have made a skeletal implementation of my custom Test Report. The report is in JSON text format. It is plain; I mean the report is not ornamented at all. I have produced a jar file named `kazurayam-ks-plainreport-x.x.x.jar` for general use. You can download and use it in your own Katalon project. I used the [Gson](https://github.com/google/gson) library to _jsonize_ anonymous Groovy objects. Gson is bundled in Katalon Studio. # Description ## How to install the plugin - create your own Katalon Studio project - visit the [Releases](https://github.com/kazurayam/kazurayam-ks-plainreport/releases) page and download the latest version of kazurayam-ks-plainreport-x.x.x.jar` - save the downloaded jar file into your project’s `Plugins` folder - stop/restart Katalon Studio ## Add a Test Listener - In the `Test Listener` directory, create a new Test Listenner with any name. - Copy the sample code [Test Listeners/TLPlainReport](https://github.com/kazurayam/kazurayam-ks-plainreport/blob/master/Test%20Listeners/TLPlainReport.groovy) and paste it into you new Test Listener, save it to disk. - The `TLPlainReport` is common to every case. No code editing is required. This new Test Listener is required. ## Create your Test Cases and Test Suite. You can create your Test Cases and Test Suite as you like. For demo purpose, you can reuse the following test cases as example. - [Test Cases/TC1](https://github.com/kazurayam/kazurayam-ks-plainreport/blob/master/Scripts/TC1/Script1608774761027.groovy) - [Test Cases/TC2](https://github.com/kazurayam/kazurayam-ks-plainreport/blob/master/Scripts/TC2/Script1608774808886.groovy) Here I assume you will create a Test Suite named “TS1” which binds the TC1 and TC2. ## Running the demo Select the Test Suite “TS1” and run it. ## Demo output ### Output location Once the `Test Suites/TS1` finished, a new folder `/PlainReport` will be created. Inside it you will find a 3 files. - `execution0.log` - `messages...txt` - `report...json` The [`execution0.log`](https://github.com/kazurayam/kazurayam-ks-plainreport/blob/master/PlainReport/execution0.log) file is copied from the Reports folder just to for easier reference. In the log file you can find all messages emitted by your tests with a lot of additives including timestamp. The [`report.TestSuiteID.yyyyMMdd_hhmmss.json`](https://github.com/kazurayam/kazurayam-ks-plainreport/blob/master/PlainReport/plainreport.TS1.20210430_211217.json) file contains information from: - the TestSuiteContext object - the TestCaseContext objects - the `execution.properties` file in the `Report` folder The [`messages.TestSuiteID.yyyyMMdd_hhmmss.txt`](https://github.com/kazurayam/kazurayam-ks-plainreport/blob/master/PlainReport/messages.TS1.20210430_211217.txt) file contains the messages emitted by failed Test Cases in Java printStackTrace format. [![generatedReport](https://europe1.discourse-cdn.com/katalon/original/3X/e/8/e8001d9d987a4e45090cff3e4f24858eceae4620.jpeg)](https://github.com/kazurayam/kazurayam-ks-plainreport/blob/master/docs/images/generatedPlainReport.png) ### Console log If you look at the console log, you can find output like this: ```auto @AfterTestSuite Reports/20210101_133234/TS1/20210101_133234/execution0.log 30098bytes Reports/20210101_133234/TS1/20210101_133234/execution0.log.lck 0bytes Reports/20210101_133234/TS1/20210101_133234/execution.properties 2660bytes Reports/20210101_133234/TS1/20210101_133234/testCaseBinding 128bytes ``` This message proves that **2 files in the Report folder ( `execution.properites` and `execution0.log` ) are available at the event of `@AfterTestSuite` .** In these 2 files you can find almost all information out of Katalon Studio how the test suite was configured and how it ran. ## How the demo designed Please read the source of the project to find how the demo designed. 1. [`Test Listeners/TLPlainReport.groovy`](https://github.com/kazurayam/kazurayam-ks-plainreport/blob/master/Test%20Listeners/TLPlainReport.groovy) 2. [`Keywords/com/kazurayam/ks/plainreport/ReportGenerator`](https://github.com/kazurayam/kazurayam-ks-plainreport/blob/master/Keywords/com/kazurayam/ks/plainreport/ReportGenerator.groovy) 3. [`Keywords/com/kazurayam/ks/plainreport/PlainReport`](https://github.com/kazurayam/kazurayam-ks-plainreport/blob/master/Keywords/com/kazurayam/ks/plainreport/PlainReport.groovy) ---
**Author:** ![Russ\_Thomas](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/russ_thomas/32/11884_2.png) [@Russ\_Thomas](https://forum.katalon.com/u/Russ_Thomas)\ **Post date:** [December 24, 2020, 1:37pm UTC](https://forum.katalon.com/t/custom-test-report-compiled-by-testlistener-kazurayam-ks-plainreport/50537/2 "2020-12-24T13:37:55Z")
> [@kazurayam](#): > > But he did not describe how his code looks like. He/I did, many times. For one of those times, @devalex88 mentioned he may use it or a derivation of it in a future release (not sure if that’s still the case, though). There is, in my view, an ill-defined cut-off point when it comes to sharing code. My reporting system is well beyond the “it’s just too big” to share, pretty much only because it would take me way too long to document it clearly for a 3rd party to follow. And then there’s the support. I frankly don’t have the time. > [@kazurayam](#): > > So, I am afraid, the readers may feel lost where to go for developing custom reports as Russ did. Really? What a strange thing to say, especially under **Problem to solve** and while summing up the problem at that. I could go so far as to say that’s more than a little disingenuous - there is, as you well know, a search facility on the forum… > [@kazurayam](#): > > However, I would remind you that it would involve a lot of effort compiling your custom report. Exactly, like I said. And now I’m adding what I said above. _You_ have to support it. That is quite significant and the author of such a system should not consider it a trivial task. ---
**Author:** ![anon46315158](https://avatars.discourse-cdn.com/v4/letter/a/73ab20/32.png) [@anon46315158](https://forum.katalon.com/u/anon46315158)\ **Post date:** [December 24, 2020, 2:27pm UTC](https://forum.katalon.com/t/custom-test-report-compiled-by-testlistener-kazurayam-ks-plainreport/50537/3 "2020-12-24T14:27:37Z")
@Russ_Thomas @kazurayam i think sometime it is enough to just prove ‘it is possible’ and both of you did it. In the end it is a community forum, nobody can be enforced here to provide ‘out-of-the-box’ solutions. (or to share/document/maintain his own code) And IMHO, providing just guidance it is by far more constructive, let the end user explore and develop his own version. Lazy people should just open a feature request and wait for it to be considered. Now they know what to ask 😛 (or not) ---
**Author:** ![gengland](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/gengland/32/20464_2.png) [@gengland](https://forum.katalon.com/u/gengland)\ **Post date:** [December 28, 2020, 4:17pm UTC](https://forum.katalon.com/t/custom-test-report-compiled-by-testlistener-kazurayam-ks-plainreport/50537/4 "2020-12-28T16:17:33Z")
Thanks for taking the time to put this together, I’ll definitely have a play around with your idea and I think that it (along with your initial suggestion of having a test collection to copy the built-in results) are great workarounds for a feature that probably should be already available within Katalon out of the box 🙂 ---
**Author:** ![kazurayam](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kazurayam/32/333_2.png) [@kazurayam](https://forum.katalon.com/u/kazurayam)\ **Post date:** [March 18, 2021, 1:37am UTC](https://forum.katalon.com/t/custom-test-report-compiled-by-testlistener-kazurayam-ks-plainreport/50537/5 "2021-03-18T01:37:08Z")
I updated the demo project with tag 0.3 > **[kazurayam/CompilingCustomReportInKatalonStudio](https://github.com/kazurayam/CompilingCustomReportInKatalonStudio)** > > Contribute to kazurayam/CompilingCustomReportInKatalonStudio development by creating an account on GitHub. I changed my Custom Report in JSON to make it easier to read; the keys of JSON Objects are sorted in alpha-numerically ascending order while ignoring cases. Sample output: ```auto { "executedBrowser": "Chrome", "executionProfile": "default", "executionProperties": { "allowCustomizeRequestResponseSizeLimit": true, "allowCustomizeRequestTimeout": true, "allowUsingSelfHealing": true, "allowUsingTimeCapsule": true, "description": "", "execution": { "drivers": { "preferences": { "WebUI": {} }, "system": { "WebUI": { "browserType": "CHROME_DRIVER", "chromeDriverPath": "/Applications/Katalon Studio.app/Contents/Eclipse/configuration/resources/drivers/chromedriver_mac/chromedriver" } } }, "general": { "actionDelay": 0, "autoApplyNeighborXpaths": false, "defaultFailureHandling": "STOP_ON_FAILURE", "defaultPageLoadTimeout": 30, "enablePageLoadTimeout": false, "excludeKeywords": [ "verifyElementPresent", "verifyElementNotPresent" ], "executionProfile": "default", "ignorePageLoadTimeoutException": false, "methodsPriorityOrder": [ { "left": "XPATH", "right": true }, { "left": "BASIC", "right": true }, { "left": "CSS", "right": true }, { "left": "IMAGE", "right": true } ], "proxy": "{\"proxyOption\":\"NO_PROXY\",\"proxyServerType\":\"HTTP\",\"username\":\"\",\"password\":\"\",\"proxyServerAddress\":\"\",\"proxyServerPort\":0,\"exceptionList\":\"\",\"applyToDesiredCapabilities\":true}", "report": { "reportFolder": "/Users/kazuakiurayama/katalon-workspace/CompilingCustomReportInKatalonStudio/Reports/20210318_125423/TS1/20210318_125423", "screenCaptureOption": true, "videoRecorderOption": { "allowedRecordIfFailed": true, "allowedRecordIfPassed": false, "enable": false, "recordAllTestCases": false, "useBrowserRecorder": true, "videoFormat": "AVI", "videoQuality": "LOW" } }, "selfHealingEnabled": true, "terminateDriverAfterTestCase": false, "terminateDriverAfterTestSuite": true, "testDataInfo": {}, "timeCapsuleEnabled": false, "timeout": 30, "useActionDelayInSecond": "SECONDS", "xpathsPriority": [ { "left": "xpath:attributes", "right": true }, { "left": "xpath:idRelative", "right": true }, { "left": "dom:name", "right": true }, { "left": "xpath:link", "right": true }, { "left": "xpath:neighbor", "right": true }, { "left": "xpath:href", "right": true }, { "left": "xpath:img", "right": true }, { "left": "xpath:position", "right": true } ] }, "globalSmartWaitEnabled": true, "logTestSteps": false }, "host": { "hostAddress": "192.168.43.135", "hostName": "kazuakiurayama - 192.168.43.135", "hostPort": 53056, "os": "Mac OS X 64bit" }, "id": "Test Suites/TS1", "logbackConfigFileLocation": "/Applications/Katalon Studio.app/Contents/Eclipse/configuration/org.eclipse.osgi/73/0/.cp/resources/logback/logback-console.xml", "Name": "Chrome", "name": "TS1", "pluginTestListeners": [ "KatalonReportListener" ], "projectDir": "/Users/kazuakiurayama/katalon-workspace/CompilingCustomReportInKatalonStudio", "runningMode": "GUI", "sessionServer.host": "localhost", "sessionServer.port": 52680, "source": "/Users/kazuakiurayama/katalon-workspace/CompilingCustomReportInKatalonStudio/Test Suites/TS1.ts", "testops": {} }, "testCases": [ { "isMainTestCase": true, "isSkipped": false, "message": "", "testCaseId": "Test Cases/TC1", "testCaseIndex": 0, "testCaseStatus": "PASSED", "testCaseVariables": { "var1": "foo", "var2": 99 } }, { "isMainTestCase": true, "isSkipped": false, "message": "Test Cases/TC2 FAILED.\nReason:\ncom.kms.katalon.core.exception.StepFailedException: Failed intentionally\n\tat com.kms.katalon.core.util.KeywordUtil.markFailed(KeywordUtil.java:19)\n\tat com.kms.katalon.core.util.KeywordUtil$markFailed.call(Unknown Source)\n\tat TC2.run(TC2:6)\n\tat com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)\n\tat com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)\n\tat com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:398)\n\tat com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:389)\n\tat com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:368)\n\tat com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:360)\n\tat com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:255)\n\tat com.kms.katalon.core.main.TestSuiteExecutor.accessTestCaseMainPhase(TestSuiteExecutor.java:203)\n\tat com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:152)\n\tat com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:95)\n\tat com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)\n\tat com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)\n\tat TempTestSuite1616039663642.run(TempTestSuite1616039663642.groovy:39)\n", "testCaseId": "Test Cases/TC2", "testCaseIndex": 1, "testCaseStatus": "FAILED", "testCaseVariables": {} } ], "testSuite": { "status": "COMPLETE", "testSuiteId": "Test Suites/TS1" } } ``` ---
**Author:** ![kazurayam](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kazurayam/32/333_2.png) [@kazurayam](https://forum.katalon.com/u/kazurayam)\ **Post date:** [March 18, 2021, 7:31am UTC](https://forum.katalon.com/t/custom-test-report-compiled-by-testlistener-kazurayam-ks-plainreport/50537/6 "2021-03-18T07:31:56Z")
I have made a further improvement in [https://github.com/kazurayam/CompilingCustomReportInKatalonStudio](https://github.com/kazurayam/CompilingCustomReportInKatalonStudio) tagged as 0.4 In the JSON file, newlines and tab characters in a string must be escaped as `\n` and `\t` like this: ```auto "message": "Test Cases/TC2 FAILED.\nReason:\ncom.kms.katalon.core.exception.StepFailedException: Failed intentionally\n\tat com.kms.katalon.core.util.KeywordUtil.markFailed(KeywordUtil.java:19)\n\tat com.kms.katalon.core.util.KeywordUtil$markFailed.call(Unknown Source)\n\tat TC2.run(TC2:6)\n\tat com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)\n\tat com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)\n\tat com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:398)\n\tat com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:389)\n\tat com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:368)\n\tat com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:360)\n\tat com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:255)\n\tat com.kms.katalon.core.main.TestSuiteExecutor.accessTestCaseMainPhase(TestSuiteExecutor.java:203)\n\tat com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:152)\n\tat com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:95)\n\tat com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)\n\tat com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)\n\tat TempTestSuite1616039663642.run(TempTestSuite1616039663642.groovy:39)\n", ``` A message which contains newlines and tabs escaped in this way is very difficult to read. So, I extended my reporting script to create another file `messages_TS1.txt`. Its content is as this (as everyone would expect): ```auto Test Cases/TC2 FAILED. Reason: com.kms.katalon.core.exception.StepFailedException: Failed intentionally at com.kms.katalon.core.util.KeywordUtil.markFailed(KeywordUtil.java:19) at com.kms.katalon.core.util.KeywordUtil$markFailed.call(Unknown Source) at TC2.run(TC2:6) at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194) at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119) at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:398) at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:389) at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:368) at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:360) at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:255) at com.kms.katalon.core.main.TestSuiteExecutor.accessTestCaseMainPhase(TestSuiteExecutor.java:203) at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:152) at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:95) at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157) at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source) at TempTestSuite1616051889386.run(TempTestSuite1616051889386.groovy:39) ``` ---
**Author:** ![kazurayam](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kazurayam/32/333_2.png) [@kazurayam](https://forum.katalon.com/u/kazurayam)\ **Post date:** [March 18, 2021, 10:56am UTC](https://forum.katalon.com/t/custom-test-report-compiled-by-testlistener-kazurayam-ks-plainreport/50537/7 "2021-03-18T10:56:42Z")
I have made one more improvement in [https://github.com/kazurayam/CompilingCustomReportInKatalonStudio/](https://github.com/kazurayam/CompilingCustomReportInKatalonStudio/) tagged as 0.5 The output [**plainreport.XXX.yyyyMMdd\_hhmmss.json**](https://github.com/kazurayam/kazurayam-ks-plainreport/tree/master/PlainReport) file now includes GlobalVariables: ```auto { "executedBrowser": "Chrome", "executionProfile": "default", "executionProfile_GlobalVariables": { "DEBUG_MODE": false, "foo": "bar", "Password": "", "Products": [ 88, 99 ], "URL": "http://demoaut.katlaon.com/", "Username": "John Doe" }, ... ``` ---
**Author:** ![kazurayam](https://dub1.discourse-cdn.com/katalon/user_avatar/forum.katalon.com/kazurayam/32/333_2.png) [@kazurayam](https://forum.katalon.com/u/kazurayam)\ **Post date:** [June 9, 2021, 10:55pm UTC](https://forum.katalon.com/t/custom-test-report-compiled-by-testlistener-kazurayam-ks-plainreport/50537/8 "2021-06-09T22:55:06Z")
The version 0.6.0 of “kazurayam-ks-plainreport” publishes a plugable jar file. See the doc [README How to install the plugin](https://github.com/kazurayam/kazurayam-ks-plainreport).