KS 8.1.0 stucked in running (after more than 5 days of running long haul test) - need help to generate partial report

well, of-course, one may use python too, provided is already installed.
just do:

$python -m SimpleHTTPServer <port_number>

in whatever folder you like.
no need to grab any third-party script, this module is already provided with a standard installation.

but the idea of the current topic was that the reporting part has to be improved, in regards with memory efficiency.
note that the final html report it is not generated until the test suite is ended gracefully.

Precisely. (He gets it !) We want it as it is being constructed, not wait until it’s flushed to disk - that’s the point. This requires new code in Katalon to flush the steps to disk, step by step (not TC by TC and certainly not per suite, which is what we have right now). I’d even settle for simple “outcomes” - pass/fail/error.

Great post though, @kazurayam. I’m sure it will help somebody. But like @bionel said, browsing local folders is easy. I use file:/// stuff all the time – I have seven tiddlyservers (node) open right now.

Katalon Studio emits runtime logs into execution0.log file.

$ pwd
/Users/kazurayam/katalon-workspace/ReportWriterDriver
:~/katalon-workspace/ReportWriterDriver (master #)
$ tree Reports -s
Reports
├── [         96]  20211117_145442
│   └── [         96]  TS1
│       └── [        320]  20211117_145442
│           ├── [     197916]  20211117_145442.html
│           ├── [       1803]  JUnit_Report.xml
│           ├── [       2663]  execution.properties
│           ├── [         36]  execution.uuid
│           ├── [     127247]  execution0.log   <====== THIS!
│           ├── [        144]  testCaseBinding
│           ├── [          0]  tsc_id.txt
│           └── [        128]  videos
│               ├── [     861216]  screen_recording_1_0.avi
│               └── [      10079]  screen_recording_1_0.srt
└── [         96]  Self-healing
    └── [         31]  broken-test-objects.json

5 directories, 10 files

The content of the execution0.log file is something like this:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE log SYSTEM "logger.dtd">
<log>
<record>
  <date>2021-05-31T11:54:33</date>
  <millis>1622429673295</millis>
  <sequence>0</sequence>
  <level>START</level>
  <class>com.kms.katalon.core.logging.XmlKeywordLogger</class>
  <method>startSuite</method>
  <thread>1</thread>
  <message>Start Test Suite : Test Suites/test/All</message>
  <nestedLevel>0</nestedLevel>
  <escapedJava>false</escapedJava>
  <property name="name">All</property>
  <property name="description"></property>
  <property name="id">Test Suites/test/All</property>
  <property name="stopImmediately">false</property>
</record>
...

The execution0.log file is generated by the Java Standard Logging framework: java.util.logging.XMLFormatter etc.

The content of execution0.log file is exactly the same as what we see in the Log Viewer in the Katalon Studio GUI. This means, the execution0.log file contains the on-going results as a test suite is running.

Katalon Studio transforms the execution0.log file to what we call “Basic Report as HTML” and “JUnitReport as XML”.

Therefore, if you want a means to access on-going results as a test/suite is running, then you want 2 things:

  1. You want to trigger Katalon Studio to do execution0.log --> HTML report transformation intentionally even when a Test Suite has crashed for some reason.

  2. When you forcefully stoped Katalon Studio because the Test Suite got frozen, then the execution0.log file might become a mal-formed XML document. For example,

    • the file may miss the closing </log> tag at the end of the file
    • a <record> tag may be broken at the end of the file.

Possibly, the execution0.log file is generated when a Test Suite has finished (regardless with failure or not). I am not sure how the log file will be generated when the OS process was stopped irregularly.

Do you want Katalon studio to be capable of transforming a mal-formed execution0.log to a well-formed Basic HTML report? I do not think it is possible to implement.

A super-skillful Java programmer may be able to write a purifier tool that accept a mal-formed execution0.log file to make it a well-formed XML. Then the magic may become possible. — sorry, I am not capable.

In addition to what has been said above, perhaps I have to be a bit more precise.
When I am saying ‘reporting’ I refer also to the log viewer.
And that i know for sure it is memory hungry, since it is populating the GUI ‘on the fly’ but seems like it is not releasing the memory until the test finishes (or sometime not at all).
So the log viewer definitely need some love.

1 Like

Yes, it is.

as @duyluong wrote yesterday as follows, the Log Viewer seems to have problem.

A Katalon Studio Enterprise license holder can disable the “step execution logs”, but this option is not available to the Katalon Studio Free license holder. I am uncomfortable about this design decision. In Dec 2020, I wrote that Katalon should change this option to be free.

But the team has not taken this seriously.

I am afraid, I should advise those who want to drive a large scale test (like you @david.casiano) should not use Katalon Studio Free version. They need to pay for the Enterprise license. Otherwise, they should choose other UI-testing software.

@Jass

Let me tell this you. I think this was a bad design decision.

And this is were we are starting to converge.
This option it is available only for enterprise users, as you correctly pointed.

And, even if this will be made available for free version, the user may still like to have an option to watch the reports ‘on the fly’, particularly for such cases with large datasets / long running suites.

So, no matter if i am an enterprise user or free user, I should have the option to turn off the log viewer, but refreshing the browser some time to time (I won’t mind doing this manually)

This is why both requests should be considered.

Of-course, running the tests with KRE may be a solution (since the console runner shouldn’t bother with the log viewer) but KRE … guess what, it is yet another licensed only feature …

to the product owner of Katalon Studio, @Jass

Let me tell you, I see Katalon Studio is not good for large scale testing.

I am already on the ‘dark-side’ (pyhton + robot framework).
And actually … funny, the main reason for this was not the development inconsistencies and licensing schema … but something more serious for me … the Linux support.
Yet again, starting from 7.9.0, the linux version is broken, at least on Fedora, due to the eclipse upgrade most probably.
But I won’t bother to submit another bug report, looks like the Linux usage it is simply ignored … some people are thinking that Linux is running only in the cloud …

Have you ever reported a bug report of Katalon GUI on Fedora? Which URL? Let me bookmark it!

Is the Eclipse project on a decline nowadays?

here is one of them (fixed by accident by a previous eclipse upgrade):

Is Eclipse project on a decline nowadays?

In theory, upgrades to newest eclipse engines should fix issues.
But in practice, for katalon just breaks more stuff.

The main reason is, I think, Linux development seems to be targeted for some ancient ubuntu only distribution, but does not consider some other big players on the linux desktop scene.
Therefore, lack of testing …

“Linux development” of what?

Do you mean “Katalon Studio GUI for Linux”? or “Eclipse for Linux desktop”?

yes, I am refering to Katalon Studio GUI for Linux.
of-course, could be that the root causes are due to the changes in eclipse engine himself (or at least in the gtk runner) but still, the final product should be tested on different environments, at least the important ones.

you know ‘It is working on my machine …’
well, we don’t ship your machine …

@binel

Thank you for sharing

I know some people want to go for other ‘dark-side’ (TypeScript + Playwright).

… well, i should stop this off-topics.

I have also somewhere a POC project using groovy + spock which is mimicking the ‘global variable profile’ behavior.
It can run both with Maven or Gradle.
I created that for API testing just to have it at hand in case everything will go wrong with katalon but i have to quickly migrate to a java/groovy opensource solution.
In the mean time I moved to another company which is more python oriented.

That can be easily extended with Geb to have a ui own framework at hand too.

:stuck_out_tongue:

GEB + Spock + WebDriver + JUnit + Gradle … that is my favourite tools combination as well. This tool set requires solid skill for Java programming.

A few years ago I used to develop UI tests with this golden tools combination while I worked for my one-man project. Later I was assigned to work in a team where most of the members are non-programmers. I found it is impossible to ask them to use this tool set. So I switched to Katalon Studio. KS was easy to introduce to them. KS got them feel started with UI test automation.

I will send what i did in a private message, just not to offend somebody.
It is kind of old, i did that 2 years ago but feel free to use it for your inspiration.

@bionel

Let’s suspend our off-topics.

We should to go back to the original issue of @david.casiano.