Katalon Studio 8.0.0 much slower in opening and saving files

@kazurayam @duyluong
You need to add one variable at a time to the variables tab, and save after each. Once you reach 5, it will start to be noticeably slower, The more you add, the slower… This does not happen in the 7.8.2 release, and I still need to verify it did not happen in the 7.9.x releases, but I do not recall any issues . I really notice it in the 8.0.x versions.

In Katalon Studio v7.9.1 I made new project, a Test Case where I copy & pasted the code you previous ly shared to us. In the variable tab, I defined 6 variables. I could close, open, edit it just quickly. I could not reproduce your problem.

I think you should create a minimum project where you can reproduce your slowness problem and violates no security concern; and then share it to us.

I did say 8.0.x.

I just confirmed that the 7.9.x releases do not have the issue either. It only started with 8.0.0, and still exists in 8.0.1

Create an empty test case in 8.0.x add variables one by one to the variables tab and save after each. From 5 onwards the save get slower and slower.

When a test case with many variables is opened, it takes a long time as well

@duyluong
@ThanhTo
@danpoleary

I could reproduce the slowness problem of Katalon Studio v8.0.x.
I used Mac OSX 11.4.
I compared Katalon Studio 7.9.1 and 8.0.1.
Here I attache a sample project zipped:

v8.0.x_Slowness_demo.zip (93.3 KB)

I created Test Cases/Slowness_demo.

It’s script is minimalistic:

import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

WebUI.openBrowser("http://demoaut.katalon.com/")
WebUI.delay(3)
WebUI.closeBrowser()

I added 20 variables in the Variables tab of the Test Case:

I double click the Test case to open it. I measured the time seconds taken.

With v7.9.1 it took 5 seconds or shorter. I saw no progress indicator displayed.

With v8.0.1 it took 15 seconds or longer. With v8.0.1 when I double click the file icon, an progress indicator sunadokei came up for seconds long enough to irritate me. With v8.0.1, I checked the .log file and found no Exception was raised.

It looks to me that v8.0.1 has become just slower than v7.9.1 when I try to open a Test Case with 20 variables defined in the Variables tab.


What I guess

According to the Relase note #8.0.x it includes:

  • Improve Performance: Load large projects faster and consume less memory during execution.

To me it looks the work done for performance improvement has a unforeseen side effect which causes the problem that @danpoleary pointed out.

2 Likes

In the demo project, I found a file: v8.0.x_Slowness_demo/Test Cases/Slowness_demo.tc. The file content looks as follows:

 <?xml version="1.0" encoding="UTF-8"?>
<TestCaseEntity>
   <description></description>
   <name>Slowness_demo</name>
   <tag></tag>
   <comment></comment>
   <testCaseGuid>742e496e-e28a-4404-8cd7-0d99cb680735</testCaseGuid>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>b693b8c4-3873-4219-bc2a-081118c88edf</id>
      <masked>false</masked>
      <name>variable</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>283b4410-94dd-4fa2-a73e-e1ef4c27c98d</id>
      <masked>false</masked>
      <name>variable_0</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>1c62f908-5bf8-4922-a9bc-225a8c88f938</id>
      <masked>false</masked>
      <name>variable_1</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>bbaf780c-8ed7-43be-9baf-c9da2786593a</id>
      <masked>false</masked>
      <name>variable_2</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>22b64e13-9fe9-4077-a31b-960972da95ba</id>
      <masked>false</masked>
      <name>variable_3</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>be3196cb-9677-4326-9234-a2325b5633fb</id>
      <masked>false</masked>
      <name>variable_4</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>b9648af0-6e58-4b1c-956b-28a2e70886c3</id>
      <masked>false</masked>
      <name>variable_5</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>30859709-b123-4872-bd6b-a318d24c033f</id>
      <masked>false</masked>
      <name>variable_6</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>03c2c73b-6e16-496c-98b0-03ccaa25c3d3</id>
      <masked>false</masked>
      <name>variable_7</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>4df96971-bc6a-4f72-ae86-36e343772ebc</id>
      <masked>false</masked>
      <name>variable_8</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>31534d86-f30e-4da3-b2fe-cf805d754ac4</id>
      <masked>false</masked>
      <name>variable_9</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>e4f7be89-1052-4c65-849c-b4f7d9695a94</id>
      <masked>false</masked>
      <name>variable_10</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>5fba34a3-b5cd-45c5-91ad-692a49489e24</id>
      <masked>false</masked>
      <name>variable_11</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>85b1809c-1a9e-41b3-af75-e5cd3e140a64</id>
      <masked>false</masked>
      <name>variable_12</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>08c1eb98-f8dc-4d45-a83e-a8c916aee0ad</id>
      <masked>false</masked>
      <name>variable_13</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>d243a5c5-d0df-477b-a28b-eeee4a58e310</id>
      <masked>false</masked>
      <name>variable_14</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>d897ad5f-9eae-441c-a427-56fc7d78d4ee</id>
      <masked>false</masked>
      <name>variable_15</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>1821fadf-0f81-4675-88fb-c9d25c9f3d0c</id>
      <masked>false</masked>
      <name>variable_16</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>fb50cac3-3c46-4a43-9d8e-420ad91104b8</id>
      <masked>false</masked>
      <name>variable_17</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>f889bed7-0652-4f8a-aa2a-1da9c881c8f0</id>
      <masked>false</masked>
      <name>variable_18</name>
   </variable>
   <variable>
      <defaultValue>''</defaultValue>
      <description></description>
      <id>7c5ba178-3b50-496a-9174-0e285f816e3e</id>
      <masked>false</masked>
      <name>variable_19</name>
   </variable>
</TestCaseEntity>

I can see what it is. The content of “Variables” tab is serialised into this file.

Just I guess, with no backings, that Katalon Studio v8.0.1 reads and parses this XML file 20 times when I double clicked the Test Case icon to open it in the Editor. It reads the file and recognise 1 entry, and forget it. It reads the file again and recognise the 2nd entry, and forget it. … continue until the end of the XML file.

@whisstuan.nguyen reported an behavior change of Xerces XML Parser in between 7.9.1 and v 8.0.1:

I’m not sure but could be related

@duyluong @kazurayam

I tried with the latest 8.0.5, and no luck. I also tried switching jre to Zulu from Oracle, and it also had no effect on the long open and saves of Test cases with many variables defined in the variable tab. All of our users will be staying with 7.9.1 until this is resolved.

Let me know if their is any other information I can provide.

Dan

1 Like

Good morning @duyluong @ThanhTo, Any news on this? Our project team is stuck on version 7.9.1 and we would like to take advantage of new features, fixes, and improvements in the latest releases.

Hi @danpoleary,

Create an empty test case in 8.0.x add variables one by one to the variables tab and save after each. From 5 onwards the save get slower and slower.

We will investigate this issue by following your steps. We hope to find out the root cause and release the fix soon

@kazurayam Thanks for your investigation.

Just for reference, I am seeing the same slow down in version 8.

If I try opening a test suite Katalon isn’t responding anymore for 2 minutes. 25% CPU (so 1 complete core) until something happens again. Katalon 8.0.5
yes, Katalon 8 isn’t usable for us. According to release notes performance should be enhanced but it is way way dicreased.

@Pascal_Donauer

Please provide the .log file

Katalon Studio logs

  • Windows logs folder: \config.metadata.log

  • macOS logs folder: file:///Applications/Katalon%20Studio.app/Contents/MacOS/config/.metadata/log

Copy.log (256.2 KB)

Hi @kazurayam

I opened this morning 07:15 German Time Katalon and it was finished after 4 minutes.
After that I opened Payment_Flow_Login test which took another 2 minutes.

This log is attached

I have a i5 8th gen, SSD and 16gb RAM.

Katalon was always very slow on my PC. But a lot faster than now. Also a colleague faced this problem now with Katalon 8.

Once @duyluong suggested deleting %KATALON_INSTALLED_DIR%/config/.metadata/.plugins/org.eclipse.core.resources

for other Similar case:

This would not be harmful much. Would you try it?

Hi @Pascal_Donauer,

I opened this morning 07:15 German Time Katalon and it was finished after 4 minutes.
After that I opened Payment_Flow_Login test which took another 2 minutes.

Does Payment_Flow_Login test case have variables or not?

Sorry for the late reply. I waited for a notification via mail but didn’t receive any.

So @kazurayam I deleted the folder but without any advantages in loading times.

@duyluong Yes! We use variables extensively. Payflow Login Test has 48 variables partially used multiple times in 10 sub test cases with additional sub test cases.

But I see the issue is know according to analysis of @kazurayam. I only thought it’s solved in 8.0.5 and i am not fully aware of what amout of variables is normal as I don’t know any other company which is using Katalon but us ^^

HI @danpoleary, @Pascal_Donauer, @jmartinelli, @kazurayam

We confirm that this is an performance issue on test case that has many variables since v8.0+. This ticket is fixed in the upcoming release v8.1.0.

Sorry for your inconvenience.

1 Like

I am interested what was the cause of this issue. Could you brief us what?

At least I want to know since which version of KS got this problem. KS 8.0.0? or earlier versions also?

@kazurayam

The root cause is very technical problem that relates to the function that displays test case variables. When calculating some expressions in Groovy, we need to use GroovyClassLoader that need to load the open project classpath. The classpath need to be cached instead of calculating every time to prevent performance issue whenever we use it. Our devs did a mistake when merging code of improving loading project performance. This maybe an issue of our process, we are sorry about that.

This issue happened in v8.0 onward and didn’t happen in v7.9.1 or earlier.

So, with KS 8.0.0 when we opened a test case with 20 variables defined, GroovyClassLoader was invoked 20 times. Is it correct understanding?