GC overhead limit exceeded & Heap memory Issue while running more mobile test cases

Product Forums Katalon Studio
,
1

We faced few memory issues while running our test suites in KSE, mostly after ~15 TC runs after upgrading our Android application to AndroidX[Updated with latest sdk’s]. The same scripts/test cases are working fine and stable in our non Android X version.

Following were the errors displayed:

  1. java.lang.OutOfMemoryError: GC overhead limit exceeded

2) java.lang.OutOfMemoryError: heap memory issue

In order to debug further, we did a memory profiling for both versions of application for the same scenario and following is attached here for your reference. Based on this, memory consumed by applications seems more similar.

Based on further investigations and research, we got a few insights from a community forum to increase Katalon memory in katalon.ini and to add launch arguments to clear GC by adding JVM parameters. Currently, we made changes according to that and restarted our execution to check further.

Changes made to katalon.ini:

Updated Xmx and Xms to following:

  • Xmx8192m

  • Xms1024m

Launch commands added in Katalon > Project settings:

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=./java_pid.hprof -XX:+UseGCOverheadLimit

Our machine specification is as follows:

  • 3.2GHz 6‑core 8th‑generation Intel Core i7 (Turbo Boost up to 4.6GHz)
  • 16GB 2666MHz DDR4
  • Intel UHD Graphics 630
  • 128GB SSD storage
  • Gigabit Ethernet (10/100/1000BASE-T Gigabit Ethernet using RJ-45 connector)

KSE, adb and Appium information:

  • KSE version 7.5.5
  • Appium 1.16.0
  • Android Debug Bridge version 1.0.41
  • Version 30.0.3-6597393

Note: We have tried with different versions of Katalon and Appium also, and the issue was still there.

We would appreciate it if you could share this finding with the team and let us know your feedback on this issue. As for our regression, we might be running more TC’s together and if this memory issue is happening again it would be a challenge for us.

Your findings will help us to identify whether we need to make some changes from our side(on application/Katalon) or we should await for a fix from your side.

2

Recently I struggled with Memory issue in my Katalon Studio project which takes screenshots of hundreds of URLs and do some post-processing over the image files. There were a lot of small issues that I needed to study. One of the most significant defect I made was this: https://github.com/kazurayam/VisualTestingInKatalonStudio/issues/13

My mistake was that my test script left a lot (some hundreds) of java.io.InputStream/java.io.OutputStream objects unclosed. This caused IO buffer inside JVM left unable to be garbage-collected, resulting huge increase of VM memory usage. I changed my scripts to close all the InputStream/OutputStream objects as soon as reading/writing finished. This improved things drastically.

1 Like
3

Hi @kazurayam,

Thanks for your response and for sharing the information about “Java VisutalVM”.
Regarding, “my test script left a lot (some hundreds) of java.io.InputStream/java.io.OutputStream objects enclosed” I am afraid we have any Input/output streams in our tests are we are not processing any huge data like the scenario you mention. And for closing the application sessions, I have added some commands to TestSuite and TestCase TearDown just to make sure the session is cleared as expected even though the test is failed. But, yet unfortunate to fix the issue :frowning:

4

you may wan’t to set xms to at least half, or even equal to xmx, otherwise you put a lot of pressure on the garbage collector.

5

for this statement … mhm. OK, you don’t have IO’s … but how many objects do you create in your testcases?
without a look at the full code, we can only do assumptions. my two cent’s at this moment is your code not being optimized. creating multiple instances vs reusing an instance of a certain class is the most known mistake with java (and not only in java)
think about your backyard, in time you throw there a lot of ‘no longer needed’ objects once you buy new stuff, but who will throw them away?
being more specific, the usual mistake with katalon is to create a huge object repository vs using parameterized objects. you may think are just safe to stay there, as long as you don’t use them they do no harm. but they do, they take your space :stuck_out_tongue:
also, using data-driven approach with very large input is also known to crash katalon. because java.

6

15 runs in the same session? or each run is a new session? the reporting plugin it is also famous for eating a lot of memory.

7

@anon46315158 Thanks for your sharing your insights, really appreciate that. Will try with increasing Xms1024m to half or equal to xmx, but the allocation should be done based on the available physical memory right? Our machine is having 16GB RAM.

Regarding point 2: We are reusing the objects based on parameterization rather than creating multiple objects. And one thing to highlight here is the same set of test cases works fine with our * non-android-X* builds(we never encountered this issue before as the scripts were running for 6+ months) with similar configurations. And in the framework setup also, we tried to reuse mainly rather than creating multiple instances as per the best practices

Point 3: Session you mean same test suite right? It’s a same test suite, but we clear the process after each testcase so each test case will have a new session(appium), and ~15 is the run which we did after this memory issue, previously more than 60 also worked fine

8

yeah, so 8GB for xmx and xms will give you enough space for the remaining processes.
you can fine tune it later, it is not an exact science, but xms 1 GB looks too small, this is why you get the garbage collector error.
setting xms higher (but not above xmx) will force JVM to allocate the memory from the beginning, not to wait / decide if more is needed, so GC will have more space to play.
you can try also 4 GB for both or other combinations, and keep an eye on the usage as long as the tests are running

9

Hi @nithuamz

Please share us more information by answering these questions:

  • How many concurrent test suite executions do you have?
  • Does your problem occur on both emulators and real devices?
  • When the crash happens, please observe the task manager, share us the CPU and RAM utilization of Katalon Studio, Java processes as child processes of Katalon Studio, node processes (Appium server)
10
  1. Wee are running test suites one after other, but the collection has 3 test suites which executes sequentially
  2. Yes occur in both
  3. Noted that
12

Hi,

We we were running scripts for older version these days because of the memory issue with new version. Will be executing the scripts in Android X version over this weekend and will share the memory usage information after that by Monday for your further investigation. Apart from that I hope all the necessary information have been provided from our side

13

Hi,

Please find the attached VisualVM log based on our last execution. Hope the data is sufficient. Please let us know if anything else is required.

Thank youheapmemory.zip (9.2 KB)

14

hi @rampraesath
are you facing the ‘exact’ same issue? i doubt. so no, your data is not sufficient

15

Yes @anon46315158, we are facing same issue(we are from same team). Could you please let us know what other data you require to debug?