Then, you would want to decrease the size of files in the “Report” folder. That may result less memory occupied.
If you have a payed license of Katalon Studio Enterprise, then you can disable “log executed steps”, which will significantly reduce the size of report files. The “step execution logs” occupies approximately 90% of bytes of report files; but that information is not useful (I suppose you do not need the step execution logs). See the following post also: