2 gb is quite small, java is resource hungry.
katalon has by default xms set to 512 mb and xmx (max heap size) to 2 gb. now, figure out that only katalon himself can grab all available memory … but you need resources also for system processes.
you can try to limit the mem allocation on the docker container himself (note thar this will however affect the execution performance, but it can prevent the machine to crash. about katalon … dunno. it may crash too)
@ThanhTo will be nice to be possible to pass some java_opts via comandline execution to override such settings, to have a better control on katalon / java processess too.
any known way to do it?