Headless mode Test cases passed in local URL but fails while running against VM URL

Web Testing
,
1

Hi All,

I have been using Katalon for about 3 months now.

The biggest challenge I am facing right now is that the tests in headless mode pass in local URL but fails while running against VM URL. The DB, data set and everything are the same in both environments.

Has anyone faced similar challenges?

Regards,

Murali

2 Likes
2

whats the error while running VM url

3

Share the error or any snapshot?

4

Yes, @muralikumar.us in a VM server setup there can be multiple contributing factors such as network latency, bandwidth constraints, infrastructure throttling, and shared resources. These issues frequently impact automation stability and often lead to flaky test behavior.

In Katalon, test flakiness is typically managed through workarounds rather than being inherently prevented by the framework design.

For example –> “Sometimes due to network issue element doesn’t become visible in expected time”

In this case the execution engine will not

:white_check_mark: Wait for DOM to update
:white_check_mark: Wait for network-driven rendering
:white_check_mark: Auto-retry polling internally
:white_check_mark: Retry action when element becomes ready

In case of Katalon, the test automation will suffer with the below steps

WebUI.click(findTestObject('SubmitButton'))

If:

  • Page is still loading

  • API response is delayed

  • JS rendering is incomplete

  • Element exists but not interactable

So solution to handle the issue in VM server may be just add either of the below listed steps before the action, to add some more room before the action

  • waitForElementVisible

  • waitForElementClickable

  • delay(2)

  • retry logic

  • custom keywords

  • pageload(10)

Try the above it will surely resolve your issue, or better elaborate your issue here

5

It is a frequently asked question: “my test passes in an environment A but fails in another environment B, why?”

Have a look at the following article:

Even if “The DB, data set and everything are the same in both environments”, your test will perform in different speed in the milli-seconds order. The speed difference may trigger a lot of errors such as 'NoSuchElementException ', 'ElementNotFoundException ', 'ElementNotVisibleException '. Possibly your test is not coded defensive enough for timing issue — it is quite likely to happen in Selenium. In short, your test is fragile; it is not robust enough. You need to change your test so that it does “wait for ***” defensively.

Unfortunately the Record Web utility of Katalon Studio does NOT generate any waitFor* statements automatically. The tool is not as sophisticated as you may expect. You are supposed to edit your tests after code generation and improve them.

How to rewrite your code? — It entirely depends on the code you have and the errors you got. If you want us to help more, you need to disclose your Test Case script and the error messages with Java Stacktrace you got.

2 Likes