Yes, I do have a working code. It works w/o any problem in Sequential mode but I’ve seen one of the tests failing during login in Parallel mode (in the screenshot it looked like only @domain.com was entered as the user name instead of user@domain.com , as a result, the login failed) :
Well… That’s an interresting case.
My two cents here (without deep diving in code) is that, the issue is created by the ‘private’ stuff which are just instances to test objects.
Since the login method is using the private fields (note the btnSignIn stuff) , you may have a race condition here in parralel mode.
Just a guess…
So… Mhm. Here maybe static will help, since in theory should re-instantiate a new object…
However i dont have a full understanding how many instaces of the class are created without a look at the entire code
Sometime the devil is in details
Currently, the test collection has 3 tests each of which uses this login. As for how many instances of the class are created, I’m not sure how all these keywords work, when their classes get instantiated and how static in" theory should re-instantiate a new object".
You have this:
class Login {
TestObject buttonSignIn1 = ...
TestObject inputUserName = ...
TestObject inputPassword = ...
TestObject buttonSignIn2 = ...
def login(String userName, String password, booleann isPasswordEncrypted) {
WebUI.navigateToUrl(GlobalVariable.URL)
WebUI.click(buttonSignIn1, FailureHandling.STOP_ON_FAILURE)
WebUI.setText(inputUserName, userName)
if (isPasswordEncrypted) {
WebUI.setEncryptedText(inputPassword, password)
} else {
WebUI.setText(inputPassword, password)
}
WebUI.click(buttonSignIn2)
}
}
I found this code is fragile because it makes no “wait” for elements to manipulate.
When you run multiple test processes in parallel mode, it is likely that your Application Under Test (web server) slows down and becomes less responsive due to instantaneous high traffic. You should make sure each HTML nodes are ready to manipulate before every single WebUI actions. Something like this
def login(String userName, String password, booleann isPasswordEncrypted) {
WebUI.navigateToUrl(GlobalVariable.URL)
WebUI.waitForElementClickable(buttonSignIn1, 10)
WebUI.click(buttonSignIn1, FailureHandling.STOP_ON_FAILURE)
WebUI.waitForElementPresent(inputUserName, 10)
WebUI.setText(inputUserName, userName)
WebUI.waitForElementPresent(inputPassword)
if (isPasswordEncrypted) {
WebUI.setEncryptedText(inputPassword, password)
} else {
WebUI.setText(inputPassword, password)
}
WebUI.waitForElementClickable(buttonSignIn2, 10)
WebUI.click(buttonSignIn2)
}
If you make a search in this forum with keyword “parallel”, you can find a lot of posts that say “my test works fine in sequential mode, but it breaks in parallel mode”. In most of cases the root cause is, I suppose, that people do not write necessary “waitForXXXXX” lines.
Some other testing frameworks like Playwright does Auto-Waiting, but Katalon Studio requires you to code lines for “wait” explicitly where necessary.
By the way, I believe that the static modifier to the login() method is not relevant to your problem.
1 Like
Thank you for your reply, kazurayam. WebUI.waitForElementClickable(buttonSignIn1, 10), WebUI.waitForElementClickable(buttonSignIn2, 10) sound like a good idea. Not sure how ebUI.waitForElementPresent(inputUserName, 10) and WebUI.waitForElementPresent(inputPassword) can help. My understanding is that their effect would be no different from :
Also, there is no waits the Katalon sample project (Shoping Cart) 
If " Katalon Studio requires you to code lines for “wait” explicitly where necessary", should not they be used in the Katalon Sample?
You can see, their example has 1 line of “wait”.
WebUI.waitForElementVisible(findTestObject('Pages/MyAccount page/nav_HomeMyaccout'), GlobalVariable.waitPresentTimeout)
I suppose this single line of “wait” is effective enough. This waitForElementVisible() call ensures that the HTML has been fully loaded. Then we can be sure that the following 2 lines of sendKeys and a single click will run successfully. We need not to insert redundant lines of “wait” before sendKeys and click.
Your code is as this:
WebUI.click(buttonSignIn1, FailureHandling.STOP_ON_FAILURE)
// WebUI.waitForElementPresent(inputUserName, 10)
WebUI.setText(inputUserName, userName)
I guess, when the buttonSignIn1 is clicked, then a new HTML will be loaded. It will take certain seconds. Here you need to wait for the HTML to be fully loaded. Therefore you should insert a “wait” here for the inputUserName field present in the new page. If you do not make explicit wait until the page is fully loaded, then you would likely see something unsatisfactory like
only
@domain.comwas entered as the user name instead ofuser@domain.com
How can we write a robust test?
The biggest challenge in writing a robust test on top of Selenium WebDriver is in knowing the loading time for the web page. WebDriver looks at the DOM of web pages and that’s all it can do. WebDriver is not informed by browser of internal events of HTTP Request/Response sent/received. Therefore users of WebDriver can only do sufficient “wait” for the changes to appear in Page DOM.
1 Like
I noticed you have ever chatted with @Brandon_Hein about this at What exactly is "Default wait for element timeout (in seconds) " (in the Settings) and how exactly does it work?
I think that the Default Wait in KS does not apply to WebUI.setText().
I think that WebUI.setText(testObject) does not implicitly check and wait for the targeted HTML element <input type="text"> to be present and manipulatable. It immediately tries to manipulate it, and when the HTML element is not yet there, it fails.
In order to make your test robust, you should make explicit “wait” for the HTML node inputUserName which is subject to “WebUI.setText()”.
I think that KS should have internally applied the Default Wait to WebUI.setText() as well. There would be no harm in doing so. It would make easier for users to write stable tests — @gdearest07 you would have been helped. But KS is not designed as such. I regret that KS is not designed very consistent for “wait”. I think that newer testing framework Playwright is better in design for “wait”.
any comment?
1 Like
Thanks a lot, kazurayam, for your help! I never knew that was the case.
Sorry for the “off-topic”, do you happen to know what can be done about this as well? webui.autoUpdateDrivers=true is not working
No, I have no experience about this.
IMHO, I would rather doubt if this option works. If I were you, I would not use this option. I would look for other approaches.
I think I have 2 options to make my CI/CD environment run stable:
- freeze the version of browsers and webdrivers; turn off Chrome browser auto updates.
- use Docker where I fully control the versions manually.
Option 1 would be easy to implement. And I think it is most pragmatic.
As for option2, I would maintain day-by-day the Docker image for myself where I maintain browsers and WebDriver modules up to date. When I find my tests fail due to driver version problem, then I will stop my other jobs and update my Docker image. I would open KSE GUI and manually click the “Tools > Update Webdrivers > Chrome”, then build updated Docker image, upload the image to Docker Hub for distribution to my environments. Complicated? Yes, it is. But if you want to learn Docker, this would be a good small-sized practice.
1 Like