Sauce Labs Integration


This is a companion discussion topic for the original entry at https://docs.katalon.com/katalon-studio/docs/saucelabs-plugin.html

Hello,

I was evaluating SauceLabs with Katalon ver. 6.3.3, using the Katalon plugin for SauceLabs. Though I was able to connect to SauceLabs and start the execution of tests, I faced issues like the execution of the test case wouldn’t stop and would just go on till max timeout set for the test case at SauceLabs. This, even after it shows execution being completed in Katalon Studio.
On the other hand, in the SauceLabs dashboard, I see that after the last instruction from Katalon, the execution doesn’t cease, unless the browser is closed; it keeps calling GET on the page title, till the timeout happens at SauceLabs.
If the browser is closed at the end in Katalon script, the execution shows as ‘Completed’ in SauceLabs but the status like ‘Passed’ or ‘failed’ is not displayed.
I checked with SauceLabs support but they say that the things are fine at their end and the test is somehow not finishing at Katalon’s end.
Please help!

Regards,
Vaibhav

I have a test that runs fine from Katalon Studio but when I send it through Sauce Labs it fails the step that is verifying an element is visible. I can see in the video playback that the element is actually visible. Can you tell me why this might be failing?

The error message in Katalon is:

12-12-2020 04:01:13 PM verifyElementVisible(findTestObject(“Header/a_MacArthur Foundation”), CONTINUE_ON_FAILURE)

Elapsed time: 2.286s

Object ‘Object Repository/Header/a_MacArthur Foundation’ is NOT visible (Root cause: com.kms.katalon.core.exception.StepFailedException: Object ‘Object Repository/Header/a_MacArthur Foundation’ is NOT visible

at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:64)

at com.kms.katalon.core.webui.keyword.builtin.VerifyElementVisibleKeyword$_verifyElementVisible_closure1.doCall(VerifyElementVisibleKeyword.groovy:83)

at com.kms.katalon.core.webui.keyword.builtin.VerifyElementVisibleKeyword$_verifyElementVisible_closure1.call(VerifyElementVisibleKeyword.groovy)

at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:20)

at com.kms.katalon.core.webui.keyword.builtin.VerifyElementVisibleKeyword.verifyElementVisible(VerifyElementVisibleKeyword.groovy:97)

at com.kms.katalon.core.webui.keyword.builtin.VerifyElementVisibleKeyword.execute(VerifyElementVisibleKeyword.groovy:67)

at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:73)

at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.verifyElementVisible(WebUiBuiltInKeywords.groovy:359)

at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords$verifyElementVisible$3.call(Unknown Source)

at Basic Page Test - Headless.run(Basic Page Test - Headless:27)

at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)

at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)

at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:339)

at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:330)

at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:309)

at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:301)

at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:235)

at com.kms.katalon.core.main.TestSuiteExecutor.accessTestCaseMainPhase(TestSuiteExecutor.java:191)

at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:141)

at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:90)

at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)

at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)

at TempTestSuite1607806801786.run(TempTestSuite1607806801786.groovy:39)

)

The error on sauce labs is:

COMMAND

POST execute_async

PARAMETERS

{
	"args": [],
	"script": "\tvar callback = arguments[arguments.length - 1].bind(this);\r\n\twindow.katalonWaiter.katalon_smart_waiter_do_ajax_wait(callback);"
}

RESPONSE

HTTP Status: 500

{
“message”: “A JavaScript exception occured: undefined is not an object (evaluating ‘window.katalonWaiter.katalon_smart_waiter_do_ajax_wait’)”
}

I guess, in your test case you have this now.

WebUI.verifyElementVisible(findTestObject(“Header/a_MacArthur Foundation”), CONTINUE_ON_FAILURE)

I would suggest inserting a line:

WebUI.verifyElementPresent(findTestObject(“Header/a_MacArthur Foundation”),
    30, STOP_ON_FAILURE)

WebUI.verifyElementVisible(findTestObject(“Header/a_MacArthur Foundation”), CONTINUE_ON_FAILURE)

Call to verifyElementPresent() will let your code wait for the element to appear in the DOM for maximum 30 seconds, that is just to make sure its presence before checking its visibility.

Thank you. I tried adding a check for the element first with a 30 second wait. The first test passes and the second fails as before. Again, this test passes just fine when run directly from Katalon Studio but fails running through sauce labs. I am using the default browser setting in the plugin (MacOS 10.13 / Safari 11.1)

A couple other issues I’m having…

I can’t see the saved sauce labs profile created through the plugin anywhere to name or edit. I only seem to be able to modify the default one. The article screen shot indicates the ability to name and save more than one. Can you help me see where to do that?

I haven’t been able to change the browser settings to anything other than the default. I can see in sauce labs what’s available but how those choices need to be entered into the sauce labs profile plugin in katalon is unclear.

Any guidance would be much appreciated. Thank you.

How about adding waitForElementVisible keyword?

WebUI.verifyElementPresent(findTestObject(“Header/a_MacArthur Foundation”),
    30, STOP_ON_FAILURE)

WebUI.waitForElementVisible(findTestObject(“Header/a_MacArthur Foundation”),
    30, STOP_ON_FAILURE)

WebUI.verifyElementVisible(findTestObject(“Header/a_MacArthur Foundation”), CONTINUE_ON_FAILURE)

I do not use Saurce Labs at all. I can not do anything about Source Labs issues, sorry.

My Sauce Labs issue was resolved by disabling “Default Smart Wait” in the project settings under Execution / WebUI

Thank you.

2 Likes

I have a diff. question
in Sauce Labs it shows the test name …what if I execute test suite from Katalon , how to show each test name from Katalon in sauce lab results or at lease group the results under the suite name?

As i wrote, I do not use Saurce Labs at all. I can not do anything about Source Labs issues, sorry.

The question is not about sauce labs …it is more about Katalon.

How do I manage the custom browser capabilities from the script instead of using the Project settings?

i have no idea.