Test Suite Collection: navigateToUrl() not working


#1

Hi,

I want to start my test suite in Browserstack via a test suite collection.

If I start the testsuite with the corresponding profile, it starts up fine remotely as well as locally.

The test suite collection starts also locally.

However, as soon as I want to execute the test suite, with the same profile and remote.properties, within a test suite collection the testlistener starts the browser instance successfully, but fails while calling the method navigateToUrl():

Test Cases/ProfileFunctionalities/Login-Logout FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Unable to navigate to 'null'
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:64)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:26)
	at com.kms.katalon.core.webui.keyword.builtin.NavigateToUrlKeyword.navigateToUrl(NavigateToUrlKeyword.groovy:83)
	at com.kms.katalon.core.webui.keyword.builtin.NavigateToUrlKeyword.execute(NavigateToUrlKeyword.groovy:67)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:56)
	at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.navigateToUrl(WebUiBuiltInKeywords.groovy:183)
	at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords$navigateToUrl$0.call(Unknown Source)
	at Login-Logout.run(Login-Logout:42)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
	at com.kms.katalon.core.main.TestSuiteExecutor.accessTestCaseMainPhase(TestSuiteExecutor.java:150)
	at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:123)
	at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:87)
	at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
	at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
	at TempTestSuite1571659961461.run(TempTestSuite1571659961461.groovy:35)
Caused by: java.lang.IllegalArgumentException: Url cannot be null or empty
	at com.kms.katalon.core.webui.keyword.builtin.NavigateToUrlKeyword$_navigateToUrl_closure1.doCall(NavigateToUrlKeyword.groovy:75)
	at com.kms.katalon.core.webui.keyword.builtin.NavigateToUrlKeyword$_navigateToUrl_closure1.call(NavigateToUrlKeyword.groovy)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:20)
	... 19 more

I pass a global variable with the URL behind it to the method and as I mentioned, it works fine usually.

All the subsequent test cases fail of course, as the start up fails.

I looked up the selenium logs and found the following exception:

org.openqa.selenium.remote.server.DriverServlet - Exception: Could not initialize class org.openqa.selenium.os.Kernel32

After some reading I altered the initial URL from an internal URL http://internaldomain.com to https://www.beuth.de, but this didn’t work either. Other quick solutions didn’t occur.

I already altered the logic of the testlistener, so that the initial navgation to the starting point occurs within the first test case and not within the listener, but the exception was thrown anyways:

So, I’m pretty stuck with this problem at the moment and I would be glad, if someone had any idey, how to solve this.

Best,
Florian


#2

@Florian_Braun That’s a great, well-worded report. :clap:

I have no idea what’s wrong, but these guys might:

@Ibus @ThanhTo @devalex88


#3

i never used browserstack but … since the problem seems it is not strictly related to it … let’s try a quick workaround.
@Florian_Braun if you hardcode the url value in the testcase, it will work? (or as as a testcase variable) (most probably will fail too)
could be a bug on how the globalvariable is resolved in a testcollection … or an unresolved assignment, therefore the ‘null’

LE: a sample code of the listener will help also to debug. could be also that the selenium driver is not initialized due to a wrong/limited scope/other bug … (i am guessing you call the driver constructor in the listener, and further in the testcase you simply call the .navigate() method)


#4

Hi @Ibus,
thanks fpr the hints. Tried something out and here are my results:

Yep. exactly.

I hardcoded the url - public and internal - in the testlistener, and it worked.

BUT: The execution with a custom and/or remote profile trying to connect to Browserstack still fails as I am calling some keywords within the testlistener, which are expecting properly initialized global variables, as well as in the test cases. Tried to hardcode them as well, and then it worked.

So I pretty much narrowed it down and maybe you were right @Ibus: The test suite collection does not seem to assign the value of a global variable to the variable itself correctly. Hence the failures.

Is this a known bug? I haven’t done research yet.

PS: My KS-Version is 6.3.0. These findings are only related to the useage of a remote/custom-capabilities profile whicht tries to connect to browserstack. Locally, everyting is fine.


#5

@ThanhTo @devalex88 @duyluong any hints here?
@Florian_Braun can you share the code of the listener too? if any sensitive data, you can censore it, we are interrested only on the functionality


#6

Hi,

thanks for the reply. Find below the code of the testlistener.

One more hint: I tried to pass to the openBrowser()-method the GlobalVariable (which was the original state), then I tried to pass it a single quoted, yet empty String (’’) and now I pass, as you can see, a double quoted, empty String. When I passed the Global Variable directly to the openBrowser()-method, the result and exception was the same.

import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject

import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testobject.TestObject as TestObject

import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile

import internal.GlobalVariable as GlobalVariable

import com.kms.katalon.core.annotation.BeforeTestCase
import com.kms.katalon.core.annotation.BeforeTestSuite
import com.kms.katalon.core.annotation.AfterTestCase
import com.kms.katalon.core.annotation.AfterTestSuite
import com.kms.katalon.core.context.TestCaseContext
import com.kms.katalon.core.context.TestSuiteContext

class TSuiteOpenCloseBrowser {
	/**
	 * Executes before every test suite starts.
	 * @param testSuiteContext: related information of the executed test suite.
	 */
	@BeforeTestSuite
	def sampleBeforeTestSuite(TestSuiteContext testSuiteContext) {
		println testSuiteContext.getTestSuiteId()

		WebUI.openBrowser("")
		
		WebUI.navigateToUrl(GlobalVariable.G_URL_Home)
		
		WebUI.click(findTestObject('BWR17_Overlay/span_acceptCookiesButton'))
		
		WebUI.maximizeWindow()
		
		CustomKeywords.'com.at.util.LoginLogout.login'(GlobalVariable.G_User1, GlobalVariable.G_PW_MwendlerG_User1)
		
		CustomKeywords.'com.at.util.FinalCheck.checkOverlay'()
		
		CustomKeywords.'com.at.util.LoginLogout.logout'()
		
		CustomKeywords.'com.at.util.LoginLogout.login'(GlobalVariable.G_User2, GlobalVariable.G_PW_User2)
		
		CustomKeywords.'com.at.util.FinalCheck.checkOverlay'()
		
		CustomKeywords.'com.at.util.LoginLogout.logout'()
		
		WebUI.navigateToUrl(GlobalVariable.G_URL_Home)
		
	}

	/**
	 * Executes after every test suite ends.
	 * @param testSuiteContext: related information of the executed test suite.
	 */
	@AfterTestSuite
	def sampleAfterTestSuite(TestSuiteContext testSuiteContext) {
		println testSuiteContext.getTestSuiteId()
		
		WebUI.navigateToUrl(GlobalVariable.G_URL_Home)
		
		CustomKeywords.'com.at.util.LoginLogout.login'(GlobalVariable.G_User1, GlobalVariable.G_PW_User1)
		
		CustomKeywords.'com.at.util.FinalCheck.checkOverlay'()
		
		CustomKeywords.'com.at.util.LoginLogout.logout'()
		
		CustomKeywords.'com.at.util.LoginLogout.login'(GlobalVariable.G_User2, GlobalVariable.G_PW_User2)
		
		CustomKeywords.'com.at.util.FinalCheck.checkOverlay'()
		
		CustomKeywords.'com.at.util.LoginLogout.logout'()
		
		WebUI.closeBrowser()
	}
}

#7

Florian, try this…

Take the code out of the suite listener, put it in a Keyword class/method. Call the method from the suite listener.

I had an issue with a global map which didn’t work directly in the suite listener but worked when moved to a class/method.


#8

Hi,
thanks for your reply @Russ_Thomas.

As far as I understand your proposed option, this won’t solve the problem. The problem is, that I cannot use any Global Variable in my whole testsuite (including test cases, keywords and testlistener), while trying to connect to browserstack. If I do so, the mentioned exceptions are thrown.

I tried it anyways to verify my assumption and commented out everything in my test listener, that used a Global Variable, so that basically only
WebUI.openBrowser("")

WebUI.maximizeWindow()

was left. The testlistner was passed. But as soon as a Global Variable was called within the test cases, or in a keyword, the exception is being thrown.

So the problem is not only restricted to the test listener, but the whole test suite, including test cases and keywords. Maybe I wasn’t that clear before. If so, sorry, for that.

So, do you have another possible solution?

Best,
Florian


#9

is this working fine with browserstack execution?
if yes, i think is an acceptable workaround to initiate the webdriver with empty url and further navigate to the desired page.
i usualy choose this workflow (altough my experience with UI testing is limited, i mostly do backend testing)

not sure how the driver is actualy instantiated when a remote profile is needed (i never used browserstack). here some devs may help … or perhaps @kazurayam, he is skilled with digging under the hood of libs.


#10

Yes, If I only execute the test suite with the remote profile and not the test suite collection, this works fine. Also with any other combination of method, variable and value.

In my attempts to find a workaround, I already tried this and it seems, that non of the Global Variables are assigned correctly when used within a test suite collection. Therefor, any keyword or method will fail, in which I reference them.


#11

can we see a screenshot of the test collection setup too? something does not fit …


#12

Hi,

unfortunately, I am on vacation now and can’t access the tests from here. A colleague of mine will post some screenshots soon and I will get back to you as soon as possible.

Best,
Florian


#14

Let me just make sure about a configuration of Test Suite Collection.

In the definition of a Test Suite Collection, you are supposed to assign a Execution Profile name to each Test Suites bundled in the collection, as the following screenshot shows.


The default value of “Profile” column is “default”. If you have your own Execution Profiles, you should explicitly rewrite the names.

@Florian_Braun

Are you sure you have assigned appropriate name of Execution Profile for Test Suite here?

If you leave the “Profile” to be something like “default” which does not include “GlobalVariable.G_URL_Home”, then the following code

        WebUI.navigateToUrl(GlobalVariable.G_URL_Home)

will certainly fail with a message Unable to navigate to null.


#15

this is why i asked for a screenshot. i remember it was a glitch in a certain version (not remember exactly which one) where that column is not visible so the user has to do a trick on the header. could be the case
@Florian_Braun please check if you have such set-up
also make sure the needed variables are defined in the selected profile


#16

@Florian_Braun here you go


image


#17

@Florian_Braun

Are you behind your organization’s Proxy?

If so, the following post might be interesting for you. There are many cases, with various reasons, which give you the message Unable to navigate to null.


#18

Hi all,

back in the office and I will try to summarize the issue:

The problem is, that I cannot use any Global Variable in my whole testsuite collection (including test suites, test cases, keywords and testlistener), while trying to connect to browserstack. If I do so, this exception is thrown:

Test Cases/ProfileFunctionalities/Login-Logout FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Unable to navigate to 'null'
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:64)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:26)
	at com.kms.katalon.core.webui.keyword.builtin.NavigateToUrlKeyword.navigateToUrl(NavigateToUrlKeyword.groovy:83)
	at com.kms.katalon.core.webui.keyword.builtin.NavigateToUrlKeyword.execute(NavigateToUrlKeyword.groovy:67)
	at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:56)
	at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.navigateToUrl(WebUiBuiltInKeywords.groovy:183)
	at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords$navigateToUrl$0.call(Unknown Source)
	at Login-Logout.run(Login-Logout:42)
	at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
	at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
	at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
	at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
	at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
	at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
	at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
	at com.kms.katalon.core.main.TestSuiteExecutor.accessTestCaseMainPhase(TestSuiteExecutor.java:150)
	at com.kms.katalon.core.main.TestSuiteExecutor.accessTestSuiteMainPhase(TestSuiteExecutor.java:123)
	at com.kms.katalon.core.main.TestSuiteExecutor.execute(TestSuiteExecutor.java:87)
	at com.kms.katalon.core.main.TestCaseMain.startTestSuite(TestCaseMain.java:157)
	at com.kms.katalon.core.main.TestCaseMain$startTestSuite$0.call(Unknown Source)
	at TempTestSuite1571659961461.run(TempTestSuite1571659961461.groovy:35)
Caused by: java.lang.IllegalArgumentException: Url cannot be null or empty
	at com.kms.katalon.core.webui.keyword.builtin.NavigateToUrlKeyword$_navigateToUrl_closure1.doCall(NavigateToUrlKeyword.groovy:75)
	at com.kms.katalon.core.webui.keyword.builtin.NavigateToUrlKeyword$_navigateToUrl_closure1.call(NavigateToUrlKeyword.groovy)
	at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:20)
	... 19 more

What works:

  1. Test suite that is run remotely on Browserstack and global variables stored in a profile.
  2. Test suite collection that is run with a locally installed browser instance and global variables stored in a profile.

What I already tried to narrow it down

  1. As the test suite is already running on Browserstack and the exception is not the same as in the problem behind a proxy I suppose, that this problem has nothing to do with it. Thanks anyways @kazurayam, this restriction is good to know about as well.
  2. The setup of the profiles and the test suite colltection is as far as I am aware correct, as you can see here. At least locally, it all works fine. The test suite for itself runs remotely with the profile as well fine.
  3. I tried to hardcode the values of the global variables that are stored in a profile in the test listener and test cases and the test suite collection worked not only locally, but also remotely (i.e. the URL can be resolved etc.).
  4. I tried to call a global variable not in the test listener (here’s the setup), but only in the test cases and the mentioned exception is thrown. This means, the problem is not restricted to the test listener. Test cases will throw the same exception.

My conclusion
This problem is only restricted to the combination of a

  1. test suite collection,
  2. Browserstack and
  3. the usage of global varibales stored in a profile.

So I guess, that this is the best guess at the moment:

@Ibus, @Russ_Thomas, @ThanhTo, @devalex88, @duyluong: Have you any hints, how to solve this?

Many thanks,
Florian


#19

for the sake of my mind sanity, have you checked if the right profile is set in your test collection?
the profile himself looks OK from your previous screenshot, but are you 100 % sure the right one it is used?
see the bellow pic:

in your sshots, you posted the profile with Stage something, but in the collection settings a different one is choosen (blah DEV) for the Remote execution


#20

Hi @Ibus,
yes this profile is a valid profile and works with every test suite.
Best,
Florian


#21

@Florian_Braun
i just wanted to be sure … sometime i am hit by such mistakes, therefore more eyes are always welcomed.
unfortunately i am not using browserstack so i cannot attempt to reproduce your issue … let’s wait for other people input.