Request: More reliable "Record" feature, global variables that work across multiple tests, test run speed control

David_Eckert · 2019-02-01 16:03:25 UTC

I tried Katalon Studio more than a year ago when I could no longer get Selenium IDE to work in Firefox. Back then I made numerous attempts to import my test cases into Katalon with virtually no success.

I just updated to v. 5.10.1 and see new options for importing test cases. Again, no luck. I can import them but then get get errors without explanation as to what went wrong. I’m just told to fix the errors.

I then tried the “Record Web” option (with Chrome v.71.0.3578.98). I tried to record simple steps such as logging into my asp.net application then navigating to a particular area. But when I tried to run the test it fails at the navigation step.

I also tried to replace my typed in username and password field values with both local and global variables but neither of those worked either.

For all of it’s bells and whistles Katalon can’t seem to accomplish the same tasks as Selenium IDE could.

Russ_Thomas · 2019-02-01 16:30:13 UTC

EDIT: @devalex88, @Dung_Ngo: This thread appeared originally under the Web Testing category. In agreement with the original author I moved it to the Suggestions category since I believe Katalon could benefit from addressing at least some of the concerns @David_Eckert has raised.
The rest of this discussion is left here as an aid to Katalon Devs in understanding the original discourse, what was covered, its history/chronology and how we ended up “here”.
Thanks for reading.

I feel your pain, David, really. However, I’ve never tried to import Se tests so I can’t help much with that.

Well, that’s obviously not cool.

Was there a specific error reported in Katalon?

FWIW, I test a humongous ASP.NET app in Katalon and I can assure you, it works well. So what I suggest you do, is stick to your single test case (login, then navigate) and we’ll see if we can get you up and running. And remember, we’re approaching this one step at a time (follow the advice here and in less than a week, you’ll be testing again).

So, first, follow the advice in this article. It’ll help us to help you:

Brandon_Hein · 2019-02-01 17:24:27 UTC

Don’t let your familiarity with IDE, and unfamiliarity with Katalon, cloud your judgement. Katalon does everything that IDE could do, except it does it much better, more consistently, and in a more robust fashion.

I can’t say this with certainty, because I don’t know what your IDE scripts look like, but I will say that it’s probably not at all worth your time to try and import them.

If you spend the time learning how to build things yourself in the studio, and just use the Recorder as a helpful resource instead of a paradigm, you’ll be much better served. You will have more control over what’s going on, and you will be more confident that your scripts are doing what they’re supposed to be doing. In fact eventually, you will probably never use a recording feature again, regardless of which tool you are using.

Russ_Thomas · 2019-02-01 19:24:55 UTC

Mantra.

(… is all I wanted to say but the 10 char limit… )

David_Eckert · 2019-02-01 21:00:39 UTC

Thanks for the reply!

In Selenium IDE I could set global variables such as usernames, passwords, test speed, and even create strings such as filename+current time (so as to get a unique value each time a test was run). I could store all of these in a separate script and as long as it was run first, the rest of the tests in the current session could use those variables.

As for the record function, yes I recorded in Selenium IDE but then I would go back and tweak things manually. For this test, I recorded logging in, clicking on a parent menu, then it’s sub-menu to open a page. That’s it.

When I ran the recorded Katalon script (after editing half of the recorded 10 steps) it took 1 min. 55 sec. to open the browser, enter username and password and make two navigation clicks.

I also never found a way to get global variables to work. The test would puke trying to use ${AdminUsername} as the username. (After I set it up as a variable)

I then discovered that Selenium IDE has an extension for Chrome so I tried it. I recorded the exact same steps as I had with Katalon. Then I ran them… lightning fast, no errors. But it too doesn’t seem to handle stored global variables the same way it used to either.

I totally understand (and appreciate) your point about not letting Selenium IDE cloud my judgement. But so far, I’m not seeing the ease of use, customization, reliability and speed that I saw with the old Selenium IDE. I’ve also found the tutorials to be very Microsoft-ish… totally accurate yet totally useless. I feels as if Katalon makes it wonky on purpose to sell support contracts.

Russ_Thomas · 2019-02-01 21:12:26 UTC

Hmm. I could be wrong, but it seems to me you might be using the Katalon plugin? (i.e. not Katalon Studio). In which case, you’re in the wrong forum category. I’ll back off until you confirm one way or the other…

David_Eckert · 2019-02-01 21:15:29 UTC

I tried again after your post still encountered issues. Below are the recorded scripts and below that are the errors.

scripts`import static com.kms.katalon.core.checkpoint.CheckpointFactory.findCheckpoint
import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import static com.kms.katalon.core.testdata.TestDataFactory.findTestData
import static com.kms.katalon.core.testobject.ObjectRepository.findTestObject
import com.kms.katalon.core.checkpoint.Checkpoint as Checkpoint
import com.kms.katalon.core.cucumber.keyword.CucumberBuiltinKeywords as CucumberKW
import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile
import com.kms.katalon.core.model.FailureHandling as FailureHandling
import com.kms.katalon.core.testcase.TestCase as TestCase
import com.kms.katalon.core.testdata.TestData as TestData
import com.kms.katalon.core.testobject.TestObject as TestObject
import com.kms.katalon.core.webservice.keyword.WSBuiltInKeywords as WS
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
import internal.GlobalVariable as GlobalVariable
import org.openqa.selenium.Keys as Keys

WebUI.openBrowser(’’)

WebUI.navigateToUrl(‘https://pm-dev.pubshub.com/Login.aspx?ReturnUrl=%2F’)

WebUI.setText(findTestObject(‘Object Repository/Page_PubsHub PMSolution DEV environ/input_Username_ctl00ContentPan’), ‘phadmin’)

WebUI.setEncryptedText(findTestObject(‘Object Repository/Page_PubsHub PMSolution DEV environ/input_Password_ctl00ContentPan’),
'EDITED BY RUSS)

WebUI.sendKeys(findTestObject(‘Object Repository/Page_PubsHub PMSolution DEV environ/input_Password_ctl00ContentPan’), Keys.chord(
Keys.ENTER))

WebUI.click(findTestObject(‘Object Repository/Page_PubsHub PMSolution DEV environ/a_User List’))

WebUI.click(findTestObject(‘Object Repository/Page_User List/img’))

WebUI.setText(findTestObject(‘Object Repository/Page_Edit User/input__ctl00ContentPaneUserNam’), ‘theman 2’)

WebUI.click(findTestObject(‘Object Repository/Page_Edit User/input_No_ctl00ContentPaneSaveB’))

Errors
02-01-2019 04:06:49 PM Test Cases/test user area

Elapsed time: 51.062s

Test Cases/test user area FAILED.
Reason:
com.kms.katalon.core.exception.StepFailedException: Unable to click on object ‘Object Repository/Page_PubsHub PMSolution DEV environ/a_User List’ (Root cause: org.openqa.selenium.ElementNotVisibleException: element not interactable
(Session info: chrome=71.0.3578.98)
(Driver info: chromedriver=2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a),platform=Windows NT 10.0.14393 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
Build info: version: ‘3.7.1’, revision: ‘8a0099a’, time: ‘2017-11-06T21:07:36.161Z’
System info: host: ‘CHAD-BD6VBH2’, ip: ‘10.1.179.71’, os.name: ‘Windows 10’, os.arch: ‘amd64’, os.version: ‘10.0’, java.version: ‘1.8.0_181’
Driver info: com.kms.katalon.selenium.driver.CChromeDriver
Capabilities {acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: chrome, chrome: {chromedriverVersion: 2.43.600210 (68dcf5eebde371…, userDataDir: C:\Users\eckertd\AppData\Lo…}, cssSelectorsEnabled: true, databaseEnabled: false, goog:chromeOptions: {debuggerAddress: localhost:51385}, handlesAlerts: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: XP, platformName: XP, rotatable: false, setWindowRect: true, takesHeapSnapshot: true, takesScreenshot: true, unexpectedAlertBehaviour: , unhandledPromptBehavior: , version: 71.0.3578.98, webStorageEnabled: true}
Session ID: aa8b032727727466e1184d87fdf8235f)
at com.kms.katalon.core.keyword.internal.KeywordMain.stepFailed(KeywordMain.groovy:36)
at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.stepFailed(WebUIKeywordMain.groovy:65)
at com.kms.katalon.core.webui.keyword.internal.WebUIKeywordMain.runKeyword(WebUIKeywordMain.groovy:27)
at com.kms.katalon.core.webui.keyword.builtin.ClickKeyword.click(ClickKeyword.groovy:86)
at com.kms.katalon.core.webui.keyword.builtin.ClickKeyword.execute(ClickKeyword.groovy:67)
at com.kms.katalon.core.keyword.internal.KeywordExecutor.executeKeywordForPlatform(KeywordExecutor.groovy:53)
at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords.click(WebUiBuiltInKeywords.groovy:616)
at com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords$click$4.call(Unknown Source)
at test user area.run(test user area:29)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:319)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:298)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:290)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:224)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:106)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:97)
at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
at TempTestCase1549055191513.run(TempTestCase1549055191513.groovy:22)

Brandon_Hein · 2019-02-01 21:16:25 UTC

Global variables are used like this:

1.) Create a new Execution Profile and add your vars:

2.) Utilize them in your script:

import internal.GlobalVariable as GlobalVariable

System.out.println(GlobalVariable.url);
System.out.println(GlobalVariable.username);
System.out.println(GlobalVariable.password);

Brandon_Hein · 2019-02-01 21:24:16 UTC

This is the important bit. You need to wait for the element to become clickable before trying to click it.

.
.
.
WebUI.sendKeys(findTestObject(‘Object Repository/Page_PubsHub PMSolution DEV environ/input_Password_ctl00ContentPan’), Keys.chord(
Keys.ENTER))

// Add this line:
    WebUI.waitForElementClickable(findTestObject(‘Object Repository/Page_PubsHub PMSolution DEV environ/a_User List’), 30)

WebUI.click(findTestObject(‘Object Repository/Page_PubsHub PMSolution DEV environ/a_User List’))
.
.
.

David_Eckert · 2019-02-01 21:44:08 UTC

where, how?

David_Eckert · 2019-02-01 21:45:52 UTC

Ok, so do I need to add a click and wait type command?

Brandon_Hein · 2019-02-01 21:48:11 UTC

To follow up on Russ’s question: Are you using Katalon Studio or…?

If so, you should have your script (Test Case) open, and sitting on the Script tab:

Russ_Thomas · 2019-02-01 21:48:15 UTC

David - look at Brandon’s last post. He’s shown you what to add and where to add it. Try it and let us know.

(B: I’ll back out now - no sense in two of us being here)

David_Eckert · 2019-02-01 21:58:13 UTC

No, I’m using the desktop Katalon Studio. It pops out to the web version when you try to record. Another annoyance is that each time I select New Session it prompts me to install the Katalon extension (even though it’s already installed).

David_Eckert · 2019-02-01 21:59:42 UTC

Also got this after trying to run an imported Selenium IDE script.

02-01-2019 04:56:18 PM Test Cases/New Test Case

Elapsed time: 29.440s

Test Cases/New Test Case FAILED.
Reason:
groovy.lang.MissingPropertyException: No such property: Medium for class: Script1549057951247
at New Test Case.run(New Test Case:35)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:319)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:298)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:290)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:224)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:106)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:97)
at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
at TempTestCase1549058160149.run(TempTestCase1549058160149.groovy:22)

Brandon_Hein · 2019-02-01 22:02:13 UTC

Yeah, again, neither Russ nor I have messed with importing an IDE script, so I can’t really offer an advice other than: don’t try it. The time it will take you to import and tweak is probably better spent learning the tool and building your script step by step. Just my opinion of course, it’s your project.

Russ_Thomas · 2019-02-01 22:17:53 UTC

I couldn’t agree more. Plus, when I said “Mantra” in response to this, I meant it:

But I’m also a bit concerned about:

Is that something in the Recorder? Sorry David, but most of us (so-called) “Leaders” don’t use it. We, like Brandon mentioned, build our tests from scratch, work through them line by line in a logical, tried and tested manner. It’s the only true way to know your tests and understand (and reason about) what they’re doing. When problems occur (and yes, they will) you need this level of understanding.

And hey, we’re around most days. Whatever issues you have one of us (more than just Brandon and me) will know how to help.

This wasn’t meant to sound so “pep rally”. Sorry

So, bite the bullet and jump in (the water’s not too cold, promise!)

David_Eckert · 2019-02-04 13:57:10 UTC

We have many scripts that would take weeks to reproduce if we needed to rebuild them by hand. Isn’t the reason we use automated scripts is to save time?

We have a very small team and thus do not have the resources to dedicated that much time. (Again, one of the reasons we loved Selenium IDE).

We would love to be able to:

Use the record feature to initially create the tests as handwriting them is far too time consuming ( A few manual tweaks are to be expected).
Set the speed at which the tests run (some can be run faster than others).
Use stored global variables in order save time adding them to each test.
Ability to generate unique global variables on the fly. I.e. “text + current date/time” . This prevents the tests from trying to add an item with a name that may already exist.

Are these possible with Katalon?

Finally, I truly appreciate the feedback and your support of Katalon. I am trying hard not to compare the two but for our needs, we need simple, fast and reliable.

Russ_Thomas · 2019-02-04 14:11:03 UTC

Fair enough, David. You make a number of valid and well-reasoned points.

I guess Katalon (in it’s current form) is not filling your needs so well.

Just a suggestion: Perhaps edit your original post slightly to make it sound more like a suggestion-post and I’ll move the whole discussion to https://forum.katalon.com/c/katalon-studio/katalon-studio-feature-improvement-suggestions. Your call…

David_Eckert · 2019-02-04 15:03:08 UTC

Done. Thanks again for your effort. I really wished it had worked for us.