Unable to click submenu item for desktop applications

Product Forums Katalon Studio
, ,
1

Hi,

Im trying to test my desktop application using 7.6.6. Unfortunately the element is not clickable.

I tried to record the objects using windows action recorder. The main menu item is identified and clickable. But the submenu item is not clickable. T

In the recorder window it shows below error

com.kms.katalon.composer.windows.exception.WindowsComposerException: org.openqa.selenium.WebDriverException: An element command could not be completed because the element is not pointer- or keyboard interactable. (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
Build info: version: ‘3.141.59’, revision: ‘e82be7d358’, time: ‘2018-11-14T08:25:53’
System info: host: ‘CKUMAR-L03’, ip: ‘172.28.60.33’, os.name: ‘Windows 10’, os.arch: ‘amd64’, os.version: ‘10.0’, java.version: ‘1.8.0_181’
Driver info: io.appium.java_client.windows.WindowsDriver
Capabilities {app: C:., javascriptEnabled: true, platform: WINDOWS, platformName: WINDOWS}
Session ID: 9DEFECE2-15A1-4B0D-B94D-EF9EE3DDABE3
at com.kms.katalon.composer.windows.action.WindowsActionHandler$BaseActionObjectHandler.perform(WindowsActionHandler.java:313)
at com.kms.katalon.composer.windows.action.WindowsActionHandler.perform(WindowsActionHandler.java:85)
at com.kms.katalon.composer.windows.dialog.WindowsRecorderDialog.performAction(WindowsRecorderDialog.java:454)
at com.kms.katalon.composer.windows.dialog.WindowsRecorderDialog.lambda$2(WindowsRecorderDialog.java:412)
at com.kms.katalon.composer.windows.action.WindowsActionButtonWrapper$1.widgetSelected(WindowsActionButtonWrapper.java:33)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:249)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4418)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1079)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4236)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3824)
at org.eclipse.jface.window.Window.runEventLoop(Window.java:818)
at org.eclipse.jface.window.Window.open(Window.java:794)
at com.kms.katalon.composer.windows.dialog.WindowsRecorderDialog.open(WindowsRecorderDialog.java:189)
at com.kms.katalon.composer.windows.handler.RecordWindowsObjectHandler.execute(RecordWindowsObjectHandler.java:74)
at org.eclipse.e4.core.internal.di.MethodRequestor.execute(MethodRequestor.java:55)
at org.eclipse.e4.core.internal.di.InjectorImpl.invokeUsingClass(InjectorImpl.java:282)
at org.eclipse.e4.core.internal.di.InjectorImpl.invoke(InjectorImpl.java:264)
at org.eclipse.e4.core.contexts.ContextInjectionFactory.invoke(ContextInjectionFactory.java:132)
at org.eclipse.e4.ui.workbench.renderers.swt.DirectContributionItem.executeItem(DirectContributionItem.java:130)
at org.eclipse.e4.ui.workbench.renderers.swt.AbstractContributionItem.handleWidgetSelection(AbstractContributionItem.java:454)
at org.eclipse.e4.ui.workbench.renderers.swt.AbstractContributionItem$3.handleEvent(AbstractContributionItem.java:482)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4418)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1079)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4236)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3824)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$4.run(PartRenderingEngine.java:1121)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1022)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:150)
at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:693)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:610)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:148)
at com.kms.katalon.core.application.WorkbenchApplicationStarter.start(WorkbenchApplicationStarter.java:23)
at com.kms.katalon.application.Application.runGUI(Application.java:184)
at com.kms.katalon.application.Application.start(Application.java:98)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:388)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:243)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:673)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:610)
at org.eclipse.equinox.launcher.Main.run(Main.java:1519)
Caused by: org.openqa.selenium.WebDriverException: An element command could not be completed because the element is not pointer- or keyboard interactable. (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
Build info: version: ‘3.141.59’, revision: ‘e82be7d358’, time: ‘2018-11-14T08:25:53’
System info: host: ‘CKUMAR-L03’, ip: ‘172.28.60.33’, os.name: ‘Windows 10’, os.arch: ‘amd64’, os.version: ‘10.0’, java.version: ‘1.8.0_181’
Driver info: io.appium.java_client.windows.WindowsDriver
Capabilities {app: C:.., javascriptEnabled: true, platform: WINDOWS, platformName: WINDOWS}
Session ID: 9DEFECE2-15A1-4B0D-B94D-EF9EE3DDABE3
at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:214)
at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:166)
at org.openqa.selenium.remote.http.JsonHttpResponseCodec.reconstructValue(JsonHttpResponseCodec.java:40)
at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:80)
at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:44)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
at io.appium.java_client.remote.AppiumCommandExecutor.execute(AppiumCommandExecutor.java:239)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)
at io.appium.java_client.DefaultGenericMobileDriver.execute(DefaultGenericMobileDriver.java:42)
at io.appium.java_client.AppiumDriver.execute(AppiumDriver.java:1)
at io.appium.java_client.windows.WindowsDriver.execute(WindowsDriver.java:1)
at org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:285)
at org.openqa.selenium.remote.RemoteWebElement.click(RemoteWebElement.java:84)
at com.kms.katalon.core.windows.keyword.helper.WindowsActionHelper.click(WindowsActionHelper.java:255)
at com.kms.katalon.composer.windows.action.WindowsActionHandler$ClickActionHandler.performAction(WindowsActionHandler.java:541)
at com.kms.katalon.composer.windows.action.WindowsActionHandler$BaseActionObjectHandler$1.run(WindowsActionHandler.java:269)
at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:119)

Even in native windows recorder ,the main menu item is identified but the sub menu items (itemr1,item2 etc) all altogether identified but not individually. Means ,item1 and item2 should be highlighted seperately.
Below is the picture how items are highlighted
imgsrc
Here im trying to click on Open menu item

Note: The same application i could able to test it with Teststackwhite(another open source tool). I think there are few applications which are not identified by the katalon for menu item. May be the way menu items built in code are not identified. May be it needs different way to check these kind of menu items.

Also using Windows App driver in c# was resolved by using action builder. Not sure how to achieve this here in katalon

3

Hi @chandan.kumar,

As you said, it still can be clickable by using Action Builder. So I would suggest you try to click in this way (using Action Builder in Katalon).

import org.openqa.selenium.WebElement
import org.openqa.selenium.interactions.Actions

WebElement submenuItem = Windows.findElement(findWindowsObject('Object Repository/Mic 01/Submenu_Item'))

Actions builder = new Actions(Windows.getDriver());
builder.click(submenuItem)
builder.perform()

And about the submenu item object, maybe you have to write its locator on your own, as both Windows Recorder and Native Windows Recorder were not able to capture it.

image
image984×433 10 KB

~ Hope it could help! /=)

4

@thongnmtran
Thanks. But im not sure what you mean by own locator.
I have tried the steps you suggested. But still no luck.

Here is my keyword

@Keyword
def clickmenuname(WindowsTestObject testObject) {
WebElement submenuItem;
try
{
submenuItem = Windows.findElement(findWindowsObject(testObject))
}
catch(Exception e)
{
System.out.println(" Debugging here")
System.out.println(e.printStackTrace())
}
Actions builder = new Actions(Windows.getDriver());
//builder.moveToElement(submenuItem)
builder.click(submenuItem)
builder.perform()
}
}

DesktopApp
DesktopApp1875×900 86.1 KB

The below is the console error

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Work/Automation/Katalon/Katalon-7.6.6/Katalon_Studio_Windows_64-7.6.6/plugins/ch.qos.logback.classic_1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Work/Automation/Katalon/Katalon-7.6.6/Katalon_Studio_Windows_64-7.6.6/configuration/plugin/install/183327/1.0.0/katalon-studio-jmeter-integration-plugin-1.0.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See SLF4J Error Codes for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2020-09-14 12:20:48.828 INFO c.k.katalon.core.main.TestCaseExecutor - --------------------
2020-09-14 12:20:48.834 INFO c.k.katalon.core.main.TestCaseExecutor - START Test Cases/DiscoverDevice
2020-09-14 12:20:49.730 DEBUG testcase.DiscoverDevice - 1: startApplicationWithTitle(“C:\DesktopApp.exe”, “DesktopApp”)
2020-09-14 12:20:49.992 INFO c.k.k.c.w.driver.WindowsDriverFactory - Starting application C:\DesktopApp.exe on the local machine at address http://127.0.0.1:4723
2020-09-14 12:20:50.015 INFO c.k.k.c.w.driver.WindowsDriverFactory - desiredCapabilities = {}
Sep 14, 2020 12:20:54 PM io.appium.java_client.remote.AppiumCommandExecutor$1 lambda$0
INFO: Detected dialect: OSS
2020-09-14 12:20:55.034 DEBUG testcase.DiscoverDevice - 2: click(findWindowsObject(“SensroDemo/Spy/ImgSrc”))
2020-09-14 12:20:55.519 INFO c.k.k.c.w.k.helper.WindowsActionHelper - Clicking on element: Object Repository/SensroDemo/Spy/ImgSrc
2020-09-14 12:20:56.377 DEBUG testcase.DiscoverDevice - 3: mycustom.clickmenuitem.clickmenuname(findWindowsObject(“SensroDemo/Spy/Opn”))
Debugging exception here
groovy.lang.MissingMethodException: No signature of method: static com.kms.katalon.core.testobject.ObjectRepository.findWindowsObject() is applicable for argument types: (com.kms.katalon.core.testobject.WindowsTestObject) values: [com.kms.katalon.core.testobject.WindowsTestObject@155d1021]
Possible solutions: findWindowsObject(java.lang.String), findWindowsObject(java.lang.String, java.util.Map)
at groovy.lang.MetaClassImpl.invokeStaticMissingMethod(MetaClassImpl.java:1503)
at groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1489)
at org.codehaus.groovy.runtime.callsite.StaticMetaClassSite.callStatic(StaticMetaClassSite.java:65)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:206)
at mycustom.clickmenuitem.clickmenuname(clickmenuitem.groovy:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1215)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1024)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:812)
at groovy.lang.DelegatingMetaClass.invokeMethod(DelegatingMetaClass.java:144)
at mycustom.clickmenuitem.invokeMethod(clickmenuitem.groovy)
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.invokeStaticMethod(CustomKeywordDelegatingMetaClass.java:50)
at org.codehaus.groovy.runtime.callsite.StaticMetaClassSite.call(StaticMetaClassSite.java:53)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
at Script1598950979188.run(Script1598950979188.groovy:25)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:339)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:330)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:309)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:301)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:235)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at TempTestCase1600066244145.run(TempTestCase1600066244145.groovy:25)
at groovy.lang.GroovyShell.runScriptOrMainOrTestOrRunnable(GroovyShell.java:263)
at groovy.lang.GroovyShell.run(GroovyShell.java:518)
at groovy.lang.GroovyShell.run(GroovyShell.java:507)
at groovy.ui.GroovyMain.processOnce(GroovyMain.java:653)
at groovy.ui.GroovyMain.run(GroovyMain.java:384)
at groovy.ui.GroovyMain.process(GroovyMain.java:370)
at groovy.ui.GroovyMain.processArgs(GroovyMain.java:129)
at groovy.ui.GroovyMain.main(GroovyMain.java:109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.groovy.tools.GroovyStarter.rootLoader(GroovyStarter.java:109)
at org.codehaus.groovy.tools.GroovyStarter.main(GroovyStarter.java:131)
null
2020-09-14 12:20:56.545 ERROR k.k.c.m.CustomKeywordDelegatingMetaClass - :x: null
2020-09-14 12:20:56.548 ERROR c.k.katalon.core.main.TestCaseExecutor - :x: Test Cases/DiscoverDevice FAILED.
Reason:
java.lang.NullPointerException
at org.openqa.selenium.interactions.PointerInput$Origin.fromElement(PointerInput.java:222)
at org.openqa.selenium.interactions.Actions.moveInTicks(Actions.java:416)
at org.openqa.selenium.interactions.Actions.click(Actions.java:322)
at org.openqa.selenium.interactions.Actions$click.call(Unknown Source)
at mycustom.clickmenuitem.clickmenuname(clickmenuitem.groovy:64)
at mycustom.clickmenuitem.invokeMethod(clickmenuitem.groovy)
at com.kms.katalon.core.main.CustomKeywordDelegatingMetaClass.invokeStaticMethod(CustomKeywordDelegatingMetaClass.java:50)
at DiscoverDevice.run(DiscoverDevice:25)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:339)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:330)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:309)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:301)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:235)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
at TempTestCase1600066244145.run(TempTestCase1600066244145.groovy:25)

2020-09-14 12:20:56.620 INFO c.k.katalon.core.main.TestCaseExecutor - END Test Cases/DiscoverDevice

This how its done on Teststackwhite (pseudo code below)

string path = { “Image source”, “Open”, “Sensor on USB …” };
MenuItem(“menuitem”,path);
public void MenuItem(object obj)
{
string path = obj as string;
var menubar = SensorGui_Wnd.MenuBar;
//automation_sleep(3000);
//automation_sleep(1000);
try
{
menubar.MenuItem(path).Click();
}
catch
{
automation_sleep(1000);
menubar.MenuItem(path).Click();
TestIntfUtils.postTestStepData(“Trying to clikc again Menu item”, “”, “”);
}
//automation_sleep(1000);
menu_thrd.Abort();
//automation_sleep(1000);
}

This how its done in WinAppdriver in c#
DemoSession = new WindowsDriver(new Uri(“http://127.0.0.1:4723”), appiumOption, TimeSpan.FromMinutes(5));
var AllMenus = DemoSession.FindElementsByTagName(“MenuItem”);

Loop through all elements and find element
var openMenu = mainMenuItems.FindElementByName(“Open”);
OpenQA.Selenium.Interactions.Actions Click = new OpenQA.Selenium.Interactions.Actions(DemoSession);
Click.MoveToElement(openMenu);
Click.Click();
Click.Perform();

5

Hi @chandan.kumar,

As the error log, you just need to change this line

submenuItem = Windows.findElement(findWindowsObject(testObject))

to

submenuItem = Windows.findElement(testObject)

because the testObject already is a suitable parameter for the Windows.findElement(…) keyword. /=)

6

@thongnmtran

Perfect. it worked. Initially it didnt identify the object when i used XPATH. it worked when i used object identification using Name .
I used the katalon Windows SPY for identifying the name of the object.

Do you have any plans on having built int keyword for these kind issues.

2 Likes