Failure to synchronize object capture / Usage of testIds / XCUIElement Tree

We, a colleague of mine and I, are building prototypes for all areas of web and mobile testing. We are at it nw for a good three weeks. Our focus, having to test some iOS apps, is to get it all working with Mac OS X.

After some inital problems we now set up with:

  • Mac OS X* 10.14.2 (Mojave)
  • Xcode 9.4.1
  • Appium 1.8.1
  • Jenkins 2.150.2 LTS

So far we got the web, api and Android testing working with the Katalon Studion GUI and within Jenkins. The last obstacle is to get the iOS testing to work.

We have to support testing for React Native Android/iOS applications. The Android part is working. Alas we are stuck with the iOS part. We have a setup where we can build our .app files locall with the Bitrise CLI or within the online Bitrise build environment. The result in using the genrated .app files is invariant to the build environment.

Katalon will install the App and the the WebDriverAgentRunner on the selected Simulator an then successfully start the application - and the device view.

After this start sequence the device view will show a different screen (the splash screen) than the actual simulator (the first app screen). And the captured objects will, at least partially, reflect the device view and not the simulator/app view.

After recapturing the objects the device view will updated and synchronized with the simulator/app. The captured object will reflect the device view but again, only partially.

Has anybody got similar experiences or even better a corresponding solution?

Hi @Uwe_Heyroth,

First, it looks like you have the correct supported versions of Xcode and Appium, which is good!

As for your first screenshot from above, it’s not uncommon that the simulator and what you see in the “Device View” panel are out of sync. You’ll need to make sure you press the “Capture Object” button any time you change screens or interact with an element in the simulator in order to get the latest in your inspector.

That being said, from your second screenshot, it does look like you’re in sync, but the objects are all XCUIElementTypeOther. Obviously, that’s not very helpful for finding objects. I’m guessing this is a side effect of creating a React Native app; Appium 1.8.1 and iOS may not be playing nice together to give you a detailed picture of what’s on the screen.

I think this is a case where you are going to need to make sure you give every element on the screen that you want to interact with an accessibiltyId (shows up as name when you capture the object). That way, you can refer to elements as their id instead of relying on the type.

If you want to see some more detail about exactly what Appium/Katalon detects on the screen you can log out the XML contents of the screen. You should be able to do that by putting the following import statements at the top of your test case:

import com.kms.katalon.core.logging.KeywordLogger
import com.kms.katalon.core.mobile.keyword.internal.MobileDriverFactory
import io.appium.java_client.AppiumDriver

Then in the code for your test, add the following after you navigate to a screen:

AppiumDriver<?> driver = MobileDriverFactory.getDriver()
KeywordLogger log = new KeywordLogger()
log.logInfo(driver.getPageSource())

When the test runs, you should see an entry in the Log Viewer for “Statement - log.logInfo(driver.getPageSource())” that shows the start of the XML for the screen. If you tap on it, you’ll see all of the XML for the screen on the right-hand pane.

This might help you better define the elements that you want to detect.

Hope this helps,

Chris

Hi Chris,

It took us a while, but finally our supplier provided us with an updated view class and I now know how to add testID myself.

Albeit, as you can see in the attached screen shot it didn’t really take us forward.

We can see the testID “countrySelection” and it doesn’t discriminate to a degree where we can identify th attached XCUI Element.

  • Is our supplier providing us with faulty code?
  • Is there a problem in the interaction between React Native and Katalon?
  • Is there a problem with our test setup?

Can you help us chasing down the problem and

get closer to a solution?

Cheers

Uwe & Sameer

Problem Solved!

After putting in the right search words I found THE hint.
Some additional prototyping and testing lead to the expected results then.

React Native Context
With React Native comes an Accessibility concept.
Every view element can be tagged with an accesible attribute.
In short that means, if at any level in the GUI hierarchy an element’s accessible attribute is set to true all its child elements are not accessible individually.
In the following example the views accessible attribute is set to true which implies, that the text of the embedded text elements is not accessible sepeartely. I. e. the texts cannot be selected for cut copy paste operations.

Accessibility Eample

text one
text two
</View

In terms of testing it is essential to make any elements, that are to be eaxamined or tested invariant to code changes.
I. e. that requires to orchestrate them with resusable ids like

Accessibility Eample

<Text accessibilityLabel={“testId-1”} ={“testId-1”}}>text one
<Text accessibilityLabel={“testId-2”} ={“testId-2”}}>text two
</View
This should enable searching for the text fields maybe like …findbbyId(“testId-1”)
With the accessible attribute is set to true at the view level this will not work since that exactly disables physical and thereby programmtic access to these element.
The solution here is, to set every elements accessible attribute, in the hierarchy of the element that you want to access, to false.

Thus the example view has to be orchestrated like this:

Accessibility Eample

<Text accessible={true} accessibilityLabel={“testId-1”} ={“testId-1”}}>text one
<Text accessible={true} accessibilityLabel={“testId-2”} ={“testId-2”}}>text two
</View

Coding Requirements
Default: accessible={false}
With the previuos explanation in mind, we request that every view element is tagged with accessible={false} by default and only the “operational” elements will be accessible.
Common function testId
In order to ease the orchestration process a general pupose testId method has to be provided.
Fehler beim Rendern des Makros ‘code’: Ungültiger Wert für den Parameter ‘firstline’
export default function testId(id) {
return { accessible: true, accessibilityLabel: id, testID: id };
}

Then the view example can be rewritten like this:

Accessibility Eample

<Text {…testId(‘testId-1’)}>text one
<Text {…testId(‘testId-1’)}}>text two

Orchestrated Sample

export default class CountrySelectorWidget extends Component {

render() {
const { country, messages } = this.props;
return (

<TouchableOpacity
{…testId(‘ddlbCountrySelector’)}
onPress={() => {
this.props.onCountryPress();
this.setState({ isCountryFieldDirty: true });
}}
style={style.container}>

{country
? localisedCountryName(country.countryCode, messages)
: messages.SG_COUNTRY_SELECTION_LABEL}

      <Icon
        style={{ position: 'absolute', bottom: 5, right: -10 }}
        name={'downArrow'}
        fill={ThemeColors.BLACK}
        height={15}
        widht={15}
      />
    </TouchableOpacity>
    <VaillantText
      style={{
        padding: 5,
        paddingHorizontal: 0,
        color: ThemeColors.ERROR
      }}>
      {this.props.showCountryError
        ? messages.SG_COUNTRY_SELECTION_LABEL_ERROR
        : ' '}
    </VaillantText>
    {this.props.showDropdown && (
      <View style={style.itemContainer}>
        <ScrollView accessible={false}>
            {countries[Config.COMPANY_NAME].map((country, key) => {
            return (
              <TouchableOpacity
                {...testId('ddlbCountrySelection')}
                key={key}
                style={{ height: 40, justifyContent: 'center' }}
                onPress={() => this.props.onCountryPress(country)}>
                <VaillantText>
                  {localisedCountryName(country.countryCode, messages)}
                </VaillantText>
              </TouchableOpacity>
            );
          })}
        </ScrollView>
      </View>
    )}
  </View>
);

}
}

If you need any additinal infos, don’t be afraid to ask.


1 Like

Hi!

How did you guys integrate Katalon with Bitrise?

Thanx!
/Goran

Hi @Uwe_Heyroth,

In the beginning of this post you wrote: “We have to support testing for React Native Android/iOS applications. The Android part is working.”
How do you find elements in the Android tests?

We are making an app with react native too, and I am trying to test it with Katalon, but I cannot manage to find the elements consistently. We added ‘testID’ to the elements, and I can find the element in the mobile object spy, but when running a test I see that it is still using the wrong elements. (see TestID not used when running test)

I am pretty stuck and hope you can help me.
Thanks,
Emine van Graafeiland