MOBILE: How to select image 1 when Katalon keeps selecting image 2

Mobile Testing
1

I am using Katalon 9.7.3 but my co-worker is using Katalon 10+. I am not a developer. This is on a Android device.

We are having issues selecting images within a Composeview. For whatever reason, the app always chooses the second picture even though we have captured image 1. In other words, we click image 1 but when we run the test it clicks image 2. We have tried to capture the objects using CTRL to switch between layers. There are two layers colored red. This was our experience in the past and we just excepted image 2 for our test.

But now we are trying to get away from capture objects and using locators? How do we select images? We cannot use class because they are all named “android.view.View”. We cannot use Xpaths. We haven’t been successfully using bounds? Any ideas? Any help is appreciated.

Pic1
Pic1887Ă—561 228 KB

Pic2
Pic2718Ă—656 9.52 KB

**`katalon-studio`** **`mobile-testing`**, **`images`**
2 Likes
2

To resolve the issue where Katalon keeps selecting the second image instead of the first in your Android ComposeView, follow these steps:

1. Use Image Recognition (Recommended)

Since standard locators fail, leverage Katalon’s image-based recognition:

groovy

import com.kms.katalon.core.mobile.keyword.MobileBuiltInKeywords as Mobile

// Capture reference image of Pic1 (store in project dir)
String refImage = 'Path/to/your/pic1_reference.png'

// Set similarity threshold (adjust as needed)
float similarity = 0.75

// Tap on matching image
Mobile.tapOnImage(refImage, similarity)

2. Positional Targeting via Bounds

If images are in a grid, calculate coordinates:

groovy

// Get screen dimensions
int screenWidth = Mobile.getDeviceWidth()
int screenHeight = Mobile.getDeviceHeight()

// Calculate position for first image (e.g., top-left quarter)
int tapX = screenWidth / 4
int tapY = screenHeight / 4

Mobile.tapAtPosition(tapX, tapY)

3. Accessibility Properties (Compose-Specific)

Add test tags in your Compose UI code:

kotlin

// In Android app code
Image(
    painter = ...,
    contentDescription = "Content image",
    modifier = Modifier.testTag("image_1") // Add this
)

Then use in Katalon:

groovy

Mobile.tap(findTestObject('Object Repository/image_1'), 0)

4. Child Index Targeting

If images share a parent container:

groovy

TestObject parent = findTestObject('parent_container')
TestObject firstImage = Mobile.getChildObject(parent, 0) // 0 = first child
Mobile.tap(firstImage, 0)

5. Dynamic Object Identification

Use Groovy to filter by visible content:

groovy

List<TestObject> allImages = Mobile.findObjects(findTestObject('Generic_View'))

for (TestObject image : allImages) {
    String contentDesc = Mobile.getAttribute(image, 'content-desc')
    if (contentDesc?.contains("unique_text_near_pic1")) {
        Mobile.tap(image, 0)
        break
    }
}

Configuration Tips:

  1. Enable advanced spy mode:
  • In Katalon: Settings > Mobile > Record > Enable advanced spy mode
  • Use “Capture Elements with Hierarchy” during spying
  1. Adjust object properties:

java

// In object properties
[{
  "class": "android.view.View",
  "index": "0",       // First match
  "bounds": "[0,0][500,500]" // Specific area
}]
  1. Disable view recycling (if dynamic):

groovy

Mobile.switchToNative()
Mobile.scrollToTop()

Troubleshooting Workflow:

  1. Use Mobile.startExistingApplication() to reset state
  2. Run Mobile.delay(3) before image selection
  3. Capture UI hierarchy via:

groovy

String xml = Mobile.getDeviceOS() == 'android' ? 
             Mobile.getAttribute(findTestObject('*'), 'xpath') : 
             Mobile.getAllViews()
println xml

Alternative for Flaky Tests:

groovy

int attempts = 0
while (attempts < 3) {
    try {
        Mobile.tapOnImage(refImage, 0.7)
        break
    } catch (Exception e) {
        Mobile.swipe(0, 300, 0, 500) // Adjust swipe if needed
        attempts++
    }
}

Key Notes:

  • Use image recognition for highest accuracy
  • Test tags (Modifier.testTag) are the most reliable if you control app code
  • For pixel-perfect taps, combine getChildObject() with bounds
  • Avoid XPath in Compose - use accessibility hierarchy instead

Example of full test case:

groovy

'Select first image': {
    String refImage = 'TestData/Images/pic1_ref.png'
    float similarity = 0.8
    
    try {
        Mobile.tapOnImage(refImage, similarity)
    } catch (NotFoundException e) {
        // Fallback to coordinates
        int width = Mobile.getDeviceWidth()
        int height = Mobile.getDeviceHeight()
        Mobile.tapAtPosition(width/4, height/4)
    }
}
3 Likes
3

Why?

Do you mean that your organization has an internal policy that prohibits you from XPath? — I think it is ridiculous.

Or, do you mean that you haven’t studied XPath so that you don’t understand it?

In my humble opinion, the skill of XPath is mandatory for the Selenium-based technology. You should spend several days learning it. If you are willing, see

As long as you stay ignorant of XPath, I think that you wouldn’t achieve anything in the Selenium-based testing jobs.

4 Likes
4

For Mobile, we were told they cause issues with performance.

2 Likes
5

Thank you for your reply.
Now I understand you.

2 Likes
7

I redirected your question to ChatGPT. It answered as follows. It suggests that you need your dev team to help you by modifying the app. I think that this suggestion is appropriate.

Thanks for the detailed explanation. Your issue is a common challenge when testing Jetpack Compose UI elements on Android, especially when elements have no unique identifiers or accessible properties, which is the case when everything is rendered as android.view.View.

Let’s walk through why this happens and then discuss how you can reliably select the right image using better locator strategies in Katalon (even without coding experience).

:magnifying_glass_tilted_left: Why Is Image 1 Selecting Image 2?

  • In Jetpack Compose, UI elements often lack unique view IDs unless explicitly set by developers.
  • Compose often draws components over the same root android.view.View, which leads to multiple elements sharing the same class and layer.
  • The order or position of elements in the view hierarchy becomes critical, especially when you use tools like Katalon that rely on those hierarchies.

So what’s happening is:

  • You’re clicking the first image.
  • But at runtime, the test engine matches a different (but similarly named or placed) element, likely due to view overlap or dynamic positioning.

:white_check_mark: Recommended Strategies to Fix It

1. Use Test Tags (if your dev team can help) :white_check_mark:

If you have access to the dev team, ask them to add a Modifier.testTag() to each image in the Compose UI.

Example (Kotlin):

Image(
    painter = painterResource(id = R.drawable.image1),
    contentDescription = "Image 1",
    modifier = Modifier.testTag("image1")
)

Then in Katalon, use this test tag:

locator = //*[@content-desc='image1']

:green_circle: This is the most stable, long-term fix because test tags persist and are specifically made for UI testing.

2 Likes
8

Thanks so much for everyone’s time. I am gonna look into these.

1 Like