Why Are My Mobile App Tests Failing Intermittently on Different Devices?

Mobile Testing
,
1

Hi everyone,

I’m currently working on mobile app testing using automation tools, and I’m facing an issue where my test cases pass on some devices but fail intermittently on others. The failures seem inconsistent and are hard to reproduce, especially when running tests across different OS versions and screen sizes.

Here are a few things I’ve noticed:

Tests fail randomly during UI interactions
Timing and synchronization issues seem to occur
Differences in device performance may be affecting results

I’d like to understand:

What are the best practices for handling flaky mobile test cases?
How do you ensure consistency across multiple devices and environments?
Are there specific strategies or tools to improve test stability?
How do you manage waits, retries, or dynamic elements effectively?

Any suggestions, real-world solutions, or debugging tips would be really helpful.

Thanks in advance!

2

Are you using Katalon Studio IDE? Which version?

Can you share any error log and test case script that failed? Concrete information needed to understand you.

3

The hyper link “mobile app” lead me to a web page

What are you doing with this web page at all? Do you want to test this web page?

4

In most cases, flakiness isn’t really a multi‑device issue—it’s a synchronization and test design problem that becomes more visible on slower devices or different OS versions. When tests are written to wait intelligently, use reliable locators, account for device variability, and capture meaningful failure evidence, they tend to behave consistently across different screen sizes and operating systems.

So make sure the below some best practices

  1. Wait intelligently
    Uses stable locators
  2. Assumes device variability
  3. Captures enough failure evidence

But yes, please let us know whihc version of Katalon studio IDE are you usong ?

5

hi @aartijangid372

intermittent mobile failures are almost always sync issues that slower devices expose
drop the delay() calls and use Mobile.waitForElementPresent / waitForElementClickable with timeouts tuned for your slowest device

stick to accessibility IDs or resource IDs for locators. XPath gets brittle across different screen sizes and OS versions

6

Mobile flaky tests from device variance + timing—fix with explicit waits + retries.

Root Causes & Fixes

Issue Fix
UI timing Mobile.waitForElementVisible(element, 10) not delay()
Screen size Dynamic locators: //android.widget.Button[contains(@text, 'Submit')]
Performance Test Suite > Retry failed: 2x immediate
OS diff Device farm (AWS/Kobiton) + matrix (Android 13/14)

Keyword Template

@Keyword
void robustTap(TestObject to) {
    Mobile.waitForElementVisible(to, 10)
    Mobile.tap(to, FailureHandling.CONTINUE_ON_FAILURE)
}

Suite: Execution > Retry failed Test Cases: 2x

Debug: Run isolated, logs + screenshots. Tag flaky for quarantine

7

Do you handle retries only at the suite level or also inside custom keywords like robustTap?