Can any one explain the advantages of Katalon over espresso for testing UI of android applications?

Archive
,
1

I heard that Espresso developed by Google community runs faster than any UI automation tools and does depend on the Id’s associated with the codebase and requires little maintenance.
Can you please clarify whether Katalon also serves this purpose?

1 Like
2

Hi @umarfarooq.techwiz,

Before choosing Katalon Studio as our testing platform, I looked at several other options, including Espresso for Android and XCUITest for iOS. Key factors that I looked at were:

  • Ease of writing tests - I wanted non-developers to be able to write new tests
  • Shared test suites across all platforms: iOS, Android, Web
  • Ease of maintaining tests - As the screen layouts changed, I didn’t want to be writing brittle tests that required a lot of test changes
  • Tests should be QA driven: Ideally with a separation of code from the tests
  • Ease of running in a Continuous Integration environment

Katalon Studio was able to accomplish this with:

  • Interactive test recording and playback with Mobile Record and object capture with Mobile Spy
  • Optional scripting of tests in Groovy (a language that is isn’t very verbose and is easily approachable by non-developers)
  • Reusable Test Objects that can be modified independently from the test code itself, so that tests don’t need to change when text on the screen changes
  • Extensible - you can add third party helper libraries to make testing easier
  • Test suites, that with a little bit of logic can be used to switch between iOS and Android using the exact same test code
  • Tests are run based on a compiled .app or .apk file, separating the application code from the tests

I will say that Android Espresso does look promising:

However, by Google’s own documents (https://developer.android.com/training/testing/espresso/), Espresso is targeted at developers. Even with the Test Recorder, the test code lives alongside the application code. This requires a bit more coordination with the development team. It will also more tightly couple the tests to the code, which could be seen as a benefit, depending on your team’s comfort with the application code.

Espresso is also targeted only at Android testing, so if you plan to test iOS or Web in the future, you’ll need to create separate test suites using specific test tools for those platforms.

Espresso tests are also written in Kotlin or Java, so you’ll need to make sure you have a resource to support those (there is lots of documentation/courses online for those languages).

In the end, I think it depends on your intended use and responsibilities for who will be writing and maintaining the tests.

I hope this helps give you some more information,

Chris

5 Likes
3

Hello Chris,

Thank you for such a brief explanation about the comparison between Katalon Studio Vs Espresso. However wanted to know a few more clarifications on below items,

  1. I wanted to hear also about the execution time of Katalon test suites vs espresso in your experience.
  2. Can you share the maintenance effort in terms of UI testing when the data points part of the UI are rearranged completely in terms of XPath change etc.,
  3. Also how effective is Katalon Studio in terms of sanity testing in terms of test coverage on feature build and integration build.
  4. Should we consider any architectural changes in our development life cycle to embrace Katalon Studio as our unique testing tool for Web, Andriod, and iOS?
    As we also wanted to have a single automation tool apart from currently espresso, XCUIT, and selenium with testNG.

Thanks,
Umar

4

Hi @umarfarooq.techwiz,

  1. I wanted to hear also about the execution time of Katalon test suites vs espresso in your experience.

I haven’t written any Espresso tests, but I have written UI tests using Xcode’s XCUITest. The native UI testing frameworks are faster to execute because they don’t go through a “bridge” like Appium, which needs to read the screen and then put the contents into something digestible by tests.

You can do some things to speed up UI tests in general, like disabling or limiting animation times.

  1. Can you share the maintenance effort in terms of UI testing when the data points part of the UI are rearranged completely in terms of XPath change etc.

In general, with any UI testing, I would highly recommend against building any tests that rely on xpath. As you imply in your question, the xpaths are subject to change frequently, so are not reliable for object definition. I also recommend against using display text/labels as object identifiers, as those could change for version to version and/or be internationalized.

Instead, I would recommend using a unique resource-id (or group of ids for list items) that won’t change. You’ll want to coordinate with the development team to define these, stressing the importance that they stay constant.

In the case where you do need to change an id or other definition for an object, Katalon Studio has the concept of Test Objects, which make this very easy. Test Objects are obfuscated screen elements (think of them as variables) that your tests will use. Test Objects have properties that Katalon and Appium will use behind the scenes to find elements on the screen, but your tests never need to know those underlying properties. So, in the event that an object id (or even xpath) changes in the app, you don’t need to change your tests, just the Test Object, creating a very nice layer of isolation.

  1. Also how effective is Katalon Studio in terms of sanity testing in terms of test coverage on feature build and integration build.

Because the UI tests are run separately on a pre-built app/apk, I don’t think there is code coverage available for Katalon Tests. I will say that test output from Katalon Studio is Junit compatible, so you can produce nice test result reports for visibility in continuous integration.

I typically don’t try for 100% code coverage with UI tests anyway, because of the amount of time they take to run. Hitting every error scenario doesn’t seem like a good use of time. Instead, we use UI tests as regression tests of the happy path; making sure that the normal flow continues to work. We mostly use it to test that the things we expect to be on a screen after an action are on the screen, and that navigation from screen to screen works as intended.

  1. Should we consider any architectural changes in our development life cycle to embrace Katalon Studio as our unique testing tool for Web, Andriod, and iOS?

The main architectural change that we had to make was to make sure that all elements on the screen that we wanted to interact with had unique ids. It did help that our iOS and Android apps have very similar flows, so we could reuse the same tests across both, by having 2 sets of Test Objects with the platform-specific attributes for each. There are some small cases that are handled differently, e.g. Android has a slide-over menu, whereas iOS has a tab bar at the bottom of the screen. We had to add a couple of lines of conditional code to account for this in a few tests, but other than that, we were able to reuse 95% of the test code for both iOS and Android platforms.

Some other interesting things we encountered related to how the apps were built. For iOS, we can’t use an app built for distribution for testing because of the architecture for which it’s built and the code signing. For Android, we encountered some issues where all of the resource ids are prefixed with the build type, e.g. “staging”, “pullrequest”, “production”, so we needed to make sure our tests used the same build type each time.

Finally, because these are UI tests that execute in a real app, you’ll want to make sure any API calls don’t hit production databases and/or clean up after themselves and don’t destroy production data.

I hope this helps give you some more info. Please let me know if you have any other questions,

Chris

2 Likes