App Center Integration


This is a companion discussion topic for the original entry at https://docs.katalon.com/katalon-studio/docs/app-center.html

Hi,

I’m trying to integrate our test cases into AzureDevOps pipeline & run them on AppCenter Test Runs.

I’ve followed your sideload project, both with your demo KatalonDemoProject & our own test cases - our project does not work in the AppCenter test runs.

When we compared the 2 test projects, your Demo has a pom.xml file, which is missing in our Katalon test project.
My DevOps lead is under the impression that if we can get the missing pom file into the project, the tests will run in AppCenter.

How do we create this pom.xml file in an ordinary Katalon Test project?
Does the test project need to be packaged in JUnit first for it to work using the sideload steps?
If so, how would I do this?

Thanks

Hi @anne.mccloghry,

The pom.xml file is only required for the sideload project. And your project just needs to be zipped and placed inside the sideload project (at the path src/test/resources).

In summary, the steps are:

  1. Update your Katalon project to create a custom Appium Driver as in the instruction.
  2. Zip your Katalon project and place it inside the folder src/test/resources of the sideload project.
  3. Update the Katalon running command in src\test\java\com\katalon\sideload\SideloadTest.java (and some other parameters as described in this section).
  4. Pack the sideload project with command: "mvn clean -DskipTests -P prepare-for-upload package" (Run this command at the sideload folder)
  5. Upload and run sideload project on the App Center with command "appcenter test run appium --app <app_name> --devices <device_id/device_name> --app-path <path_to_app_file> --test-series "master" --locale "en_US" --build-dir target/upload" (Run this command at the sideload folder)

~ Hope this could solves your problem /=)

1 Like

Thanks, that’s great.

What is the probability of Katalon making this integration project into an official plugin (as part of KRE)?
The fact that its still in PoC stage is making my management team a bit uneasy. :slightly_smiling_face:

Thanks
Anne

Hi @thongnmtran,

thanks for the help with the sideload project.
We got it running on AppCenter but the tests failed. I’ve attached the logs from AppCenter.

  • Since we are still conducting research for a PoC for using KSE & KRE, we don’t yet have a full license. So I generated the API Key in Katalon Studio using my trial license. This is what I put into the apiKey value in src\test\java\com\katalon\sideload\SideloadTest.java
  • my default global ExecutionProfile is empty as I have all my variables in their corresponding ExProfiles. Does this matter?

Can you possibly help us work out what the issue might be? We are so close… :slight_smile:
Thanks
Anne

AppiumLogs.txt (31.8 KB) TestFailures.txt (2.4 KB) TestLogs.txt (36.5 KB)

Hi @anne.mccloghry,

Your test log looks pretty good, the license check is ok, and the ExProfiles wouldn’t be an issue /=). The problem is you just passing an inappropriate “appActivity” to the desired capabilities.

Possible workarounds are:

  • Remove appActivity from desired capabilities.
  • Update the appActivity to "com.theyield.sensingplusv2.MainActivity"
  • Update the appActivity to "com.theyield.sensingplusv2.*"
  • Double-check your main activity name :smiley:

For more details on troubleshooting, you could refer to this section.


From test log:

INFO: org.openqa.selenium.SessionNotCreatedException: Unable to create a new remote session. Please check the server log for more details. Original error: An unknown server-side error occurred while processing the command. Original error: Cannot start the ‘com.theyield.sensingplusv2’ application. Visit https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/activity-startup.md for troubleshooting. Original error: Activity name ‘.MainActivity’ used to start the app doesn’t exist or cannot be launched! Make sure it exists and is a launchable activity


And one more thing. About your concern yesterday, that does we have any plan to make this integration project into an official plugin. I have sent your concern to my team, and we’re still discussing it. Because the App Center has a different approach to Mobile testing vs Kobiton, Sauce Labs… So it could take a lot of effort to build a plugin to integrate with App Center in a fully compatible way.

We will continue to find ways to improve it. But hope you can sympathize with this inconvenience /=)

1 Like

Hi @thongnmtran,

It worked, thank you!

So I have a question about how it works.

My test case was a very simple one - logging in to the app, then switching tabs & logging out.
if the login is unsuccessful, then the logout will fail & the test will fail.
As I mentioned, all of the variables such as Username & Password were in environment-specific Execution Profiles (not the default one, its completely empty).
How does the AppCenter test run of my Katalon tests know which execution profile to use? How does it extract the correct variables?

Regards
Anne

Hi @anne.mccloghry,

Currently, the App Center doesn’t directly run the test, but the Katalon does. The App Center will run Katalon, and Katalon runs your tests. So the way you passing variables to Katalon in the local environment will also work like that when running on App Center through the sideload.

hmm… I would like to describe in more detail but I’m not really sure about your “environment-specific Execution Profiles”. Hope this could give you some clues to get a better understanding of how it work /=)

Thanks @thongnmtran again for your explanation.
And I would really like to understand this example with more detail.

Say I have 3 execution profiles “Dev”, “Stag”, “Prod” - with a Username & Password that are different across the different environments.
My default Exe Profile is empty.

When I run the tests in KSE, if I don’t switch to my “DevProfile”, for example, it will just try to use the default profile - and the test fails.
So if I am including my TestSuite in the sideload project, where does it specify which ExeProfile to use? I haven’t defined it anywhere in the TestSuite, which one to select.

When I ran sideload in AppCenter using “MyApp_Dev”, how did it know to use the Dev Execution Profile?

Thanks
Anne

Hi @anne.mccloghry,

I’m sure you’ve seen it, but maybe you just missed that part /=). In the step update sideload, you can specify which ExeProfile will be used along with the execution command by set it to the “-executionProfile” argument.

For example, if you want to run a test for the staging environment, you can set “-executionProfile” to something like -executionProfile=“Stag”. And the full command would be

-retry=0 -testSuitePath="Test Suites/Regression Tests" -executionProfile="Stag" -browserType="Chrome" -apiKey="********"

image


For more convenience, you can also use the command generator to generate an execution command with your preferred profile / arguments :smiley: .


image

1 Like

@thongnmtran - that is such a great help to my understanding!
I did miss that part in the script where you specify the executionProfile!
Thanks again, I hope we continue to use the sideload project - I think it fits our needs really well :wink:

1 Like

@anne.mccloghry, yeah! Glad to hear that! Hope it could help your work.

~ and Thanks for using Katalon! /=)

1 Like

Hi again @thongnmtran!

We had a good result in getting the tests to run on AppCenter, when I kicked them off from my local machine (pointing to a saved .ipa build that I had downloaded from AppCenter).

Now we are at the stage where we are trying to automate the test run from Azure DevOps pipeline.
We are getting this error:

com.katalon.sideload.SideloadTest.sideload
java.lang.AssertionError: Failed to execute Katalon
at org.junit.Assert.fail(Assert.java:88)
at com.katalon.sideload.SideloadTest.sideload(SideloadTest.java:44)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)

Do you have any idea where we have gone wrong, that’s its
Failed to execute Katalon
?

thanks
Anne

Hi again @anne.mccloghry,

The error log you sent is the sideload’s test log. It just indicates that your test has failed. Could you help me to find the Katalon’s log, or maybe Appium’s log too /=). Thanks…

thanks @thongnmtran - I’ve attached here

AppiumLogs.txt (1.5 KB) TestLogs.txt (5.9 KB)

By the way - I can’t get them to run through a local CLI command now either :frowning:

Hi @anne.mccloghry,

Sorry for the late reply /=). I got stuck with some hot tickets last week. Are you still having trouble with that? I have checked it again and it still works fine. I’ve checked your test logs too, but still have no clue. If you still get trouble with that, maybe you should try to re-clone the sideload and setup again.

Hi @thongnmtran
I am strying automate the test run with Appcenter , I am get gettin this error

[INFO] Running com.katalon.sideload.SideloadTest

Logging xtc events to /Volumes/Data/xamarin/workspaces/e50b4449-dd8b-44c6-8d03-0e09a892fae1/workspace/event_stream.log

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.002 s <<< FAILURE! - in com.katalon.sideload.SideloadTest

[ERROR] com.katalon.sideload.SideloadTest.sideload Time elapsed: 0.665 s <<< FAILURE!

java.lang.AssertionError: Failed to execute Katalon

at com.katalon.sideload.SideloadTest.sideload(SideloadTest.java:44

I dont know where i have wrong, do you have any idea?

=>>>java.lang.AssertionError: Failed to execute Katalon

Hi,

How can I retrieve and publish the result as Azure DevOps pipeline report?

Hi, i have problems too.

When i try to run locally the sideload it gives me the error: java.net.MalformedURLException: no protocol: nullwd/hub

At the line: String remoteServerUrl = System.getenv(‘XTC_SERVICE_ENDPOINT_APPIUM’) + ‘wd/hub’

Could you please help?

Thank you