This is a companion discussion topic for the original entry at https://docs.katalon.com/katalon-studio/docs/app-center.html
This is a companion discussion topic for the original entry at https://docs.katalon.com/katalon-studio/docs/app-center.html
Hi,
Iâm trying to integrate our test cases into AzureDevOps pipeline & run them on AppCenter Test Runs.
Iâve followed your sideload project, both with your demo KatalonDemoProject & our own test cases - our project does not work in the AppCenter test runs.
When we compared the 2 test projects, your Demo has a pom.xml file, which is missing in our Katalon test project.
My DevOps lead is under the impression that if we can get the missing pom file into the project, the tests will run in AppCenter.
How do we create this pom.xml file in an ordinary Katalon Test project?
Does the test project need to be packaged in JUnit first for it to work using the sideload steps?
If so, how would I do this?
Thanks
Hi @anne.mccloghry,
The pom.xml file is only required for the sideload project. And your project just needs to be zipped and placed inside the sideload project (at the path src/test/resources
).
In summary, the steps are:
- Update your Katalon project to create a custom Appium Driver as in the instruction.
- Zip your Katalon project and place it inside the folder
src/test/resources
of the sideload project. - Update the Katalon running command in
src\test\java\com\katalon\sideload\SideloadTest.java
(and some other parameters as described in this section). - Pack the sideload project with command:
"mvn clean -DskipTests -P prepare-for-upload package"
(Run this command at the sideload folder) - Upload and run sideload project on the App Center with command
"appcenter test run appium --app <app_name> --devices <device_id/device_name> --app-path <path_to_app_file> --test-series "master" --locale "en_US" --build-dir target/upload"
(Run this command at the sideload folder)
~ Hope this could solves your problem /=)
Thanks, thatâs great.
What is the probability of Katalon making this integration project into an official plugin (as part of KRE)?
The fact that its still in PoC stage is making my management team a bit uneasy.
Thanks
Anne
Hi @thongnmtran,
thanks for the help with the sideload project.
We got it running on AppCenter but the tests failed. Iâve attached the logs from AppCenter.
- Since we are still conducting research for a PoC for using KSE & KRE, we donât yet have a full license. So I generated the API Key in Katalon Studio using my trial license. This is what I put into the apiKey value in src\test\java\com\katalon\sideload\SideloadTest.java
- my default global ExecutionProfile is empty as I have all my variables in their corresponding ExProfiles. Does this matter?
Can you possibly help us work out what the issue might be? We are so closeâŚ
Thanks
Anne
AppiumLogs.txt (31.8 KB) TestFailures.txt (2.4 KB) TestLogs.txt (36.5 KB)
Hi @anne.mccloghry,
Your test log looks pretty good, the license check is ok, and the ExProfiles wouldnât be an issue /=). The problem is you just passing an inappropriate âappActivityâ to the desired capabilities.
Possible workarounds are:
- Remove appActivity from desired capabilities.
- Update the appActivity to
"com.theyield.sensingplusv2.MainActivity"
- Update the appActivity to
"com.theyield.sensingplusv2.*"
- Double-check your main activity name
For more details on troubleshooting, you could refer to this section.
From test log:
INFO: org.openqa.selenium.SessionNotCreatedException: Unable to create a new remote session. Please check the server log for more details. Original error: An unknown server-side error occurred while processing the command. Original error: Cannot start the âcom.theyield.sensingplusv2â application. Visit https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/activity-startup.md for troubleshooting. Original error: Activity name â.MainActivityâ used to start the app doesnât exist or cannot be launched! Make sure it exists and is a launchable activity
And one more thing. About your concern yesterday, that does we have any plan to make this integration project into an official plugin. I have sent your concern to my team, and weâre still discussing it. Because the App Center has a different approach to Mobile testing vs Kobiton, Sauce Labs⌠So it could take a lot of effort to build a plugin to integrate with App Center in a fully compatible way.
We will continue to find ways to improve it. But hope you can sympathize with this inconvenience /=)
Hi @thongnmtran,
It worked, thank you!
So I have a question about how it works.
My test case was a very simple one - logging in to the app, then switching tabs & logging out.
if the login is unsuccessful, then the logout will fail & the test will fail.
As I mentioned, all of the variables such as Username & Password were in environment-specific Execution Profiles (not the default one, its completely empty).
How does the AppCenter test run of my Katalon tests know which execution profile to use? How does it extract the correct variables?
Regards
Anne
Hi @anne.mccloghry,
Currently, the App Center doesnât directly run the test, but the Katalon does. The App Center will run Katalon, and Katalon runs your tests. So the way you passing variables to Katalon in the local environment will also work like that when running on App Center through the sideload.
hmm⌠I would like to describe in more detail but Iâm not really sure about your âenvironment-specific Execution Profilesâ. Hope this could give you some clues to get a better understanding of how it work /=)
Thanks @thongnmtran again for your explanation.
And I would really like to understand this example with more detail.
Say I have 3 execution profiles âDevâ, âStagâ, âProdâ - with a Username & Password that are different across the different environments.
My default Exe Profile is empty.
When I run the tests in KSE, if I donât switch to my âDevProfileâ, for example, it will just try to use the default profile - and the test fails.
So if I am including my TestSuite in the sideload project, where does it specify which ExeProfile to use? I havenât defined it anywhere in the TestSuite, which one to select.
When I ran sideload in AppCenter using âMyApp_Devâ, how did it know to use the Dev Execution Profile?
Thanks
Anne
Hi @anne.mccloghry,
Iâm sure youâve seen it, but maybe you just missed that part /=). In the step update sideload
, you can specify which ExeProfile will be used along with the execution command by set it to the â-executionProfileâ argument.
For example, if you want to run a test for the staging environment, you can set â-executionProfileâ to something like -executionProfile=âStagâ. And the full command would be
-retry=0 -testSuitePath="Test Suites/Regression Tests" -executionProfile="Stag" -browserType="Chrome" -apiKey="********"
For more convenience, you can also use the command generator to generate an execution command with your preferred profile / arguments .
@thongnmtran - that is such a great help to my understanding!
I did miss that part in the script where you specify the executionProfile!
Thanks again, I hope we continue to use the sideload project - I think it fits our needs really well
@anne.mccloghry, yeah! Glad to hear that! Hope it could help your work.
~ and Thanks for using Katalon! /=)
Hi again @thongnmtran!
We had a good result in getting the tests to run on AppCenter, when I kicked them off from my local machine (pointing to a saved .ipa build that I had downloaded from AppCenter).
Now we are at the stage where we are trying to automate the test run from Azure DevOps pipeline.
We are getting this error:
com.katalon.sideload.SideloadTest.sideload
java.lang.AssertionError: Failed to execute Katalon
at org.junit.Assert.fail(Assert.java:88)
at com.katalon.sideload.SideloadTest.sideload(SideloadTest.java:44)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
Do you have any idea where we have gone wrong, thatâs its
Failed to execute Katalon
?
thanks
Anne
Hi again @anne.mccloghry,
The error log you sent is the sideloadâs test log. It just indicates that your test has failed. Could you help me to find the Katalonâs log, or maybe Appiumâs log too /=). ThanksâŚ
thanks @thongnmtran - Iâve attached here
AppiumLogs.txt (1.5 KB) TestLogs.txt (5.9 KB)
By the way - I canât get them to run through a local CLI command now either
Hi @anne.mccloghry,
Sorry for the late reply /=). I got stuck with some hot tickets last week. Are you still having trouble with that? I have checked it again and it still works fine. Iâve checked your test logs too, but still have no clue. If you still get trouble with that, maybe you should try to re-clone the sideload and setup again.
Hi @thongnmtran
I am strying automate the test run with Appcenter , I am get gettin this error
[INFO] Running com.katalon.sideload.SideloadTest
Logging xtc events to /Volumes/Data/xamarin/workspaces/e50b4449-dd8b-44c6-8d03-0e09a892fae1/workspace/event_stream.log
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.002 s <<< FAILURE! - in com.katalon.sideload.SideloadTest
[ERROR] com.katalon.sideload.SideloadTest.sideload Time elapsed: 0.665 s <<< FAILURE!
java.lang.AssertionError: Failed to execute Katalon
at com.katalon.sideload.SideloadTest.sideload(SideloadTest.java:44
I dont know where i have wrong, do you have any idea?
=>>>java.lang.AssertionError: Failed to execute Katalon
Hi,
How can I retrieve and publish the result as Azure DevOps pipeline report?
Hi, i have problems too.
When i try to run locally the sideload it gives me the error: java.net.MalformedURLException: no protocol: nullwd/hub
At the line: String remoteServerUrl = System.getenv(âXTC_SERVICE_ENDPOINT_APPIUMâ) + âwd/hubâ
Could you please help?
Thank you