Device Farm Integration [Android]

Product Forums Katalon Studio
,
1

Hi Katalon Team,

I am trying to use aws device farm to execute test cases I followed steps mentioned in katalon device farm for android but I am getting this error can someone please help me with issue

[DEVICEFARM] Setting up your device. This usually takes 2-3 minutes.
[DEVICEFARM] ########### Start executing testspec ###########
[DEVICEFARM] ########### Entering phase install ###########
[DEVICEFARM] ########### Entering phase pre_test ###########
[DeviceFarm] export CLASSPATH=$CLASSPATH:$DEVICEFARM_TESTNG_JAR
[DeviceFarm] export CLASSPATH=$CLASSPATH:$DEVICEFARM_TEST_PACKAGE_PATH/*
[DeviceFarm] export CLASSPATH=$CLASSPATH:$DEVICEFARM_TEST_PACKAGE_PATH/dependency-jars/*
[DeviceFarm] echo “Start appium server”
Start appium server
[DeviceFarm] appium --log-timestamp --device-name $DEVICEFARM_DEVICE_NAME --platform-name $DEVICEFARM_DEVICE_PLATFORM_NAME --app $DEVICEFARM_APP_PATH --udid $DEVICEFARM_DEVICE_UDID --chromedriver-executable $DEVICEFARM_CHROMEDRIVER_EXECUTABLE >> $DEVICEFARM_LOG_DIR/appiumlog.txt 2>&1 &
[DeviceFarm] start_appium_timeout=0; while [ true ]; do
if [ $start_appium_timeout -gt 60 ];
then
echo “appium server never started in 60 seconds. Exiting”;
exit 1;
fi;
grep -i “Appium REST http interface listener started on 0.0.0.0:4723” $DEVICEFARM_LOG_DIR/appiumlog.txt >> /dev/null 2>&1;
if [ $? -eq 0 ];
then
echo “Appium REST http interface listener started on 0.0.0.0:4723”;
break;
else
echo “Waiting for appium server to start. Sleeping for 1 second”;
sleep 1;
start_appium_timeout=$((start_appium_timeout+1));
fi;
done;
Waiting for appium server to start. Sleeping for 1 second
Waiting for appium server to start. Sleeping for 1 second
Appium REST http interface listener started on 0.0.0.0:4723
[DEVICEFARM] ########### Entering phase test ###########
[DeviceFarm] echo “Navigate to test package directory”
Navigate to test package directory
[DeviceFarm] cd $DEVICEFARM_TEST_PACKAGE_PATH
[DeviceFarm] echo “Start Appium JUnit test”
Start Appium JUnit test
[DeviceFarm] java -Dappium.screenshots.dir=$DEVICEFARM_SCREENSHOT_PATH org.testng.TestNG -junit -testjar *-tests.jar -d $DEVICEFARM_LOG_DIR/test-output -verbose 10
[TestNG] Couldn’t find the testng.xml in the jar file, running all the classes
[TestRunner] Running the tests in ‘Command line test eb6f987f-214d-4626-9288-5b73e517470b’ with parallel mode:false
[RunInfo] Adding method selector: org.testng.internal.XmlMethodSelector@270421f5 priority: 10
[TestNG] Running:
Command line suite
[SuiteRunner] Created 1 TestRunners
[TestRunner] Running test Command line test eb6f987f-214d-4626-9288-5b73e517470b on 0 classes, included groups: excluded groups:
[TestNG] RUNNING: Suite: “Command line test eb6f987f-214d-4626-9288-5b73e517470b” containing “0” Tests (config: null)
[TestNG] INVOKING: “Command line test eb6f987f-214d-4626-9288-5b73e517470b” - com.kms.example.aws_ios.test.TestIos.test()
Nov 29, 2022 4:07:20 PM com.kms.example.aw_ios.utils.ConsoleLogger info
INFO: KATALON_VERSION: 8.1.0
Nov 29, 2022 4:07:20 PM com.kms.example.aw_ios.utils.ConsoleLogger info
INFO: KATALON_PROJECT_PACKAGE_FILE: app-empresarial-pruebas-automatizadas.zip
Nov 29, 2022 4:07:20 PM com.kms.example.aw_ios.utils.ConsoleLogger info
INFO: KATALON_PROJECT_PATH:
Nov 29, 2022 4:07:20 PM com.kms.example.aw_ios.utils.ConsoleLogger info
INFO: DEVICEFARM_LOG_DIR: /tmp/customer_log_directory5WMESp
[TestNG] FAILED: “Command line test eb6f987f-214d-4626-9288-5b73e517470b” - com.kms.example.aws_ios.test.TestIos.test() finished in 389 ms
[TestNG] java.lang.AssertionError: Failed to execute Katalon
[TestNG] at org.junit.Assert.fail(Assert.java:88)
[TestNG] at com.kms.example.aws_ios.test.TestIos.test(TestIos.java:84)
[TestNG] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[TestNG] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[TestNG] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[TestNG] at java.lang.reflect.Method.invoke(Method.java:498)
[TestNG] at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
[TestNG] at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
[TestNG] at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
[TestNG] at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
[TestNG] at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
[TestNG] at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
[TestNG] at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
[TestNG] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
[TestNG] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
[TestNG] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
[TestNG] at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
[TestNG] at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
[TestNG] at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
[TestNG] at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
[TestNG] at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
[TestNG] at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
[TestNG] at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
[TestNG] at org.testng.junit.JUnit4TestRunner.start(JUnit4TestRunner.java:81)
[TestNG] at org.testng.junit.JUnit4TestRunner.run(JUnit4TestRunner.java:69)
[TestNG] at org.testng.TestRunner$1.run(TestRunner.java:682)
[TestNG] at org.testng.TestRunner.runWorkers(TestRunner.java:1008)
[TestNG] at org.testng.TestRunner.privateRunJUnit(TestRunner.java:713)
[TestNG] at org.testng.TestRunner.run(TestRunner.java:614)
[TestNG] at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
[TestNG] at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:343)
[TestNG] at org.testng.SuiteRunner.privateRun(SuiteRunner.java:305)
[TestNG] at org.testng.SuiteRunner.run(SuiteRunner.java:254)
[TestNG] at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
[TestNG] at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
[TestNG] at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
[TestNG] at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
[TestNG] at org.testng.TestNG.run(TestNG.java:1057)
[TestNG] at org.testng.TestNG.privateMain(TestNG.java:1364)
[TestNG] at org.testng.TestNG.main(TestNG.java:1333)
[JUnit4TestMethod] Method ‘initializationError’ not found in class ‘com.kms.example.aw_ios.utils.KatalonDownloadUtils’: com.kms.example.aw_ios.utils.KatalonDownloadUtils.initializationError()
[JUnit4TestMethod] Method ‘initializationError’ not found in class ‘com.kms.example.aw_ios.utils.KatalonDownloadUtils’: com.kms.example.aw_ios.utils.KatalonDownloadUtils.initializationError()
[JUnit4TestMethod] Method ‘initializationError’ not found in class ‘com.kms.example.aw_ios.utils.KatalonDownloadUtils’: com.kms.example.aw_ios.utils.KatalonDownloadUtils.initializationError()
[JUnit4TestMethod] Method ‘initializationError’ not found in class ‘com.kms.example.aw_ios.utils.KatalonDownloadUtils$1’: com.kms.example.aw_ios.utils.KatalonDownloadUtils$1.initializationError()
[JUnit4TestMethod] Method ‘initializationError’ not found in class ‘com.kms.example.aw_ios.utils.KatalonDownloadUtils$1’: com.kms.example.aw_ios.utils.KatalonDownloadUtils$1.initializationError()
[JUnit4TestMethod] Method ‘initializationError’ not found in class ‘com.kms.example.aw_ios.utils.KatalonDownloadUtils$1’: com.kms.example.aw_ios.utils.KatalonDownloadUtils$1.initializationError()
[JUnit4TestMethod] Method ‘initializationError’ not found in class ‘com.kms.example.aw_ios.utils.SideloadUtils’: com.kms.example.aw_ios.utils.SideloadUtils.initializationError()
[JUnit4TestMethod] Method ‘initializationError’ not found in class ‘com.kms.example.aw_ios.utils.KatalonUtils’: com.kms.example.aw_ios.utils.KatalonUtils.initializationError()
[JUnit4TestMethod] Method ‘initializationError’ not found in class ‘com.kms.example.aw_ios.utils.ConsoleLogger’: com.kms.example.aw_ios.utils.ConsoleLogger.initializationError()
[JUnit4TestMethod] Method ‘initializationError’ not found in class ‘com.kms.example.aw_ios.utils.ConsoleLogger’: com.kms.example.aw_ios.utils.ConsoleLogger.initializationError()
===== Invoked methods
TestIos.test()[pri:0, instance:test(com.kms.example.aws_ios.test.TestIos)] 1497973285

Creating /tmp/customer_log_directory5WMESp/test-output/Jar suite/Command line test eb6f987f-214d-4626-9288-5b73e517470b.html
Creating /tmp/customer_log_directory5WMESp/test-output/Jar suite/Command line test eb6f987f-214d-4626-9288-5b73e517470b.xml
FAILED: test on test(com.kms.example.aws_ios.test.TestIos)
java.lang.AssertionError: Failed to execute Katalon
at org.junit.Assert.fail(Assert.java:88)
at com.kms.example.aws_ios.test.TestIos.test(TestIos.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at org.testng.junit.JUnit4TestRunner.start(JUnit4TestRunner.java:81)
at org.testng.junit.JUnit4TestRunner.run(JUnit4TestRunner.java:69)
at org.testng.TestRunner$1.run(TestRunner.java:682)
at org.testng.TestRunner.runWorkers(TestRunner.java:1008)
at org.testng.TestRunner.privateRunJUnit(TestRunner.java:713)
at org.testng.TestRunner.run(TestRunner.java:614)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:343)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:305)
at org.testng.SuiteRunner.run(SuiteRunner.java:254)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
at org.testng.TestNG.run(TestNG.java:1057)
at org.testng.TestNG.privateMain(TestNG.java:1364)
at org.testng.TestNG.main(TestNG.java:1333)

Command line test eb6f987f-214d-4626-9288-5b73e517470b
Tests run: 1, Failures: 1, Skips: 0

===============================================
[TestNG]
[TestNG] ===============================================
[TestNG] Command line test eb6f987f-214d-4626-9288-5b73e517470b
[TestNG] Tests run: 1, Failures: 1, Skips: 0
[TestNG] ===============================================

Jar suite
Total tests run: 1, Failures: 1, Skips: 0

[TestNG] Time taken by org.testng.reporters.XMLReporter@7382f612: 51 ms
Creating /tmp/customer_log_directory5WMESp/test-output/testng-failed.xml
Creating /tmp/customer_log_directory5WMESp/test-output/Jar suite/testng-failed.xml
[TestNG] Time taken by [FailedReporter passed=0 failed=0 skipped=0]: 8 ms
[TestNG] Time taken by org.testng.reporters.jq.Main@39a054a5: 24 ms
[TestNG] Time taken by org.testng.reporters.EmailableReporter2@67f89fa3: 5 ms
Creating /tmp/customer_log_directory5WMESp/test-output/junitreports/TEST-com.kms.example.aws_ios.test.TestIos.xml
[TestNG] Time taken by org.testng.reporters.JUnitReportReporter@43556938: 4 ms
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/toc.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/Command line test eb6f987f-214d-4626-9288-5b73e517470b.properties
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/index.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/main.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/groups.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/methods.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/methods.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/methods.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/methods-alphabetical.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/methods-alphabetical.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/methods-alphabetical.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/classes.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/reporter-output.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/methods-not-run.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/Jar suite/testng.xml.html
Creating /tmp/customer_log_directory5WMESp/test-output/old/index.html
[TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@77468bd9: 6 ms
[DEVICEFARM] ########### Entering phase post_test ###########
[DEVICEFARM] ########### Finish executing testspec ###########
[DEVICEFARM] ########### Setting upload permissions ###########
[DEVICEFARM] Tearing down your device. Your tests report will come shortly.

2

Hi @osiris007x ,

Thank you for letting us know your problem. I will shorten your error to be Couldn’t find the testng.xml. You can find the similar answer via this post: [AWS Device Farm Integration] Katalon test for Android does not work with some error - #4 by rie.ota.

Thank you.

3

Hi thanks for the reply I have doubt if I use Java Junit in aws then does it require the testng.xml to execute

4

@osiris007x you can give it a try, but i doubt also this is the root cause for the failure.
according to your logs and some docs, the presence of testng.xml file is optional.
e.g: Extending custom test environments in Device Farm - AWS Device Farm

If I read your log properly, if such is not present then will execute all tests found.
You have to inspect the generated reports and look into what test has failed and why, since something was executed:

Tests run: 1, Failures: 1, Skips: 0

What you actually posted is not an error, but the execution log.
I will split the log in some relevant info’s:

// since the xml is not present will execute on 0 selected classes and no included / excluded stuff---

[TestRunner] Running test Command line test eb6f987f-214d-4626-9288-5b73e517470b on 0 classes, included groups: excluded groups:
[TestNG] RUNNING: Suite: “Command line test eb6f987f-214d-4626-9288-5b73e517470b” containing “0” Tests (config: null)

// looking for all test methods and picks 
// com.kms.example.aws_ios.test.TestIos.test()
// most probably is the only one in the code ---

[TestNG] INVOKING: “Command line test eb6f987f-214d-4626-9288-5b73e517470b” - com.kms.example.aws_ios.test.TestIos.test()
Nov 29, 2022 4:07:20 PM com.kms.example.aw_ios.utils.ConsoleLogger info
INFO: KATALON_VERSION: 8.1.0
Nov 29, 2022 4:07:20 PM com.kms.example.aw_ios.utils.ConsoleLogger info
INFO: KATALON_PROJECT_PACKAGE_FILE: app-empresarial-pruebas-automatizadas.zip
Nov 29, 2022 4:07:20 PM com.kms.example.aw_ios.utils.ConsoleLogger info
INFO: KATALON_PROJECT_PATH:
Nov 29, 2022 4:07:20 PM com.kms.example.aw_ios.utils.ConsoleLogger info
INFO: DEVICEFARM_LOG_DIR: /tmp/customer_log_directory5WMESp

// Test finishes after 389 ms ---

[TestNG] FAILED: “Command line test eb6f987f-214d-4626-9288-5b73e517470b” - com.kms.example.aws_ios.test.TestIos.test() finished in 389 ms

// The test is failed with an Assertion error
[TestNG] java.lang.AssertionError: Failed to execute Katalon
[TestNG] at org.junit.Assert.fail(Assert.java:88)