[AWS Device Farm Integration] Katalon test for Android does not work with some error

I’m trying to integrate Katalon and AWS device farm using this document as a reference.

  • Device info
    • Google Pixel 3a XL
    • OS: Android 12

Currently, the test fails with the following log.
What are the possible causes?

Logs about “_.DS_Store” are output, so I deleted all .DS_Store under my project and compressed it, but it is not resolved.

[DEVICEFARM] Setting up your device. This usually takes 2-3 minutes.
[DEVICEFARM] ########### Start executing testspec ###########
 
 
[DEVICEFARM] ########### Entering phase install ###########
 
[DeviceFarm] export NVM_DIR=$HOME/.nvm
[DeviceFarm] . $NVM_DIR/nvm.sh
[DeviceFarm] nvm install 14.19.1
v14.19.1 is already installed.
Now using node v14.19.1 (npm v6.14.16)
[DeviceFarm] export APPIUM_VERSION=1.22.2
[DeviceFarm] avm $APPIUM_VERSION
/usr/bin/avm: line 261: appium: command not found
  e[36m    existse[0m : e[90me[0m
[DeviceFarm] ln -s /usr/local/avm/versions/$APPIUM_VERSION/node_modules/.bin/appium  /usr/local/avm/versions/$APPIUM_VERSION/node_modules/appium/bin/appium.js
 
[DEVICEFARM] ########### Entering phase pre_test ###########
 
[DeviceFarm] export CLASSPATH=$CLASSPATH:$DEVICEFARM_TESTNG_JAR
[DeviceFarm] export CLASSPATH=$CLASSPATH:$DEVICEFARM_TEST_PACKAGE_PATH/*
[DeviceFarm] export CLASSPATH=$CLASSPATH:$DEVICEFARM_TEST_PACKAGE_PATH/dependency-jars/*
[DeviceFarm] echo "Start appium server"
Start appium server
[DeviceFarm] appium --log-timestamp --default-capabilities "{\"deviceName\": \"$DEVICEFARM_DEVICE_NAME\", \"platformName\":\"$DEVICEFARM_DEVICE_PLATFORM_NAME\", \"app\":\"$DEVICEFARM_APP_PATH\", \"udid\":\"$DEVICEFARM_DEVICE_UDID\", \"platformVersion\":\"$DEVICEFARM_DEVICE_OS_VERSION\", \"chromedriverExecutable\":\"$DEVICEFARM_CHROMEDRIVER_EXECUTABLE\"}" >> $DEVICEFARM_LOG_DIR/appiumlog.txt 2>&1 &
[DeviceFarm] start_appium_timeout=0; while [ true ]; do
    if [ $start_appium_timeout -gt 60 ];
    then
        echo "appium server never started in 60 seconds. Exiting";
        exit 1;
    fi;
    grep -i "Appium REST http interface listener started on 0.0.0.0:4723" $DEVICEFARM_LOG_DIR/appiumlog.txt >> /dev/null 2>&1;
    if [ $? -eq 0 ];
    then
        echo "Appium REST http interface listener started on 0.0.0.0:4723";
        break;
    else
        echo "Waiting for appium server to start. Sleeping for 1 second";
        sleep 1;
        start_appium_timeout=$((start_appium_timeout+1));
    fi;
done;
Waiting for appium server to start. Sleeping for 1 second
Waiting for appium server to start. Sleeping for 1 second
Waiting for appium server to start. Sleeping for 1 second
Waiting for appium server to start. Sleeping for 1 second
Appium REST http interface listener started on 0.0.0.0:4723
 
[DEVICEFARM] ########### Entering phase test ###########
 
[DeviceFarm] echo "Navigate to test package directory"
Navigate to test package directory
[DeviceFarm] cd $DEVICEFARM_TEST_PACKAGE_PATH
[DeviceFarm] echo "Start Appium JUnit test"
Start Appium JUnit test
[DeviceFarm] java -Dappium.screenshots.dir=$DEVICEFARM_SCREENSHOT_PATH org.testng.TestNG -junit -testjar *-tests.jar -d $DEVICEFARM_LOG_DIR/test-output -verbose 10
[TestNG] Couldn't find the testng.xml in the jar file, running all the classes
[TestRunner] Running the tests in 'Command line test 8a5d353c-34f6-4707-8902-a457e82f8732' with parallel mode:false
[RunInfo] Adding method selector: org.testng.internal.XmlMethodSelector@270421f5 priority: 10
[TestNG] Running:
  Command line suite

[SuiteRunner] Created 1 TestRunners
[TestRunner] Running test Command line test 8a5d353c-34f6-4707-8902-a457e82f8732 on 0  classes,  included groups:[] excluded groups:[]
[TestNG] RUNNING: Suite: "Command line test 8a5d353c-34f6-4707-8902-a457e82f8732" containing "0" Tests (config: null)
[TestNG] INVOKING: "Command line test 8a5d353c-34f6-4707-8902-a457e82f8732" - com.kms.example.aws_ios.test.TestIos.test()
Nov 08, 2022 1:16:29 AM com.kms.example.aw_ios.utils.ConsoleLogger info
INFO: KATALON_VERSION: 8.1.0

Nov 08, 2022 1:16:29 AM com.kms.example.aw_ios.utils.ConsoleLogger info
INFO: KATALON_PROJECT_PACKAGE_FILE: myproject.zip

Nov 08, 2022 1:16:29 AM com.kms.example.aw_ios.utils.ConsoleLogger info
INFO: KATALON_PROJECT_PATH: 

Nov 08, 2022 1:16:29 AM com.kms.example.aw_ios.utils.ConsoleLogger info
INFO: DEVICEFARM_LOG_DIR: /tmp/customer_log_directory8Lmdnb

/tmp/scratchylTcFv.scratch/test-packagewCxDVZ/tmp/test-3495561724695806999.zip
java.io.FileNotFoundException: /tmp/scratchylTcFv.scratch/test-packagewCxDVZ/tmp/__MACOSX/myproject/._.DS_Store (No such file or directory)
	at java.io.FileOutputStream.open0(Native Method)
	at java.io.FileOutputStream.open(FileOutputStream.java:270)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
	at com.kms.example.aw_ios.utils.SideloadUtils.unzip(SideloadUtils.java:66)
	at com.kms.example.aw_ios.utils.SideloadUtils.executeKatalon(SideloadUtils.java:40)
	at com.kms.example.aws_ios.test.TestIos.test(TestIos.java:74)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.testng.junit.JUnit4TestRunner.start(JUnit4TestRunner.java:81)
	at org.testng.junit.JUnit4TestRunner.run(JUnit4TestRunner.java:69)
	at org.testng.TestRunner$1.run(TestRunner.java:682)
	at org.testng.TestRunner.runWorkers(TestRunner.java:1008)
	at org.testng.TestRunner.privateRunJUnit(TestRunner.java:713)
	at org.testng.TestRunner.run(TestRunner.java:614)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:343)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:305)
	at org.testng.SuiteRunner.run(SuiteRunner.java:254)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
	at org.testng.TestNG.run(TestNG.java:1057)
	at org.testng.TestNG.privateMain(TestNG.java:1364)
	at org.testng.TestNG.main(TestNG.java:1333)
[TestNG] FAILED: "Command line test 8a5d353c-34f6-4707-8902-a457e82f8732" - com.kms.example.aws_ios.test.TestIos.test() finished in 701 ms
[TestNG] java.lang.AssertionError: Failed to execute Katalon
[TestNG] 	at org.junit.Assert.fail(Assert.java:88)
[TestNG] 	at com.kms.example.aws_ios.test.TestIos.test(TestIos.java:84)
[TestNG] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[TestNG] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[TestNG] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[TestNG] 	at java.lang.reflect.Method.invoke(Method.java:498)
[TestNG] 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
[TestNG] 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
[TestNG] 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
[TestNG] 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
[TestNG] 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
[TestNG] 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
[TestNG] 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
[TestNG] 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
[TestNG] 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
[TestNG] 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
[TestNG] 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
[TestNG] 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
[TestNG] 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
[TestNG] 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
[TestNG] 	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
[TestNG] 	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
[TestNG] 	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
[TestNG] 	at org.testng.junit.JUnit4TestRunner.start(JUnit4TestRunner.java:81)
[TestNG] 	at org.testng.junit.JUnit4TestRunner.run(JUnit4TestRunner.java:69)
[TestNG] 	at org.testng.TestRunner$1.run(TestRunner.java:682)
[TestNG] 	at org.testng.TestRunner.runWorkers(TestRunner.java:1008)
[TestNG] 	at org.testng.TestRunner.privateRunJUnit(TestRunner.java:713)
[TestNG] 	at org.testng.TestRunner.run(TestRunner.java:614)
[TestNG] 	at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
[TestNG] 	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:343)
[TestNG] 	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:305)
[TestNG] 	at org.testng.SuiteRunner.run(SuiteRunner.java:254)
[TestNG] 	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
[TestNG] 	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
[TestNG] 	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
[TestNG] 	at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
[TestNG] 	at org.testng.TestNG.run(TestNG.java:1057)
[TestNG] 	at org.testng.TestNG.privateMain(TestNG.java:1364)
[TestNG] 	at org.testng.TestNG.main(TestNG.java:1333)
[JUnit4TestMethod] Method 'initializationError' not found in class 'com.kms.example.aw_ios.utils.KatalonDownloadUtils$1': com.kms.example.aw_ios.utils.KatalonDownloadUtils$1.initializationError()
[JUnit4TestMethod] Method 'initializationError' not found in class 'com.kms.example.aw_ios.utils.KatalonDownloadUtils$1': com.kms.example.aw_ios.utils.KatalonDownloadUtils$1.initializationError()
[JUnit4TestMethod] Method 'initializationError' not found in class 'com.kms.example.aw_ios.utils.KatalonDownloadUtils$1': com.kms.example.aw_ios.utils.KatalonDownloadUtils$1.initializationError()
[JUnit4TestMethod] Method 'initializationError' not found in class 'com.kms.example.aw_ios.utils.KatalonDownloadUtils': com.kms.example.aw_ios.utils.KatalonDownloadUtils.initializationError()
[JUnit4TestMethod] Method 'initializationError' not found in class 'com.kms.example.aw_ios.utils.KatalonDownloadUtils': com.kms.example.aw_ios.utils.KatalonDownloadUtils.initializationError()
[JUnit4TestMethod] Method 'initializationError' not found in class 'com.kms.example.aw_ios.utils.KatalonDownloadUtils': com.kms.example.aw_ios.utils.KatalonDownloadUtils.initializationError()
[JUnit4TestMethod] Method 'initializationError' not found in class 'com.kms.example.aw_ios.utils.KatalonUtils': com.kms.example.aw_ios.utils.KatalonUtils.initializationError()
[JUnit4TestMethod] Method 'initializationError' not found in class 'com.kms.example.aw_ios.utils.ConsoleLogger': com.kms.example.aw_ios.utils.ConsoleLogger.initializationError()
[JUnit4TestMethod] Method 'initializationError' not found in class 'com.kms.example.aw_ios.utils.ConsoleLogger': com.kms.example.aw_ios.utils.ConsoleLogger.initializationError()
[JUnit4TestMethod] Method 'initializationError' not found in class 'com.kms.example.aw_ios.utils.SideloadUtils': com.kms.example.aw_ios.utils.SideloadUtils.initializationError()
===== Invoked methods
    TestIos.test()[pri:0, instance:test(com.kms.example.aws_ios.test.TestIos)] 540159270
=====
Creating /tmp/customer_log_directory8Lmdnb/test-output/Jar suite/Command line test 8a5d353c-34f6-4707-8902-a457e82f8732.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/Jar suite/Command line test 8a5d353c-34f6-4707-8902-a457e82f8732.xml
FAILED: test on test(com.kms.example.aws_ios.test.TestIos)
java.lang.AssertionError: Failed to execute Katalon
	at org.junit.Assert.fail(Assert.java:88)
	at com.kms.example.aws_ios.test.TestIos.test(TestIos.java:84)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.testng.junit.JUnit4TestRunner.start(JUnit4TestRunner.java:81)
	at org.testng.junit.JUnit4TestRunner.run(JUnit4TestRunner.java:69)
	at org.testng.TestRunner$1.run(TestRunner.java:682)
	at org.testng.TestRunner.runWorkers(TestRunner.java:1008)
	at org.testng.TestRunner.privateRunJUnit(TestRunner.java:713)
	at org.testng.TestRunner.run(TestRunner.java:614)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:343)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:305)
	at org.testng.SuiteRunner.run(SuiteRunner.java:254)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
	at org.testng.TestNG.run(TestNG.java:1057)
	at org.testng.TestNG.privateMain(TestNG.java:1364)
	at org.testng.TestNG.main(TestNG.java:1333)


===============================================
    Command line test 8a5d353c-34f6-4707-8902-a457e82f8732
    Tests run: 1, Failures: 1, Skips: 0
===============================================

[TestNG] 
[TestNG] ===============================================
[TestNG]     Command line test 8a5d353c-34f6-4707-8902-a457e82f8732
[TestNG]     Tests run: 1, Failures: 1, Skips: 0
[TestNG] ===============================================

===============================================
Jar suite
Total tests run: 1, Failures: 1, Skips: 0
===============================================

[TestNG] Time taken by org.testng.reporters.XMLReporter@7382f612: 19 ms
Creating /tmp/customer_log_directory8Lmdnb/test-output/testng-failed.xml
Creating /tmp/customer_log_directory8Lmdnb/test-output/Jar suite/testng-failed.xml
[TestNG] Time taken by [FailedReporter passed=0 failed=0 skipped=0]: 6 ms
[TestNG] Time taken by org.testng.reporters.jq.Main@39a054a5: 21 ms
[TestNG] Time taken by org.testng.reporters.EmailableReporter2@67f89fa3: 6 ms
Creating /tmp/customer_log_directory8Lmdnb/test-output/junitreports/TEST-com.kms.example.aws_ios.test.TestIos.xml
[TestNG] Time taken by org.testng.reporters.JUnitReportReporter@43556938: 3 ms
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/toc.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/Command line test 8a5d353c-34f6-4707-8902-a457e82f8732.properties
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/index.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/main.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/groups.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/methods.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/methods.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/methods.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/methods-alphabetical.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/methods-alphabetical.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/methods-alphabetical.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/classes.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/reporter-output.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/methods-not-run.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/Jar suite/testng.xml.html
Creating /tmp/customer_log_directory8Lmdnb/test-output/old/index.html
[TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@77468bd9: 6 ms
 
[DEVICEFARM] ########### Entering phase post_test ###########
 
 
[DEVICEFARM] ########### Finish executing testspec ###########
 
[DEVICEFARM] ########### Setting upload permissions ###########
 
 
[DEVICEFARM] Tearing down your device. Your tests report will come shortly.

Thank you for sharing this. Our team is looking into the issue and get back to you soon.

1 Like

Hi @rie.ota ,

Thank you for raising your issue. Based on the error log, you can try 2 below ways to solve the problem:

  1. Appium Java TestNG requires testng.xml file at the root of the *-tests.jar file to execute. Please follow the steps mentioned in this AWS document to deal with that issue.

  2. You should not delete .DS_Store file because it is a system file hidden by MacOS. This means to support MacOS to display folders. Based on the error log, it is a compile error log. Java did not find that file in file input/output stream (literally opening and closing files). (Reference). Please bring it back if the first way still did not work.

Hope this answer can help the issue work. Please feel free to reach out to us if you need more help. Thank you!

1 Like

@Elly_Tran Sorry for being a newbie.
What should I put in the contents of the testng.xml file?
It does not work with just the following

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >

It’s ok. Initially, please make sure you have verified that your testng.xml file is in the root of your Appium test package’s *-tests.jar file (step 5 in the previous AWS document link). In terms of the content of the testing.xml file, it will vary depending on the output you expect. Just need to add some classes or description by referring to this link. Thank you!

1 Like