Hello there.
I’ve set up Jenkins and Katalon Studio on the same machine. I’m running an iOS .app-file on a simulator.
I have a test suite consisting of two test cases, where the first case is on-boarding an app and closing it, and the second case is opening the app again and logging in. The first case passes, while the second case fails, because the simulator is always cleaned and thus the wrong screen appears when opening the app.
The following is part of the console output accessible in Jenkins for the project:
Project ‘iOS Mobile Tests with Katalon Studio’ opened
Test Suites/Onboarding and login - iOS - 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 -
20190125_143912…0/2(0%)Request sent successfully.
2019-01-25 14:39:13.996 INFO c.k.katalon.core.main.TestSuiteExecutor - START Test Suites/Onboarding and login
2019-01-25 14:39:14.045 INFO c.k.katalon.core.main.TestSuiteExecutor - hostName = admin - 10.50.10.158
2019-01-25 14:39:14.046 INFO c.k.katalon.core.main.TestSuiteExecutor - os = Mac OS X 64bit
2019-01-25 14:39:14.046 INFO c.k.katalon.core.main.TestSuiteExecutor - hostAddress = 10.50.10.158
2019-01-25 14:39:14.047 INFO c.k.katalon.core.main.TestSuiteExecutor - katalonVersion = 5.10.1.1
2019-01-25 14:39:14.291 INFO c.k.katalon.core.main.TestCaseExecutor - --------------------
2019-01-25 14:39:14.292 INFO c.k.katalon.core.main.TestCaseExecutor - START Test Cases/OpenClose_onboarding_no-uninstall
2019-01-25 14:39:14.645 DEBUG t.OpenClose_onboarding_no-uninstall - 1: startApplication(app_path, false)
2019-01-25 14:39:14.846 INFO c.k.k.c.m.k.i.MobileDriverFactory - User set preference: [‘noReset’, ‘true’]
2019-01-25 14:39:14.847 INFO c.k.k.c.m.k.i.MobileDriverFactory - User set preference: [‘fullReset’, ‘false’]
2019-01-25 14:39:14.847 INFO c.k.k.c.m.k.i.MobileDriverFactory - User set preference: [‘deviceId’, ‘37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95’]
2019-01-25 14:39:15.577 INFO c.k.k.c.a.driver.AppiumDriverManager - ios_webkit_debug_proxy server started on port 52908
2019-01-25 14:39:18.619 INFO c.k.k.c.a.driver.AppiumDriverManager - Appium server started on port 52,910
e[35m[Appium]e[39m Welcome to Appium v1.8.1
e[35m[Appium]e[39m Non-default server args:
e[35m[Appium]e[39m port: 52910
e[35m[Appium]e[39m loglevel: info
e[35m[Appium]e[39m chromeDriverPort: 52911
e[35m[Appium]e[39m tmpDir: /var/folders/ck/nvhzb8n96p193ztpl_ty6kvw0000gn/T//Katalon/Appium/Temp1548423555581
e[35m[Appium]e[39m webkitDebugProxyPort: 52908
e[35m[Appium]e[39m Appium REST http interface listener started on 0.0.0.0:52910
e[35m[HTTP]e[39m e[37m–>e[39m e[37mGETe[39m e[37m/wd/hub/statuse[39m
e[35m[HTTP]e[39m e[90m{}e[39m
e[35m[HTTP]e[39m e[37m<-- GET /wd/hub/status e[39me[32m200e[39m e[90m18 ms - 83e[39m
e[35m[HTTP]e[39m e[90me[39m
e[35m[HTTP]e[39m e[37m–>e[39m e[37mPOSTe[39m e[37m/wd/hub/sessione[39m
e[35m[HTTP]e[39m e[90m{“desiredCapabilities”:{“app”:“/Users/admin/Documents/appfiles/MySuperApp.app”,“automationName”:“XCUITest”,“deviceId”:“37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95”,“deviceName”:“iPhone 6 Plus”,“fullReset”:false,“newCommandTimeout”:1800,“noReset”:true,“platform”:“IOS”,“platformName”:“iOS”,“platformVersion”:“11.4”,“realDeviceLogger”:“/Applications/Katalon Studio.app/Contents/Eclipse/configuration/resources/tools/deviceconsole/deviceconsole”,“waitForAppScript”:“true;”,“wdaLocalPort”:52907},“capabilities”:{“desiredCapabilities”:{“app”:“/Users/admin/Documents/appfiles/MySuperApp.app”,“automationName”:“XCUITest”,“deviceId”:“37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95”,“deviceName”:“iPhone 6 Plus”,“fullReset”:false,“newCommandTimeout”:1800,“noReset”:true,“platform”:“IOS”,“platformName”:“iOS”,“platformVersion”:“11.4”,“realDeviceLogger”:“/Applications/Katalon Studio.app/Contents/Eclipse/configuration/resources/tools/deviceconsole/deviceconsole”,“waitForAppScript”:“true;”,“wdaLocalPort”:52907},"firstMatche[39m
e[35m[Appium]e[39m Could not parse W3C capabilities: ‘deviceName’ can’t be blank. Falling back to JSONWP protocol.
e[35m[Appium]e[39m The following capabilities were provided in the JSONWP desired capabilities that are missing in W3C capabilities: [“app”,“automationName”,“deviceId”,“deviceName”,“fullReset”,“newCommandTimeout”,“noReset”,“platform”,“platformName”,“platformVersion”,“realDeviceLogger”,“waitForAppScript”,“wdaLocalPort”]. Falling back to JSONWP protocol.
e[35m[Appium]e[39m Creating new XCUITestDriver (v2.84.0) session
e[35m[Appium]e[39m Capabilities:
e[35m[Appium]e[39m app: /Users/admin/Documents/appfiles/MySuperApp.app
e[35m[Appium]e[39m automationName: XCUITest
e[35m[Appium]e[39m deviceId: 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95
e[35m[Appium]e[39m deviceName: iPhone 6 Plus
e[35m[Appium]e[39m fullReset: false
e[35m[Appium]e[39m newCommandTimeout: 1800
e[35m[Appium]e[39m noReset: true
e[35m[Appium]e[39m platform: IOS
e[35m[Appium]e[39m platformName: iOS
e[35m[Appium]e[39m platformVersion: 11.4
e[35m[Appium]e[39m realDeviceLogger: /Applications/Katalon Studio.app/Contents/Eclipse/configuration/resources/tools/deviceconsole/deviceconsole
e[35m[Appium]e[39m waitForAppScript: true;
e[35m[Appium]e[39m wdaLocalPort: 52907
e[35m[BaseDriver]e[39m The following capabilities were provided, but are not recognized by appium: deviceId, platform.
e[35m[BaseDriver]e[39m Session created with session id: bd5f9f3a-518d-4e46-8804-ef30c126dcf3
e[35m[iOSSim]e[39m Constructing iOS simulator for Xcode version 9.4.1 with udid ‘37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95’
e[35m[XCUITest]e[39m Determining device to run tests on: udid: ‘37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95’, real device: false
e[35m[BaseDriver]e[39m Using local app ‘/Users/admin/Documents/appfiles/MySuperApp.app’
e[35m[XCUITest]e[39m Continuing without capturing device logs: iOS Simulator with udid 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 is not running
e[35m[XCUITest]e[39m Setting up simulator
e[35m[iOSSim]e[39m Booting Simulator with UDID 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95…
e[35m[iOSSim]e[39m Starting Simulator UI with command: open -Fn /Applications/Xcode9.4.1.app/Contents/Developer/Applications/Simulator.app --args -CurrentDeviceUDID 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 -ConnectHardwareKeyboard 0Test Suites/Onboarding and login - iOS - 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 -
20190125_143912…0/2(0%)e[35m[iOSSim]e[39m Simulator with UDID 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 booted in 12 seconds
e[35m[XCUITest]e[39m Using WDA path: ‘/usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent’
e[35m[XCUITest]e[39m Using WDA agent: ‘/usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent/WebDriverAgent.xcodeproj’
e[35m[XCUITest]e[39m Launching WebDriverAgent on the deviceTest Suites/Onboarding and login - iOS - 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 -
20190125_143912…0/2(0%)Jan 25, 2019 2:39:44 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: OSS
2019-01-25 14:39:44.478 INFO c.k.k.c.a.driver.AppiumRequestService - appiumVersion = 1.8.1
2019-01-25 14:39:44.504 INFO c.k.k.c.a.driver.AppiumDriverManager - deviceId = null
2019-01-25 14:39:44.505 INFO c.k.k.c.a.driver.AppiumDriverManager - deviceName = iPhone 6 Plus
2019-01-25 14:39:44.505 INFO c.k.k.c.a.driver.AppiumDriverManager - deviceModel = Not available for Simulator
2019-01-25 14:39:44.506 INFO c.k.k.c.a.driver.AppiumDriverManager - deviceManufacturer = Apple
2019-01-25 14:39:44.506 INFO c.k.k.c.a.driver.AppiumDriverManager - deviceOS = iOS
2019-01-25 14:39:44.506 INFO c.k.k.c.a.driver.AppiumDriverManager - deviceOSVersion = 11.4
2019-01-25 14:39:44.509 DEBUG t.OpenClose_onboarding_no-uninstall - 2: if (waitForElementPresent(findTestObject(“Login/login_presentation_Button_next”), 0))
2019-01-25 14:39:44.549 WARN c.kms.katalon.core.helper.KeywordHelper - Timeout ‘0’ is invalid. Using default page load timeout: ‘30’
e[35m[XCUITest]e[39m Skipping setting of the initial display orientation. Set the “orientation” capability to either “LANDSCAPE” or “PORTRAIT”, if this is an undesired behavior.
e[35m[Appium]e[39m New XCUITestDriver session created successfully, session bd5f9f3a-518d-4e46-8804-ef30c126dcf3 added to master session list
[…]
(Test case 1 finishes)
Now, for test case 2, follows more output:
[…]
[35m[Appium]e[39m Creating new XCUITestDriver (v2.84.0) session
e[35m[Appium]e[39m Capabilities:
e[35m[Appium]e[39m app: /Users/admin/Documents/appfiles/MySuperApp.app
e[35m[Appium]e[39m automationName: XCUITest
e[35m[Appium]e[39m deviceId: 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95
e[35m[Appium]e[39m deviceName: iPhone 6 Plus
e[35m[Appium]e[39m fullReset: true
e[35m[Appium]e[39m newCommandTimeout: 1800
e[35m[Appium]e[39m noReset: false
e[35m[Appium]e[39m platform: IOS
e[35m[Appium]e[39m platformName: iOS
e[35m[Appium]e[39m platformVersion: 11.4
e[35m[Appium]e[39m realDeviceLogger: /Applications/Katalon Studio.app/Contents/Eclipse/configuration/resources/tools/deviceconsole/deviceconsole
e[35m[Appium]e[39m waitForAppScript: true;
e[35m[Appium]e[39m wdaLocalPort: 52989
e[35m[BaseDriver]e[39m The following capabilities were provided, but are not recognized by appium: deviceId, platform.
e[35m[BaseDriver]e[39m Session created with session id: 7fb1e27f-27a8-47b0-956b-1688fc60e431
e[35m[iOSSim]e[39m Constructing iOS simulator for Xcode version 9.4.1 with udid ‘37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95’
e[35m[XCUITest]e[39m Determining device to run tests on: udid: ‘37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95’, real device: false
e[35m[BaseDriver]e[39m Using local app ‘/Users/admin/Documents/appfiles/MySuperApp.app’
e[35m[iOSSim]e[39m Constructing iOS simulator for Xcode version 9.4.1 with udid ‘37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95’
e[35m[XCUITest]e[39m Determining device to run tests on: udid: ‘37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95’, real device: false
e[35m[BaseDriver]e[39m Using local app ‘/Users/admin/Documents/appfiles/MySuperApp.app’Test Suites/Onboarding and login - iOS - 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 -
20190125_143912…1/2(50%)e[35m[iOSSim]e[39m Cleaning simulator 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95
e[35m[iOSSim]e[39m Cleaning simulator 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95
e[35m[XCUITest]e[39m Continuing without capturing device logs: iOS Simulator with udid 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 is not running
e[35m[XCUITest]e[39m Setting up simulator
e[35m[iOSSim]e[39m Booting Simulator with UDID 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95…
e[35m[XCUITest]e[39m Continuing without capturing device logs: iOS Simulator with udid 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 is not running
e[35m[XCUITest]e[39m Setting up simulator
e[35m[iOSSim]e[39m Booting Simulator with UDID 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95…Test Suites/Onboarding and login - iOS - 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 -
20190125_143912…1/2(50%)e[35m[iOSSim]e[39m Simulator with UDID 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 booted in 18 seconds
e[35m[iOSSim]e[39m Simulator with UDID 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 booted in 18 seconds
e[35m[XCUITest]e[39m Using WDA path: ‘/usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent’
e[35m[XCUITest]e[39m Using WDA agent: ‘/usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent/WebDriverAgent.xcodeproj’
e[35m[XCUITest]e[39m Using WDA path: ‘/usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent’
e[35m[XCUITest]e[39m Using WDA agent: ‘/usr/local/lib/node_modules/appium/node_modules/appium-xcuitest-driver/WebDriverAgent/WebDriverAgent.xcodeproj’
e[35m[XCUITest]e[39m Launching WebDriverAgent on the device
e[35m[XCUITest]e[39m Launching WebDriverAgent on the deviceTest Suites/Onboarding and login - iOS - 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95 -
20190125_143912…1/2(50%)Jan 25, 2019 2:42:06 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: OSS
2019-01-25 14:42:06.833 INFO c.k.k.c.a.driver.AppiumRequestService - appiumVersion = 1.8.1
2019-01-25 14:42:06.856 INFO c.k.k.c.a.driver.AppiumDriverManager - deviceId = null
2019-01-25 14:42:06.856 INFO c.k.k.c.a.driver.AppiumDriverManager - deviceName = iPhone 6 Plus
2019-01-25 14:42:06.856 INFO c.k.k.c.a.driver.AppiumDriverManager - deviceModel = Not available for Simulator
2019-01-25 14:42:06.857 INFO c.k.k.c.a.driver.AppiumDriverManager - deviceManufacturer = Apple
2019-01-25 14:42:06.857 INFO c.k.k.c.a.driver.AppiumDriverManager - deviceOS = iOS
2019-01-25 14:42:06.857 INFO c.k.k.c.a.driver.AppiumDriverManager - deviceOSVersion = 11.4
2019-01-25 14:42:06.863 DEBUG testcase.InstalledLogin - 2: waitForElementPresent(findTestObject(“InstalledLogin/unlock_StaticText_header”), 0)
2019-01-25 14:42:06.868 WARN c.kms.katalon.core.helper.KeywordHelper - Timeout ‘0’ is invalid. Using default page load timeout: ‘30’
e[35m[XCUITest]e[39m Skipping setting of the initial display orientation. Set the “orientation” capability to either “LANDSCAPE” or “PORTRAIT”, if this is an undesired behavior.
e[35m[Appium]e[39m New XCUITestDriver session created successfully, session 7fb1e27f-27a8-47b0-956b-1688fc60e431 added to master session list
e[35m[HTTP]e[39m e[37m<-- POST /wd/hub/session e[39me[32m200e[39m e[90m41671 ms - 760e[39m
e[35m[HTTP]e[39m e[90me[39m
e[35m[HTTP]e[39m e[37m–>e[39m e[37mGETe[39m e[37m/wd/hub/statuse[39m
e[35m[HTTP]e[39m e[90m{}e[39m
e[35m[HTTP]e[39m e[37m<-- GET /wd/hub/status e[39me[32m200e[39m e[90m24 ms - 55e[39m
e[35m[HTTP]e[39m e[90me[39m
e[35m[HTTP]e[39m e[37m–>e[39m e[37mPOSTe[39m e[37m/wd/hub/session/7fb1e27f-27a8-47b0-956b-1688fc60e431/timeoutse[39m
e[35m[HTTP]e[39m e[90m{“type”:“implicit”,“ms”:30000}e[39m
e[35m[HTTP]e[39m e[37m<-- POST /wd/hub/session/7fb1e27f-27a8-47b0-956b-1688fc60e431/timeouts e[39me[33m400e[39m e[90m17 ms - 4203e[39m
e[35m[HTTP]e[39m e[90me[39m
e[35m[HTTP]e[39m e[37m–>e[39m e[37mGETe[39m e[37m/wd/hub/session/7fb1e27f-27a8-47b0-956b-1688fc60e431/window/current/sizee[39m
e[35m[HTTP]e[39m e[90m{}e[39m
e[35m[HTTP]e[39m e[37m<-- GET /wd/hub/session/7fb1e27f-27a8-47b0-956b-1688fc60e431/window/current/size e[39me[32m200e[39m e[90m63 ms - 98e[39m
e[35m[HTTP]e[39m e[90me[39m
e[35m[HTTP]e[39m e[37m–>e[39m e[37mPOSTe[39m e[37m/wd/hub/session/7fb1e27f-27a8-47b0-956b-1688fc60e431/elementse[39m
e[35m[HTTP]e[39m e[90m{“using”:“xpath”,“value”:“//XCUIElementTypeApplication/XCUIElementTypeWindow[2]/XCUIElementTypeOther[1]/XCUIElementTypeOther[1]/XCUIElementTypeOther[2]/XCUIElementTypeStaticText[1]”}e[39m
[…]
I’m especially confused by the lines of the latter output quote that says “Cleaning simulator 37C0481A-6BFC-4BD9-BAC9-D73D84ADBA95” considering that fullReset = false and noReset = true according to the first output quoted.
I also notice:
[35m[BaseDriver]e[39m Session created with session id: bd5f9f3a-518d-4e46-8804-ef30c126dcf3
from quote 1
e[35m[BaseDriver]e[39m Session created with session id: 7fb1e27f-27a8-47b0-956b-1688fc60e431
from quote 2. Also the wdaLocalPort values are different for the two XCUITestDriver sessions.
Do you have any insight on what I can do to avoid cleaning the simulator, and in test case 2 just open the closed application from test case 1? Maybe I can keep the same driver session through the whole test suite? Where can I specify this?
When launching the suite directly from Katalon Studio (not through Jenkins) it passed nicely, so the inconsistency in behaviour confuses me.
Thank you.