Browser timeouts when using Katalon CLI or Docker

Product Forums Katalon Studio
,
1

We are trying run a test suite collection that is configured to run the same test suite and BrowserStack configuration against 5 different websites in parallel (so 5 test suites are to run in parallel).

When I run the test suite collection via the Katalon Studio GUI, all of the tests run correctly and finish in about 50 minutes. However, every time I run the same tests via the CLI or Docker container (on my Macbook Pro or via CircleCI) with a parallelism set to 2 or more, BrowserStack ends up generating “IDLE TIMEOUT” errors after about 20 minutes. This happens if BrowserStack sees no network activity from Katalon for 90 seconds. If I run the test suite collection with the CLI or Docker container with a parallelism set to 1, the tests seem to run fine, but this would take 250 minutes to run the tests instead of 50.

System Notes:

  • Test Suite contains 42 test cases
  • TestListener set up to run “WebUI.openBrowser(null);” before a test case and “WebUI.closeBrowser();” after a test case.
  • Plugins: Basic Reporting and Slack Integration

Things I’ve tried but failed:

  • Running tests via CLI without Docker
  • Running tests with Katalon Docker container
  • Reordered tests in the test suite
  • Ran first half of tests in one test suite collection and second half of tests in another test suite collection
  • On the “Execution” tab on “Project–>Settings”, ran test suite collection with the, “Terminate drivers after each Test Case” and “Terminate drivers after each Test Suite” option enabled and disabled.
  • Changed the BrowserStack version of Windows 10 Chrome to 76
  • Increased Docker resources to 4 CPU and 6 GB of memory (The Katalon container often used less than 1 CPU and its memory use stayed around 2 GB so I didn’t really expect this to change anything)

There seems to be a difference in how Katalon CLI and Docker container handle WebDriver connections vs how Katalon GUI handles them.

Operating System:

  • Alpine Linux (CircleCI) + Docker 18.09.8
  • OS X (local Machine) + Docker Desktop 2.1.0.2

Katalon Studio Version:

  • GUI version: 6.3.3
  • CLI version: 6.3.3
  • Docker image tags: “latest”, “6.3.0”, “6.2.2”, and “6.3.0.rc5”

Katalon Studio logs:
N/A

Browser and Browser version:
BrowserStack Windows 10: Chrome 77

Steps to reproduce
Here’s the docker compose file we use to run the Katalon Docker container. Our docker image just adds our test code to the /user/src/app directory. PRODUCT is simply the name of one of our products under test.

version: 3.6'

services:
  katalon:
  image: "our-katalon-based-docker-image:latest"
  container_name: 'katalon'
  working_dir: "/usr/src/app/${PRODUCT}"
  entrypoint:
    ["katalon-execute.sh", -apiKey=${KATALON_API_KEY}","-testSuiteCollectionPath=Test Suites/${TEST_SUITE_COLLECTION}" ]

Expected Behavior:
All test suite collections run without BrowserStack timeout errors.

Actual Behavior:

The following error message is displayed in all test runs inBrowserStack dashboard after about 20 minutes:
Timeout(BROWSERSTACK_IDLE_TIMEOUT: Session idle for more than 90 seconds

2

define ‘we’. please …

3

Thanks for responding,

By, “we,” I meant the QA team.

6

i closed this as it is inactive