Katalon and AWS device farm integration

Product Forums Katalon Studio
,
1

Hi Katalon Community :waving_hand:,

I’m looking to integrate Katalon Studio with AWS Device Farm in order to execute my mobile test cases (Android/iOS) on real devices hosted in the cloud.

So far, I’m familiar with uploading Appium-based tests (e.g., in Python) to AWS Device Farm using their CLI/SDK. However, I’d like to understand:

:red_question_mark: My Questions:

  1. Is there a native or recommended way to integrate Katalon Studio with AWS Device Farm?
  • Can we export Katalon test cases in a compatible format for Device Farm (e.g., as an Appium Java JAR or APK)?
  1. Has anyone successfully run their Katalon-generated mobile test cases (Android/iOS) on AWS Device Farm?
  • If so, what was the setup? Any steps or examples would be very helpful.
  1. Can Katalon TestOps or CLI Runner (Katalonc) be used in combination with AWS Device Farm?
  • For example, packaging a test suite and uploading it as a zip?
1 Like
2

Hi @mandeep.singh1,

Welcome to our community. Thank you for sharing your questions

  1. You can read more about Katalon integration with AWS device farm here: AWS Device Farm integration | Katalon Docs. As far as I know, I cannot find any guide on exporting a test case to the Device Farm format
  2. Welcome everyone to share
  3. You can integrate with Katalon TestOps following this: Integrate Katalon Platform with Katalon Studio | Katalon Docs and using CLI to triggle the test: Katalon CLI | Katalon Docs.

Thank you

3

Here’s a structured guide to integrating Katalon Studio with AWS Device Farm for running mobile tests on real devices, including workarounds and best practices from the community:

1. Is Native Integration Possible?

  • No direct integration exists between Katalon Studio and AWS Device Farm. However, since Katalon uses Appium under the hood, you can repackage Katalon tests into a format AWS Device Farm accepts (e.g., Java JAR + Appium).

2. Export Katalon Tests for AWS Device Farm

AWS Device Farm requires tests to be in Appium Java JAR or Python/Node.js format. Here’s how to adapt Katalon tests:

Option 1: Export as JAR via Katalon CLI (katalonc)

  • Step 1: Generate an executable JAR from Katalon tests:
katalonc -noSplash -runMode=console -projectPath="path/to/project.prj" -retry=0 -testSuitePath="Test Suites/YourTestSuite" -browserType="Android" -executionProfile="default" -apiKey="YOUR_KATALON_API_KEY" -exportJAR="path/to/output.jar"
  • Step 2: Package the JAR with dependencies (AWS requires a self-contained JAR):
    • Use Maven/pom.xml to bundle dependencies (e.g., Appium, Selenium).
    • Example pom.xml for AWS Device Farm here.

Option 2: Use Katalon’s Appium-Compatible Output

  • Katalon generates Appium-compatible test scripts in Groovy/Java. Manually refactor these into a Maven/Java project.
  • Example directory structure for AWS:
├── src/
│   └── test/
│       └── java/
│           └── YourTestClass.java (refactored Katalon scripts)
├── pom.xml (with Appium/Selenium dependencies)
└── test_spec.yml (AWS test spec)

3. AWS Device Farm Setup

Prerequisites

  • AWS CLI installed and configured.
  • Test Spec File: Define how AWS should execute your tests:
# test_spec.yml
version: 0.1
phases:
  install:
    commands:
      - echo "Installing dependencies..."
  test:
    commands:
      - java -jar your-katalon-tests.jar

Upload and Run

  1. Zip your JAR, test_spec.yml, and dependencies:
zip -r katalon-aws-tests.zip your-katalon-tests.jar test_spec.yml
  1. Use AWS CLI to schedule a run:
aws devicefarm schedule-run \
    --project-arn "arn:aws:devicefarm:us-west-2:123456789:project:..." \
    --app-arn "arn:aws:devicefarm:us-west-2::upload:..." \
    --device-pool-arn "arn:aws:devicefarm:us-west-2::devicepool:..." \
    --name "Katalon Test Run" \
    --test type=APPIUM_JAVA_JUNIT,testPackageArn="arn:aws:devicefarm:us-west-2::upload:..."

4. Katalon TestOps & CLI Integration

  • TestOps: Use TestOps to trigger AWS Device Farm runs via API/Webhooks. Example flow:
    1. Upload tests to AWS via TestOps CI/CD pipeline.
    2. Fetch results back into TestOps for reporting.
  • Katalon CLI (katalonc): Automate JAR generation and AWS upload:
# Generate JAR
katalonc -noSplash -runMode=console ... -exportJAR="katalon.jar"

# Upload to AWS via AWS CLI
aws devicefarm create-upload --project-arn "your-project-arn" --name "katalon-tests.zip" --type APPIUM_JAVA_JUNIT_TEST_PACKAGE

5. Community Success Stories

  • Workaround: Users have successfully run Katalon tests by:
    1. Refactoring Katalon scripts into Maven projects.
    2. Using AWS’s custom environment variables to match Katalon’s capabilities.
    3. Adjusting desired capabilities in Katalon to align with AWS’s device farm requirements (e.g., udid, platformName).
  • Example capabilities in Katalon:
"appium:deviceName" => "Any",  // AWS overrides this
"appium:platformVersion" => "12.0",
"appium:app" => "your_app.apk"

6. Troubleshooting Tips

  • Dependency Conflicts: Ensure your JAR’s dependencies match AWS’s supported versions (check AWS Appium docs).
  • Logs: Use AWS’s devicefarm console to download logcat and Katalon execution logs.
  • Permissions: Grant AWS IAM roles access to Device Farm and S3.

7. Alternatives

  • Third-Party CI/CD: Use Jenkins/GitHub Actions to trigger Katalon CLI executions and AWS runs in parallel.
  • Dockerize Katalon: Run Katalon in a Docker container on AWS EC2/Batch (not Device Farm).

By repackaging Katalon tests into AWS-compatible formats and leveraging CLI tools, you can execute Katalon mobile tests on AWS Device Farm