Please can someone clarify exactly how TestOps remote execution works, particularly in terms of configuring Plans in the Grid, and how the code repo works?
We have a runtime license, a dedicated ‘test execution’ machine with an Agent installed, with Katalon Runtime downloaded on that machine.
Some other possible relevant background information - we use GIT to synchronise our project between test engineers PCs, and also to the dedicated execution machine. I can successfully kick-off tests locally on this machine, using a Katalon command (katalonc…) so I’m satisfied the ‘Runtime’ aspect is good.
The problem I faced was that when I started creating a Plan in the Grid, the available selection of Test Suite Collections was not the same as the Test Suite Collections I had created in the project. Reading the help documentation (https://docs.katalon.com/katalon-analytics/docs/code-repo.html) helped me understand the significance of the code repo, in that it seems I’d need to upload a zip file of the project folder each and every time I had new Test Case / Suites / Collections etc.
Is that the only way to do this?
I had envisaged being able to select a Test Suite Collection within a plan, and for the project file synchronised on our ‘execution machine’ to be executed (rather than a project from an uploaded zip file). But I think that currently this is a misunderstanding on my part?
This leaves me with three main questions…
Are there plans by Katalon to enable us to select Test Suite Collections that reside in the project (i.e. NOT an exported zip file of the project)? (I have TestOps integration enabled, so surely TestOps Cloud ‘knows’ about my projects, right?)
If I use choose to use ‘Katalon Command’ in the plan, and I paste in a command from Katalon Studio, will this again take the details from the code repo, or from the local data on the machine with the agent installed?
NOTE: currently, when I select this option, I see errors like “Found multiple arguments for option projectPath, but you asked for only one”…but then if I remove the projectPath option from the command, it fails (as you would expect). How should this work?
- Are there any potential workarounds for this, perhaps using the ‘Generic Command’ option?
Basically, I’d like our Katalon test planning and scheduling to be seamless, and not have to manually obtain, then upload a zip file prior to each scheduled execution (this seems completely counter-intuitive to me), so other than perhaps scripting a GIT Pull, I just want the tests to execute on the latest project data.
Any suggestions or corrections of any misunderstandings I have would be greatly appreciated, as would confirmation from Katalon that such a feature is being developed, or is planned to be developed. As it stands, this is the missing piece that probably means we will have to use something else for scheduling our tests.