TestOps Remote Execution (Grid / Test Plans / Repo)

Please can someone clarify exactly how TestOps remote execution works, particularly in terms of configuring Plans in the Grid, and how the code repo works?

We have a runtime license, a dedicated ‘test execution’ machine with an Agent installed, with Katalon Runtime downloaded on that machine.

Some other possible relevant background information - we use GIT to synchronise our project between test engineers PCs, and also to the dedicated execution machine. I can successfully kick-off tests locally on this machine, using a Katalon command (katalonc…) so I’m satisfied the ‘Runtime’ aspect is good.

The problem I faced was that when I started creating a Plan in the Grid, the available selection of Test Suite Collections was not the same as the Test Suite Collections I had created in the project. Reading the help documentation (https://docs.katalon.com/katalon-analytics/docs/code-repo.html) helped me understand the significance of the code repo, in that it seems I’d need to upload a zip file of the project folder each and every time I had new Test Case / Suites / Collections etc.

Is that the only way to do this?

I had envisaged being able to select a Test Suite Collection within a plan, and for the project file synchronised on our ‘execution machine’ to be executed (rather than a project from an uploaded zip file). But I think that currently this is a misunderstanding on my part?

This leaves me with three main questions…

  1. Are there plans by Katalon to enable us to select Test Suite Collections that reside in the project (i.e. NOT an exported zip file of the project)? (I have TestOps integration enabled, so surely TestOps Cloud ‘knows’ about my projects, right?)

  2. If I use choose to use ‘Katalon Command’ in the plan, and I paste in a command from Katalon Studio, will this again take the details from the code repo, or from the local data on the machine with the agent installed?

NOTE: currently, when I select this option, I see errors like “Found multiple arguments for option projectPath, but you asked for only one”…but then if I remove the projectPath option from the command, it fails (as you would expect). How should this work?

  1. Are there any potential workarounds for this, perhaps using the ‘Generic Command’ option?

Basically, I’d like our Katalon test planning and scheduling to be seamless, and not have to manually obtain, then upload a zip file prior to each scheduled execution (this seems completely counter-intuitive to me), so other than perhaps scripting a GIT Pull, I just want the tests to execute on the latest project data.

Any suggestions or corrections of any misunderstandings I have would be greatly appreciated, as would confirmation from Katalon that such a feature is being developed, or is planned to be developed. As it stands, this is the missing piece that probably means we will have to use something else for scheduling our tests.

Many thanks.

1 Like

Hi @kevin.mcandrew,

Please find below the information you needed.

Yes, this is the only way to do with Code Repo.

Currently, you need to make sure your data in TestOps is up-to-date. However, we will take this into consideration as a feature for TestOps and will let you know as soon as the final decision is made.

The details will be taken from the latest .zip file of the Test Project associated with your configured Plan. Please refer to the screenshot below as an example.

Would you mind sending me the command that you are using?

Similar to Katalon Command, Generic Command downloads .zip file from your Test Project. However, Generic Command allows you to execute commands from other tools instead of Katalon.

Thank you and hope this helps.

@Beo

Thank you for your response, and apologies for the delay in replying - I missed this one.

If I’m honest, I don’t see the current solution as workable, at least not for any scale. We have hours and hours of test executions even evening/night, and they need to be fully unattended.

Even if I manually uploaded a zip file of the Katalon Project every afternoon, how would this work on my days off?! I think perhaps this is missing the point of test ‘automation’.

Unless I’m missing something, this isn’t a workable solution for anything of any scale, so I would politely urge Katalon to work on an improved offering - e.g. using GIT repositories - in the future.

Thanks,
Kevin

@ThanhTo - this does indeed seem like what we were looking for - is this new? I can’t seem to find the 'Create Git Test Project button as described in your documentation.

Hi @kevin.mcandrew

Apparently the doc is released early, it’s an up-coming feature, I’ll notify you when this is available. Sorry for the confusion.

Thanks @ThanhTo - this will indeed be potentially useful for the future - no problem re confusion, it’s good to know that the feature will be coming :slight_smile:

1 Like