Jenkins-Plugin that uses Lisa Invoke API

Idea created by stephan.burkard on Apr 11, 2016
    Delivered
    Score36

    Hi

     

    Update 2016-04-15: I updated the description below with first findings on Invoke-API.

     

    The approach of integrating Lisa tests into CI through the Lisa Ant task has the problem of remoting. The Ant task must run on a Lisa server while the trigger to run the tests and the report files must come and go to the CI server. I guess nobody wants to install Lisa on the CI server(s).

     

    There are a lot of questions around how to integrate Lisa into CI. I think it would be great to have a Jenkins plugin for Lisa that uses the Lisa Invoke API and can do the following:

    1. Execute tests or testsuites on a remote machine via REST call
      1. This is possible with Invoke API (for example "runSuite" command)
    2. Executions can be made synchronous
      1. Test results are immediately available in the response
      2. BUT I think the result is unfortunately NOT JUnit compatible
      3. E.g. if a testsuite is executed and 1 test succeeds and 1 fails, I don't see which one failed
    3. Executions can also made asynchronous
      1. Tests or even suites can take a long time to run, so the REST call cannot wait for every result
      2. There is a parameter to run an execution async
      3. The response has no results, but a "callbackKey" that can be used to poll the results
    4. Then the plugin should poll the test results with the given callbackKey in a configurable interval and number of times
      1. Tests or suites can massively differ how long they run, so it would be nice to configure the interval and the number of times to poll per test/suite execution
      2. The number of times to poll is important to avoid infinite polling
    5. The test results could easily be transformed to a JUnit compatible format, so Jenkins can generate graphics etc as for other tests
      1. see http://stackoverflow.com/questions/4922867/junit-xml-format-specification-that-hudson-supports for the JUnit XML report format that is needed for Jenkins

     

    Stephan