Jenkins

Overview

The Endtest Jenkins plugin lets you start Endtest automated tests directly from a Jenkins Pipeline, wait for the executions to finish, and use the results in later Pipeline steps.

The plugin supports:

• Secure storage of Endtest API credentials in Jenkins
• Single Endtest executions
• Multiple executions started by the same API request
• Label-based Endtest API requests
• Automatic polling until execution results are available
• Aggregate results across multiple executions
• Optional Jenkins build failure when Endtest reports failures or errors
• Resuming result polling after a Jenkins controller restart
• json and json-light result formats

Requirements

Before configuring the integration, make sure you have:

• Jenkins 2.528.3 or newer
• Permission to install Jenkins plugins and create credentials
• An Endtest account
• An Endtest App ID and App Code
• An Endtest API request that starts the test suite, configuration, or label you want to run

For more information about creating an API request, see How to use the Endtest API.

Install the Endtest plugin

1. In Jenkins, open Manage Jenkins.
2. Select Plugins.
3. Open the Available plugins tab.
4. Search for Endtest.
5. Select the Endtest plugin and install it.
6. Restart Jenkins if your Jenkins installation asks you to do so.

The plugin page is available at:

https://plugins.jenkins.io/endtest/

Add Endtest credentials to Jenkins

Storing the App ID and App Code in Jenkins Credentials is recommended. This keeps the credentials out of the Jenkinsfile and allows the same credential entry to be reused by multiple jobs.

1. In Jenkins, open Manage Jenkins.
2. Select Credentials.
3. Open the credential store and domain where the Pipeline should have access.
4. Select Add Credentials.
5. For Kind, select Endtest API credentials.
6. Enter the following values:

The examples in this guide use the credential ID endtest-api.

The Jenkins credential ID is not the Endtest App ID. It is the identifier Jenkins uses to locate the stored credential.

Prepare the Endtest API request

Create or copy an Endtest API request for the test suite, configuration, or label you want Jenkins to execute.

The request must:

• Use HTTPS
• Use the app.endtest.io host
• Start an Endtest execution
• Include all execution settings required by your Endtest project

A request may contain appId and appCode, but storing those values in Jenkins Credentials is safer. When a credential ID is provided, the plugin automatically adds any missing appId or appCode values to the request.

When credentials are present both in the API request and in Jenkins Credentials, the values in the API request take precedence.

Add Endtest to a Jenkins Pipeline

The plugin provides the Pipeline step:

endtestRun(...)

A basic Declarative Pipeline example looks like this:

pipeline {
    agent any

    stages {
        stage('Run Endtest') {
            steps {
                script {
                    def result = endtestRun(
                        credentialsId: 'endtest-api',
                        apiRequest: 'https://app.endtest.io/api.php?<YOUR_ENDTEST_API_PARAMETERS>'
                    )

                    echo "Executions: ${result.executionCount}"
                    echo "Test cases: ${result.testCases}"
                    echo "Passed: ${result.passed}"
                    echo "Failed: ${result.failed}"
                    echo "Errors: ${result.errors}"
                }
            }
        }
    }
}

Replace the value of apiRequest with the API request generated for your Endtest tests.

By default, the plugin:

• Waits for up to 30 minutes
• Checks the Endtest results every 30 seconds
• Uses the json-light results format
• Fails the Jenkins build when Endtest reports failed assertions or errors

Complete Pipeline example

The following example displays aggregate results and a link to each Endtest execution:

pipeline {
    agent any

    stages {
        stage('Run Endtest') {
            steps {
                script {
                    def result = endtestRun(
                        credentialsId: 'endtest-api',
                        apiRequest: 'https://app.endtest.io/api.php?<YOUR_ENDTEST_API_PARAMETERS>',
                        timeoutMinutes: 45,
                        pollIntervalSeconds: 30,
                        failBuild: true,
                        resultsFormat: 'json-light'
                    )

                    echo "Endtest completed ${result.executionCount} execution(s)"
                    echo "Test cases: ${result.testCases}"
                    echo "Passed: ${result.passed}"
                    echo "Failed: ${result.failed}"
                    echo "Errors: ${result.errors}"

                    result.executions.eachWithIndex { execution, index ->
                        echo "Execution ${index + 1}"
                        echo "Test suite: ${execution.testSuiteName}"
                        echo "Configuration: ${execution.configuration}"
                        echo "Hash: ${execution.hash}"
                        echo "Results: ${execution.resultsUrl}"
                    }
                }
            }
        }
    }
}

Pipeline parameters

Parameter limits

• timeoutMinutes must be between 1 and 1440.
• pollIntervalSeconds must be between 5 and 300.
• resultsFormat must be json or json-light.
• credentialsId is optional only when both appId and appCode are already included in apiRequest.

Control the Jenkins build result manually

When failBuild is set to true, the plugin stops the Pipeline step with an error when Endtest reports one or more failed assertions or errors.

Set failBuild to false when you want the Pipeline to inspect the results and decide what to do:

script {
    def result = endtestRun(
        credentialsId: 'endtest-api',
        apiRequest: 'https://app.endtest.io/api.php?<YOUR_ENDTEST_API_PARAMETERS>',
        failBuild: false
    )

    if (result.errors > 0) {
        error("Endtest reported ${result.errors} execution errors")
    }

    if (result.failed > 0) {
        unstable("Endtest reported ${result.failed} failed assertions")
    }
}

This is useful when failed tests should mark a build as unstable instead of immediately failing it.

Run multiple Endtest executions

A single Endtest API request can start more than one execution. For example, an API request that runs a label may start multiple test suites or configurations.

When Endtest returns multiple execution hashes, the plugin automatically:

• Stores every execution hash
• Polls all executions
• Waits until all results are available
• Returns the result of each execution
• Calculates aggregate totals
• Exposes a results URL for each execution

No Jenkinsfile changes are required to support multiple executions.

You can check whether the request started multiple executions using:

if (result.multipleExecutions) {
    echo "The Endtest request started ${result.executionCount} executions"
}

Returned result

The endtestRun step returns a map that can be used by later Pipeline logic.

Aggregate fields

Fields for each execution

Each item in result.executions can contain:

For a single execution, the plugin also exposes the individual execution fields directly on the returned result. For example:

echo "Test suite: ${result.testSuiteName}"
echo "Results: ${result.resultsUrl}"

Use credentials already included in the API request

For compatibility with existing Jenkinsfiles, the plugin also accepts an API request that already contains appId and appCode:

endtestRun(
    apiRequest: 'https://app.endtest.io/api.php?<PARAMETERS>&appId=<APP_ID>&appCode=<APP_CODE>'
)

This works without credentialsId, but it is not recommended because the credentials become part of the Jenkinsfile and may be exposed through source control, build configuration, or logs.

Prefer Jenkins Credentials whenever possible.

Jenkins restart behavior

After Endtest returns the execution hashes, the plugin stores its polling state in Jenkins. If the Jenkins controller restarts while the tests are running, result polling resumes after Jenkins comes back online.

If Jenkins restarts while the initial request is still being sent and no execution hash has been received yet, the step fails safely instead of repeating the start request. This prevents the plugin from accidentally starting the same Endtest tests twice.

In that situation:

1. Check Endtest to see whether an execution was created.
2. Review the Jenkins console output.
3. Start the Pipeline again when appropriate.

Console output

The plugin writes status information to the Jenkins build log, including:

• When the Endtest request starts
• The number of executions started
• Each execution hash
• A results link for each execution
• Current Endtest execution status
• Final totals for passed, failed, and errored assertions

Sensitive Jenkins credential values are not intentionally printed by the plugin.

Troubleshooting

credentialsId is required when appId or appCode is missing from apiRequest

The API request does not contain complete credentials and no Jenkins credential ID was provided.

Add credentialsId to the Pipeline step:

credentialsId: 'endtest-api'

Make sure the referenced Jenkins credential is of the type Endtest API credentials.

Could not find Endtest credentials with ID

Jenkins could not find an Endtest credential matching the provided ID.

Check:

• The credential ID for spelling differences
• The credential scope
• The credential domain
• Whether the Pipeline job has permission to access it

The Endtest API request must use HTTPS

The value of apiRequest starts with http:// or does not contain a valid scheme.

Use an HTTPS API request generated by Endtest.

The Endtest API request must use the app.endtest.io host

The plugin only accepts API requests sent to app.endtest.io.

Check that the request was copied correctly and was not replaced with a proxy, shortened URL, or different host.

Timed out waiting for Endtest

The execution did not finish before timeoutMinutes was reached.

Increase the timeout for longer test suites:

timeoutMinutes: 60

Also check the Endtest Results page to determine whether the execution is still running or encountered a problem.

Jenkins build failed after the Endtest execution completed

With the default failBuild: true setting, the Jenkins build fails when Endtest reports failed assertions or execution errors.

Use failBuild: false when you want to handle those results yourself.

Endtest returned an HTTP or invalid response error

Check:

• The App ID and App Code
• The API request parameters
• Network access from the Jenkins controller to https://app.endtest.io
• Whether the Endtest service is reachable
• Whether the request works when used directly according to the Endtest API documentation

Security recommendations

• Store the App Code in Jenkins Credentials.
• Do not commit appId and appCode values to source control.
• Restrict access to the Jenkins credential domain.
• Use a dedicated Endtest API credential for CI/CD when appropriate.
• Rotate the App Code if it is exposed.
• Avoid printing the full authenticated API request in Pipeline logs.

Additional resources

• Endtest API documentation
• Endtest Jenkins plugin
• Endtest Jenkins plugin source code