# Jenkins

Run Endtest end-to-end tests from a Jenkins Pipeline with the official Endtest plugin.

Part of the Endtest documentation (Integrations). Canonical URL: https://endtest.io/docs/integrations/jenkins

---
## Overview

The Endtest Jenkins plugin lets you start Endtest automated tests directly from a Jenkins Pipeline, wait for the executions to finish, and use the results in later Pipeline steps.

The plugin supports:

- Secure storage of Endtest API credentials in Jenkins
- Single Endtest executions
- Multiple executions started by the same API request
- Label-based Endtest API requests
- Automatic polling until execution results are available
- Aggregate results across multiple executions
- Optional Jenkins build failure when Endtest reports failures or errors
- Resuming result polling after a Jenkins controller restart
- `json` and `json-light` result formats

## Requirements

Before configuring the integration, make sure you have:

- Jenkins 2.528.3 or newer
- Permission to install Jenkins plugins and create credentials
- An Endtest account
- An Endtest App ID and App Code
- An Endtest API request that starts the test suite, configuration, or label you want to run

For more information about creating an API request, see [How to use the Endtest API](https://endtest.io/docs/advanced/how-to-use-the-endtest-api).

## Install the Endtest plugin

1. In Jenkins, open **Manage Jenkins**.
2. Select **Plugins**.
3. Open the **Available plugins** tab.
4. Search for **Endtest**.
5. Select the Endtest plugin and install it.
6. Restart Jenkins if your Jenkins installation asks you to do so.

The plugin page is available at:

[https://plugins.jenkins.io/endtest/](https://plugins.jenkins.io/endtest/)

## Add Endtest credentials to Jenkins

Storing the App ID and App Code in Jenkins Credentials is recommended. This keeps the credentials out of the Jenkinsfile and allows the same credential entry to be reused by multiple jobs.

1. In Jenkins, open **Manage Jenkins**.
2. Select **Credentials**.
3. Open the credential store and domain where the Pipeline should have access.
4. Select **Add Credentials**.
5. For **Kind**, select **Endtest API credentials**.
6. Enter the following values:

| Field | Description |
| --- | --- |
| App ID | Your Endtest App ID |
| App Code | Your Endtest App Code |
| ID | A Jenkins credential ID, such as `endtest-api` |
| Description | An optional description for the credential |

The examples in this guide use the credential ID `endtest-api`.

> The Jenkins credential ID is not the Endtest App ID. It is the identifier Jenkins uses to locate the stored credential.

## Prepare the Endtest API request

Create or copy an Endtest API request for the test suite, configuration, or label you want Jenkins to execute.

The request must:

- Use HTTPS
- Use the `app.endtest.io` host
- Start an Endtest execution
- Include all execution settings required by your Endtest project

A request may contain `appId` and `appCode`, but storing those values in Jenkins Credentials is safer. When a credential ID is provided, the plugin automatically adds any missing `appId` or `appCode` values to the request.

When credentials are present both in the API request and in Jenkins Credentials, the values in the API request take precedence.

## Add Endtest to a Jenkins Pipeline

The plugin provides the Pipeline step:

```groovy
endtestRun(...)
```

A basic Declarative Pipeline example looks like this:

```groovy
pipeline {
    agent any

    stages {
        stage('Run Endtest') {
            steps {
                script {
                    def result = endtestRun(
                        credentialsId: 'endtest-api',
                        apiRequest: 'https://app.endtest.io/api.php?<YOUR_ENDTEST_API_PARAMETERS>'
                    )

                    echo "Executions: ${result.executionCount}"
                    echo "Test cases: ${result.testCases}"
                    echo "Passed: ${result.passed}"
                    echo "Failed: ${result.failed}"
                    echo "Errors: ${result.errors}"
                }
            }
        }
    }
}
```

Replace the value of `apiRequest` with the API request generated for your Endtest tests.

By default, the plugin:

- Waits for up to 30 minutes
- Checks the Endtest results every 30 seconds
- Uses the `json-light` results format
- Fails the Jenkins build when Endtest reports failed assertions or errors

## Complete Pipeline example

The following example displays aggregate results and a link to each Endtest execution:

```groovy
pipeline {
    agent any

    stages {
        stage('Run Endtest') {
            steps {
                script {
                    def result = endtestRun(
                        credentialsId: 'endtest-api',
                        apiRequest: 'https://app.endtest.io/api.php?<YOUR_ENDTEST_API_PARAMETERS>',
                        timeoutMinutes: 45,
                        pollIntervalSeconds: 30,
                        failBuild: true,
                        resultsFormat: 'json-light'
                    )

                    echo "Endtest completed ${result.executionCount} execution(s)"
                    echo "Test cases: ${result.testCases}"
                    echo "Passed: ${result.passed}"
                    echo "Failed: ${result.failed}"
                    echo "Errors: ${result.errors}"

                    result.executions.eachWithIndex { execution, index ->
                        echo "Execution ${index + 1}"
                        echo "Test suite: ${execution.testSuiteName}"
                        echo "Configuration: ${execution.configuration}"
                        echo "Hash: ${execution.hash}"
                        echo "Results: ${execution.resultsUrl}"
                    }
                }
            }
        }
    }
}
```

## Pipeline parameters

| Parameter | Required | Default | Description |
| --- | --- | --- | --- |
| `apiRequest` | Yes | None | The Endtest API request used to start one or more executions |
| `credentialsId` | Conditional | None | The ID of an **Endtest API credentials** entry stored in Jenkins |
| `timeoutMinutes` | No | `30` | Maximum number of minutes Jenkins waits for the results |
| `pollIntervalSeconds` | No | `30` | Number of seconds between result checks |
| `failBuild` | No | `true` | Fails the Pipeline when Endtest reports failures or errors |
| `resultsFormat` | No | `json-light` | Endtest result format, either `json` or `json-light` |

### Parameter limits

- `timeoutMinutes` must be between `1` and `1440`.
- `pollIntervalSeconds` must be between `5` and `300`.
- `resultsFormat` must be `json` or `json-light`.
- `credentialsId` is optional only when both `appId` and `appCode` are already included in `apiRequest`.

## Control the Jenkins build result manually

When `failBuild` is set to `true`, the plugin stops the Pipeline step with an error when Endtest reports one or more failed assertions or errors.

Set `failBuild` to `false` when you want the Pipeline to inspect the results and decide what to do:

```groovy
script {
    def result = endtestRun(
        credentialsId: 'endtest-api',
        apiRequest: 'https://app.endtest.io/api.php?<YOUR_ENDTEST_API_PARAMETERS>',
        failBuild: false
    )

    if (result.errors > 0) {
        error("Endtest reported ${result.errors} execution errors")
    }

    if (result.failed > 0) {
        unstable("Endtest reported ${result.failed} failed assertions")
    }
}
```

This is useful when failed tests should mark a build as unstable instead of immediately failing it.

## Run multiple Endtest executions

A single Endtest API request can start more than one execution. For example, an API request that runs a label may start multiple test suites or configurations.

When Endtest returns multiple execution hashes, the plugin automatically:

- Stores every execution hash
- Polls all executions
- Waits until all results are available
- Returns the result of each execution
- Calculates aggregate totals
- Exposes a results URL for each execution

No Jenkinsfile changes are required to support multiple executions.

You can check whether the request started multiple executions using:

```groovy
if (result.multipleExecutions) {
    echo "The Endtest request started ${result.executionCount} executions"
}
```

## Returned result

The `endtestRun` step returns a map that can be used by later Pipeline logic.

### Aggregate fields

| Field | Description |
| --- | --- |
| `executionCount` | Number of Endtest executions started by the request |
| `hashes` | List of Endtest execution hashes |
| `executions` | List containing the result of each execution |
| `resultsUrls` | List of links to the Endtest Results pages |
| `testCases` | Total number of test cases |
| `passed` | Total number of passed assertions |
| `failed` | Total number of failed assertions |
| `errors` | Total number of errors |
| `multipleExecutions` | `true` when the request started more than one execution |

### Fields for each execution

Each item in `result.executions` can contain:

| Field | Description |
| --- | --- |
| `testSuiteName` | Name of the Endtest test suite |
| `configuration` | Endtest execution configuration |
| `testCases` | Number of test cases |
| `passed` | Number of passed assertions |
| `failed` | Number of failed assertions |
| `errors` | Number of errors |
| `startTime` | Execution start time |
| `endTime` | Execution end time |
| `hash` | Endtest execution hash |
| `resultsUrl` | Link to the execution in Endtest |
| `detailedLogs` | Detailed execution log data when included in the selected result format |
| `screenshotsAndVideo` | Screenshot and video data when available |
| `testCaseManagement` | Test case management data when available |

For a single execution, the plugin also exposes the individual execution fields directly on the returned result. For example:

```groovy
echo "Test suite: ${result.testSuiteName}"
echo "Results: ${result.resultsUrl}"
```

## Use credentials already included in the API request

For compatibility with existing Jenkinsfiles, the plugin also accepts an API request that already contains `appId` and `appCode`:

```groovy
endtestRun(
    apiRequest: 'https://app.endtest.io/api.php?<PARAMETERS>&appId=<APP_ID>&appCode=<APP_CODE>'
)
```

This works without `credentialsId`, but it is not recommended because the credentials become part of the Jenkinsfile and may be exposed through source control, build configuration, or logs.

Prefer Jenkins Credentials whenever possible.

## Jenkins restart behavior

After Endtest returns the execution hashes, the plugin stores its polling state in Jenkins. If the Jenkins controller restarts while the tests are running, result polling resumes after Jenkins comes back online.

If Jenkins restarts while the initial request is still being sent and no execution hash has been received yet, the step fails safely instead of repeating the start request. This prevents the plugin from accidentally starting the same Endtest tests twice.

In that situation:

1. Check Endtest to see whether an execution was created.
2. Review the Jenkins console output.
3. Start the Pipeline again when appropriate.

## Console output

The plugin writes status information to the Jenkins build log, including:

- When the Endtest request starts
- The number of executions started
- Each execution hash
- A results link for each execution
- Current Endtest execution status
- Final totals for passed, failed, and errored assertions

Sensitive Jenkins credential values are not intentionally printed by the plugin.

## Troubleshooting

### `credentialsId is required when appId or appCode is missing from apiRequest`

The API request does not contain complete credentials and no Jenkins credential ID was provided.

Add `credentialsId` to the Pipeline step:

```groovy
credentialsId: 'endtest-api'
```

Make sure the referenced Jenkins credential is of the type **Endtest API credentials**.

### `Could not find Endtest credentials with ID`

Jenkins could not find an Endtest credential matching the provided ID.

Check:

- The credential ID for spelling differences
- The credential scope
- The credential domain
- Whether the Pipeline job has permission to access it

### `The Endtest API request must use HTTPS`

The value of `apiRequest` starts with `http://` or does not contain a valid scheme.

Use an HTTPS API request generated by Endtest.

### `The Endtest API request must use the app.endtest.io host`

The plugin only accepts API requests sent to `app.endtest.io`.

Check that the request was copied correctly and was not replaced with a proxy, shortened URL, or different host.

### `Timed out waiting for Endtest`

The execution did not finish before `timeoutMinutes` was reached.

Increase the timeout for longer test suites:

```groovy
timeoutMinutes: 60
```

Also check the Endtest Results page to determine whether the execution is still running or encountered a problem.

### Jenkins build failed after the Endtest execution completed

With the default `failBuild: true` setting, the Jenkins build fails when Endtest reports failed assertions or execution errors.

Use `failBuild: false` when you want to handle those results yourself.

### Endtest returned an HTTP or invalid response error

Check:

- The App ID and App Code
- The API request parameters
- Network access from the Jenkins controller to `https://app.endtest.io`
- Whether the Endtest service is reachable
- Whether the request works when used directly according to the Endtest API documentation

## Security recommendations

- Store the App Code in Jenkins Credentials.
- Do not commit `appId` and `appCode` values to source control.
- Restrict access to the Jenkins credential domain.
- Use a dedicated Endtest API credential for CI/CD when appropriate.
- Rotate the App Code if it is exposed.
- Avoid printing the full authenticated API request in Pipeline logs.

## Additional resources

- [Endtest API documentation](https://endtest.io/docs/advanced/how-to-use-the-endtest-api)
- [Endtest Jenkins plugin](https://plugins.jenkins.io/endtest/)
- [Endtest Jenkins plugin source code](https://github.com/jenkinsci/endtest-plugin)