# Argo

Integrate Endtest with Argo Rollouts and Argo CD as deployment gates and post-deployment validation checks.

Part of the Endtest documentation (Integrations). Canonical URL: https://endtest.io/docs/integrations/argo

---
## Overview

Integrate Endtest with Argo Rollouts and Argo CD to run automated end-to-end tests as deployment gates and post-deployment validation checks.

The official integration can start one Endtest test suite or multiple test suites in a single API request. It waits for every returned execution to finish and exposes the combined result as a Kubernetes Job exit code:

- exit code `0` when every Endtest execution finishes with no failed assertions and no execution errors
- a non-zero exit code when any execution reports failed assertions, execution errors, an invalid response, or a timeout

With Argo Rollouts, that result determines whether a canary or blue-green deployment continues. With Argo CD, the same integration can run as a `PostSync` hook after an application is deployed.

The integration source code and Kubernetes manifests are available in the [Endtest Argo repository](https://github.com/endtest-technologies/endtest-argo).

## Prerequisites

Before configuring the integration, make sure you have:

- an Endtest account
- an Endtest App ID and App Code
- an Endtest API request that starts the test execution or executions you want to use
- a Kubernetes cluster
- Argo Rollouts, Argo CD, or both

For more information about API requests, see the [Endtest API documentation](https://endtest.io/docs/advanced/how-to-use-the-endtest-api).

## Run one or multiple Endtest test suites

To start one test suite, use an API request containing `suite`:

```text
https://app.endtest.io/api.php?action=runWeb&appId=APP_ID&appCode=APP_CODE&suite=SUITE_ID&platform=windows&os=windows11&browser=chrome&browserVersion=latest&resolution=1280x1024&geolocation=sanfrancisco&cases=all&notes=
```

To start multiple test suites, assign the same label to those suites and use `label` instead of `suite`:

```text
https://app.endtest.io/api.php?action=runWeb&appId=APP_ID&appCode=APP_CODE&label=Critical&platform=windows&os=windows11&browser=chrome&browserVersion=latest&resolution=1280x1024&geolocation=sanfrancisco&cases=all&notes=
```

A request that starts multiple suites returns a comma-separated list of execution hashes:

```text
HASH_ONE,HASH_TWO
```

The integration sends all returned hashes to the Endtest `getResults` API in one request:

```text
https://app.endtest.io/api.php?action=getResults&appId=APP_ID&appCode=APP_CODE&hash=HASH_ONE,HASH_TWO&format=json-light
```

For one completed execution, Endtest returns one JSON object. For multiple completed executions, Endtest returns a JSON array containing one result object per execution.

While multiple executions are still running, the JSON array can temporarily contain status strings such as `Test is still running.`, `Processing video recording.`, or `Stopping.`. The integration treats those entries as pending executions and continues polling until every execution has produced a final result object.

The Argo gate passes only when every final result object contains:

```text
failed = 0
errors = 0
```

If any suite has failed assertions or execution errors, the complete deployment gate fails.

## How the integration works

1. Argo creates a Kubernetes Job from the Endtest analysis template or Argo CD hook.
2. The Job reads the Endtest App ID, App Code, and API request from a Kubernetes Secret.
3. The runner sends the API request and receives one hash or a comma-separated list of hashes.
4. The runner polls Endtest with all returned hashes.
5. The runner accepts a single result object, a completed array of result objects, or a temporary array containing pending status strings.
6. When any execution is still pending, the runner waits and polls again.
7. The Job succeeds only when every execution contains zero failed assertions and zero execution errors.
8. Argo continues or blocks the deployment based on the Job result.

## Create the Endtest Secret

Create a Kubernetes Secret in the same namespace where the Rollout, AnalysisRun, or Argo CD application runs:

```bash
kubectl create secret generic endtest-credentials \
  --from-literal=ENDTEST_APP_ID='your-app-id' \
  --from-literal=ENDTEST_APP_CODE='your-app-code' \
  --from-literal=ENDTEST_API_REQUEST='your-complete-endtest-api-request'
```

The Secret must contain these keys:

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: endtest-credentials
type: Opaque
stringData:
  ENDTEST_APP_ID: "your-app-id"
  ENDTEST_APP_CODE: "your-app-code"
  ENDTEST_API_REQUEST: "your-complete-endtest-api-request"
```

Do not commit a populated Secret to source control. In production, use your existing secret-management solution, such as External Secrets, Sealed Secrets, AWS Secrets Manager, Azure Key Vault, Google Secret Manager, or another Kubernetes-compatible secret provider.

## Install the Argo Rollouts analysis template

The recommended option is the cluster-scoped template, which can be reused by Rollouts in multiple namespaces:

```bash
kubectl apply \
  -f https://raw.githubusercontent.com/endtest-technologies/endtest-argo/v0.2.1/manifests/cluster-analysis-template.yaml
```

Verify the template:

```bash
kubectl get clusteranalysistemplate endtest
```

A namespace-scoped version is also available:

```bash
kubectl apply \
  -f https://raw.githubusercontent.com/endtest-technologies/endtest-argo/v0.2.1/manifests/analysis-template.yaml
```

Use the namespace-scoped version when you want the template to be available only inside one namespace.

## Use Endtest in a canary rollout

Add an Endtest analysis step after exposing the candidate version to a percentage of traffic:

```yaml
strategy:
  canary:
    steps:
      - setWeight: 20
      - pause:
          duration: 2m
      - analysis:
          templates:
            - templateName: endtest
              clusterScope: true
          args:
            - name: secret-name
              value: endtest-credentials
            - name: number-of-loops
              value: "20"
            - name: poll-interval-seconds
              value: "30"
      - setWeight: 50
      - pause:
          duration: 5m
      - setWeight: 100
```

When every Endtest execution passes, Argo proceeds to the next rollout step.

When any Endtest execution reports failed assertions or execution errors, the analysis fails and the rollout does not continue to the next traffic weight.

A complete canary example is available in the repository:

```bash
kubectl apply \
  -f https://raw.githubusercontent.com/endtest-technologies/endtest-argo/v0.2.1/examples/canary-rollout.yaml
```

You can monitor the rollout with:

```bash
kubectl argo rollouts get rollout example-app --watch
```

## Use Endtest in a blue-green rollout

For blue-green deployments, run Endtest against the preview environment before Argo promotes it to the active Service:

```yaml
strategy:
  blueGreen:
    activeService: example-app-active
    previewService: example-app-preview
    prePromotionAnalysis:
      templates:
        - templateName: endtest
          clusterScope: true
      args:
        - name: secret-name
          value: endtest-credentials
        - name: number-of-loops
          value: "20"
```

Configure the Endtest API request so the test suite or label targets the preview URL.

When every execution passes, Argo can promote the preview version. When any execution fails, the active Service remains on the previous version.

A complete blue-green example is available in the repository:

```bash
kubectl apply \
  -f https://raw.githubusercontent.com/endtest-technologies/endtest-argo/v0.2.1/examples/blue-green-rollout.yaml
```

## Test the integration with a standalone AnalysisRun

Before adding the integration to a production Rollout, test it independently:

```bash
kubectl argo rollouts create analysisrun \
  --global \
  --from endtest \
  --name endtest-smoke-test \
  --argument secret-name=endtest-credentials
```

Print its final phase:

```bash
kubectl get analysisrun endtest-smoke-test \
  -o jsonpath='{.status.phase}{"\n"}'
```

A successful Endtest execution should produce:

```text
Successful
```

Find the Kubernetes Job created by the AnalysisRun:

```bash
kubectl get jobs \
  -l app.kubernetes.io/name=endtest
```

Inspect its logs:

```bash
kubectl logs job/<job-name> -c endtest
```

For multiple executions, the logs first show pending progress when necessary, then every completed suite separately, followed by a combined result:

```text
Endtest executions are still running: 2 pending (1/20).
Endtest executions completed: 2.
Execution 1 of 2
Test Suite: Checkout
Failed: 0
Errors: 0

Execution 2 of 2
Test Suite: Login
Failed: 0
Errors: 0

Combined result
Executions: 2
Failed: 0
Errors: 0
Endtest release gate passed.
```

## Use Endtest as an Argo CD PostSync hook

For Argo CD applications that do not use Argo Rollouts, run Endtest after the application has been synchronized and becomes healthy.

Add the official hook manifest to the same Git repository that Argo CD synchronizes:

```bash
curl -o endtest-post-sync-job.yaml \
  https://raw.githubusercontent.com/endtest-technologies/endtest-argo/v0.2.1/manifests/argocd-post-sync-job.yaml
```

The Job contains these annotations:

```yaml
annotations:
  argocd.argoproj.io/hook: PostSync
  argocd.argoproj.io/hook-delete-policy: BeforeHookCreation,HookSucceeded
```

Argo CD runs the Job after the application resources have been synchronized.

When every Endtest execution passes, the hook succeeds. When any execution reports failures or errors, the hook fails and the Argo CD sync operation is marked as failed.

The default hook reads the `endtest-credentials` Secret:

```yaml
envFrom:
  - secretRef:
      name: endtest-credentials
```

The Secret must exist in the namespace where the hook Job runs.

## Template parameters

The Endtest analysis template supports these parameters:

| Parameter | Default | Description |
| --- | --- | --- |
| `secret-name` | `endtest-credentials` | Kubernetes Secret containing the Endtest App ID, App Code, and API request. |
| `image` | `ghcr.io/endtest-technologies/endtest-argo:0.2.1` | Endtest Argo runner container image. |
| `results-format` | `json-light` | Endtest result format. Supported values are `json-light` and `json`. |
| `number-of-loops` | `20` | Maximum number of result checks. |
| `poll-interval-seconds` | `30` | Delay between result checks. |
| `request-timeout-seconds` | `30` | Timeout for each Endtest API request. |

The integration uses `json-light` by default because the deployment gate only needs summary fields. Use `json` when you also want the full logs and media URLs returned by the API.

You can override the format or image version in a Rollout:

```yaml
args:
  - name: image
    value: ghcr.io/endtest-technologies/endtest-argo:0.2.1
  - name: results-format
    value: json
```

Pinning a specific image version is recommended for production deployments.

## Result behavior

The release gate passes only when every Endtest result object contains:

```text
failed = 0
errors = 0
```

A test execution with no assertions is considered successful when it completes with no execution errors.

The release gate fails when:

- any Endtest execution reports one or more failed assertions
- any Endtest execution reports one or more execution errors
- the start request returns an invalid hash or comma-separated hash list
- the number of returned result objects does not match the number of execution hashes
- the result API returns malformed or unexpected data
- an execution returns the terminal `Erred.` status
- the executions do not complete before the polling limit
- a required Secret value is missing
- the runner cannot reach the Endtest API

## Troubleshooting

### The AnalysisRun remains Running

Check the Job and Pod:

```bash
kubectl get analysisruns
kubectl get jobs
kubectl get pods
```

Inspect the Job logs:

```bash
kubectl logs job/<job-name> -c endtest
```

Check recent Kubernetes events:

```bash
kubectl get events \
  --sort-by=.lastTimestamp \
  | tail -30
```

### The container is waiting in ContainerCreating

The cluster may still be downloading the Endtest image.

Inspect the Pod:

```bash
kubectl describe pod <pod-name>
```

The official image is public:

```text
ghcr.io/endtest-technologies/endtest-argo:0.2.1
```

### The AnalysisRun is Failed or Error

Inspect the Job logs:

```bash
kubectl logs job/<job-name> -c endtest
```

Common causes include:

- incorrect App ID or App Code
- an invalid or incomplete API request
- one or more executions containing failures
- one or more executions containing errors
- a mismatch between execution hashes and result objects
- a polling timeout
- a missing Kubernetes Secret

### The Secret cannot be found

Make sure the Secret exists in the same namespace as the AnalysisRun or Rollout:

```bash
kubectl get secret endtest-credentials
```

When using a `ClusterAnalysisTemplate`, the template is cluster-scoped, but the generated Job still reads the Secret from the namespace where the AnalysisRun executes.

## Security

The official integration includes the following security defaults:

- Endtest credentials are read at runtime from a Kubernetes Secret
- credentials and the API request are not printed in Job logs
- the container runs as a non-root user
- the Kubernetes manifests disable service-account token mounting
- the container uses a read-only root filesystem
- Linux capabilities are dropped
- the official container image is published through GitHub Container Registry

For production environments, restrict access to the Secret and use your standard Kubernetes secret-management and RBAC policies.

## Source code

The integration is open source:

[github.com/endtest-technologies/endtest-argo](https://github.com/endtest-technologies/endtest-argo)