Argo

Overview

Integrate Endtest with Argo Rollouts and Argo CD to run automated end-to-end tests as deployment gates and post-deployment validation checks.

The official integration can start one Endtest test suite or multiple test suites in a single API request. It waits for every returned execution to finish and exposes the combined result as a Kubernetes Job exit code:

• exit code 0 when every Endtest execution finishes with no failed assertions and no execution errors
• a non-zero exit code when any execution reports failed assertions, execution errors, an invalid response, or a timeout

With Argo Rollouts, that result determines whether a canary or blue-green deployment continues. With Argo CD, the same integration can run as a PostSync hook after an application is deployed.

The integration source code and Kubernetes manifests are available in the Endtest Argo repository.

Prerequisites

Before configuring the integration, make sure you have:

• an Endtest account
• an Endtest App ID and App Code
• an Endtest API request that starts the test execution or executions you want to use
• a Kubernetes cluster
• Argo Rollouts, Argo CD, or both

For more information about API requests, see the Endtest API documentation.

Run one or multiple Endtest test suites

To start one test suite, use an API request containing suite:

https://app.endtest.io/api.php?action=runWeb&appId=APP_ID&appCode=APP_CODE&suite=SUITE_ID&platform=windows&os=windows11&browser=chrome&browserVersion=latest&resolution=1280x1024&geolocation=sanfrancisco&cases=all&notes=

To start multiple test suites, assign the same label to those suites and use label instead of suite:

https://app.endtest.io/api.php?action=runWeb&appId=APP_ID&appCode=APP_CODE&label=Critical&platform=windows&os=windows11&browser=chrome&browserVersion=latest&resolution=1280x1024&geolocation=sanfrancisco&cases=all&notes=

A request that starts multiple suites returns a comma-separated list of execution hashes:

HASH_ONE,HASH_TWO

The integration sends all returned hashes to the Endtest getResults API in one request:

https://app.endtest.io/api.php?action=getResults&appId=APP_ID&appCode=APP_CODE&hash=HASH_ONE,HASH_TWO&format=json-light

For one completed execution, Endtest returns one JSON object. For multiple completed executions, Endtest returns a JSON array containing one result object per execution.

While multiple executions are still running, the JSON array can temporarily contain status strings such as Test is still running., Processing video recording., or Stopping.. The integration treats those entries as pending executions and continues polling until every execution has produced a final result object.

The Argo gate passes only when every final result object contains:

failed = 0
errors = 0

If any suite has failed assertions or execution errors, the complete deployment gate fails.

How the integration works

1. Argo creates a Kubernetes Job from the Endtest analysis template or Argo CD hook.
2. The Job reads the Endtest App ID, App Code, and API request from a Kubernetes Secret.
3. The runner sends the API request and receives one hash or a comma-separated list of hashes.
4. The runner polls Endtest with all returned hashes.
5. The runner accepts a single result object, a completed array of result objects, or a temporary array containing pending status strings.
6. When any execution is still pending, the runner waits and polls again.
7. The Job succeeds only when every execution contains zero failed assertions and zero execution errors.
8. Argo continues or blocks the deployment based on the Job result.

Create the Endtest Secret

Create a Kubernetes Secret in the same namespace where the Rollout, AnalysisRun, or Argo CD application runs:

kubectl create secret generic endtest-credentials \
  --from-literal=ENDTEST_APP_ID='your-app-id' \
  --from-literal=ENDTEST_APP_CODE='your-app-code' \
  --from-literal=ENDTEST_API_REQUEST='your-complete-endtest-api-request'

The Secret must contain these keys:

apiVersion: v1
kind: Secret
metadata:
  name: endtest-credentials
type: Opaque
stringData:
  ENDTEST_APP_ID: "your-app-id"
  ENDTEST_APP_CODE: "your-app-code"
  ENDTEST_API_REQUEST: "your-complete-endtest-api-request"

Do not commit a populated Secret to source control. In production, use your existing secret-management solution, such as External Secrets, Sealed Secrets, AWS Secrets Manager, Azure Key Vault, Google Secret Manager, or another Kubernetes-compatible secret provider.

Install the Argo Rollouts analysis template

The recommended option is the cluster-scoped template, which can be reused by Rollouts in multiple namespaces:

kubectl apply \
  -f https://raw.githubusercontent.com/endtest-technologies/endtest-argo/v0.2.1/manifests/cluster-analysis-template.yaml

Verify the template:

kubectl get clusteranalysistemplate endtest

A namespace-scoped version is also available:

kubectl apply \
  -f https://raw.githubusercontent.com/endtest-technologies/endtest-argo/v0.2.1/manifests/analysis-template.yaml

Use the namespace-scoped version when you want the template to be available only inside one namespace.

Use Endtest in a canary rollout

Add an Endtest analysis step after exposing the candidate version to a percentage of traffic:

strategy:
  canary:
    steps:
      - setWeight: 20
      - pause:
          duration: 2m
      - analysis:
          templates:
            - templateName: endtest
              clusterScope: true
          args:
            - name: secret-name
              value: endtest-credentials
            - name: number-of-loops
              value: "20"
            - name: poll-interval-seconds
              value: "30"
      - setWeight: 50
      - pause:
          duration: 5m
      - setWeight: 100

When every Endtest execution passes, Argo proceeds to the next rollout step.

When any Endtest execution reports failed assertions or execution errors, the analysis fails and the rollout does not continue to the next traffic weight.

A complete canary example is available in the repository:

kubectl apply \
  -f https://raw.githubusercontent.com/endtest-technologies/endtest-argo/v0.2.1/examples/canary-rollout.yaml

You can monitor the rollout with:

kubectl argo rollouts get rollout example-app --watch

Use Endtest in a blue-green rollout

For blue-green deployments, run Endtest against the preview environment before Argo promotes it to the active Service:

strategy:
  blueGreen:
    activeService: example-app-active
    previewService: example-app-preview
    prePromotionAnalysis:
      templates:
        - templateName: endtest
          clusterScope: true
      args:
        - name: secret-name
          value: endtest-credentials
        - name: number-of-loops
          value: "20"

Configure the Endtest API request so the test suite or label targets the preview URL.

When every execution passes, Argo can promote the preview version. When any execution fails, the active Service remains on the previous version.

A complete blue-green example is available in the repository:

kubectl apply \
  -f https://raw.githubusercontent.com/endtest-technologies/endtest-argo/v0.2.1/examples/blue-green-rollout.yaml

Test the integration with a standalone AnalysisRun

Before adding the integration to a production Rollout, test it independently:

kubectl argo rollouts create analysisrun \
  --global \
  --from endtest \
  --name endtest-smoke-test \
  --argument secret-name=endtest-credentials

Print its final phase:

kubectl get analysisrun endtest-smoke-test \
  -o jsonpath='{.status.phase}{"\n"}'

A successful Endtest execution should produce:

Successful

Find the Kubernetes Job created by the AnalysisRun:

kubectl get jobs \
  -l app.kubernetes.io/name=endtest

Inspect its logs:

kubectl logs job/<job-name> -c endtest

For multiple executions, the logs first show pending progress when necessary, then every completed suite separately, followed by a combined result:

Endtest executions are still running: 2 pending (1/20).
Endtest executions completed: 2.
Execution 1 of 2
Test Suite: Checkout
Failed: 0
Errors: 0

Execution 2 of 2
Test Suite: Login
Failed: 0
Errors: 0

Combined result
Executions: 2
Failed: 0
Errors: 0
Endtest release gate passed.

Use Endtest as an Argo CD PostSync hook

For Argo CD applications that do not use Argo Rollouts, run Endtest after the application has been synchronized and becomes healthy.

Add the official hook manifest to the same Git repository that Argo CD synchronizes:

curl -o endtest-post-sync-job.yaml \
  https://raw.githubusercontent.com/endtest-technologies/endtest-argo/v0.2.1/manifests/argocd-post-sync-job.yaml

The Job contains these annotations:

annotations:
  argocd.argoproj.io/hook: PostSync
  argocd.argoproj.io/hook-delete-policy: BeforeHookCreation,HookSucceeded

Argo CD runs the Job after the application resources have been synchronized.

When every Endtest execution passes, the hook succeeds. When any execution reports failures or errors, the hook fails and the Argo CD sync operation is marked as failed.

The default hook reads the endtest-credentials Secret:

envFrom:
  - secretRef:
      name: endtest-credentials

The Secret must exist in the namespace where the hook Job runs.

Template parameters

The Endtest analysis template supports these parameters:

The integration uses json-light by default because the deployment gate only needs summary fields. Use json when you also want the full logs and media URLs returned by the API.

You can override the format or image version in a Rollout:

args:
  - name: image
    value: ghcr.io/endtest-technologies/endtest-argo:0.2.1
  - name: results-format
    value: json

Pinning a specific image version is recommended for production deployments.

Result behavior

The release gate passes only when every Endtest result object contains:

failed = 0
errors = 0

A test execution with no assertions is considered successful when it completes with no execution errors.

The release gate fails when:

• any Endtest execution reports one or more failed assertions
• any Endtest execution reports one or more execution errors
• the start request returns an invalid hash or comma-separated hash list
• the number of returned result objects does not match the number of execution hashes
• the result API returns malformed or unexpected data
• an execution returns the terminal Erred. status
• the executions do not complete before the polling limit
• a required Secret value is missing
• the runner cannot reach the Endtest API

Troubleshooting

The AnalysisRun remains Running

Check the Job and Pod:

kubectl get analysisruns
kubectl get jobs
kubectl get pods

Inspect the Job logs:

kubectl logs job/<job-name> -c endtest

Check recent Kubernetes events:

kubectl get events \
  --sort-by=.lastTimestamp \
  | tail -30

The container is waiting in ContainerCreating

The cluster may still be downloading the Endtest image.

Inspect the Pod:

kubectl describe pod <pod-name>

The official image is public:

ghcr.io/endtest-technologies/endtest-argo:0.2.1

The AnalysisRun is Failed or Error

Inspect the Job logs:

kubectl logs job/<job-name> -c endtest

Common causes include:

• incorrect App ID or App Code
• an invalid or incomplete API request
• one or more executions containing failures
• one or more executions containing errors
• a mismatch between execution hashes and result objects
• a polling timeout
• a missing Kubernetes Secret

The Secret cannot be found

Make sure the Secret exists in the same namespace as the AnalysisRun or Rollout:

kubectl get secret endtest-credentials

When using a ClusterAnalysisTemplate, the template is cluster-scoped, but the generated Job still reads the Secret from the namespace where the AnalysisRun executes.

Security

The official integration includes the following security defaults:

• Endtest credentials are read at runtime from a Kubernetes Secret
• credentials and the API request are not printed in Job logs
• the container runs as a non-root user
• the Kubernetes manifests disable service-account token mounting
• the container uses a read-only root filesystem
• Linux capabilities are dropped
• the official container image is published through GitHub Container Registry

For production environments, restrict access to the Secret and use your standard Kubernetes secret-management and RBAC policies.

Source code

The integration is open source:

github.com/endtest-technologies/endtest-argo