Troubleshooting

Webhook problems

GitHub says delivery failed

Check that:

• the webhook URL includes the right projectId,
• the webhook secret matches ReleaseReady,
• and the app can receive GitHub requests at the configured URL.

Signature verification fails

Make sure the GitHub webhook secret matches the project webhook secret or the fallback GITHUB_WEBHOOK_SECRET configuration.

If you call the webhook endpoint manually from GitHub Actions, X-Hub-Signature-256 must be an HMAC SHA-256 digest of the raw payload (sha256=<digest>), not the plain webhook secret value.

Recommended command in Actions (avoids parsing text output from OpenSSL):

RELEASEREADY_SIGNATURE="sha256=$(openssl dgst -sha256 -hmac "$RELEASEREADY_WEBHOOK_SECRET" -binary "$GITHUB_EVENT_PATH" | xxd -p -c 256)"

Recommended full webhook step (copy/paste safe, with HTTP status handling):

set -euo pipefail
RELEASEREADY_WEBHOOK_URL="$(printf '%s' "$RELEASEREADY_WEBHOOK_URL" | tr -d '\r\n')"
RELEASEREADY_WEBHOOK_SECRET="$(printf '%s' "$RELEASEREADY_WEBHOOK_SECRET" | tr -d '\r\n')"
RELEASEREADY_SIGNATURE="sha256=$(openssl dgst -sha256 -hmac "$RELEASEREADY_WEBHOOK_SECRET" -binary "$GITHUB_EVENT_PATH" | xxd -p -c 256)"
RESPONSE_FILE="$(mktemp)"
STATUS_CODE="$(curl -sS -o "$RESPONSE_FILE" -w "%{http_code}" -X POST "$RELEASEREADY_WEBHOOK_URL" \
  -H "X-Hub-Signature-256: $RELEASEREADY_SIGNATURE" \
  -H "X-GitHub-Event: pull_request" \
  -H "X-ReleaseReady-Playwright-Image: mcr.microsoft.com/playwright:v1.58.0-noble" \
  -H "X-ReleaseReady-Command: npm test" \
  -H "Content-Type: application/json" \
  --data-binary "@$GITHUB_EVENT_PATH")"
cat "$RESPONSE_FILE"
if [ "$STATUS_CODE" -ge 400 ]; then
  echo "ReleaseReady webhook failed with HTTP $STATUS_CODE" >&2
  exit 1
fi

Also send X-GitHub-Event: pull_request; otherwise the endpoint may return { ok: true } but skip run creation because the event type is missing.

If the first attempt works and a re-run later fails with invalid_signature, verify these points:

• use a GitHub secret (${{ secrets.RELEASEREADY_WEBHOOK_SECRET }}), not a hardcoded literal in the YAML;
• keep the same value in both places ReleaseReady may validate against (project webhook secret and optional GITHUB_WEBHOOK_SECRET);
• compute the signature from the exact file sent to curl ($GITHUB_EVENT_PATH) in the same step.

When the response shows candidateSecretCount: 2, ReleaseReady is checking two possible secrets; if GitHub does not match either one, signature verification fails.

If GitHub Actions fails with curl: (3) URL rejected: Malformed input to a URL function, your webhook URL secret likely contains a hidden newline. Re-save the secret as a single-line value and strip CR/LF before calling curl.

If you store the webhook URL as a GitHub **variable**, reference it as ${{ vars.RELEASEREADY_WEBHOOK_URL }}. ${{ secrets.RELEASEREADY_WEBHOOK_URL }} only works when the value is actually stored as a secret.

If you need to enforce a Playwright image during webhook-triggered runs, send it via X-ReleaseReady-Playwright-Image. If omitted, ReleaseReady falls back to the configured integration image or mcr.microsoft.com/playwright:v1.58.0-noble.

If you want webhook-triggered runs to behave like manual Trigger Run command selection, send X-ReleaseReady-Command (for example npm test, pnpm test, or pytest) so the run does not rely on the default Docker command.

Manual run problems

Run creation works but execution fails

Common causes:

• invalid repository URL,
• missing branch,
• missing runner configuration,
• or repository-specific dependencies not available in the execution environment.

Docker command fails in runtime

If the webhook returns Failed to trigger checklist run for opened PR with runnerStatus: 500 and a message like Docker no está disponible en el runtime, webhook validation already succeeded and the failure is in execution.

Use one of these options:

• provide an explicit non-Docker test command for that project/template (for example npm test, pnpm test, pytest, go test ./...);
• or run through a delegated runner environment that has Docker available.

Playwright browsers are missing

ReleaseReady already includes fallback behavior that can attempt a Playwright browser install when output clearly indicates missing browsers. If the install step also fails, inspect the output and runtime limitations.

Runner integration problems

Delegated runner cannot be reached

Check RUNNER_URL, networking between the app and runner, and RUNNER_SECRET alignment.

Runner results are rejected

Check the payload shape sent to POST /api/runner/results. ReleaseReady accepts legacy aggregate payloads, progress payloads, and execution-report payloads.

Continue

• FAQ
• Interpreting a run