---
icon: 🎭
---

# E2E Tests & Monitors

One Playwright suite in `packages/tests-e2e` feeds three consumers that fail independently: CI on a fresh throwaway instance, Checkly monitors against **production Cloud**, and a single BetterStack monitor. A change that only breaks one of them looks green everywhere else, so it is worth knowing which reads what.

How to write a spec lives in one place: "Writing a spec" in `packages/tests-e2e/README.md`. This page holds the gotchas only.

**Local / CI suite** — `playwright.config.ts` runs every `scenarios/**/*.spec.ts`; `AP_EDITION=ce` skips `scenarios/ee`. Boots the whole stack itself via the `webServer` block unless `E2E_INSTANCE_URL` is set. Global setup signs in once and saves the session to `.auth/user.json`, so specs start signed in.
**Checkly monitors** — `checkly.config.ts` picks up the *same* `**/scenarios/**/*.spec.ts` files and runs them every 10 minutes with `baseURL: https://cloud.activepieces.com`, signing in with `E2E_EMAIL` / `E2E_PASSWORD`. It runs the specs from its last deploy, not the specs on `main`.
**BetterStack monitor** — one standalone file, `scenarios/betterstack/*.flat.spec.js`.

## Gotchas

- **Checkly runs the shared specs against production Cloud**, so anything the page objects assume about the login screen has to hold on Cloud too — not just on the SMTP-less instance CI boots. This is the usual reason a UI change breaks the monitors but not CI. See the auth-card gotchas on [CE Authentication](../connections-auth/ce-authentication.md).
- **Checkly deploys only by hand.** The `Deploy-Checkly` job in `emergency-cloud-deploy.yml` is the only path; nothing has called `e2e-tests-checkly.yml` since September 2025, and the `test-checkly` / `deploy-checkly` scripts are gone. A spec merged to `main` is not monitored until someone runs that workflow.
- **BetterStack does not read the repo — the repo pushes to it.** `.github/workflows/sync-betterstack-playwright.yml` fires on push to `main` and `PATCH`es the file's contents into the hardcoded monitor `4211060` as `playwright_script`. One-way and `main`-only: the monitor updates at *merge*, never on the PR, and any edit made in the BetterStack UI is silently overwritten by the next push.
- **The BetterStack file is deliberately flat and duplicated.** BetterStack executes one self-contained script, so it cannot `require` the `pages/` objects — its sign-in is a copy. Fix the page object and you have *not* fixed the monitor; both files need the change.
- **The `.flat.spec.js` runs nowhere else.** Playwright matches `*.spec.ts` and Checkly matches `**/scenarios/**/*.spec.ts`, so a broken flat file is invisible locally and in CI until it fails in BetterStack.
- **PRs do not run E2E unless they touch the suite or carry `ready-for-e2e`.** `e2e-matrix.yml` runs on its own only when `packages/tests-e2e/**` or its compose files change, so an unlabeled server change merges with no E2E check and the first signal is the Cloud gate's matrix. A missing check is not a pass. The label makes `e2e.yml` call the same matrix.
- **The label runs on `pull_request`, not `pull_request_target`.** A fork PR gets no `EE_LICENSE_KEY`, so its two EE configs fail; read the CE ones. Do not move the trigger back to `pull_request_target` to "fix" that: it hands PR code the secret.
- **Turbo strict env mode silently strips most of `.env.e2e`.** `globalPassThroughEnv` in `turbo.json` is an allow-list, so vars not named there never reach the `serve` tasks — verify with `tr '\0' '\n' < /proc/<api-pid>/environ`. `AP_ENVIRONMENT` is among the casualties, so CI falls back to the `prod` default. Widening it to `AP_*` does forward them, but that alone broke worker→API Socket.IO auth (jobs queue up unconsumed), so the passthrough and the worker's `AP_WORKER_TOKEN` have to be sorted out together.
- **`AP_DEV_PIECES` loads from `packages/pieces/**/dist`, which `npm run dev` does not build.** Only pieces that happen to be build dependencies of api/worker have a `dist`, so a default dev instance serves **0 pieces** and every spec that picks a trigger times out on the piece search. Build them explicitly: `npx turbo run build --filter=@activepieces/piece-webhook --filter=@activepieces/piece-store`.
- **CE sign-up is invitation-only once a platform exists** (`INVITATION_ONLY_SIGN_UP`), so the suite's sign-up path only works on a genuinely fresh instance. Against a dev-seeded database, set `E2E_EMAIL` / `E2E_PASSWORD` instead — `global-setup.ts` prefers them and signs in rather than signing up.
- **Every workspace declares its own deps.** `@faker-js/faker` was imported by the page objects for months while only `server/api` declared it; under Bun's isolated linker that means the suite cannot import its own page objects at all.
- **A property description shows only its first 70 characters.** `ReadMoreDescription` hides the rest behind "show more", so a `getByText` on a long description can pass in the matrix and fail on staging: the base URL `https://stg.activepieces.com/api/v1/webhooks/` is 10 characters longer than `http://172.30.0.10/api/v1/webhooks/`. Check a part that sits in the first 70 characters on every host, such as the URL origin (#17586).
- **`npx playwright test` can resolve a globally installed Playwright** instead of the repo's, and the two copies fail with `Playwright Test did not expect test.describe() to be called here` on *every* spec, including ones that have not been touched. The message points at your file and the cause is elsewhere. Run `./node_modules/.bin/playwright test --config=packages/tests-e2e/playwright.config.ts` instead.
- **Specs own their data.** Each flow and connection a spec makes is named `E2E …` and deleted by the `cleanup` fixture (`cleanup.track({ resource, id })`), also on failure; global setup only sweeps `E2E ` flows and connections older than an hour. The old setup deleted every flow, which put staging in the empty state on every run and once wiped a shared account.
- **Automations renders "Create New" while it loads, then may swap to the empty state.** Clicking it during the load opens a menu that disappears. `AutomationsPage.createFlow` waits for `automations-loading` to detach before it picks an entry.
- **"Test Flow" does not flush pending edits.** Step edits save through a 1 s debounce, and a test run started inside that window runs the old draft. The page object waits for the `POST /api/v1/flows/<id>` save after each edit; Publish flushes on its own.
- **Playwright sees every button in the builder breadcrumb as disabled.** `BreadcrumbPage` renders `role="link" aria-disabled="true"`, which Playwright inherits, so the flow menu button never clicks. Click the flow name text to rename instead.
- **Checkly runs no global setup.** It matches the same `*.spec.ts` files, so a spec that relies on the saved session fails there.
- **A spec that needs a live third-party connection cannot live in `scenarios/`.** Checkly and CI both pick up every `*.spec.ts` there, and neither has the connection, so it fails for everyone but its author. Keep piece-connection specs outside the repo and pin the regression in the piece's own unit tests instead. To cover the connection path itself, `connection-step.spec.ts` points the Activepieces Platform piece's Base URL at a webhook flow of its own, then reads the `Authorization` header in that flow's run.
- **A `FormLabel` names its input only through `FormControl`.** `FormControl` puts the label's id on its child, so an input outside it, or a wrapper that drops `id`, leaves `getByLabel` with nothing. Fix the markup rather than locate the field by nearby text.
- **A step must not call a `/sync` webhook on the same instance.** The step test holds a worker slot while the called flow waits for one, so with one slot the call times out (408). The matrix split configs run one slot: `benchmark/docker-compose.yml` passes `AP_WORKER_CONCURRENCY` as an empty string, which the worker reads as invalid and replaces with 1, not the default 5. Call the async webhook and check the called flow's run instead.
- **Custom API Call leaves its URL empty in the builder**, although the action's default is the connection's base URL. "Test Step" stays disabled until the spec fills the URL.
- **The EE configs run without a working license, so AI Center is locked there.** The server does not read the `AP_LICENSE_KEY` the matrix sets, so the platform sits on `AUTUMN_FREE_PLAN` with `aiProvidersEnabled: false` and no provider card shows Connect. The EE configs also run every spec under `scenarios/ce/`. An AI-provider spec lives in `ce/` and opens with `test.skip(process.env.AP_EDITION === 'ee', …)`.
- **No CI run uses `AP_EDITION=cloud`, and `Piece isolation` runs nowhere.** The matrix runs `ce` and `ee` only, so `scenarios/cloud/` never runs, and its `grepInvert` skips `Piece isolation` because the stack turns piece sync off. The `e2e-staging` job ran the `scenarios/ce/` specs against staging until 2026-11-18; it was removed because staging-only mismatches (no `aiProvidersEnabled` on the test account, a longer host) stopped Cloud releases with no product bug behind them. The LiteLLM provider spec was one of them (ENG-1524).
- **Only the EE configs send mail.** `benchmark/docker-compose.mail.yml` adds Mailpit to the EE configs of `e2e-matrix.yml`. With SMTP set, the server mails the invitation link and leaves `link` out of the invite response, so the dialog shows no link to copy. The CE configs keep no SMTP and test that path. A spec that reads mail uses the `mailbox` fixture and lives under `scenarios/ee/`.
- **Never default an `AP_SMTP_*` variable to empty in a compose file.** The env reader keeps an empty string and `isSmtpConfigured()` checks only for nil, so `AP_SMTP_HOST: ${AP_SMTP_HOST:-}` makes the app treat SMTP as set, and every send fails. The variables live in the mail overlay only.
- **Mail links arrive HTML-escaped.** The templates write `{{setupLink}}`, so Mustache turns `/`, `=` and `&` into entities. `fixtures/mail.ts` decodes them; code that reads the `HTML` of a Mailpit message itself must too.
- **The matrix serves the app over plain HTTP on an IP, which is not a secure context.** On `http://172.30.0.10`, `navigator.clipboard` and the other secure-context APIs are undefined, while `localhost` counts as secure, so a clipboard spec passes locally and fails every CI retry; `grantPermissions` does not help. When `E2E_INSTANCE_URL` is set, `playwright.config.ts` launches the full Chromium (`channel: 'chromium'`) with `--unsafely-treat-insecure-origin-as-secure`. The default headless shell ignores that flag.
- **`@quarantine` specs are skipped only in CI.** The config's `grepInvert` applies when `CI` is set, and a CLI `--grep-invert` replaces it, so `e2e-matrix.yml` repeats `@quarantine` in its own pattern.

## Key files
- `packages/tests-e2e` — `playwright.config.ts` (local/CI), `checkly.config.ts` (Cloud monitors), `global-setup.ts` (signs in or signs up, saves the session, sweeps stale `E2E ` flows and connections), `fixtures/` (`session`, `cleanup`, `mailbox`, `e2eFlows`, and one fixture file per feature), `pages/` (one page object per feature, imported directly; no barrel), `scenarios/betterstack/` (the standalone monitor script)
- `.github/workflows/e2e-matrix.yml` — the only E2E suite that gates a release: the built image in four CE/EE configurations
- `.github/workflows/e2e.yml` — the `ready-for-e2e` label trigger for that matrix
- `.github/workflows/sync-betterstack-playwright.yml` — the push-to-`main` upload
