Skip to content

feat(e2e): use PLAPI to dynamically create with-email-codes instance #9455

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dstaley wants to merge 5 commits into
base: main
Choose a base branch
from
Open

feat(e2e): use PLAPI to dynamically create with-email-codes instance #9455

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
59c6629
feat(e2e): use PLAPI to dynamically create with-email-codes instance
dstaley Aug 14, 2026
22e28b9
fix(e2e): encode run keys
dstaley Aug 14, 2026
7a1b10e
fix(e2e): don't use platform api key in hash
dstaley Aug 14, 2026
c298b65
fix(e2e): make instance keys side-effect free
dstaley Aug 14, 2026
b40bb46
fix(e2e): update import
dstaley Aug 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .changeset/tidy-env-awaits.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
---
---
132 changes: 68 additions & 64 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ concurrency:
jobs:
# Check triggering actor permissions to prevent PRs from forks accessing secrets by default, preventing them from exfiltrating secrets for malicious purposes
check-permissions:
runs-on: "blacksmith-8vcpu-ubuntu-2204"
runs-on: 'blacksmith-8vcpu-ubuntu-2204'
defaults:
run:
shell: bash
Expand Down Expand Up @@ -48,7 +48,7 @@ jobs:
pre-checks:
needs: [check-permissions]
name: Formatting | Dedupe | Changeset
runs-on: "blacksmith-8vcpu-ubuntu-2204"
runs-on: 'blacksmith-8vcpu-ubuntu-2204'
defaults:
run:
shell: bash
Expand All @@ -61,7 +61,7 @@ jobs:
persist-credentials: false
fetch-depth: 100 # Enough for changeset status comparison, much faster than full history
fetch-tags: false
filter: "blob:none"
filter: 'blob:none'
show-progress: false

- name: Fetch main branch for changeset comparison
Expand Down Expand Up @@ -104,7 +104,7 @@ jobs:
build-packages:
needs: [check-permissions]
name: Build Packages
runs-on: "blacksmith-8vcpu-ubuntu-2204"
runs-on: 'blacksmith-8vcpu-ubuntu-2204'
permissions:
contents: read
defaults:
Expand All @@ -122,7 +122,7 @@ jobs:
persist-credentials: false
fetch-depth: 1
fetch-tags: false
filter: "blob:none"
filter: 'blob:none'
show-progress: false

- name: Setup
Expand Down Expand Up @@ -152,7 +152,7 @@ jobs:
name: Bundle size
permissions:
contents: read
runs-on: "blacksmith-8vcpu-ubuntu-2204"
runs-on: 'blacksmith-8vcpu-ubuntu-2204'
defaults:
run:
shell: bash
Expand All @@ -165,7 +165,7 @@ jobs:
persist-credentials: false
fetch-depth: 1
fetch-tags: false
filter: "blob:none"
filter: 'blob:none'
show-progress: false

- name: Setup
Expand Down Expand Up @@ -196,7 +196,7 @@ jobs:
permissions:
contents: read
actions: write # needed for actions/upload-artifact
runs-on: "blacksmith-8vcpu-ubuntu-2204"
runs-on: 'blacksmith-8vcpu-ubuntu-2204'
defaults:
run:
shell: bash
Expand All @@ -212,7 +212,7 @@ jobs:
persist-credentials: false
fetch-depth: 1
fetch-tags: false
filter: "blob:none"
filter: 'blob:none'
show-progress: false

- name: Setup
Expand Down Expand Up @@ -249,7 +249,7 @@ jobs:
permissions:
contents: read
actions: write # needed for actions/upload-artifact
runs-on: "blacksmith-8vcpu-ubuntu-2204"
runs-on: 'blacksmith-8vcpu-ubuntu-2204'
defaults:
run:
shell: bash
Expand All @@ -263,11 +263,11 @@ jobs:
matrix:
include:
- node-version: 24.15.0
test-filter: "**"
filter-label: "**"
test-filter: '**'
filter-label: '**'
- node-version: 20.19.0
test-filter: "**"
filter-label: "**"
test-filter: '**'
filter-label: '**'

steps:
- name: Checkout Repo
Expand All @@ -276,7 +276,7 @@ jobs:
persist-credentials: false
fetch-depth: 1
fetch-tags: false
filter: "blob:none"
filter: 'blob:none'
show-progress: false

- name: Setup
Expand Down Expand Up @@ -328,7 +328,7 @@ jobs:
permissions:
contents: read
actions: write # needed for actions/upload-artifact
runs-on: "blacksmith-8vcpu-ubuntu-2204"
runs-on: 'blacksmith-8vcpu-ubuntu-2204'
defaults:
run:
shell: bash
Expand All @@ -339,46 +339,46 @@ jobs:
matrix:
test-name:
[
"generic",
"express",
"fastify",
"ap-flows",
"localhost",
"sessions",
"sessions:staging",
"handshake",
"handshake:staging",
"astro",
"tanstack-react-start",
"vue",
"nuxt",
"react-router",
"custom",
"hono",
"chrome-extension",
"electron",
'generic',
'express',
'fastify',
'ap-flows',
'localhost',
'sessions',
'sessions:staging',
'handshake',
'handshake:staging',
'astro',
'tanstack-react-start',
'vue',
'nuxt',
'react-router',
'custom',
'hono',
'chrome-extension',
'electron',
]
test-project: ["chrome"]
test-project: ['chrome']
include:
- test-name: "billing"
test-project: "chrome"
- test-name: "machine"
test-project: "chrome"
- test-name: "nextjs"
test-project: "chrome"
next-version: "15"
- test-name: "nextjs"
test-project: "chrome"
next-version: "16"
- test-name: "quickstart"
test-project: "chrome"
next-version: "15"
- test-name: "quickstart"
test-project: "chrome"
next-version: "16"
- test-name: "cache-components"
test-project: "chrome"
next-version: "16"
- test-name: 'billing'
test-project: 'chrome'
- test-name: 'machine'
test-project: 'chrome'
- test-name: 'nextjs'
test-project: 'chrome'
next-version: '15'
- test-name: 'nextjs'
test-project: 'chrome'
next-version: '16'
- test-name: 'quickstart'
test-project: 'chrome'
next-version: '15'
- test-name: 'quickstart'
test-project: 'chrome'
next-version: '16'
- test-name: 'cache-components'
test-project: 'chrome'
next-version: '16'

steps:
- name: Checkout Repo
Expand All @@ -387,7 +387,7 @@ jobs:
persist-credentials: false
fetch-depth: 1
fetch-tags: false
filter: "blob:none"
filter: 'blob:none'
show-progress: false

- name: Setup
Expand All @@ -413,8 +413,8 @@ jobs:
env:
E2E_APP_CLERK_JS_DIR: ${{runner.temp}}
E2E_APP_CLERK_UI_DIR: ${{runner.temp}}
E2E_CLERK_JS_VERSION: "latest"
E2E_CLERK_UI_VERSION: "latest"
E2E_CLERK_JS_VERSION: 'latest'
E2E_CLERK_UI_VERSION: 'latest'
E2E_NEXTJS_VERSION: ${{ matrix.next-version }}
E2E_PROJECT: ${{ matrix.test-project }}
INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
Expand Down Expand Up @@ -478,8 +478,8 @@ jobs:
- name: Write all ENV certificates to files in integration/certs
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7
env:
INTEGRATION_CERTS: "${{secrets.INTEGRATION_CERTS}}"
INTEGRATION_ROOT_CA: "${{secrets.INTEGRATION_ROOT_CA}}"
INTEGRATION_CERTS: '${{secrets.INTEGRATION_CERTS}}'
INTEGRATION_ROOT_CA: '${{secrets.INTEGRATION_ROOT_CA}}'
with:
script: |
const fs = require('fs');
Expand Down Expand Up @@ -523,11 +523,13 @@ jobs:
pnpm turbo test:integration:${{ matrix.test-name }} $TURBO_ARGS
fi
env:
E2E_DEBUG: "1"
CLERK_E2E_OAUTH_PROVIDER_CLIENT_SECRET: ${{ secrets.CLERK_E2E_OAUTH_PROVIDER_CLIENT_SECRET }}
CLERK_PLATFORM_API_KEY: ${{ secrets.CLERK_PLATFORM_API_KEY }}
E2E_DEBUG: '1'
E2E_APP_CLERK_JS_DIR: ${{runner.temp}}
E2E_APP_CLERK_UI_DIR: ${{runner.temp}}
E2E_CLERK_JS_VERSION: "latest"
E2E_CLERK_UI_VERSION: "latest"
E2E_CLERK_JS_VERSION: 'latest'
E2E_CLERK_UI_VERSION: 'latest'
E2E_NEXTJS_VERSION: ${{ matrix.next-version }}
E2E_PROJECT: ${{ matrix.test-project }}
INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
Expand All @@ -539,6 +541,8 @@ jobs:
timeout-minutes: 4
run: pnpm test:integration:cleanup
env:
CLERK_E2E_OAUTH_PROVIDER_CLIENT_SECRET: ${{ secrets.CLERK_E2E_OAUTH_PROVIDER_CLIENT_SECRET }}
CLERK_PLATFORM_API_KEY: ${{ secrets.CLERK_PLATFORM_API_KEY }}
INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem

Expand All @@ -562,7 +566,7 @@ jobs:
pkg-pr-new:
name: Publish with pkg-pr-new
needs: [check-permissions, build-packages]
runs-on: "blacksmith-8vcpu-ubuntu-2204"
runs-on: 'blacksmith-8vcpu-ubuntu-2204'
defaults:
run:
shell: bash
Expand All @@ -577,7 +581,7 @@ jobs:
persist-credentials: false
fetch-depth: 1
fetch-tags: false
filter: "blob:none"
filter: 'blob:none'
show-progress: false

- name: Setup Node
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/e2e-cleanups.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,5 +38,6 @@ jobs:
- name: Run cleanup
run: pnpm test:integration:cleanup
env:
CLERK_PLATFORM_API_KEY: ${{ secrets.CLERK_PLATFORM_API_KEY }}
INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
INTEGRATION_STAGING_INSTANCE_KEYS: ${{ secrets.INTEGRATION_STAGING_INSTANCE_KEYS }}
17 changes: 17 additions & 0 deletions .github/workflows/nightly-checks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,13 @@ jobs:
pnpm init
pkglab add @clerk/ui

- name: Configure test run cleanup
run: echo "INTEGRATION_TEST_RUN_KEY=${RUN_ID}-${RUN_ATTEMPT}-nightly-${TEST_NAME}" >> "$GITHUB_ENV"
env:
RUN_ID: ${{ github.run_id }}
RUN_ATTEMPT: ${{ github.run_attempt }}
TEST_NAME: ${{ matrix.test-name }}

- name: Run Integration Tests
id: integration_tests
continue-on-error: true
Expand All @@ -71,6 +78,8 @@ jobs:
fi
echo "exit_code=${PIPESTATUS[0]}" >> $GITHUB_OUTPUT
env:
CLERK_E2E_OAUTH_PROVIDER_CLIENT_SECRET: ${{ secrets.CLERK_E2E_OAUTH_PROVIDER_CLIENT_SECRET }}
CLERK_PLATFORM_API_KEY: ${{ secrets.CLERK_PLATFORM_API_KEY }}
E2E_APP_CLERK_JS_DIR: ${{runner.temp}}
E2E_APP_CLERK_UI_DIR: ${{runner.temp}}
# Don't set E2E_CLERK_JS_VERSION - let it use '*' (via linkPackage)
Expand Down Expand Up @@ -118,6 +127,14 @@ jobs:
test-results/
retention-days: 7

- name: Delete integration-test resources
if: ${{ always() && steps.integration_tests.outcome != 'skipped' }}
timeout-minutes: 4
run: pnpm test:integration:cleanup
env:
CLERK_PLATFORM_API_KEY: ${{ secrets.CLERK_PLATFORM_API_KEY }}
INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}

- name: Report Status
if: always()
uses: ravsamhq/notify-slack-action@4ed28566c2bdcdaee6dca2b46b9666d01b4ed8a4 # v1
Expand Down
25 changes: 23 additions & 2 deletions integration/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -331,7 +331,7 @@ If you need to run a test suite inside a different environment (e.g. a different
1. Inside `presets/envs.ts`, create a new environment config wrapped with `withInstanceKeys`:

```ts
const yourConciseName = withInstanceKeys('your-concise-name', base.clone().setId('yourConciseName'));
const yourConciseName = await withInstanceKeys('your-concise-name', base.clone().setId('yourConciseName'));
```

The `withInstanceKeys` wrapper sets PK/SK from the instance keys map and automatically swaps to staging keys when `E2E_STAGING=1` is set. The first argument is the production key name — the staging key is looked up as `clerkstage-your-concise-name`. See [Running tests against staging](#running-tests-against-staging) for more details.
Expand Down Expand Up @@ -562,7 +562,7 @@ await app.withEnv(appConfigs.envs.withEmailCodes);
Inside [`presets/envs.ts`](../integration/presets/envs.ts) you can also create a completely new environment config. All new configs should be wrapped with `withInstanceKeys` to enable staging environment swapping:

```ts
const withCustomRoles = withInstanceKeys(
const withCustomRoles = await withInstanceKeys(
'with-custom-roles',
base
.clone()
Expand All @@ -572,6 +572,27 @@ const withCustomRoles = withInstanceKeys(
);
```

For non-staging tests, when `CLERK_PLATFORM_API_KEY` is set and `integration/configs/<key-name>.js` exists, `withInstanceKeys` creates a development application through the Platform API. It applies the exported configuration and uses the keys from the new application. If the file does not exist, the wrapper uses the existing instance keys map.

Use `defineConfig` for configuration files. JavaScript configuration files can read environment variables. The optional `setup` function receives a `ClerkClient` after the instance configuration is applied. Omit `setup` when no additional operations are required.

```js
import { defineConfig } from '../presets/platformApplication.js';

export default defineConfig({
config: {
session: {
lifetime: Number(process.env.CLERK_E2E_SESSION_LIFETIME || 3600),
},
},
setup: async clerkClient => {
await clerkClient.allowlistIdentifiers.createAllowlistIdentifier({ identifier: 'allowed@example.com' });
},
});
```

The application name includes a deterministic, letter-only token derived from `INTEGRATION_TEST_RUN_KEY` when it is available. Otherwise, it uses a random letter-only suffix. Application keys are cached during `E2E_APP_ID` runs because their setup and test workers must use the same instance. Global teardown removes the cache file.

When `E2E_STAGING=1`, this will automatically look up `clerkstage-with-custom-roles` from the staging keys and swap the PK, SK, and API URL. If the staging key doesn't exist, the config will not be staging-ready and any long running apps using it will be gracefully skipped.

Read [creating a new environment config](#creating-a-new-environment-config) and [running tests against staging](#running-tests-against-staging) to learn more.
Expand Down
Loading
Loading