fix(telemetry): disable telemetry in tests for local runs.

[Hweinstock]

Problem

We are seeing test data show up in our metrics:

{
  "OTelLib": "agentcore-cli",
  "Version": "1",
  "_aws": {
    "CloudWatchMetrics": [
      {
        "Namespace": "agentcore-cli",
        "Dimensions": [
          [],
          [
            "exit_reason"
          ],
          [
            "service.version"
          ]
        ],
        "Metrics": [
          {
            "Name": "cli.command_run",
            "Unit": "",
            "StorageResolution": 60
          }
        ]
      }
    ],
    "Timestamp": 1788963541491
  },
  "agentcore-cli.installation_id": "9f6e8e7f-9bfc-4231-a15f-0354b97ca689",
  "agentcore-cli.session_id": "ffffffff-1111-2222-3333-444444444444",
  "cli.command_run": {
    "Max": 123,
    "Min": 123,
    "Count": 1,
    "Sum": 123
  },
  "command_path": "/agentcore",
  "exit_reason": "failure",
  "host.arch": "x64",
  "is_tui": "false",
  "node.version": "v26.3.0",
  "os.type": "Linux",
  "os.version": "6.12.103-127.188.amzn2023.x86_64",
  "service.name": "agentcore-cli",
  "service.version": "1.0.0-rc.1"
}

This is likely coming from https://github.com/aws/agentcore-cli/blob/refactor/src/telemetry/client.test.tsx#L109, on local runs of the tests where telemetry is not disabled.

Solution

• Add a pre-test hook in testing/setup.ts the sets AGENTCORE_TELEMETRY_DISABLED to 1.
• Removed AGENTCORE_TELEMETRY_DISABLED from the CI workflow since its not baked into the tests.

Verification

Setup a basic local collector:

const outFile = process.env.COLLECTOR_OUT!;
const port = Number(process.env.COLLECTOR_PORT ?? 4318);
await Bun.write(outFile, "");
Bun.serve({
  port,
  async fetch(req) {
    const body = await req.text();
    await Bun.write(outFile, (await Bun.file(outFile).text()) + body + "\n");
    return new Response("", { status: 200 });
  },
});
console.log(`collector listening on ${port}`);

Run the collector via COLLECTOR_OUT=collector-out.txt COLLECTOR_PORT=4318 bun collector.ts &

update the default endpoint used in the tests at

agentcore-cli/src/globalConfig/config.tsx

Line 10 in 0caec7f

endpoint: "https://telemetry.agentcore.aws.dev",

to point to localhost:4318.

Run the tests with and without the env var injection to verify that it limits what is emitted.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[agentcore-devx-automation]

AgentCore Harness Review

Verdict: Changes requested

Nice, focused change — moving AGENTCORE_TELEMETRY_DISABLED into src/testing/setup.ts correctly covers all local bun test runs (setup.ts is preloaded via bunfig.toml and client.tsx:118 reads that env var), and the workflow-level env becomes redundant for the bun test step.

One issue

.github/workflows/verify.yml (removed top-level env block) — removing the workflow-level env also removes it from the Smoke test binary step (${{ matrix.binary }} --help, line 63-64). That step runs the compiled binary outside of bun test, so it no longer picks up the preload. Per src/index.ts:105, every invocation emits a cli.command_run metric in the finally block, so this PR will start shipping one telemetry event per platform per CI run from ephemeral runners.

Options:

• Restore the workflow-level env: AGENTCORE_TELEMETRY_DISABLED: "1" (simplest — mirrors prior behavior).
• Set the env var only on the smoke-test step.
• Set AGENTCORE_TELEMETRY_DISABLED=1 in the CodeBuild runner environment so it applies to any binary invocation in CI.

Everything else looks good to merge once the smoke-test path is covered.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.04%. Comparing base (d9ab959) to head (d5b136d).
⚠️ Report is 1 commits behind head on refactor.

Additional details and impacted files

@@            Coverage Diff            @@
##           refactor    #2253   +/-   ##
=========================================
  Coverage     97.04%   97.04%           
=========================================
  Files           566      566           
  Lines         39409    39409           
=========================================
  Hits          38244    38244           
  Misses         1165     1165           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[agentcore-devx-automation]

Claude Security Review: no high-confidence findings. (run)

[Hweinstock]

Addressed harness comment by re-applying change so that smoketest does not emit telemetry.