fix: require live smoke opt-in (#7)

Co-authored-by: CometAPI <support@cometapi.com>

Author: tensornull

.github/workflows/live-smoke.yml

@@ -27,7 +27,7 @@ jobs:
     name: Bounded Chat Completions and Responses smoke
     if: >-
       github.ref == format('refs/heads/{0}', github.event.repository.default_branch) &&
-      (github.event_name == 'workflow_dispatch' || vars.LIVE_SMOKE_ENABLED == 'true')
+      vars.LIVE_SMOKE_ENABLED == 'true'
     runs-on: ubuntu-latest
     timeout-minutes: 10
     environment: live-smoke

.github/workflows/publish.yml

@@ -93,8 +93,8 @@ jobs:
     timeout-minutes: 10
     permissions:
       contents: read
-    # Required repository configuration: protect this environment, require reviewers, and
-    # configure COMETAPI_KEY plus the approved COMETAPI_LIVE_MODEL variable.
+    # Required repository configuration: protect this environment without required reviewers,
+    # and configure COMETAPI_KEY plus the approved COMETAPI_LIVE_MODEL variable.
     environment: live-smoke
     env:
       COMETAPI_LIVE_CONCURRENCY: "1"

AGENTS.md

@@ -46,8 +46,9 @@ Before the first remote push:
    violations in one run while returning non-zero if any violation exists.
    Keep checks for canonical identity, contacts, repository metadata, public-
    safe language, and standalone content.
-4. Gate scheduled live smoke with a `LIVE_SMOKE_ENABLED` repository variable.
-   An unset or non-true value must prevent live execution. Keep
+4. Gate scheduled and manually dispatched live smoke with a
+   `LIVE_SMOKE_ENABLED` repository variable. An unset or non-true value must
+   prevent live execution. Keep
    `RELEASE_PLEASE_ENABLED` disabled through the initial manual alpha.
 5. Make the release live-model setting use `gpt-5.4` when
    `COMETAPI_LIVE_MODEL` is unset or empty; never allow an empty model value.

RELEASING.md

@@ -39,11 +39,11 @@ The package manifest uses `authors = [{ name = "CometAPI" }]`. Remove
 `.github/CODEOWNERS` and its validation dependencies; it is not required while
 the project has one active maintainer.
 
-Before the first push, require `LIVE_SMOKE_ENABLED=true` for scheduled live
-execution and keep `RELEASE_PLEASE_ENABLED` disabled through the initial manual
-alpha. An unset or non-true value prevents the corresponding workflow from
-running. The release live-model configuration resolves an unset or empty
-`COMETAPI_LIVE_MODEL` to `gpt-5.4`.
+Before the first push, require `LIVE_SMOKE_ENABLED=true` for scheduled and
+manually dispatched live execution, and keep `RELEASE_PLEASE_ENABLED` disabled
+through the initial manual alpha. An unset or non-true value prevents the
+corresponding gated job from executing. The release live-model configuration
+resolves an unset or empty `COMETAPI_LIVE_MODEL` to `gpt-5.4`.
 
 The private stage validates sanitized history, the complete local gate, and
 real credential-free default-branch CI only. Do not configure or exercise
@@ -128,8 +128,8 @@ violations in one run and still returns non-zero when any violation exists.
   is ongoing monitoring only and cannot satisfy a release gate. It is capped at
   four requests, 16 output tokens
   per generation, a 30-second request timeout, concurrency one, a ten-minute
-  workflow timeout, and stop on the first failure. Scheduled execution also
-  requires `LIVE_SMOKE_ENABLED=true`.
+  workflow timeout, and stop on the first failure. Every trigger requires
+  `LIVE_SMOKE_ENABLED=true`.
 - `release-please.yml` maintains a human-reviewed version and changelog pull
   request from Conventional Commits after maintainers enable the
   `RELEASE_PLEASE_ENABLED` repository variable. Keep it disabled until the

ROADMAP.md

@@ -216,8 +216,9 @@ All workflow files must pass local `actionlint` 1.7.12. This is static
 validation only. Remote behavior remains unverified until each workflow runs
 successfully in the canonical GitHub repository.
 
-Scheduled live smoke must additionally require `LIVE_SMOKE_ENABLED=true`; an
-unset or other value prevents live execution. Release Please requires
+Scheduled and manually dispatched live smoke must require
+`LIVE_SMOKE_ENABLED=true`; an unset or other value prevents live execution.
+Release Please requires
 `RELEASE_PLEASE_ENABLED=true` and remains disabled through the initial manual
 alpha. Release jobs must resolve an unset or empty `COMETAPI_LIVE_MODEL` to
 `gpt-5.4` rather than attempt a request with an empty model.

scripts/check_workflows.py

@@ -23,13 +23,13 @@ def _require_pattern(text: str, pattern: str, message: str) -> None:
         raise CheckError(message)
 
 
-def _job(text: str, name: str) -> str:
+def _job(text: str, name: str, *, source: str = "publish workflow") -> str:
     match = re.search(
         rf"(?ms)^  {re.escape(name)}:\n(?P<body>.*?)(?=^  [a-zA-Z0-9_-]+:\n|\Z)",
         text,
     )
     if match is None:
-        raise CheckError(f"publish workflow has no {name!r} job")
+        raise CheckError(f"{source} has no {name!r} job")
     return match.group(0)
 
 
@@ -135,20 +135,21 @@ def check_publish_workflow(text: str, live_smoke_text: str) -> None:
     if write_permissions != ["id-token"]:
         raise CheckError("id-token: write on the publish job must be the only write permission")
 
+    monitoring_live = _job(live_smoke_text, "smoke", source="live-smoke workflow")
     _require_pattern(
         live_smoke_text,
         r"(?m)^concurrency:\n  group: trusted-live-smoke\n  cancel-in-progress: false$",
         "release and monitoring live smokes must share one non-cancelling concurrency group",
     )
-    _require(
-        live_smoke_text,
-        "(github.event_name == 'workflow_dispatch' || vars.LIVE_SMOKE_ENABLED == 'true')",
-        "scheduled live smoke must require LIVE_SMOKE_ENABLED=true",
-    )
-    _require(
-        live_smoke_text,
-        "github.ref == format('refs/heads/{0}', github.event.repository.default_branch)",
-        "monitoring live smoke must run only against the canonical default branch",
+    _require_pattern(
+        monitoring_live,
+        r"(?m)^    if: >-\n"
+        r"      github\.ref == format\('refs/heads/\{0\}', "
+        r"github\.event\.repository\.default_branch\) &&\n"
+        r"      vars\.LIVE_SMOKE_ENABLED == 'true'\n"
+        r"    runs-on:",
+        "monitoring live smoke must run only against the canonical default branch and "
+        "require LIVE_SMOKE_ENABLED=true for every trigger",
     )
 
     build = _job(text, "build")

tests/test_release_workflow.py

@@ -380,13 +380,43 @@ def test_semantic_contract_rejects_split_monitoring_live_concurrency() -> None:
         )
 
 
-def test_semantic_contract_rejects_ungated_scheduled_live_smoke() -> None:
-    live_smoke = LIVE_SMOKE_WORKFLOW.read_text(encoding="utf-8").replace(
+@pytest.mark.parametrize(
+    "replacement",
+    [
+        "github.event_name == 'workflow_dispatch'",
         "(github.event_name == 'workflow_dispatch' || vars.LIVE_SMOKE_ENABLED == 'true')",
+        "vars.LIVE_SMOKE_ENABLED != 'false'",
+        "vars.LIVE_SMOKE_ENABLED == 'true'\n      || github.event_name == 'workflow_dispatch'",
+    ],
+    ids=["manual-only", "manual-bypass", "non-exact-opt-in", "continued-manual-bypass"],
+)
+def test_semantic_contract_rejects_live_smoke_gate_bypasses(replacement: str) -> None:
+    live_smoke = LIVE_SMOKE_WORKFLOW.read_text(encoding="utf-8").replace(
+        "vars.LIVE_SMOKE_ENABLED == 'true'",
+        replacement,
+        1,
+    )
+    with pytest.raises(RuntimeError, match="every trigger"):
+        check_publish_workflow(
+            PUBLISH_WORKFLOW.read_text(encoding="utf-8"),
+            live_smoke,
+        )
+
+
+def test_semantic_contract_checks_live_smoke_gate_on_smoke_job() -> None:
+    live_smoke = LIVE_SMOKE_WORKFLOW.read_text(encoding="utf-8").replace(
+        "vars.LIVE_SMOKE_ENABLED == 'true'",
         "github.event_name == 'workflow_dispatch'",
         1,
     )
-    with pytest.raises(RuntimeError, match="LIVE_SMOKE_ENABLED=true"):
+    live_smoke += """
+  decoy:
+    if: >-
+      github.ref == format('refs/heads/{0}', github.event.repository.default_branch) &&
+      vars.LIVE_SMOKE_ENABLED == 'true'
+    runs-on: ubuntu-latest
+"""
+    with pytest.raises(RuntimeError, match="every trigger"):
         check_publish_workflow(
             PUBLISH_WORKFLOW.read_text(encoding="utf-8"),
             live_smoke,