Skip to content

feat: carbon-aware codecarbon wait (forecast + greenest-window selection) - #1363

Open
davidberenstein1957 wants to merge 6 commits into
feat/intensity-providersfrom
feat/carbon-aware-scheduling
Open

feat: carbon-aware codecarbon wait (forecast + greenest-window selection)#1363
davidberenstein1957 wants to merge 6 commits into
feat/intensity-providersfrom
feat/carbon-aware-scheduling

Conversation

@davidberenstein1957

@davidberenstein1957 davidberenstein1957 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

A deliberately narrow first slice of carbon-aware scheduling (#1356).

Stacks on #1358: wait reuses that PR's cached, backed-off Electricity Maps client instead of opening a second HTTP path. Review #1358 first; this PR targets feat/intensity-providers and its diff here is only the wait feature.

What is in it

  • codecarbon/core/intensity_forecast.pyIntensityPoint / Forecast dataclasses, get_forecast() against the Electricity Maps /carbon-intensity/forecast endpoint (going through feat: cache and back off Electricity Maps carbon intensity #1358's shared request(), so a failing location backs off once for the whole process rather than once per caller), and best_window(), a pure sliding-mean window search with no I/O.
  • codecarbon/cli/wait.py + a wait command in codecarbon/cli/main.py:
# print the recommendation and exit
$ codecarbon wait --dry-run --deadline 24h --duration 90m
🌱 Best start: 2026-08-13 03:00 UTC  (112 gCO2e/kWh, now: 341)  -> saves ~67%

# block until the greenest window, then run under measurement
$ codecarbon wait --deadline 12h --duration 2h -- python train.py

# the job must be *finished* within 8 hours, and takes about 2
$ codecarbon wait --finish-by 8h --duration 2h -- python train.py

--deadline is the latest acceptable start, so --deadline 12h --duration 2h may leave the job running 14 hours from now. --finish-by is the complement most people actually mean: it bounds the end, and is implemented as finish_by - duration feeding the same search. A --finish-by shorter than --duration is rejected.

One API call. wait fetches the forecast and nothing else. The "now" figure printed, and the value --threshold compares against, is the first forecast point — the current period as the forecast sees it. An earlier revision also hit /carbon-intensity/latest for a live reading, which was a second network round trip in service of a percentage.

The blocking form delegates to the existing run_and_monitor, so measurement, CSV output and exit-code propagation are unchanged. The tracker only starts after the sleep, so a waiting process holds no lock. Ctrl-C during the wait starts the job immediately rather than aborting.

get_forecast never raises: no token, a non-200, a malformed payload or an empty forecast all return None, and every caller degrades to running now. A job is never blocked on a missing credential.

What is deliberately not in it

  • The @carbon_aware decorator / wait_for_green_window context manager. The CLI is the smaller surface and validates whether anyone wants the sleeping behaviour before we own a second one.
  • deferred_seconds / avoided_emissions on EmissionsData. Worth having, but they should ship with a proven blocking path — two always-zero CSV columns are a schema change for no reader.
  • A static fallback diurnal profile for users without a token. It would broaden reach a lot but risks systematically wrong advice: solar-heavy zones trough at midday, wind-heavy zones at night.
  • Mid-wait re-evaluation as the forecast updates, and region shifting.

What is blocked on the intensity-provider work

Electricity Maps is currently the only source that can serve a forecast, and only for users holding a paid token. That caps how much of the audience this feature can reach, and it is why this is a narrow slice rather than the full design in #1356.

get_forecast is written to be absorbed: once pluggable intensity providers land it should become an optional forecast() method on the provider protocol rather than a second HTTP client, and the HTTP lives in one place (#1358's request()) so that move is mechanical.

Tests

tests/test_intensity_forecast.py (14) and tests/cli/test_wait.py (22), all passing, no network — HTTP is stubbed with responses as in tests/test_electricitymaps_api.py, and geolocation, config and time.sleep are monkeypatched in the CLI tests. Coverage includes payload parsing, lat/lon vs countryCode selection, naive-vs-aware timestamps, horizon truncation, every failure mode returning None, the shared cooldown skipping the request, the window search (trough, flat, monotonic, deadline shorter than duration, duration longer than horizon), --finish-by bounding the end and its rejection when shorter than --duration, the duration parser, the threshold short-circuit taking no second API call, and delegation to run_and_monitor with the residual command.

uv run pytest tests/cli/test_wait.py tests/test_intensity_forecast.py -q   # 36 passed

Docs: a codecarbon wait section in docs/reference/cli.md and a pointer from docs/tutorials/cli.md.

Refs #1356

🤖 Generated with Claude Code

@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.46154% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.77%. Comparing base (70bbd45) to head (7fd8003).

Files with missing lines Patch % Lines
codecarbon/cli/main.py 50.00% 2 Missing ⚠️
Additional details and impacted files
@@                     Coverage Diff                      @@
##           feat/intensity-providers    #1363      +/-   ##
============================================================
+ Coverage                     91.54%   91.77%   +0.22%     
============================================================
  Files                            49       51       +2     
  Lines                          5122     5253     +131     
============================================================
+ Hits                           4689     4821     +132     
+ Misses                          433      432       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@davidberenstein1957
davidberenstein1957 force-pushed the feat/carbon-aware-scheduling branch from a71bcb1 to e7022e6 Compare August 13, 2026 05:28
@davidberenstein1957
davidberenstein1957 changed the base branch from master to feat/intensity-providers August 13, 2026 05:28
@davidberenstein1957
davidberenstein1957 marked this pull request as ready for review August 13, 2026 05:31
@davidberenstein1957
davidberenstein1957 requested a review from a team as a code owner August 13, 2026 05:31
davidberenstein1957 and others added 4 commits August 16, 2026 10:44
Fetch an Electricity Maps carbon intensity forecast, pick the window
with the lowest mean intensity that still meets the deadline, and
either report it (--dry-run) or sleep until it and delegate to
run_and_monitor.

Advisory/blocking only: no EmissionsData schema change, no decorator,
and no static fallback profile. Without a token, get_forecast returns
None and the job runs immediately -- a job is never blocked on a
missing credential. get_forecast should become a method on the
provider protocol once pluggable intensity providers land.

Refs #1356

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add a CLI reference section for `codecarbon wait` covering every flag and
its default, the forecast requirements, and the run-now degradation when
no forecast is available, plus one cross-link from the CLI tutorial.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`codecarbon wait` had its own HTTP path to Electricity Maps. It now goes
through `electricitymaps_api.request`, so the token lookup, the request
plumbing and the exponential failure cooldown are shared with the
current-intensity path: a failing API is backed off once, process-wide.

The forecast response is deliberately not put in the intensity cache.
That cache exists for a value refetched on every measurement tick with a
300 s TTL; a forecast is fetched once per `wait` invocation and has a
completely different useful lifetime.

`get_forecast` still never raises: a cooldown is just one more reason to
return None and run the job now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`--deadline` is documented as the maximum delay before the job must start,
but `best_window` rejected any window finishing after it, so
`--deadline 12h --duration 2h` only searched 10h of start times. The
documented semantics win: the deadline now bounds the start, the horizon
request is widened to deadline + duration, and the docs and tests say so.

Also stop stripping every argument equal to "wait" (it mangled
`codecarbon wait -- make wait`; only a leading one is ours), and compare
`--threshold` against the live carbon intensity instead of the first
forecast point.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@davidberenstein1957
davidberenstein1957 force-pushed the feat/carbon-aware-scheduling branch from 1ec74c6 to ca960ea Compare August 16, 2026 08:46
`find_green_window` fetched the forecast and then asked /latest for the
current intensity, a second HTTP call whose value only fed a "saves ~X%" line
and the --threshold short-circuit. The forecast's first point is that same
period, so use it and drop the call, the fallback and the try/except with it.

Add --finish-by as the complement to --deadline: --deadline bounds the start,
--finish-by bounds the end and is what most people mean. It is a subtraction,
not a second search path.

The Electricity Maps request extraction this branch used to carry now lives in
its base branch (#1358) where it belongs, so `clear_cooldown` is gone: request()
clears its own location's cooldown on a usable response.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@davidberenstein1957
davidberenstein1957 force-pushed the feat/carbon-aware-scheduling branch from ca960ea to 85df4b7 Compare August 16, 2026 08:47
Drop the scheduler and forecast aphorisms, keep the Ctrl+C and no-forecast
behaviour as plain statements, and remove the em-dashes the CLI reference
does not use.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant