Skip to content

feat: store CPU, GPU and RAM tracking methods in EmissionsData - #1388

Open
NoiceHax wants to merge 1 commit into
mlco2:masterfrom
NoiceHax:fix/issue-1144
Open

feat: store CPU, GPU and RAM tracking methods in EmissionsData#1388
NoiceHax wants to merge 1 commit into
mlco2:masterfrom
NoiceHax:fix/issue-1144

Conversation

@NoiceHax

Copy link
Copy Markdown

Description

CodeCarbon already logs which tracking method it picked for RAM, CPU and GPU at setup time, but that value stopped at the log line. Nothing downstream could tell whether CPU power came from RAPL, cpu_load or a TDP constant.

ResourceTracker._run_full_hardware_setup now writes ram_tracking_method, cpu_tracking_method and gpu_tracking_method into the tracker conf once both setup futures resolve, and EmissionsData carries the three values through to the CSV and the other outputs. They go through _conf rather than staying on the ResourceTracker, because the cached setup path rebuilds state from the stored plan. That is also why the keys were added to CONF_KEYS in hardware_cache.py. The new dataclass fields default to "", so existing positional construction still works.

The BoAmps mapper can now fill cpuTrackingMode and gpuTrackingMode, which it had to leave empty before. The comment explaining why they were missing is gone with it.

This covers the first bullet of the issue only. Sending the values to the API in the Experiment needs new columns on RunBase plus a carbonserver model and an alembic migration, so I left it out rather than mix two scopes in one PR. Happy to open a follow up.

Related Issue

issue #1144

Motivation and Context

The tracking method changes how much you should trust a measurement, so it belongs next to the numbers it produced. Reading it back from a CSV weeks later is not possible today.

How Has This Been Tested?

Ran the package suite locally with CODECARBON_ALLOW_MULTIPLE_RUNS=True pytest --ignore=tests/test_viz_data.py -m 'not integ_test' tests/: 629 passed, 19 skipped. The one failure, test_task_energy_with_live_update_interference, is a timing-sensitive energy sum assertion that fails the same way on a clean master checkout on my machine, so it is not from this change.

New tests: test_run_full_hardware_setup_stores_tracking_methods_in_conf, test_capture_and_apply_restore_tracking_methods for the cached path, test_offline_tracker_writes_tracking_methods for the CSV columns, and the BoAmps mapping test was split into a mapped case and an omitted case. I checked all of them fail before the source change and pass after.

Two fixtures needed the new columns: tests/test_data/emissions_valid_headers.csv, since has_valid_headers compares the whole header set, and the EmissionsData fixture in tests/output_methods/test_file.py. On the second one, leaving the fields at "" hits an existing problem in the update write path, where df[col].dtype.type(val) cannot cast an empty string on a column pandas read back as float. That already happens today with cloud_provider="" on private infrastructure, so I left it for a separate issue instead of widening this PR.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

AI Usage Disclosure

  • 🟥 AI-vibecoded: You cannot explain the logic. Car analogy : the car drive by itself, you are outside it and just tell it where to go.
  • 🟠 AI-generated: Car analogy : the car drive by itself, you are inside and give instructions.
  • â­� AI-assisted. Car analogy : you drive the car, AI help you find your way.
  • â™»ï¸� No AI used. Car analogy : you drive the car.

An AI agent wrote the code and this description while I directed it, and I reviewed and ran every change before submitting.

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the docs/how-to/contributing.md document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

The tracking method picked for each device was only written to the log
at setup time, so nothing downstream could tell whether CPU power came
from RAPL, cpu_load or a TDP constant.

ResourceTracker now writes ram_tracking_method, cpu_tracking_method and
gpu_tracking_method into the tracker conf once the hardware setup is
done, and EmissionsData carries them through to the CSV and the other
outputs. The keys are also part of the hardware cache CONF_KEYS so a
second tracker in the same process gets them back from the cached plan.

The BoAmps mapper can now fill cpuTrackingMode and gpuTrackingMode,
which it previously had to leave empty.
@NoiceHax
NoiceHax requested a review from a team as a code owner August 15, 2026 11:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant