feat(experimentation): experiment metadata on feature states - #8532
Conversation
…ent feature states
…nt pause, resume and rename
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review. 📝 WalkthroughWalkthroughThe change adds metadata for matching running experiments to feature states. Environment documents and identity responses expose the experiment ID, name, and Priority: ➖ Normal Estimated code review effort: 4 (Complex) | ~45 minutes Merge Risk: 🔵 Low · up to The generated schema may expose experiment metadata through Warning Git: CodeRabbit could not clone the repository, so clone-backed analysis was skipped and this review may be incomplete. Verify repository clone access, such as SSH credentials, before requesting another full review. If clone access is intentionally unavailable, use Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
api
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8532 +/- ##
==========================================
- Coverage 98.82% 98.81% -0.01%
==========================================
Files 1622 1620 -2
Lines 66727 66185 -542
==========================================
- Hits 65942 65400 -542
Misses 785 785 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
github-actions
Bot
added
feature
…uilder instead of prefetching
github-actions
Bot
added
feature
|
@themis-blindfold review |
github-actions
Bot
added
feature
Docker builds report
|
![themis-blindfold[bot]](https://avatars.githubusercontent.com/in/4278417?s=60&v=4){kind=small}
⚖️ Themis review: 🟠 Fix before mergeThe new experiment metadata reaches default and segment states, but not identity overrides in local-evaluation documents. That makes the metadata disappear precisely when an identity override wins evaluation. Completed API unit tests across Python 3.11–3.13 and static analysis checks passed; one third-party review check was still running.
🟠 Majors
📝 Walkthrough
🧪 How to verify
Product take: This is a solid observability improvement for experiments, but its value drops for identities with explicit overrides until the local-evaluation path matches remote evaluation. 🧭 Assumptions & unverified claimsNo unverified assumptions or claims. One metadata field, two evaluation routes — they should agree. · reviewed at 44d7bf0 |
✅ oss · depot-ubuntu-latest-arm-16 — run #20478 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
🗂️ Previous results✅ private-cloud · depot-ubuntu-latest-16 — run #20478 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
✅ private-cloud · depot-ubuntu-latest-arm-16 — run #20478 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
Skipped testsfirefox › tests/onboarding-tests.pw.ts › Onboarding › New user connects via the single-page onboarding flow @oss ✅ oss · depot-ubuntu-latest-16 — run #20478 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
✅ private-cloud · depot-ubuntu-latest-arm-16 — run #20465 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
✅ private-cloud · depot-ubuntu-latest-16 — run #20465 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
✅ oss · depot-ubuntu-latest-arm-16 — run #20465 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
✅ oss · depot-ubuntu-latest-16 — run #20465 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Advanced
Run ID: 412362eb-b73b-41b4-8c6b-89afbd4002e8
⛔ Files ignored due to path filters (1)
api/uv.lockis excluded by!**/*.lock
📒 Files selected for processing (16)
api/experimentation/feature_state_metadata.pyapi/experimentation/models.pyapi/features/serializers.pyapi/pyproject.tomlapi/tests/conftest.pyapi/tests/integration/environments/identities/test_integration_identities.pyapi/tests/unit/environments/identities/test_unit_identities_views.pyapi/tests/unit/environments/test_unit_environments_models.pyapi/tests/unit/environments/test_unit_environments_views_sdk_environment.pyapi/tests/unit/experimentation/test_models.pyapi/tests/unit/util/mappers/test_unit_mappers_dynamodb.pyapi/tests/unit/util/mappers/test_unit_mappers_engine.pyapi/util/engine_models/features/models.pyapi/util/mappers/engine.pyopenapi.yamlsdk/openapi.yaml
Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.
Visual Regression19 screenshots compared. See report for details. |
|
@themis-blindfold review |
⚖️ Themis review: ✅ Ship itThe remote identity response and environment-state mappings attach experiment metadata with lifecycle updates rebuilding the document when it can change. Completed API unit tests (Python 3.12 and 3.13) and E2E checks passed; one Python 3.11 unit job was still running. Focused local tests could not run because required dependencies could not be downloaded in this environment.
⚖️ Acknowledged
📝 Walkthrough
🧪 How to verify
Product take: Solid improvement for experiment consumers: exposure context now travels with evaluated states without changing evaluation itself. 🧭 Assumptions & unverified claimsNo unverified assumptions or claims. Experiment metadata now has a passport for its trip through evaluation. · reviewed at 44d7bf0 |
github-actions
Bot
added
feature
2 participants
Thanks for submitting a PR! Please check the boxes below:
docs/if required so people know about the feature.Changes
Feature states gain an optional
metadatadict on the wire.Its first key is
experiment { id, name, in_experiment }, present only while the feature's experiment is running.in_experimentistrueon the rollout-segment override andfalseon the feature's other states, so the object survives whichever override wins evaluation.Other features are untouched and
metadatais omitted when empty.Identity overrides never carry it: a hand-picked identity is not part of the experiment, on any evaluation route.
FeatureStateModel.metadata, excluded when empty.filter_for_document_builder.Experimenthook: republishes the environment document on pause / resume / rename, which did not touch feature states before./identities:SDKIdentityFeatureStateSerializeremits the same object,in_experimentfrom the identity's segment evaluation./flagsunchanged.flagsmith-common3.15.0 forFeatureStateMetadata,sdk/openapi.yamlregenerated.Query counts on
/identitiesand the environment document go up by one.How did you test this code?
Unit tests for the mapper, compressed document,
/identities, environment document endpoint and the hook.Covers in / out of cohort, identity override, non-running statuses and the single experiment query with many experimented flags.
Staging QA against Edge #719 to follow.