perf(parser): lazy-load envelopes to reduce Lambda cold start latency by ~900ms - #8405
Conversation
…rt latency Importing `parse` or `event_parser` from `aws_lambda_powertools.utilities.parser` previously caused all 16 envelope modules to load eagerly via the package `__init__.py`, even when no envelope was used. This added ~900ms to Lambda cold start for any function using the parser without an envelope (the common case for API Gateway handlers). The fix uses Python's module-level `__getattr__` to defer loading of `envelopes` and `BaseEnvelope` until they are actually accessed. Public API is unchanged — existing code importing from this module continues to work without modification. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need. |
leandrodamascena
left a comment
There was a problem hiding this comment.
Thanks for working on this. The lazy-loading direction is valuable, but the current implementation breaks an existing public import path. I am requesting changes so we preserve compatibility and make the import test reliably isolated.
…ests - Fix RecursionError by using importlib.import_module() and caching in globals() - Improve test isolation by running tests in fresh subprocesses - Add tests for all public import paths (parse, envelopes, BaseEnvelope, import *) - Ensure backward compatibility for all existing public APIs Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
Thank you for the thorough review and valuable feedback! I've addressed all your comments: ✅ Fixed RecursionError: Now using
All tests pass and the lazy-loading behavior is fully preserved while maintaining backward compatibility. Thanks again for taking the time to review this! 🙏 |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #8405 +/- ##
===========================================
- Coverage 96.65% 96.60% -0.05%
===========================================
Files 296 296
Lines 14806 14820 +14
Branches 1255 1257 +2
===========================================
+ Hits 14310 14317 +7
- Misses 361 366 +5
- Partials 135 137 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
leandrodamascena
left a comment
There was a problem hiding this comment.
Thank you, Erez. This was really good work.
I ran the benchmark myself using real Lambda functions and confirmed the improvement. On Python 3.13 ARM64, the median Init Duration dropped from 786.65 ms to 553.04 ms across 20 cold starts per variant. The exact number depends on the environment, but the improvement is clear.
I noticed one small compatibility gap: the lazy exports were missing from dir(parser) and inspect.getmembers(parser). I added __dir__ and a regression test directly to your branch. This keeps the envelopes lazy while preserving normal Python introspection.
I am approving this and will merge it once CI is green. Really nice contribution.
d3890ce to
6ac623c
Compare
|
51c89a3
into
aws-powertools:develop
|
Awesome work, congrats on your first merged pull request and thank you for helping improve everyone's experience! |
|
Awesome work, congrats on your first merged pull request and thank you for helping improve everyone's experience! |



Issue number: closes #8406
Summary
Changes
envelopesandBaseEnvelopeinparser/__init__.pywith a module-level__getattr__that defers loading until those names are actually accessedparsedoes not load any envelope modulesUser experience
Before: Any Lambda function importing
parseorevent_parserfromaws_lambda_powertools.utilities.parsercaused all 16 envelope modules to load eagerly at cold start — even when no envelope was ever used.After: Envelope modules are only imported on first access (e.g. when passing
envelope=SqsEnvelopetoparse()). Functions that useparse()without an envelope pay zero envelope-loading cost.Measured impact: ~900ms reduction in Lambda INIT_DURATION for API Gateway handlers using
parse()without an envelope. Profiled usingPYTHONPROFILEIMPORTTIME=1on AWS Lambda (arm64, Python 3.13, 1024MB):Public API is fully preserved —
from aws_lambda_powertools.utilities.parser import envelopesandfrom aws_lambda_powertools.utilities.parser import BaseEnvelopecontinue to work exactly as before.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.