PYTHON-5909 / PYTHON-5979 Add GA support for Queryable Encryption string queries + Add QE prefix+suffix GA and rename API to string

[aclark4life]

PYTHON-5909
PYTHON-5979

Changes in this PR

Adds general availability support for Queryable Encryption prefix, suffix, and substring string queries against MongoDB 9.0+, and renames the preview API to its GA form. Support landed in libmongocrypt per query type: prefix and suffix require 1.19.0+, substring requires 1.20.0+.

Public API

• pymongo.encryption_options.StringOpts replaces TextOpts. TextOpts is kept as a deprecated subclass of StringOpts and emits a DeprecationWarning on construction. It also stays re-exported from pymongo.encryption, so from pymongo.encryption import TextOpts continues to work for the deprecation period.
• Algorithm.STRING replaces Algorithm.TEXTPREVIEW, which is now deprecated.
• New QueryType.PREFIX, QueryType.SUFFIX, and QueryType.SUBSTRING, backing the $encStrStartsWith, $encStrEndsWith, and $encStrContains operators. The existing QueryType.PREFIXPREVIEW / SUFFIXPREVIEW / SUBSTRINGPREVIEW members remain for experimental use against servers older than 9.0.
• ClientEncryption.encrypt() and AsyncClientEncryption.encrypt() gain a string_opts parameter. text_opts is retained as a deprecated alias; passing both raises ConfigurationError.

Internal

• A shared helper resolves string_opts / text_opts into a single value so the sync and async encrypt() paths share one deprecation code path.
• _STRING_QUERY_MIN_LIBMONGOCRYPT in the test suite declares the minimum libmongocrypt version per query type in one place, so the test gates and the changelog cannot drift apart.

CI

• libmongocrypt now comes from the signed GitHub release assets. MONGOCRYPT-838 moved the per-variant release builds to a restricted bucket, which left mciuploads/.../master/latest frozen at 1.18.0 and skipped the entire prose suite on every variant. setup_tests.py now fetches libmongocrypt-{target}-{version}.tar.gz from the release page, pinned by LIBMONGOCRYPT_VERSION = "1.20.2" (1.20.0 is the floor for substring GA). Release assets are keyed by libc flavor rather than distro, so the old Debian/Ubuntu/RHEL target table collapses to arch plus glibc/musl, which also picks up ppc64le and s390x. A get_libmongocrypt_base() helper absorbs the layout difference: release archives put lib/ at the archive root, while the master builds nested everything under nocrypto/. An explicit LIBMONGOCRYPT_URL still wins.
• The released-pymongocrypt pin now reads VERSION, not MONGODB_VERSION. Evergreen never sets MONGODB_VERSION it is only assigned in-process by run_server.py so the 8.x branch of that gate was dead code and those tasks kept installing the master binding.
• The prose setup only builds the substring fixture where the query type exists. It previously encrypted it unconditionally, so on libmongocrypt 1.19.x setup raised before the substring cases could skip, failing all 11 cases including the prefix and suffix ones.
• New test-string-query-preview task on server 8.2. The preview query types need a server at least 8.2 and older than 9.0, but ALL_VERSIONS jumps straight from 8.0 to 9.0, so the preview cases skipped on the server version gate no matter which libmongocrypt was installed. create_string_query_preview_tasks() emits one replica-set task pinned to 8.2, selected by the six encryption variants. ALL_VERSIONS is untouched, so the rest of the matrix does not grow.

Test Plan

• New prose tests TestStringExplicitEncryptionProse implements the spec's "String Explicit Encryption" tests, cases 01-11: find by prefix/suffix/substring, the corresponding no-match cases, contentionFactor being required, and the new case-insensitive and diacritic-insensitive prefix/suffix/substring cases.
• Per-query-type version gating the class is gated on require_version_min(8, 2, -1) and require_libmongocrypt_min(1, 18, 1), the floor for the preview query types, so both API generations can run. Setup then encrypts with Algorithm.STRING where it is available and the deprecated Algorithm.TEXTPREVIEW below 1.19.0. Two helpers gate the individual cases, and both look their requirements up in _STRING_QUERY_MIN_LIBMONGOCRYPT rather than hardcoding them at the call site: _params(kind) returns the (query_type, collection) pair for the cases that run against the GA type on 9.0+ and the preview type on earlier servers, and _require_ga(*query_types) skips the GA-only cases, naming each query type they exercise. A case is skipped below server 9.0, or when libmongocrypt is too old for any query type it uses.
• New unit tests TestStringOptsDeprecation covers the deprecation surface directly: TextOpts warns and remains re-exported from pymongo.encryption, and the resolve helper accepts string_opts, accepts text_opts with a warning, and rejects both with ConfigurationError. These need no server or libmongocrypt, so the deprecation shims are covered on every CI task. Verified locally: 10 tests pass across both suites.

Checklist

Checklist for Author

• Did you update the changelog (if necessary)?
• Is there test coverage?
• Is any followup work tracked in a JIRA ticket? If so, add link(s).

Checklist for Reviewer

• Does the title of the PR reference a JIRA Ticket?
• Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
• Is all relevant documentation (README or docstring) updated?

[Copilot]

Pull request overview

Adds GA support for Queryable Encryption string queries (prefix/suffix/substring) targeting MongoDB 9.0+, including API updates and refreshed prose/integration coverage, and introduces CI/process changes to better manage uv.lock maintenance.

Changes:

• Introduces Algorithm.STRING, StringOpts, and GA QueryType values (PREFIX, SUFFIX, SUBSTRING) while deprecating TextOpts/Algorithm.TEXTPREVIEW and keeping preview query types for pre-9.0 servers.
• Updates encryption prose/integration tests (sync + async) to cover GA vs preview behavior, plus adds unit tests for the deprecation shims.
• Adds/adjusts dependency-management workflows and docs around uv.lock (scheduled lockfile update workflow, uv lock --check in CI, Dependabot tuning).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
test/test_encryption.py	Sync encryption tests updated for string GA/preview query behavior + deprecation shim tests.
test/asynchronous/test_encryption.py	Async encryption tests updated for string GA/preview query behavior + deprecation shim tests.
pymongo/encryption_options.py	Adds StringOpts; deprecates TextOpts as a compatibility shim.
pymongo/synchronous/encryption.py	Sync encryption API updated to accept string_opts and deprecate text_opts; adds GA string/query enums.
pymongo/asynchronous/encryption.py	Async encryption API updated to accept string_opts and deprecate text_opts; adds GA string/query enums.
doc/changelog.rst	Documents GA support and the new/Deprecated APIs.
pyproject.toml	Adjusts uv dependency constraints to avoid problematic back-solving for boto3 across forks.
CONTRIBUTING.md	Updates dependency/lockfile workflow guidance for contributors.
.pre-commit-config.yaml	Excludes uv.lock from the large-file pre-commit check.
.gitignore	Stops ignoring uv.lock so it can be committed/checked.
.github/workflows/uv-lock-update.yml	New scheduled workflow to regularly update uv.lock.
.github/workflows/test-python.yml	Adds uv lock --check and removes the custom exclude-newer action usage.
.github/dependabot.yml	Disables routine uv version-update PRs (handled by the scheduled uv-lock-update workflow).
.github/actions/set-uv-exclude-newer/action.yml	Removes the custom action previously used to set UV_EXCLUDE_NEWER.
.evergreen/run-mongodb-aws-ecs-test.sh	Stops deleting uv.lock, aligning with committed lockfile usage.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (5)

pymongo/asynchronous/encryption.py:609

• This preview-query note omits the specification's required warning that the feature's security is not guaranteed and that GA may not be backward compatible with preview payloads. Include those caveats so users do not treat SUFFIXPREVIEW as production-safe merely because it remains available.

    .. note:: The preview query types are for experimental workloads only and
       are only supported by MongoDB versions before 9.0. Use
       :attr:`QueryType.SUFFIX` instead.

pymongo/asynchronous/encryption.py:619

• This preview-query note omits the specification's required warning that the feature's security is not guaranteed and that GA may not be backward compatible with preview payloads. Include those caveats so users do not treat SUBSTRINGPREVIEW as production-safe merely because it remains available.

    .. note:: The preview query types are for experimental workloads only and
       are only supported by MongoDB versions before 9.0. Use
       :attr:`QueryType.SUBSTRING` instead.

test/asynchronous/test_encryption.py:3363

• The prose-test baseline is libmongocrypt 1.18.1, and the substringPreview cases are explicitly required to run at that version. This class-level 1.19.0 gate skips the entire suite before _params() can apply its 1.18.1 requirement, leaving that supported combination untested. Lower the class gate to 1.18.1 and keep the per-case gates for newer query types.

    @async_client_context.require_libmongocrypt_min(1, 19, 0)

pymongo/asynchronous/encryption.py:537

• The client-side encryption specification requires drivers to document that String payloads must be processed by a client configured with AutoEncryptionOpts, with bypass_auto_encryption=False (while bypass_query_analysis may be true). The new public algorithm currently omits this operational requirement, so users can follow the API docs and send an unusable payload. Add the required usage note to this docstring.

    STRING = "String"
    """String.

    .. versionadded:: 4.18

pymongo/asynchronous/encryption.py:599

• This preview-query note omits the specification's required warning that the feature's security is not guaranteed and that GA may not be backward compatible with preview payloads. Include those caveats so users do not treat PREFIXPREVIEW as production-safe merely because it remains available.

This issue also appears in the following locations of the same file:

• line 607
• line 617

    .. note:: The preview query types are for experimental workloads only and
       are only supported by MongoDB versions before 9.0. Use
       :attr:`QueryType.PREFIX` instead.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[blink1073]

Okay, we need two things to close this out:

• We need to target pymongcrypt<1.19 when MONGODB_VERSION is 8.0, in setup_tests.py, and make sure the SubstringPreview tests are not skipped on MongoDB 8.0.
• We need to find out where libmongocrypt is putting the updated release files, because f"https://s3.amazonaws.com/mciuploads/libmongocrypt/{target}/master/latest/libmongocrypt.tar.gz" is pointing to 1.18.0-20260313+git9f4f0a1382. I think mongodb/libmongocrypt@6571317 is the relevant comment. Then make sure the substring tests without preview run on MongoDB 9.0.

[aclark4life]

We need to find out where libmongocrypt is putting the updated release files,

MONGOCRYPT-838 switched release-branch builds to a restricted bucket, cdn-origin-libmongocrypt. That's the root cause: the per-variant release artifacts we were relying on are no longer public.

master/latest is still public on both hosts, but frozen:

Host	Version at macos/master/latest
s3.amazonaws.com/mciuploads	1.18.0-20260317+git9f4f0a1382
downloads.mongodb.org	1.18.0-20260317+git6571317bb8

So master builds did move to downloads.mongodb.org at that commit — but both are stuck in March either way, which is why the prose suite currently skips everywhere.

Two options:

1. Versioned "all" tarball — https://downloads.mongodb.org/libmongocrypt/all/{version}/libmongocrypt-all.tar.gz

Verified resolving for 1.19.0, 1.19.1, 1.20.0 and 1.20.2, and it uses the same target names we already map to (debian10, amazon2, macos, …), so setup_libmongocrypt() would barely change. The problem is size: 1.76 GB (content-length: 1760303227). Not viable to pull on every encryption task.

2. GitHub release assets — e.g. libmongocrypt-macos-universal-1.20.2.tar.gz

Signed, and what the 1.18.0 changelog explicitly tells drivers to migrate to. Every target is nocrypto except windows-x86_64, which has no nocrypto variant (it links CNG, so there is no OpenSSL dependency to avoid). Small (macOS is 13 MB). Two differences from what setup_tests.py expects:

• Layout is lib/libmongocrypt.dylib at the archive root — there's no nocrypto/ subdir, so BASE = ROOT / "libmongocrypt/nocrypto" needs to become conditional.
• Variant names are release names (linux-x86_64-glibc_2_7-nocrypto, linux-arm64-glibc_2_17-nocrypto, linux-x86_64-musl_1_2-nocrypto, linux-arm64-musl_1_2-nocrypto, linux-ppc64le-glibc_2_17-nocrypto, linux-s390x-glibc_2_7-nocrypto, macos-universal, windows-x86_64) rather than the distro names we map to today (debian11/12/13, rhel-70-64-bit, rhel-80-64-bit, rhel-82-arm64, windows-test).

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (4)

.evergreen/scripts/setup_tests.py:362

• Evergreen exposes the selected server as VERSION (this script already reads it at line 328); MONGODB_VERSION is only assigned inside the separate run_server.py process. Consequently this is false on the 8.x tasks, so they continue installing master instead of the released pre-1.19 binding and do not exercise the preview combination described here. Fall back to VERSION when selecting the dependency.

        use_released_pymongocrypt = os.environ.get("MONGODB_VERSION", "").startswith("8.")

pymongo/asynchronous/encryption.py:1035

• This inserts the new parameter into the positional slot formerly occupied by text_opts. Existing positional callers will therefore bind their old argument to string_opts and never receive the promised deprecation warning. Preserve the old slot by keeping text_opts before the newly appended parameter.

        string_opts: Optional[StringOpts] = None,
        text_opts: Optional[StringOpts] = None,

test/asynchronous/test_encryption.py:3435

• On MongoDB 9.0 with libmongocrypt 1.18.x (allowed by the class decorator), this selects TextPreview and setup immediately encrypts fixtures for GA collections before any per-test gate runs. The cases error during setup rather than skip. Skip the GA class below the 1.19.0 floor before selecting the algorithm.

        self.algorithm = (
            Algorithm.STRING if _libmongocrypt_at_least(1, 19, 0) else Algorithm.TEXTPREVIEW
        )

test/asynchronous/test_encryption.py:3483

• Setup always performs GA substring encryption, although the advertised 1.19.x configuration supports only prefix and suffix. It will fail here before _require_query_type("substring") can skip those cases, preventing the valid prefix/suffix tests from running. Only build this fixture for pre-9.0 preview tests or libmongocrypt 1.20+.

            string_opts=StringOpts(
                case_sensitive=True,
                diacritic_sensitive=True,
                substring=dict(strMaxLength=10, strMaxQueryLength=6, strMinQueryLength=2),

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (3)

pymongo/asynchronous/encryption.py:1035

• text_opts previously occupied this positional slot. Inserting string_opts before it makes existing positional calls bind their legacy value to string_opts, silently bypassing the deprecation warning. Keep text_opts in its original position and append the new parameter so positional compatibility is preserved.

        string_opts: Optional[StringOpts] = None,
        text_opts: Optional[StringOpts] = None,

.evergreen/scripts/setup_tests.py:392

• On every 8.x task this branch now bypasses setup_libmongocrypt() and never sets PYMONGOCRYPT_LIB, even when the documented LIBMONGOCRYPT_URL override is supplied. That makes custom libmongocrypt validation jobs silently test the wheel's bundled library instead of the requested build. Only select the released wheel when no URL override is present.

        use_released_pymongocrypt = os.environ.get("VERSION", "").startswith("8.")

pymongo/encryption_options.py:414

• Replacing the explicit public constructor with *args: Any, **kwargs: Any removes the argument names from introspection and causes type checkers to accept invalid TextOpts arguments throughout the deprecation period. Mirror StringOpts' constructor signature while emitting the warning to retain the existing API and static validation.

    def __init__(self, *args: Any, **kwargs: Any) -> None:

[blink1073]

@aclark4life, note, the new pymongocrypt only accepts string_opts, so we'll have to call it with the appropriate name based on the the pymongocrypt version.