Skip to content

gh-204: Support proxy environment variables - #387

Open
Punisheroot wants to merge 2 commits into
python:mainfrom
Punisheroot:feat/js/support-proxy-env
Open

gh-204: Support proxy environment variables#387
Punisheroot wants to merge 2 commits into
python:mainfrom
Punisheroot:feat/js/support-proxy-env

Conversation

@Punisheroot

@Punisheroot Punisheroot commented Jul 28, 2026

Copy link
Copy Markdown

Summary

  • Honor NO_PROXY, HTTP_PROXY, and HTTPS_PROXY in the BITS, WinHTTP, and PowerShell download paths.
  • Preserve the existing automatic proxy discovery behavior when none of these variables are configured.
  • Centralize proxy URL normalization and credential extraction so every backend receives the same settings.

Context

Python Manager currently uses the user's Windows proxy configuration, but it does not consistently allow that configuration to be overridden with the commonly used proxy environment variables. This is particularly limiting in environments where applications use an explicit proxy without enabling a system-wide Windows proxy.

This implementation follows the backend-specific behavior described in the maintainer's design summary.

Closes #204.

Changes

Proxy resolution

Proxy settings are resolved once when a request is created:

  • a non-empty NO_PROXY selects a direct connection;
  • HTTP_PROXY and HTTPS_PROXY produce a protocol-specific proxy list;
  • credentials embedded in proxy URLs are decoded, removed from the endpoint, and passed separately to the download backend;
  • credentials from HTTPS_PROXY take precedence, followed by credentials from HTTP_PROXY;
  • when no proxy environment variable is configured, the existing automatic detection path remains unchanged.

Proxy values without a scheme are treated as HTTP proxy URLs. Paths and query components are removed before an endpoint is passed to the native APIs.

BITS

BITS now selects BG_JOB_PROXY_USAGE_NO_PROXY for direct mode and BG_JOB_PROXY_USAGE_OVERRIDE for explicit proxy configuration. The generated proxy list follows the format documented for IBackgroundCopyJob::SetProxySettings. Explicit Basic credentials are supplied when present; otherwise, the existing Negotiate behavior is retained.

WinHTTP

WinHTTP applies a named proxy with WINHTTP_OPTION_PROXY when an override is configured. For direct mode, it explicitly applies WINHTTP_ACCESS_TYPE_NO_PROXY to the request. This is intentional because the session is opened with the default Windows proxy configuration, so merely skipping proxy setup would not reliably disable an inherited static proxy. The access modes and request-level proxy option are documented in WINHTTP_PROXY_INFO and the WinHTTP option flags.

PowerShell

The PowerShell fallback receives normalized proxy settings through process-local environment variables. It uses -ProxyCredential for explicit credentials and -ProxyUseDefaultCredentials otherwise; these options are kept mutually exclusive as required by Invoke-WebRequest.

For direct mode, the child process installs an empty WebProxy. A parameterless WebProxy has no address and bypasses every destination, allowing NO_PROXY to suppress an inherited Windows proxy without changing system settings.

Tests

The local HTTP test server can now act as a simple proxy and issue a Basic 407 Proxy Authentication Required challenge. Integration coverage verifies:

  • environment-variable precedence and URL normalization;
  • credential decoding and HTTPS-to-HTTP credential fallback;
  • explicit proxy routing through BITS, WinHTTP, and PowerShell;
  • Basic proxy authentication through all three backends;
  • direct BITS, WinHTTP, and PowerShell downloads when NO_PROXY is set and HTTP_PROXY is intentionally unreachable.

Validation

Performed on Windows with Python 3.14.6:

  • python -m pymsbuild -c _msbuild_test.py: PASS
  • python -m pytest tests/test_urlutils.py -k proxy -q: PASS (13 passed)
  • Authenticated HTTPS CONNECT smoke test through a separate local proxy process to https://www.python.org/: PASS for BITS, WinHTTP, and PowerShell
  • python -m pytest -q: 399 passed, with one environment-specific failure in tests/test_shellext.py::test_RegReadStr. The test assumes HKCU\Environment\PATH is REG_EXPAND_SZ; it is REG_SZ on the test machine. The failing shell-extension path is unchanged by this pull request.
  • git diff --check: PASS

Compatibility and risk

The native entry points accept the proxy settings as an optional final argument, preserving existing callers. The automatic proxy paths are unchanged when the environment variables are absent.

As specified in the issue's accepted design, any non-empty NO_PROXY value selects a fully direct connection; this change does not implement per-host bypass-list matching. The committed integration proxy keeps the automated suite self-contained and covers HTTP routing and authentication. A separate end-to-end smoke test verified an authenticated 407 Basic challenge, a real CONNECT tunnel, TLS negotiation, and data transfer through BITS, WinHTTP, and PowerShell. This smoke test remains manual because it uses an external HTTPS endpoint.

@python-cla-bot

python-cla-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

@Punisheroot
Punisheroot marked this pull request as ready for review July 28, 2026 23:37

@zooba zooba left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic PR. Only one minor comment, but the more critical piece is that both the _msbuild.py definitions are going to require the new .cpp file or else it won't compile.

Once that change is added, I'll trigger CI.

Comment thread src/manage/urlutils.py Outdated


class _ProxySettings:
__slots__ = ("mode", "proxy_list", "powershell_proxy", "username", "password")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Drop this - we're never going to have enough instances to care.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 383c301

@Punisheroot

Copy link
Copy Markdown
Author

Thanks for the review @zooba ! I’ll drop __slots__ for sure.
I double-checked the build definitions before updating them. Both _msbuild.py and _msbuild_test.py already include CSourceFile('*.cpp') with source='src/_native', and pymsbuild expands that wildcard during project generation. I confirmed that both generated _native.proj and _native_test.proj contain proxy.cpp; both sdists include proxy.cpp/proxy.h, and a clean test build compiles and links successfully.
Did you want an explicit CSourceFile('proxy.cpp') entry for clarity, or were you referring to another build definition?

@Punisheroot
Punisheroot requested a review from zooba July 29, 2026 15:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support HTTP_PROXY env

2 participants