Skip to content

Raise the urllib3 pool maxsize to 10 and make it configurable - #1026

Open
jakobovski-arb wants to merge 1 commit into
massive-com:masterfrom
jakobovski-arb:make-pool-maxsize-configurable
Open

Raise the urllib3 pool maxsize to 10 and make it configurable#1026
jakobovski-arb wants to merge 1 commit into
massive-com:masterfrom
jakobovski-arb:make-pool-maxsize-configurable

Conversation

@jakobovski-arb

@jakobovski-arb jakobovski-arb commented Jul 31, 2026

Copy link
Copy Markdown

Problem

BaseClient builds its PoolManager with num_pools but never maxsize, so urllib3's default of one connection per host applies:

self.client = urllib3.PoolManager(
    num_pools=num_pools,
    headers=self.headers,
    ...
)

num_pools caps the number of distinct host pools, which doesn't help a client that talks to a single host from many threads. The two are easy to conflate — passing a large num_pools looks like it should raise concurrency, but it changes nothing for a one-host client.

Under threads the result is a TLS handshake storm: threads that lose the race for the single pooled connection open their own, then can't return it (Connection pool is full, discarding connection), so it's closed and the next request renegotiates TLS.

Measurements

Against the trades endpoint, 240 calls, same machine:

CPU per call pool is full warnings
single-threaded 0.0074 s 0
20 threads 0.1309 s (18x) 70

And 40 threads were slower than 20 — the pool was the bottleneck, not the network. With maxsize raised: warnings gone, CPU per call down 2.9x, and throughput scales with threads again (25.7 → 66.3 req/sec in that benchmark).

Change

Adds a maxsize parameter to RESTClient, threaded through to the PoolManager, defaulting to 10.

The new default intentionally changes behaviour for every user, including those who never touch the parameter — that is the point of the change. maxsize=1 isn't a sensible ceiling for a client whose own constructor advertises pooling, and a user hitting it has no way to discover why their threads are slow: the symptom looks like a slow API, not a client setting. Fixing it only for people who go looking for a new argument would leave the common case broken.

10 matches both the existing num_pools default and requests' HTTPAdapter default, so it's a conventional value rather than a tuned one.

maxsize is a cap, not a preallocation — a single-threaded client still opens exactly one connection and sees no difference. The change is felt only by concurrent callers, who are the ones currently paying for it.

On BaseClient it's an optional keyword rather than a required one, so existing subclasses calling super().__init__(...) keep working.

Five lines of production change; tests cover the default, an explicit value, and that it's independent of num_pools.

Checks

  • pytest test_rest test_websocket — 44 passed, no new failures. (test_clint_headers_concat already fails on a clean master for me, unrelated to this change.)
  • black 24.8.0 --check clean on all three touched files.
  • mypy clean on both changed modules.

@jakobovski-arb
jakobovski-arb force-pushed the make-pool-maxsize-configurable branch 2 times, most recently from 431196b to c19d1e2 Compare July 31, 2026 22:51
BaseClient builds its PoolManager with num_pools but never maxsize, so urllib3's
default of ONE connection per host applies. num_pools caps the number of distinct
HOST pools, which does not help a client that talks to a single host from many
threads -- the two are often conflated, and passing a large num_pools looks like
it should raise concurrency but does nothing.

The effect under threads is a TLS handshake storm. Threads that lose the race for
the single pooled connection open their own, cannot return it ("Connection pool is
full, discarding connection"), so it is closed and the next request renegotiates
TLS. Measured against the trades endpoint over 240 calls:

    single-threaded    0.0074 CPU-s per call,   0 pool-full warnings
    20 threads         0.1309 CPU-s per call,  70 pool-full warnings  (18x CPU)

and 40 threads were SLOWER than 20. With maxsize raised the warnings disappear,
CPU per call drops 2.9x, and throughput scales with threads again (25.7 -> 66.3
requests/sec in that benchmark).

Adds a `maxsize` parameter to RESTClient, threaded through to the PoolManager,
defaulting to 10.

The new default intentionally changes behaviour for every user, including those
who never touch the parameter -- that is the point of the change. maxsize=1 is not
a sensible ceiling for a client whose own constructor advertises pooling, and a
user hitting it has no way to discover why their threads are slow: the symptom
looks like a slow API, not a client setting. Fixing it only for people who find
the new argument would leave the common case broken.

10 matches both the existing num_pools default and requests' HTTPAdapter default,
so it is a conventional value rather than a tuned one. maxsize is a cap and not a
preallocation, so a single-threaded client still opens exactly one connection and
sees no difference; the change is felt only by concurrent callers, who are the
ones currently paying for it.

On BaseClient it is an optional keyword rather than a required one, so existing
subclasses that call super().__init__ keep working.

Tests cover the default, an explicit value, and that it is independent of
num_pools.
@jakobovski-arb
jakobovski-arb force-pushed the make-pool-maxsize-configurable branch from c19d1e2 to dd4d88d Compare July 31, 2026 22:51
@jakobovski-arb jakobovski-arb changed the title Make urllib3 pool maxsize configurable (default 10) Raise the urllib3 pool maxsize to 10 and make it configurable Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants