Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 49 additions & 32 deletions docs/fork-safety.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,51 @@
# Fork safety

## Why inherited clients are rejected

wreq-ruby uses a process-wide Tokio runtime and connection pool. `fork` copies
the parent's memory, but only the thread that called `fork` continues in the
child. Tokio's worker threads are gone, and its inherited tasks, locks, and
connections are not safe to reuse.

If the parent has already loaded wreq-ruby, native HTTP operations in the child
raise `Wreq::ForkError`. This applies to new and existing clients, module
request methods, streaming request bodies, and response methods backed by native
state. Retrying the operation in the same child raises the same error. Read-only
response metadata such as status, headers, and captured TLS information remains
available.

The parent can continue using its clients. When inherited Ruby objects are
collected in the child, their native runtime state is left for the operating
system to reclaim when the process exits.

## HTTP work in forked children is unsupported

A process created with `fork` must not start or continue HTTP work through
wreq-ruby, even when it first loads the extension after the fork. If the parent
loaded wreq-ruby, native HTTP operations in the child raise `Wreq::ForkError`.

When the extension was not present in the parent, no wreq-ruby state or fork
marker reaches the child. The extension cannot reliably distinguish that child
from a newly started process, so this unsupported path cannot guarantee a Ruby
error and may fail inside platform libraries.

Prefork servers should use an `exec`- or spawn-based worker model when workers
need wreq-ruby. Requiring the extension again does not reset inherited runtime
state, and there is no `after_fork!` hook.
## Prefork checklist

- `require "wreq"` may run in the parent before workers fork.
- Create each `Wreq::Client`, `Wreq::Jar`, and `Wreq::BodySender` in the worker
that will use it.
- Keep each `Wreq::Response` in the process that received it.
- Do not start requests or push streaming body data in the parent before workers
fork.
- If the parent must use wreq-ruby first, start workers with `spawn` or `exec`
instead of `fork`.

wreq-ruby checks process ownership whenever it exposes guarded native state. It
does not copy, reset, or rebuild inherited objects.

## Loading before fork

wreq-ruby creates its process-wide Tokio runtime on the first operation that
needs it. Requiring the gem does not initialize the runtime, so a prefork server
may load wreq-ruby during boot. Each worker can then create its own runtime on
its first request without an `after_fork!` hook.

Create clients and other native-backed objects inside the worker. Each client,
response, body sender, and cookie jar belongs to the process that created it.
Using an inherited object raises `Wreq::ForkError`, even when the parent never
started the runtime. wreq-ruby does not rebuild these objects.

## Forking after runtime initialization

Once the parent starts an HTTP operation or otherwise uses the Tokio runtime, a
forked child must not reuse it. Tokio's worker threads do not survive `fork`, and
the inherited connection pool may refer to those missing threads.

Operations that need the inherited runtime raise `Wreq::ForkError`. This
includes requests through new or existing clients, module request methods, and
streaming request writes. Constructing a new client, body sender, or cookie jar
does not use the runtime, but runtime-backed operations remain unavailable in
that child. Retrying them raises the same error.

An inherited `Wreq::Response` cannot be used at all. This includes status,
headers, socket addresses, TLS information, and body methods. Values copied out
before the fork, such as a `Wreq::StatusCode` or `Wreq::TlsInfo`, are separate
objects and do not retain access to the response.

The parent remains usable. Native objects collected in the child do not destroy
state owned by the parent process.

Use a spawn- or exec-based worker when the parent must perform HTTP work before
workers start. Requiring the extension again cannot replace an inherited
runtime.
7 changes: 4 additions & 3 deletions docs/interrupt-handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,10 @@ to the Ruby-owned thread with the GVL may [`src/rt.rs`](../src/rt.rs) map a
wreq-owned cancellation to the `Wreq::InterruptError` defined in
[`src/error.rs`](../src/error.rs).

Keep this conversion centralized in `rt::try_block_on`. Request, response, and
body operations may call `try_block_on`, but they must not construct their own
Ruby cancellation exception.
Keep cancellation conversion centralized in `rt::block_on`. Request, response,
and body operations may call `block_on`, but they must not construct their own
Ruby cancellation exception. `block_on` returns a future's native error
unchanged so the caller can convert it after the GVL has been reacquired.

These forms are forbidden for wreq-owned cancellation:

Expand Down
48 changes: 36 additions & 12 deletions lib/wreq.rb
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,33 @@
require_relative "wreq_ruby/cookie"

unless defined?(Wreq)
# An HTTP client backed by a lazily initialized, process-wide Tokio runtime.
#
# Loading wreq-ruby before `fork` is supported. The parent must not send a
# request or perform another operation that starts the runtime before workers
# are forked. Create clients and begin HTTP work inside each worker so it gets
# its own runtime and connection pool. Clients, responses, body senders, and
# cookie jars belong to the process that created them and must be recreated
# in the worker. wreq-ruby does not rebuild inherited objects.
#
# Accessing an inherited native-backed object raises Wreq::ForkError even if
# the parent did not start the runtime. If the parent did start it, the child
# also cannot perform new runtime-backed operations. Retrying does not replace
# either kind of inherited state. Use `spawn` or `exec`, or move the parent's
# HTTP work until after the workers have been forked.
#
# @example Preload the extension, then start HTTP work in the worker
# require "wreq"
#
# Process.fork do
# client = Wreq::Client.new
# response = client.get("https://example.com")
# puts response.status
# end
#
# @note Fork safety Create clients, cookie jars, and body senders inside the
# worker that uses them. Do not carry responses across `fork`.
# @see https://github.com/SearchApi/wreq-ruby/blob/main/docs/fork-safety.md
module Wreq
# Current wreq gem version.
# @return [String]
Expand All @@ -29,9 +56,6 @@ module Wreq
# raise ArgumentError. Known values retain the error class from their Ruby
# or native conversion, such as TypeError or Wreq::BuilderError. Validation
# finishes before network I/O.
#
# If a child process inherits wreq-ruby from its parent, requests raise
# Wreq::ForkError. Require wreq after the worker has been forked.

# Send an HTTP request.
#
Expand Down Expand Up @@ -64,7 +88,7 @@ module Wreq
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the child inherited an initialized wreq-ruby runtime
def self.request(method, url, **options)
end

Expand Down Expand Up @@ -98,7 +122,7 @@ def self.request(method, url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the child inherited an initialized wreq-ruby runtime
def self.get(url, **options)
end

Expand Down Expand Up @@ -132,7 +156,7 @@ def self.get(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the child inherited an initialized wreq-ruby runtime
def self.head(url, **options)
end

Expand Down Expand Up @@ -166,7 +190,7 @@ def self.head(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the child inherited an initialized wreq-ruby runtime
def self.post(url, **options)
end

Expand Down Expand Up @@ -200,7 +224,7 @@ def self.post(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the child inherited an initialized wreq-ruby runtime
def self.put(url, **options)
end

Expand Down Expand Up @@ -234,7 +258,7 @@ def self.put(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the child inherited an initialized wreq-ruby runtime
def self.delete(url, **options)
end

Expand Down Expand Up @@ -268,7 +292,7 @@ def self.delete(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the child inherited an initialized wreq-ruby runtime
def self.options(url, **options)
end

Expand Down Expand Up @@ -302,7 +326,7 @@ def self.options(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the child inherited an initialized wreq-ruby runtime
def self.trace(url, **options)
end

Expand Down Expand Up @@ -336,7 +360,7 @@ def self.trace(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the child inherited an initialized wreq-ruby runtime
def self.patch(url, **options)
end
end
Expand Down
15 changes: 9 additions & 6 deletions lib/wreq_ruby/body.rb
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,12 @@ module Wreq
#
# A sender can be attached to one request. Closing it prevents further writes but
# retains queued chunks so a request attached afterward can still drain them.
# Creating or using a sender raises Wreq::ForkError if the child inherited
# wreq-ruby from its parent.
# Creating a sender does not initialize Tokio. An inherited sender raises
# Wreq::ForkError before its channel is accessed. A new sender can be
# created in a child, but pushing data also requires a usable runtime.
#
# @note Fork safety Create each sender in the worker that writes to it.
# Do not pass a sender through `fork`.
class BodySender
# Create a bounded request-body sender.
#
Expand All @@ -27,7 +31,6 @@ class BodySender
# @return [Wreq::BodySender] A streaming request body sender
# @raise [ArgumentError] if capacity is zero, negative, or too large
# @raise [TypeError] if capacity is not an Integer
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def self.new(capacity = 8)
end

Expand All @@ -36,7 +39,7 @@ def self.new(capacity = 8)
# @param data [String] binary chunk
# @return [nil]
# @raise [IOError] if the sender or receiving side is closed
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the sender or runtime belongs to the parent process
def push(data)
end

Expand All @@ -45,7 +48,7 @@ def push(data)
# This operation is idempotent.
#
# @return [nil]
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the sender belongs to the parent process
def close
end

Expand All @@ -55,7 +58,7 @@ def close
# the receiving side.
#
# @return [Boolean]
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the sender belongs to the parent process
def closed?
end
end
Expand Down
32 changes: 18 additions & 14 deletions lib/wreq_ruby/client.rb
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,14 @@ module Wreq
# native conversion, such as TypeError or Wreq::BuilderError. Request
# validation finishes before network I/O.
#
# A child process cannot create or use a client if it inherited wreq-ruby
# from its parent. These calls raise Wreq::ForkError before accessing the
# native runtime. Require wreq after the worker has been forked.
# A client belongs to the process that created it. An inherited client
# raises Wreq::ForkError before its connection pool is accessed. Loading
# the gem before fork is supported, but clients must be created inside the
# worker. If the parent already started the runtime, new clients can be
# constructed in the child but cannot send requests.
#
# @note Fork safety Create each client in the worker that uses it. An
# inherited client is never rebuilt automatically.
#
# @example Basic usage
# client = Wreq::Client.new
Expand Down Expand Up @@ -174,8 +179,7 @@ class Client
# value cannot be converted or validated.
# @raise [Wreq::BuilderError, Wreq::TlsError] if the native client cannot
# be initialized.
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
#
# @raise [Wreq::ForkError] if :cookie_provider belongs to a parent process.
# @example Minimal client
# client = Wreq::Client.new
#
Expand Down Expand Up @@ -290,7 +294,7 @@ def self.new(**options)
# or unavailable on the current platform
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the client or runtime belongs to the parent process
def request(method, url, **options)
end

Expand Down Expand Up @@ -324,7 +328,7 @@ def request(method, url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the client or runtime belongs to the parent process
def get(url, **options)
end

Expand Down Expand Up @@ -358,7 +362,7 @@ def get(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the client or runtime belongs to the parent process
def head(url, **options)
end

Expand Down Expand Up @@ -392,7 +396,7 @@ def head(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the client or runtime belongs to the parent process
def post(url, **options)
end

Expand Down Expand Up @@ -426,7 +430,7 @@ def post(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the client or runtime belongs to the parent process
def put(url, **options)
end

Expand Down Expand Up @@ -460,7 +464,7 @@ def put(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the client or runtime belongs to the parent process
def delete(url, **options)
end

Expand Down Expand Up @@ -494,7 +498,7 @@ def delete(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the client or runtime belongs to the parent process
def options(url, **options)
end

Expand Down Expand Up @@ -528,7 +532,7 @@ def options(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the client or runtime belongs to the parent process
def trace(url, **options)
end

Expand Down Expand Up @@ -562,7 +566,7 @@ def trace(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @raise [Wreq::ForkError] if the client or runtime belongs to the parent process
def patch(url, **options)
end
end
Expand Down
Loading