Skip to content

Add a hook for handling logical decoding messages on subscribers. - #415

Open
pg-hub-mirror[bot] wants to merge 1 commit into
masterfrom
pg-hub/mirror-patch-3376da510c84451b
Open

pg-hub-mirror[bot] wants to merge 1 commit into
masterfrom
pg-hub/mirror-patch-3376da510c84451b

Conversation

@pg-hub-mirror

@pg-hub-mirror pg-hub-mirror Bot commented Sep 18, 2026

Copy link
Copy Markdown

Read-only mirror. Reply and review on pgsql-hackers; activity here is not sent upstream.

  • Original author: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
  • Mailing list: pgsql-hackers
  • Message-ID: CAD21AoDrvttNr0sOywJ7+1C1pivOgj8L3QuMqpziPDRopw+UDQ@mail.gmail.com
  • Original email

Patch files:


On Thu, Aug 6, 2026 at 10:42 AM Bharath Rupireddy
<bharath(dot)rupireddyforpostgres(at)gmail(dot)com> wrote:

Hi,

On Thu, Aug 6, 2026 at 9:59 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:

2/

  • /*
    • The message doesn't belong to any remote transaction, so there is
    • no remote commit LSN nor timestamp to record. Clear the state left
    • over by the previously applied transaction so that this commit
    • doesn't inherit it.
  • */
  • replorigin_xact_clear(false);

Why is this a problem if we let the non-transactional message inherit it?

IIUC non-transactional messages would have the same commit timestamp
as the previously applied transaction, which is wrong to me.

Having replorigin_xact_clear there looks fine to me. The next
transaction commit would anyway set the origin LSN and timestamp.

Wrapping the hook with begin and end replication step is nice. This
lets the hook see the correct command ID, snapshot, and memory
context. There are callers that do the begin first and read message
next (insert), but it seems okay this way because read message doesn't
do any catalog or table accesses, so it should be fine.

begin_replication_step() switches the memory context to
ApplyMessageContext. Given logicalrep_read_message() palloc's for
messages, it should be called after begin_replication_step(). Fixed it.

Right. I verified other places and wherever the read does a palloc, it
is wrapped within begin and end replication step.

I've attached the updated patch.

Thanks. The v4 patch looks good to me. pgindent and tests are happy. I
have no further comments. I marked the CF entry RfC
(https://commitfest.postgresql.org/patch/7092/) FWIW, the CF bot
complains with "needs rebase":
https://cfbot.cputube.org/patch_7092.log.
Thanks for reviewing the patch!
While I also think the patch is in good shape, I'd like to raise a
security risk this feature might introduce, particularly around
pg_logical_emit_message():
EXECUTE on that function is granted to PUBLIC, so any role that can
connect to the publisher database can emit a logical decoding message.
With this patch the apply worker hands the message to a handler that
runs with the privileges of the subscription owner, and unlike
insert/update/delete there is no table owner to switch to. One use
case I have in mind is DDL replication. If an extension implements it
on top of this hook and the subscription is owned by a superuser, any
role on the publisher can choose what the handler is given and have it
executed with superuser privileges on the subscriber. So extensions
should carefully consider this case. I think the same applies to other
extensions that might use this feature. The real problem is that there
is no reliable way for the subscriber to tell whether a message came
from the source it expects.
A practical solution is to revoke EXECUTE on pg_logical_emit_message()
from PUBLIC on the publisher and grant it to a role created for that
purpose. I think that covers most cases I did consider having the
server record the emitting role in the message so that the subscriber
could check it, but I'm not sure this feature alone justifies it. So
my current thought is to document these risks and add nothing special
for these cases.
I've added the documentation changes and rebased the patch. Any ideas
and feedback is very welcome.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

Previously, pgoutput could emit logical decoding messages, those
written with pg_logical_emit_message(), but the built-in subscriber
never requested them from the publisher, and apply_handle_message()
was only a placeholder that discarded the MESSAGE message type. Acting
on such messages therefore meant writing a separate logical decoding
client, and giving up the apply worker's streaming, conflict handling
and progress tracking. Logical messages are a general-purpose side
channel carried in band with the data, useful for replicating deparsed
DDL, for carrying change-data-capture context that the row data does
not have, and for cache invalidation delivered in commit order
relative to the rows it refers to.

This commit adds a new "messages" subscription option, which asks the
publisher for the pgoutput "messages" option, and dispatches each
received message to a new LogicalRepMessageHandle_hook. Behavior is
unchanged when the option is off or when no handler is installed.

A transactional message is applied as a step of the remote transaction
it belongs to, so the handler's work commits atomically with that
transaction, and transaction streaming and ALTER SUBSCRIPTION ... SKIP
apply to it as they do to any other change. A non-transactional
message belongs to no remote transaction and is committed on its own.

A message may still reach the handler more than once, for example
after an apply worker restart, so handlers must be idempotent. Note
also that messages are not scoped by publication: every message
emitted in the publisher database is sent, subject only to origin
filtering, and pg_logical_emit_message() requires no special
privileges. A handler must therefore treat the payload as untrusted
input, and as an arbitrary string of bytes rather than as text in the
subscriber's encoding. The publisher must be running PostgreSQL 14 or
later, where pgoutput gained the "messages" option; against an older
publisher the option is accepted but no message is ever sent.

Bump catalog version.

Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/CAD21AoCTNGiddikkUcDKj5QLnsg-51bpr-o6L-GTHWZL4ZFYtQ@mail.gmail.com
@pg-hub-mirror pg-hub-mirror Bot added source:pgsql-hackers Mirrored from pgsql-hackers type:patch Mail thread contains a PostgreSQL patch area:executor Executor area:testing Tests and buildfarm area:security Authentication, authorization, or security area:sql SQL language or commands area:replication Physical or logical replication area:docs Documentation labels Sep 18, 2026
@pg-hub-mirror pg-hub-mirror Bot locked and limited conversation to collaborators Sep 18, 2026
@pg-hub-mirror pg-hub-mirror Bot added cf:pg20-2 PostgreSQL CommitFest status:ready-for-committer CommitFest: Ready for Committer labels Sep 18, 2026
@pg-hub-mirror pg-hub-mirror Bot added this to the PG20-2 milestone Sep 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area:docs Documentation area:executor Executor area:replication Physical or logical replication area:security Authentication, authorization, or security area:sql SQL language or commands area:testing Tests and buildfarm cf:pg20-2 PostgreSQL CommitFest source:pgsql-hackers Mirrored from pgsql-hackers status:ready-for-committer CommitFest: Ready for Committer type:patch Mail thread contains a PostgreSQL patch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant