Skip to content

fix(prompt): preserve langchain placeholder messages - #1840

Open
arpansahu wants to merge 1 commit into
langfuse:mainfrom
arpansahu:fix/prompt-langchain-placeholder-messages
Open

fix(prompt): preserve langchain placeholder messages#1840
arpansahu wants to merge 1 commit into
langfuse:mainfrom
arpansahu:fix/prompt-langchain-placeholder-messages

Conversation

@arpansahu

@arpansahu arpansahu commented Aug 27, 2026

Copy link
Copy Markdown

Problem

ChatPromptClient.compile() supports chat placeholders and already preserves dict messages, but it treats LangChain BaseMessage instances passed to a placeholder as invalid objects. Passing a natural LangChain history like [HumanMessage(...), AIMessage(...)] therefore emits a warning and appends the stringified entire history with role NOT_GIVEN, losing message roles and content structure.

Reproducer

Added test_langchain_messages_in_message_placeholder_are_preserved, which compiles a chat prompt placeholder with HumanMessage(content="Hello {{name}}") and AIMessage(content="Hi {{name}}").

Before the fix, the regression failed because the compiled output contained role="NOT_GIVEN" and a stringified placeholder list instead of separate user/assistant messages.

Fix

Add placeholder-message compilation for LangChain-style message objects by reading their content, mapping LangChain message type values such as human and ai to chat roles, compiling string content variables, and preserving optional tool-related fields.

Testing

  • Before fix: python -m pytest tests\unit\test_prompt_compilation.py::test_langchain_messages_in_message_placeholder_are_preserved -q failed with the placeholder warning and incorrect NOT_GIVEN/stringified output.
  • After fix: .\.venv\Scripts\python.exe -m pytest tests\unit\test_prompt_compilation.py::test_langchain_messages_in_message_placeholder_are_preserved -q passed.
  • After fix: .\.venv\Scripts\python.exe -m pytest tests\unit\test_prompt_compilation.py -q passed (33 passed).
  • After fix: .\.venv\Scripts\python.exe -m ruff format langfuse\model.py tests\unit\test_prompt_compilation.py --check passed.
  • After fix: .\.venv\Scripts\python.exe -m ruff check langfuse\model.py tests\unit\test_prompt_compilation.py passed.

Full tests\unit has pre-existing Windows/env failures unrelated to this change, previously observed in serializer path assertions, prompt atexit subprocess environment, and prompt mock setup.

Greptile Summary

This PR adds support for expanding LangChain message objects supplied to chat placeholders and tests variable interpolation for human and AI messages.

  • Maps LangChain message types to chat roles.
  • Compiles string content and carries selected tool-related attributes into the output.
  • Adds regression coverage for HumanMessage and AIMessage placeholder history.

Confidence Score: 4/5

The PR should not merge until LangChain message metadata outside the current allowlist is preserved during placeholder compilation.

The new object-conversion path can silently remove provider-specific function or tool context before compiled messages are sent to a model.

Files Needing Attention: langfuse/model.py

Prompt To Fix All With AI
### Issue 1
langfuse/model.py:427-430
**Message metadata is dropped**

When a LangChain message stores provider or subtype data outside `name`, `tool_call_id`, `tool_calls`, and `invalid_tool_calls`, this allowlist removes that data from the compiled history, causing downstream model calls to receive incomplete function or tool context.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "fix(prompt): preserve langchain placehol..." | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

Context used:

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: arpan sahu <28574248+arpansahu@users.noreply.github.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Comment thread langfuse/model.py
Comment on lines +427 to +430

for key in ("name", "tool_call_id", "tool_calls", "invalid_tool_calls"):
value = getattr(msg, key, None)
if value:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Message metadata is dropped

When a LangChain message stores provider or subtype data outside name, tool_call_id, tool_calls, and invalid_tool_calls, this allowlist removes that data from the compiled history, causing downstream model calls to receive incomplete function or tool context.

Knowledge Base Used: Prompt retrieval, compilation, and caching

Prompt To Fix With AI
This is a comment left during a code review.
Path: langfuse/model.py
Line: 427-430

Comment:
**Message metadata is dropped**

When a LangChain message stores provider or subtype data outside `name`, `tool_call_id`, `tool_calls`, and `invalid_tool_calls`, this allowlist removes that data from the compiled history, causing downstream model calls to receive incomplete function or tool context.

**Knowledge Base Used:** [Prompt retrieval, compilation, and caching](https://app.greptile.com/personal-org-4986/-/custom-context/knowledge-base/langfuse/langfuse-python/-/docs/prompt-retrieval-compilation-and-cache.md)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

@arpansahu

Copy link
Copy Markdown
Author

Disclosure: this contribution was prepared with AI assistance (GitHub Copilot CLI), and
I am flagging that explicitly rather than leaving you to guess.

The defect, the fix and the regression test were verified locally: the new test fails on
unmodified upstream and passes with the change applied, and the surrounding suite is green.

If your project would prefer not to take AI-assisted contributions, or you would rather this
were reworked and resubmitted by hand, please just close it -- no objection at all, and
apologies for the noise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants