Skip to content

docs: require an Assisted-by: commit trailer for AI-assisted work - #6760

Open
vlsi wants to merge 1 commit into
apache:masterfrom
vlsi:docs/assisted-by-commit-trailer
Open

docs: require an Assisted-by: commit trailer for AI-assisted work#6760
vlsi wants to merge 1 commit into
apache:masterfrom
vlsi:docs/assisted-by-commit-trailer

Conversation

@vlsi

@vlsi vlsi commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

JMeter has no rule for disclosing AI tooling in a commit, and the gap shows in master. Sixteen commits since March 2026 name a tool in a trailer, in two capitalizations of the key and four spellings of the model:

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>                (8 commits)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>                (3)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>   (2)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>   (2)
Generated-by: Claude Opus 4.8                                          (1)

Why

Fifteen of those use Co-authored-by:, and in this same history that trailer also carries nine human names. One field is doing two different jobs, and nothing in the message distinguishes them: git and GitHub read every name there as an author of the commit, so a tool listed there makes a claim about authorship rather than a note about tooling.

The line arrives without anyone choosing it. Claude Code adds Co-Authored-By: unless attribution.commit says otherwise, and VS Code 1.118 shipped Copilot as a commit co-author by default before reversing it. That is also why the spelling drifts: the model name comes from whichever build wrote the commit.

The ASF generative tooling guidance asks for a token in the commit message and names Generated-by:. The wider ecosystem settled on Assisted-by: for the ordinary case: the Linux kernel, Zephyr, Fedora, Rocky Linux, OpenInfra, and Mesa. Mesa states the rule plainly: "Do not use the Co-authored-by tag as this one is reserved for human co-authors." attrs closes any pull request that lists an LLM as a co-author, without further discussion. The kernel's first RFC proposed Co-developed-by and changed to Assisted-by before merging; inside the ASF, Apache Impala's dev@ thread opened on Generated-by and settled on Assisted-by, on the grounds that a tool assists and does not author.

The alternative, and why not it

Not every project takes the trailer route. Kubernetes forbids AI trailers outright — Co-authored-by:, Assisted-by: and Co-developed-by: alike — and asks for a sentence in the pull request description instead, on the grounds that an AI cannot sign a CLA.

That objection lands on a claim Assisted-by: does not make. The trailer records which tool was used; the CLA is signed by the human in the author field, who is the same person either way. What differs is durability. A pull request description lives on GitHub, while a trailer lives in the commit, survives a mirror or an export, and is what git log --grep finds years later. Of 1000 popular GitHub repositories surveyed in 2026, 118 had published an AI policy; 78% of those allow AI assistance and 51% require disclosure, so the question is where the disclosure goes, not whether there is one (arXiv:2605.16706).

A reviewer who weighs that trade the other way should say so: this is the part of the proposal with a real competing design behind it.

What

AGENTS.md gains a ## Commit messages section holding one sentence:

Always end commit messages, including drafts, with Assisted-by: <tool> (<model-id>), never Co-Authored-By:. Keep existing trailers and add yours when amending someone else's commit.

The heading is deliberately broader than the rule: further commit message conventions belong under it rather than in a section of their own.

That is the whole change, and the brevity is deliberate. Every sentence in this file is read by an agent and changes what it writes, so the file carries the wording the matrix below actually tested and nothing else. Drafts of this change that surrounded the rule with rationale measurably changed agent output: a paragraph offering Generated-by: as a second tier moved agents onto Generated-by: for a one-word typo fix. The reasoning therefore lives in this pull request, where it informs reviewers without steering tools.

The wording is imperative rather than descriptive, and that is load-bearing rather than a matter of taste. An agent arrives with its own harness instruction to sign commits Co-Authored-By:, and a descriptive line loses to it.

The human contributor remains the author and remains responsible for the change under the ICLA. That is why the rule forbids Co-Authored-By: for a tool rather than merely preferring Assisted-by:, and it needs no restating in AGENTS.md, since the trailer the file mandates already carries it.

Generated-by: is deliberately left out, beyond the steering problem above: making the narrow label reachable for routine work defeats the distinction it exists to draw. A second tier is worth having, but its trigger needs wording an agent does not read as covering ordinary edits, and that wording needs its own run through the matrix. That is a follow-up, not a blocker for this one.

How this was checked

The wording was chosen by running coding agents on live git repositories rather than by reading it. Each run seeds a throwaway repo with one typo, gives the agent the task sentence only with no mention of trailers, and grades the resulting commit. Asking an agent which trailer it would use measures its self-report, not its behavior, so that was never done.

Final matrix, one column per agent, at least three runs for every cell that had ever failed:

Scenario Claude Sonnet 5 Claude Opus 5 Codex gpt-5.6-terra
fix a typo and commit pass pass pass
commit with a message dictated verbatim pass 3/3 pass
draft a message, commit nothing 3/3 3/3 3/3
amend a commit carrying another tool's trailer pass, kept and added pass, kept and added
amend a commit carrying a trailer already pass pass, no duplicate 3/3, kept and added
two separate commits pass, both pass, both pass, both

No Co-Authored-By: line appeared in any run of the final wording. Earlier rungs of the ladder failed in ways worth recording, because each failure is what a shorter wording costs:

  • A descriptive line, Commit trailer: X, not Y, lost outright: Opus kept Co-Authored-By: and Sonnet emitted no trailer at all. The imperative Always end commit messages with is what beats the harness instruction.
  • <model> produced (Sonnet 5) and (Opus 5) in some runs; <model-id> produced claude-opus-5, claude-sonnet-5, and gpt-5.6-terra in every run, at no extra length. That is the axis this repository's history drifts on today.
  • Without Keep existing trailers, Sonnet replaced another tool's Assisted-by: with its own in one run of three. That is attribution loss, not formatting drift.
  • Without including drafts, Codex omitted the trailer from a drafted message in one run of three; with it, three of three carried it.

The <tool> field still varies — mostly Claude Code, sometimes Claude, and Codex or Codex CLI. The trailer key and the model id were stable in every run, so git log --grep and any machine parsing are unaffected.

One caveat on method. Codex reads AGENTS.md itself, so that column is a direct measurement. The Claude columns were produced by injecting the rule into a subagent prompt in the wrapper the harness uses for memory files, because a nested claude -p could not authenticate on the test machine. Prompt injection is a stronger channel than a memory file, so treat those cells as an upper bound and re-run them against a real session before relying on a marginal result.

Reproducing this, or extending it

A prompt for a coding agent, if you want to add a model or a scenario. It is written to be handed over as-is.

Reproduction prompt
**Goal.** Find the shortest line in an agent memory file (`AGENTS.md`, `~/.claude/CLAUDE.md`) that makes a coding agent write `Assisted-by:` instead of the `Co-Authored-By:` trailer its own harness asks for. Vary wording, model, and task; hold everything else fixed.

**What is actually being compared.** In Claude Code the `Co-Authored-By:` line is not a model habit: the CLI puts it in the system prompt, built from the `attribution.commit` setting and its default. So this measures a memory-file line against a system-prompt line. Codex has no such built-in trailer, so its control produces no trailer at all rather than `Co-Authored-By:`. A team that only wants the string changed in Claude Code should set `attribution.commit` instead of tuning prose; the wording matters for the cases that setting does not reach, which are drafts, amends, other tools, and subagents.

**Wording ladder.** Run the control and at least the first and last rung; the middle rungs are what locate the cheapest sufficient wording.

0. control, no rule
1. ``Commit trailer: `Assisted-by: <tool> (<model-id>)`, not `Co-Authored-By:`.``
2. ``Always end commit messages with `Assisted-by: <tool> (<model-id>)`, never `Co-Authored-By:`.``
3. rung 2 plus ``Keep existing trailers and add yours when amending someone else's commit.``
4. rung 3 with `including drafts` after `commit messages`

**Method.** For each (rung x model x scenario x repetition), seed a throwaway git repo, give the agent the task sentence only, and grade the repository afterwards. Never mention trailers in the task. Never ask the agent which trailer it would use: that measures its self-report, not its behavior.

**Seed.** A fresh directory under `mktemp -d`, with no `AGENTS.md` or `CLAUDE.md` in any ancestor directory up to `/` — assert this, because a project file nearby steers the agent and you will measure that file instead of your wording. Set `user.name`, `user.email`, `commit.gpgsign=false`, `core.hooksPath=/dev/null`. Write the rule to `AGENTS.md` for Codex and `CLAUDE.md` for Claude Code, but not both in the same repo: Claude Code reads either, and two copies is a stronger treatment than one. Commit the rule file in the seed commit so `git add -A` does not sweep it into the agent's commit. Record the CLI version and the resolved model id in every row; both the wrapper and the built-in trailer are version-dependent.

**Delivery.** Point the CLI at a throwaway config directory (`CLAUDE_CONFIG_DIR`, `CODEX_HOME`) so the tester's own memory files and settings stay out of the run. Where a fresh session cannot be started and the rule has to be injected into a subagent prompt instead, say so in the writeup: prompt injection is a stronger channel than a memory file, so those numbers are an upper bound, not a measurement of the memory file.

**Scenarios**, one repo each. The stated outcome is what passing means.

1. Commit. `README.md` contains `recieve`. Task: "fix the spelling mistake in README.md and commit the change". One new commit, carrying the trailer.
2. Dictated message. Same seed, task: `commit it with the message "Fix typo in README"`. The trailer is still required; the dictated text becomes the subject.
3. Draft. The fix is already unstaged. Task: "draft a commit message; do not commit". The trailer appears in the agent's final message, and `HEAD` and `git status --porcelain` are byte-identical to the pre-run snapshot. This is the one scenario whose oracle is agent output rather than repository state.
4. Amend someone else's commit. `HEAD` carries `Assisted-by: Codex (gpt-5.6-terra)`, a second typo remains. Task: "fix it and amend into the previous commit". Both trailers survive.
5. Amend a commit that already carries this agent's own trailer. Same shape. The trailer appears exactly once.
6. Two commits. Two files, one typo each, "as two separate commits". Every commit carries the trailer.
7. History that already uses `Co-Authored-By:`. Seed three commits ending in `Co-Authored-By: Claude <noreply@anthropic.com>`, then scenario 1. Agents imitate `git log`, and a rule that only wins in a pristine repo does not win in a real one.
8. Human co-author. `HEAD` carries `Co-Authored-By: Real Person <person@example.com>`, and the task is an amend. That line must survive.

**Grading.** Mechanical, no LLM judge. A trailer line matches `^Assisted-by: .+ \(.+\)$`. Record the literal `<tool>` and `<model-id>` strings rather than grading them; their variance is a finding. A run fails if a `Co-Authored-By:` line names a tool, and separately if scenario 8 loses the human one — those are different defects and must not collapse into one boolean. A run that timed out, exited non-zero, hit a permission prompt, or produced no commit is invalid rather than failed: re-run it and report the invalid count on its own.

**Reporting.** Report per-cell rates, not booleans. A 2-of-3 bar over dozens of cells passes a coin flip about half the time, so treat 3 runs as a screen and re-run any cell that fails once at n=5 or more. Quote failing commit messages in full.

**To extend.** A new model is a new column: rerun every scenario, including the control. A new scenario needs three things written down before the first run: the seed, the task sentence, and what the repository must look like to pass.

Scenarios 7 and 8 in that prompt were not run for this change and are the first thing worth adding. Scenario 7 matters here in particular: this repository's history already carries fifteen Co-Authored-By: tool lines for an agent to imitate, and the matrix above was measured on pristine repositories.

Scope

Documentation only; no code, no build change.

This brings JMeter in line with guidance that already exists rather than inventing a policy for it. The ASF asks contributors to name the tooling in the commit message; the choice of Assisted-by: over Co-authored-by: is where the projects listed above have landed, and the ASF page predates agentic tooling and names only Generated-by:. What is left for JMeter to decide is the exact wording, which is what this diff is.

The same change is open for Calcite as apache/calcite#5230, with the same sentence.

Assisted-by: Claude Code (claude-opus-5)

Sixteen commits on master since March 2026 name a tool in a trailer, in two
capitalizations of the key and four spellings of the model. Fifteen of them
use `Co-authored-by:`, which in this same history carries nine human names.
Git and GitHub read every name in that trailer as an author of the commit,
so the field now does two different jobs and a reader cannot tell them apart.

Adopt `Assisted-by:` for the disclosure the ASF generative tooling guidance
asks for, matching the Linux kernel, Fedora, Rocky Linux, Mesa, Zephyr, and
OpenInfra, and keep the author fields for people.

The section is one sentence because every sentence in this file steers an
agent. The wording is the one that passed the test matrix in the pull
request; prose added around it changed what agents wrote, so the rationale
lives in the pull request instead.

Assisted-by: Claude Code (claude-opus-5)
@milamberspace

Copy link
Copy Markdown
Contributor

Thanks for this PR, sounds good.

Before merge, the PR does not mention the ASF Responsible AI initiative (rai.apache.org). It's probably important, because RAI was created to converge on exactly this question, and it currently points a different way:

  • ASF Legal (in generative-tooling.html) names Generated-by:
  • RAI, commit-messages page also recommends Generated-by:, paired with an Apache-ai tag.
  • RAI, attribution page proposes ai-assisted-by:, and calls Co-authored-by: for a tool "acceptable as a provenance record".

So there are now three spellings in play, and this PR would add Assisted-by: as the 4th (as Apache Impala).

Perhaps that will become an issue: if JMeter use own variant, at the moment the ASF is trying to unify the ecosystem.
The RAI pages are still marked DRAFT / UNOFFICIAL, so nothing is binding yet.

I suggest theses points:

  • Align on Generated-by:, the only token in a formal ASF text, or refuse for now and choose Assisted-by: , but say in the PR comments that we don' use the current RAI draft wording (as reference)

  • Start a Assisted-by: discuss in discuss@rai.apache.org as a (better) purpose for the RAI draft. The comment of this PR find some strong cases for Assisted-by:, I can (will) send a message in discuss@rai.a.o, just for starting.

Note for reference: Impala Confluence page (writing before the official create of ASF RAI) :
https://cwiki.apache.org/confluence/spaces/IMPALA/pages/65147115/Contributing+to+Impala
If using generative tooling, please follow Apache's guidelines. Significant contributions are governed by the Apache Contributor Agreement, and generated contributions are evaluated by the same standards we use for all contributions. Please include Assisted-by: () - such as Assisted-by: Claude Opus 4.5 (Claude Code) - in your commit message to help us understand sources used in crafting the contribution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants