generate-names: default to the frontier models, not the cheap tier - #16
Merged
Conversation
The defaults were gpt-4.1-mini and claude-haiku-4-5, and the names showed it: generic startup vocabulary, the same few stems recycled, and a drift off-brief on any description longer than a sentence. Now gpt-5.6-sol and claude-fable-5. The cheap tier was never the saving it looked like. This tool makes exactly ONE call per run whatever --count says — that is the whole design, since asking a model for a thousand names directly repeats itself within a few hundred. So the model is a rounding error against the value of a name you actually ship, and paying for the weaker one bought nothing. Two things the Anthropic path needed before claude-fable-5 could be the default: - max_tokens 4096 -> 16000. Fable 5 always thinks and thinking counts against max_tokens, so a budget that was ample for a non-thinking model can be spent before the JSON starts. - stop_reason "refusal" is now reported as a refusal. It arrives as HTTP 200 with no text block, so unhandled it reads as "the model returned nothing" and sends you to the parser rather than to the answer the API actually gave. Reading the text block by type rather than by position already handled the leading thinking block; there is now a test pinning that, and one pinning that this request never grows temperature/top_p/top_k — all three were removed on Fable 5 / Opus 5 / Sonnet 5 and are rejected with a 400. Note for anyone hitting this: every Anthropic key in the team vaults is spend-capped until 2026-09-01, so --provider anthropic returns a 400 until then and OpenAI is the working path. 187 tests pass (was 180), typecheck clean. Verified end to end against both the piped domainfree run and a bare default-model run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ThreatCrush Security Scan6 finding(s) MEDIUM: 1 | LOW: 5
Snippets are redacted; ThreatCrush never prints matched credential material. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Defaults were
gpt-4.1-miniandclaude-haiku-4-5. Nowgpt-5.6-solandclaude-fable-5.Why
The names showed the cheap tier: generic startup vocabulary, the same few stems recycled, and a drift off-brief on any description longer than a sentence.
And the saving was never real. This tool makes exactly one call per run whatever
--countsays — that is the whole design, since asking a model for a thousand names directly repeats itself within a few hundred. The model is a rounding error against the value of a name you actually ship, so paying for the weaker one bought nothing.What the Anthropic path needed first
max_tokens4096 → 16000. Fable 5 always thinks, and thinking counts againstmax_tokens, so a budget that was ample for a non-thinking model can be spent before the JSON starts.stop_reason: "refusal"is now reported as a refusal. It arrives as HTTP 200 with no text block, so unhandled it reads as "the model returned nothing" — which sends you to the parser rather than to the answer the API actually gave.Reading the text block by type rather than by position already handled the leading thinking block; there is now a test pinning that, plus one pinning that this request never grows
temperature/top_p/top_k— all three were removed on Fable 5 / Opus 5 / Sonnet 5 and are rejected with a 400.Heads-up for anyone hitting this
Every Anthropic key in the team vaults is spend-capped until 2026-09-01 (
400 invalid_request_error, "You will regain access on 2026-09-01"). So--provider anthropicfails until then and OpenAI is the working path. Verified against a live call today.Checks
187 tests pass (was 180), typecheck clean.
Verified end to end: the full
generate-names … | domainfreepipeline returned 41 available of 100 checked, and a bare run with no--modelnow reportsopenai/gpt-5.6-sol.🤖 Generated with Claude Code