feat: Implement optimization code paths and functionality for initial release - #140

Merged

andrewklatzke merged 90 commits into

mainfrom

aklatzke/AIC-2263/sdk-dx-improvements

Jul 17, 2026

Commits on Mar 26, 2026

feat: implements optimize method in SDK, code moved
andrewklatzke
committed
9859d08

Commits on Mar 31, 2026

feat: implementation of agent optimization + tests
andrewklatzke
committed
1712e4f
feat: implement ability to use completions or agents for judge calls
andrewklatzke
committed
ea596a7
feat: all logs -> debug
andrewklatzke
committed
2fd55e2
fix: lints + structured output tool rename
andrewklatzke
committed
8481690
fix: lint + missed variable rename
andrewklatzke
committed
f8e5509
fix: sort imports
andrewklatzke
committed
c032aaf
fix: lint
andrewklatzke
committed
aee6aa7
chore: break up long lines, add spaces where necessary
andrewklatzke
committed
59c7ac7
chore: break up another long line
andrewklatzke
committed
59f03f2
chore: fix on_turn path
andrewklatzke
committed
e2ff561
chore: move prompts to own file, better debug info
andrewklatzke
committed
af2dd03
chore: update tests, fix cursor feedback
andrewklatzke
committed
ea43575

Commits on Apr 1, 2026

Commits on Apr 3, 2026

feat: ground truth optimization path
andrewklatzke
committed
44c8c59

Commits on Apr 7, 2026

feat: prevent overfitting via prompt changes and post-processing
andrewklatzke
committed
8f9f1e2
chore: remove some dead code
andrewklatzke
committed
a17fd6e
chore: remove provided_tool_handlers code
andrewklatzke
committed
67fdbf1

Commits on Apr 8, 2026

fix: adjust iteration logic so validation doesn't consume them
andrewklatzke
committed
3042984
feat: implement latency & token tracking for optimizations
andrewklatzke
committed
288336e
feat: add optimization for duration
andrewklatzke
committed
5d76276

Commits on Apr 9, 2026

feat: add auto-commit option
andrewklatzke
committed
4cb8859
chore: add tests
andrewklatzke
committed
ba369a2

Commits on Apr 15, 2026

Commits on Apr 16, 2026

Commits on Apr 17, 2026

chore: update types for lint
andrewklatzke
committed
a386a27
lint, cursor feedback
andrewklatzke
committed
8d1a868
chore: additional lint
andrewklatzke
committed
937542a

Commits on Apr 22, 2026

Commits on Apr 23, 2026

feat: add token limit handling
andrewklatzke
committed
e8c6692

Commits on May 4, 2026

Commits on May 5, 2026

feat: adds ability to use inverted judges
andrewklatzke
committed
0ed243e
feat: use judge_passed for all calcs
andrewklatzke
committed
a8f14de

Commits on May 6, 2026

feat: adds ability to use inverted judges (#168)
andrewklatzke
authored
23baeb4
feat: adds ability to optimize for cost
andrewklatzke
committed
94de596
fix: remove unnecessary token path
andrewklatzke
committed
dc82818

Commits on May 7, 2026

Commits on May 12, 2026

feat: adds reporting for cost and latency optimization failures (#180)
andrewklatzke
authored
d267832

Commits on May 13, 2026

fix: don't only evaluate final input in GT results
andrewklatzke
committed
9bedf9e
fix: don't only evaluate final input in GT results
andrewklatzke
committed
53f455f
fix: only strip known provider prefixes
andrewklatzke
committed
66bc1f0

Commits on May 14, 2026

fix: address cursor feedback
andrewklatzke
committed
f2f0894

Commits on May 18, 2026

Commits on May 20, 2026

Merge branch 'main' into aklatzke/AIC-2263/sdk-dx-improvements
andrewklatzke
authored
99dd7b8
chore: lint
andrewklatzke
committed
849d41f
Merge branch 'aklatzke/AIC-2263/sdk-dx-improvements' of github.com:launchdarkly/python-server-sdk-ai into aklatzke/AIC-2263/sdk-dx-improvements
andrewklatzke
committed
c51e235
chore: isort
andrewklatzke
committed
f481ed3
chore: line lengths
andrewklatzke
committed
27b248e
chore: fix agent init typing
andrewklatzke
committed
4545203

Commits on Jun 1, 2026

implement api fields
andrewklatzke
committed
b506fef
circumvents judge_config calls to make judge evaluations in optimization
andrewklatzke
committed
806b564

Commits on Jun 2, 2026

cursor feedback
andrewklatzke
committed
509240f

Commits on Jun 3, 2026

changes cost & latency optimization to post-process
andrewklatzke
committed
af4ec5d
cursor feedback
andrewklatzke
committed
de5f24f
more cursor feedback
andrewklatzke
committed
aa0a77f
fix: ensure cost data is persisted
andrewklatzke
committed
4eb0bb0

Commits on Jun 18, 2026

Commits on Jul 14, 2026

resolve merge conflicts
andrewklatzke
committed
fba29aa
cursor feedback
andrewklatzke
committed
bc2508e
feat: ability to specify variation key on config or from_options for optimization package (#162)
andrewklatzke
authored
f0c4612
fix(optimization): use run-chosen context for config judges; guard non-dict JSON

andrewklatzke
and
cursoragent
committed
64e12f2
fix(optimization): match model config by key as well as id for pricing lookup

andrewklatzke
and
cursoragent
committed
a8415be

Commits on Jul 15, 2026

Commits on Jul 16, 2026

fix(optimization): fix CI test failures and apply Bugbot-reported fixes

andrewklatzke
and
cursoragent
committed
ffad0cb
fix(optimization): retry LLM calls on transient provider errors (429/503/529)

andrewklatzke
and
cursoragent
committed
a131e4b
fix(optimization): address Bugbot comments on PR #140

andrewklatzke
and
cursoragent
committed
9e57d86
chore(optimization): remove result persistence summary warning

andrewklatzke
and
cursoragent
committed
e8ec8d1
fix(optimization): remove dead persistence counter code

andrewklatzke
and
cursoragent
committed
c76cb16
ci: re-trigger CI after GitHub outage

andrewklatzke
and
cursoragent
committed
9baa7b1
Merge branch 'main' into aklatzke/AIC-2263/sdk-dx-improvements
andrewklatzke
authored
e7f8959