fix(workflow-operator): send provider-specific model id on HF text-gen chat fallback routes

[PG1204]

What changes were proposed in this PR?

Fixes two related bugs in the HuggingFace inference operator's provider-fallback loop (_post_with_fallback in the generated Python, emitted by PythonCodegenBase.scala) that break the text-generation / image-text-to-text path, which is the operator's default task.

1. Wrong model id on provider-scoped chat routes. The chat branch posted the same payload, carrying "model": self.MODEL_ID (the HF Hub ID), to every provider's route (router.huggingface.co/{provider}/...). Provider-scoped routes require the provider's own model name (providerId, e.g. the Hub's Qwen/Qwen2.5-72B-Instruct is a different string on Groq or Fireworks), which _resolve_providers already fetches and _call_provider already uses for the other task families. Providers whose internal name differs from the Hub ID rejected every request with 400/404, so fallback only succeeded when the names happened to match. The chat branch now posts a per-attempt copy with the model overridden:

   chat_payload = {**pipeline_payload, "model": provider_id}

   (a copy, not in-place mutation, pipeline_payload is reused for the next provider
   attempt).

2. Malformed hf-inference chat URL. The chat URL for hf-inference was built as hf-inference/v1/chat/completions, but hf-inference expects the model in the URL path: hf-inference/models/{model-id}/v1/chat/completions (the form this file already uses for pipeline tasks). Since hf-inference sorts first in PROVIDER_COST_PRIORITY, the cheapest provider failed on every row, wasting a doomed request (up to the 120 s timeout) per row before falling through.

Net effect: text-gen rows now succeed on the cheapest live provider instead of failing with "All inference providers failed" or silently drifting to pricier providers. No behavior change for other task families - _call_provider already handled them correctly.

Any related issues, documentation, discussions?

Closes #6965

How was this PR tested?

Added a generated-code test to HuggingFaceInferenceOpDescSpec ("send the provider-specific model id on provider-scoped chat routes") pinning both fixes: the corrected hf-inference chat URL, and the chat branch posting the per-provider payload copy (asserted as an anchored two-line block so pipeline routes that legitimately post pipeline_payload directly stay unaffected).

Run:

sbt "WorkflowOperator/testOnly org.apache.texera.amber.operator.huggingFace.*"

Full HF package passes (123 tests, 12 suites), including PythonCodeRawInvalidTextSpec, which py_compiles the generated Python of all 117 Python operators, verifying the edited template still emits syntactically valid Python. scalafmt clean.

Was this PR authored or co-authored using generative AI tooling?

Co-authored with Claude Fable 5 in compliance with ASF.

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• No candidates found from git blame history.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.56%. Comparing base (b98faf4) to head (a53b20b).

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #6972      +/-   ##
============================================
- Coverage     79.56%   79.56%   -0.01%     
+ Complexity     3834     3833       -1     
============================================
  Files          1159     1159              
  Lines         46145    46145              
  Branches       5127     5127              
============================================
- Hits          36716    36714       -2     
- Misses         7799     7800       +1     
- Partials       1630     1631       +1     

Flag	Coverage Δ		*Carryforward flag
access-control-service	70.00% <ø> (ø)
agent-service	77.42% <ø> (ø)		Carriedforward from 8c255b8
amber	73.04% <ø> (-0.02%)	⬇️
computing-unit-managing-service	20.49% <ø> (ø)
config-service	66.66% <ø> (ø)
file-service	67.21% <ø> (ø)
frontend	83.05% <ø> (ø)		Carriedforward from 8c255b8
notebook-migration-service	78.94% <ø> (ø)
pyamber	97.36% <ø> (ø)		Carriedforward from 8c255b8
workflow-compiling-service	26.31% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

⚠️ Benchmark changes need a look

🟢 4 better · 🔴 3 worse · ⚪ 8 noise (<±5%) · 0 without baseline

Compared against main b98faf4 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	372	0.227	26,188/34,663/34,663 us	🔴 +16.1% / 🔴 +119.6%
🟢	bs=100 sw=10 sl=64	789	0.482	124,391/153,189/153,189 us	🟢 -12.2% / 🔴 +38.9%
⚪	bs=1000 sw=10 sl=64	907	0.554	1,102,793/1,208,642/1,208,642 us	⚪ within ±5% / 🔴 +14.5%

Baseline details

Latest main b98faf4 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	372 tuples/sec	410 tuples/sec	771.62 tuples/sec	-9.3%	-51.8%
bs=10 sw=10 sl=64	MB/s	0.227 MB/s	0.25 MB/s	0.471 MB/s	-9.2%	-51.8%
bs=10 sw=10 sl=64	p50	26,188 us	22,564 us	12,626 us	+16.1%	+107.4%
bs=10 sw=10 sl=64	p95	34,663 us	38,570 us	15,786 us	-10.1%	+119.6%
bs=10 sw=10 sl=64	p99	34,663 us	38,570 us	19,344 us	-10.1%	+79.2%
bs=100 sw=10 sl=64	throughput	789 tuples/sec	793 tuples/sec	971.56 tuples/sec	-0.5%	-18.8%
bs=100 sw=10 sl=64	MB/s	0.482 MB/s	0.484 MB/s	0.593 MB/s	-0.4%	-18.7%
bs=100 sw=10 sl=64	p50	124,391 us	120,980 us	103,463 us	+2.8%	+20.2%
bs=100 sw=10 sl=64	p95	153,189 us	174,409 us	110,296 us	-12.2%	+38.9%
bs=100 sw=10 sl=64	p99	153,189 us	174,409 us	118,690 us	-12.2%	+29.1%
bs=1000 sw=10 sl=64	throughput	907 tuples/sec	904 tuples/sec	1,001 tuples/sec	+0.3%	-9.4%
bs=1000 sw=10 sl=64	MB/s	0.554 MB/s	0.552 MB/s	0.611 MB/s	+0.4%	-9.4%
bs=1000 sw=10 sl=64	p50	1,102,793 us	1,102,398 us	1,008,988 us	+0.0%	+9.3%
bs=1000 sw=10 sl=64	p95	1,208,642 us	1,168,181 us	1,055,260 us	+3.5%	+14.5%
bs=1000 sw=10 sl=64	p99	1,208,642 us	1,168,181 us	1,074,689 us	+3.5%	+12.5%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,538.01,200,128000,372,0.227,26188.19,34663.17,34663.17
1,100,10,64,20,2535.11,2000,1280000,789,0.482,124390.92,153188.56,153188.56
2,1000,10,64,20,22039.34,20000,12800000,907,0.554,1102792.62,1208642.29,1208642.29

[Yicong-Huang]

@carloea2 I think the need to programmatically test operators becomes higher... There are more and more efforts on editing operators logic. It will be good to have the tests in CI soon.

[carloea2]

I am working on it, I will raise a discussion about it soon.

[PG1204]

/request-review @xuang7

[xuang7]

LGTM

[PG1204]

@xuang7 github merge bot removed the pr from merge queue, requesting to add this pr to merge queue again. Thanks!