Derive default Scala cluster runtime from DBR version

[rugpanov]

Changes

• Derive the default Scala classic-cluster Spark runtime from shared DBR and Scala version macros.
• Add regression coverage and a compact classic-template acceptance diff.
• Extend the template-version rule to cover the Scala build file and both runtime inputs.

Why

The DB Connect dependency already follows dbr_version, but the classic job template duplicated its runtime as a literal. Updating only the macro could therefore produce a client that cannot connect to the generated cluster.

Follow-up to #6407.

Tests

• go test ./libs/template -count=1
• go test ./acceptance -run TestAccept/bundle/templates/default-scala-classic (Terraform and direct variants)
• focused golangci-lint: 0 issues
• ./task checks
• ./task lint: all Go linters pass; Python lint could not download ruff because PyPI access is blocked
• ./task test: 9,786 tests pass; libs/patchwheel requires an unavailable Python 3.13 installation

This PR was written by Codex.

[eng-dev-ecosystem-bot]

Integration test report

Commit: 2e3f010

Run: 34223216447

	Env	💚​RECOVERED	✅​pass	🙈​skip	Time
💚​	aws windows	1	277	14	6:27
💚​	azure windows	1	276	14	7:09
💚​	gcp windows	1	277	14	8:42

	Test Name	aws windows	azure windows	gcp windows
💚​	TestAccept	💚​R	💚​R	💚​R

Top 4 slowest tests (at least 2 minutes):

duration	env	testname
3:39	azure windows	TestAccept
3:27	gcp windows	TestAccept
3:13	aws windows	TestAccept
2:26	gcp windows	TestFilerWorkspaceFilesExtensionsDelete

[anton-107]

Approving — reviewed end-to-end; clean, low-risk refactor with good test coverage.

Summary

Replaces the hardcoded classic-cluster spark_version: 17.3.x-scala2.13 with a spark_version macro composed from the existing dbr_version and scala_major_minor_version macros. Rendered output is byte-identical (hence no existing golden changes), and it closes a real drift hazard: bumping dbr_version previously moved only the DB Connect client and left the cluster runtime stale.

What I checked:

• Composing spark_version from the two macros default-scala already maintains — rather than introducing a third pin — is the right design and appropriate here, since both values are needed independently in build.sbt.
• The new acceptance test faithfully follows the dbt-sql-classic precedent (Phase = 1 + diff against the sibling serverless golden via $TESTDIR); the phase gating in acceptance_test.go makes the cross-test dependency sound. out.test.toml field order matches the generator, and "standard cluster" is a valid schema enum.
• Unit tests are well-factored: the composition test and the wiring test are complementary.
• Macro whitespace renders cleanly, guarded by the explicit TrimSpace assertion.

Non-blocking comments

1. The first-paragraph rules-doc edit (cli-pipelines / "default is not shipped") is accurate but logically independent of this change — consider splitting it per the repo's "keep each PR focused" convention, or at least note it in the description.
2. TestDefaultScalaRenderedVersionsUseMacros uses 99.9.x-scala9.9 as the fake spark_version, which mimics the real derived format and can read as if it tests derivation (it actually tests wiring). A clearly-synthetic sentinel would make the intent unambiguous.

Draft and currently behind main — rebase before merge.

Reviewed by Isaac (Claude Code).