Add mock-based unit test for NodeHealth cloud-mode logic

[janhoy]

I asked Claude to pick a unit test that can likely benefit from mocking instead of complex cluster setups, and it picked NodeHealthSolrCloudTest. Not because it was the slowest or anything, just an example of potential for using more mocks:

NodeHealthSolrCloudTest had two failure-scenario tests that each booted an extra Jetty node just to break its ZK connection — while the code under test (NodeHealth.getClusterState() and the cloud-mode health checks) only reads state reachable through CoreContainer and is trivially mockable.

This PR:

• Adds NodeHealthTest — a plain SolrTestCase with Mockito (no Jetty/ZK, runs in milliseconds), following the existing pattern of GetNodeCommandStatusTest in the same package. It mocks the CoreContainer → ZkController → ZkStateReader → SolrZkClient chain and covers:
  • null / shut-down CoreContainer → 500 (previously untested)
  • healthy node → OK
  • ZK client closed, and ZK client disconnected → 503 "Not connected to zk"
  • node missing from live nodes → 503 "Not in live nodes"
  • requireHealthyCores with core loading incomplete → 503 (previously untested)
  • requireHealthyCores with all cores healthy → OK
  • a recovering core → 503 with unhealthy-replica count (reusing ClusterStateMockUtil)
• Slims down NodeHealthSolrCloudTest by removing the two extra-Jetty failure tests (now covered by the mocks), keeping the happy-path tests that exercise the real HTTP/Jersey wiring on the shared 1-node cluster.

Test-only change, no changelog entry.

Measured timings (3 local runs each, averaged)

Suite	Before	After
NodeHealthSolrCloudTest	~3.1s (4 tests)	~2.1s (2 tests)
NodeHealthTest (new)	—	~0.6s (8 tests)

The two removed extra-Jetty tests accounted for ~1s (a third) of the cloud suite; their mock replacements run in milliseconds (the 0.6s suite time is almost entirely one-time Mockito/class-init overhead). Net: total time for the classes drops ~15% while going from 4 to 10 tests, including two previously untested error branches — and the ZK-connection-killing tests, the flakiest kind, no longer run against a real cluster.

[dsmiley]

I like that this increased code coverage (tested more scenarios) -- thanks.

Your opening premise seems backwards to me. Starting with a testing technique (mocking) and attempting to use it somewhere random seems backwards. Wouldn't we start with a slow test and see what that test needs to be better -- who knows what? Or target a known piece of poor infrastructure to change/elliminate. Any way, your random choice and the results turned out well here.

I confess I hate mocking; I use it very rarely. It lies to you; you create a sham world that you hope replicates reality, yet reality may be otherwise. Test passes by design; fools you into thinking bug-free, especially for something you explicitly test for. Sometimes creating the mocks is awkward; albeit wasn't too bad in this case, thankfully. It's also sad when an NPE or similar happen and we feel the need to add checks in non-test code to guard against provably impossible things. Didn't happen here, thankfully.

[janhoy]

Wouldn't we start with a slow test and see what that test needs to be better

This PR is backward on purpose. I did not attempt to delve into a formal process of assessing and improving the slowest tests (like I did a few months ago). Rather I decided to attempt LLM assisted test improvement along two axis - this one where mocks could help simplify complex failure scenarios (which is one good use of mocks). And the other where we spin up the entire sol(a)r system to test something that won't need a big cluster. So from a random set of 50 test classes I found two candidates. Which is a hint that we may have perhaps hundreds of such opportunities across the codebase.

I believe we have grown numb to writing full integration tests for every single use case. Integration tests has their place and we should not stop writing them. But we should always go with lower level tests first.

I'm hoping we can get to a place some day where gradle test runs in a few minutes and has great coverage and stability for everyday commit work. While invoking integration tests can take longer (but we should strive to keep it way below the hour) and be invoked in nightly, before releases and conditionally in PRs depending on what code is touched.

Take ExtractingRequestHandler as an example. Currently we spin up a real TikaServer with Testcontainers to test the component. This is integration testing and should ideally be opt-in. For everyday test runs it would be sufficient to mock TikaServerExtractionBackend.callTikaServer() to test the key logic without a real Tika server. I did not do that but probably should have to shave some 10-15s off of every normal test run.