3D -> Lagrange Bubbles -> Two-way Coupling -> Tracer Bubbles (4C751DAF) aborts on the Frontier CCE cpu lane with:
No Lagrangian bubbles remain in the domain. Exiting.
(src/simulation/m_data_output.fpp, in the run-time-info block beside the ICFL/VCFL NaN checks.)
It passes on amdflang GPU offload and on gfortran CPU. This is present on master as well — it has been invisible because the test suite discarded post_process/--test-all exit codes (toolchain/mfc/test/test.py:565 on master: cmd = case.run([PRE_PROCESS, SIMULATION, POST_PROCESS], ...) with no return-code check).
The abort should not be reachable in this case
The test runs one bubble (lag_params%nBubs_glb = 1), placed by create_input_lagrange (toolchain/mfc/test/case.py) at the dead centre of a unit domain with zero initial velocity and radius 8e-3:
0.5 0.5 0.5 0.0 0.0 0.0 8.0e-03 0.0
There are exactly two ways a bubble is dropped (src/simulation/m_bubbles_EL.fpp, "Remove bubbles that are no longer in a liquid"):
- it leaves the domain, or
- its cell's void fraction exceeds
lag_params%valmaxvoid (0.99 here), i.e. alpha_liquid < 1 - valmaxvoid.
Measured on gfortran CPU over the full 50 steps (dt = 1e-6, 5e-5 s total), from the case's own lag_bubble_evol output:
| quantity |
value |
| position x |
0.5 -> 0.5000076 |
| radius |
0.008000 -> 0.0079987 (shrinks; growth 1.000x) |
| max void fraction in cell |
0.0335 |
valmaxvoid threshold |
0.99 |
| growth needed to be culled |
3.1x |
The bubble is effectively stationary and stable. It is nowhere near either removal criterion: it cannot traverse 0.5 in 5e-5 s, and it would need to more than triple its radius to fill the cell.
Therefore
On CCE the bubble state appears to diverge numerically (the count reaching zero is the symptom, and the abort message names a cause that the geometry rules out). The message itself is misleading — it reports "no bubbles remain in the domain" for what is, in the valmaxvoid branch, a void-fraction cull rather than a domain exit.
Two things worth separating:
- The underlying issue: why the Lagrangian bubble state degrades under CCE when it is stable under gfortran and amdflang. Not reproducible on a non-CCE machine, so it needs someone on Frontier.
- The diagnostic: the abort text should distinguish "left the domain" from "culled by
valmaxvoid", and it may be worth reporting the offending bubble's position/radius before aborting, since right now the message actively misdirects.
Deliberately not patched by loosening the test, because the bubble has a 3.1x margin to the limiter on working platforms — making the test avoid the abort would hide whatever is actually going wrong on CCE.
3D -> Lagrange Bubbles -> Two-way Coupling -> Tracer Bubbles(4C751DAF) aborts on the Frontier CCE cpu lane with:(
src/simulation/m_data_output.fpp, in the run-time-info block beside the ICFL/VCFL NaN checks.)It passes on amdflang GPU offload and on gfortran CPU. This is present on master as well — it has been invisible because the test suite discarded
post_process/--test-allexit codes (toolchain/mfc/test/test.py:565on master:cmd = case.run([PRE_PROCESS, SIMULATION, POST_PROCESS], ...)with no return-code check).The abort should not be reachable in this case
The test runs one bubble (
lag_params%nBubs_glb = 1), placed bycreate_input_lagrange(toolchain/mfc/test/case.py) at the dead centre of a unit domain with zero initial velocity and radius 8e-3:There are exactly two ways a bubble is dropped (
src/simulation/m_bubbles_EL.fpp, "Remove bubbles that are no longer in a liquid"):lag_params%valmaxvoid(0.99 here), i.e.alpha_liquid < 1 - valmaxvoid.Measured on gfortran CPU over the full 50 steps (
dt = 1e-6, 5e-5 s total), from the case's ownlag_bubble_evoloutput:valmaxvoidthresholdThe bubble is effectively stationary and stable. It is nowhere near either removal criterion: it cannot traverse 0.5 in 5e-5 s, and it would need to more than triple its radius to fill the cell.
Therefore
On CCE the bubble state appears to diverge numerically (the count reaching zero is the symptom, and the abort message names a cause that the geometry rules out). The message itself is misleading — it reports "no bubbles remain in the domain" for what is, in the
valmaxvoidbranch, a void-fraction cull rather than a domain exit.Two things worth separating:
valmaxvoid", and it may be worth reporting the offending bubble's position/radius before aborting, since right now the message actively misdirects.Deliberately not patched by loosening the test, because the bubble has a 3.1x margin to the limiter on working platforms — making the test avoid the abort would hide whatever is actually going wrong on CCE.