Feature Request: Support simultaneous ion-cell BFGS for cell relaxation

[Growl1234]

Currently, BFGS is only used as an ionic optimizer in cell-relax. The cell itself is still optimized through the separate lattice-CG path.

In Ions_Move_Methods, the BFGS optimization space is explicitly limited to

Ions_Move_Basic::dim = natom * 3;

and both BFGS variants operate on ionic forces only. In contrast, lattice relaxation is handled separately by Lattice_Change_Methods, which uses Lattice_Change_CG with the stress tensor.

Therefore, cell-relax + bfgs currently behaves approximately as:

fixed-cell ionic BFGS
        ↓
lattice CG update
        ↓
reset ionic BFGS history
        ↓
repeat

rather than as a genuine variable-cell BFGS optimization over both ionic and cell degrees of freedom.

Current limitation

PR #7507 correctly fixed #4140 by resetting the ionic BFGS state after a cell change. Reusing the old ionic positions, gradients, Hessian/inverse-Hessian and trust-radius state after changing the lattice was inconsistent and could cause BFGS to break down.

However, this also indicates a structural limitation of the current implementation: the BFGS approximation only contains ionic curvature,

$$ B_{RR}, $$

instead of a joint variable-cell Hessian such as

$$ B = \begin{pmatrix} B_{RR} & B_{Rh}\\ B_{hR} & B_{hh} \end{pmatrix}. $$

Consequently:

• ion-cell curvature coupling is not learned;
• cell curvature is handled by a separate CG optimizer;
• all accumulated BFGS information has to be discarded after every accepted cell update;
• ionic curvature has to be learned again for the new cell.

This affects both bfgs 1 and bfgs 2; their difference is how the Hessian/inverse Hessian is represented, not which degrees of freedom are optimized.

Proposed direction

I suggest adding a simultaneous ion-cell BFGS path for cell-relax, conceptually similar to the current cg 2 implementation.

The optimization state could contain the ionic coordinates together with the active cell degrees of freedom:

$$ x=(R,h), \qquad g=(g_R,g_h), $$

with appropriate scaling/preconditioning between ionic and cell components.

The optimizer would then:

• construct a joint gradient from forces and stress;
• update ions and cell within the same quasi-Newton step;
• learn ion-ion, ion-cell and cell-cell curvature together;
• apply the existing cell constraints (fixed_axes, fixed_ibrav, etc.) in the joint optimization space;
• no longer reset BFGS history after an ordinary cell step, since the cell change would itself be part of the BFGS trajectory.

The existing simultaneous cg 2 infrastructure may provide a useful basis, since it already maintains both ionic and cell gradients/search directions and updates them together.

Expected benefits

• avoid repeatedly rebuilding the ionic BFGS history after every cell change;
• include ion-cell coupling in the quasi-Newton model;
• potentially reduce the number of force/stress evaluations for strongly coupled variable-cell relaxations;
• make relax_method = bfgs during cell-relax a genuine BFGS optimization of the full variable-cell problem rather than a nested BFGS/CG procedure.

[Growl1234]

CP2K can serve as a useful reference here. Its CELL_OPT formulates ions and cell degrees of freedom as a single optimization vector, e.g. $x=(R,h)$, and provides a common evaluator that maps this vector to the physical structure and returns the corresponding energy and joint gradient from forces and stress. BFGS, L-BFGS, and CG then operate on exactly the same generalized optimization problem. As a result, cell changes are ordinary optimizer steps rather than external events, and the quasi-Newton Hessian naturally contains ion-ion, ion-cell, and cell-cell curvature without requiring a BFGS reset after each lattice update.

I believe, extracting the relevant parts from the CG implementation should be the first step we take. And as part of the overall work we could consider deprecating cg 1.