Better synchronicity in middle-end flushing operations

[franzpoeschel]

It seems that HDF5 has become quite a bit pickier about metadata definitions in parallel setups with versions 2.0 and 2.1, leading to hangups.
Earlier, it was enough to define them consistently across ranks, now we apparently have to keep the exact same order of operations.

This is bad for the Span API which runs internal flushes for structure setup.

Idea of this PR: Temporarily add this below block to RecordComponent::resetDataset():

    /*
     * For backwards-compatibility reasons, we do not actually run the below
     * flush. But the API should be written in such a way that the entire test
     * suite passes when enabling the below code block.
     */
#if 1
    auto cleanup = auxiliary::defer([&rc, this]() {
        if (rc.m_dataset.has_value() &&
            rc.m_dataset->dtype != Datatype::UNDEFINED &&
            IOHandler()->m_seriesStatus != internal::SeriesStatus::Parsing)
        {
            seriesFlush_impl</* flush_entire_series = */ false>(
                {FlushLevel::SkeletonOnly}, /* flush_io_handler = */ true);
        }
    });
#endif

Then check for hanging or failing tests and fix the flushing logic long enough until all tests pass again.

A follow-up PR should add a new API call to run this flush operation on any RecordComponent.

• No longer flush inside the Span API, instead flush already at resetDataset()
• Try if we can only enqueue operations and run them later, should be fine for the ordering
• Ensure that only a select type of operation runs in structural setup flushes
• Remove flushParticlesPath and flushMeshesPath functions, these unnecessarily leaked attribute flushes into the structure setup
• Fix tests...
• Manually go through tests and check if any of them silently swallows some errors, the chance is high
• Guard setDirty calls
• There are some commits deactivating a handful of tests, deal with them
• Maybe document user-facing changes? probably none

Follow-Up ideas and possible design changes

• Decide how to go forward with the changes inside resetDataset(). Best idea: add the new logic to a new API call commitStructuralSetup() or so.
  • Ref. also HDF5: Close operations also upon failure #1866 for defer_type
  • Ref. also Write scientific defaults upon close (Iteration::close or Series::close) #1834 for commitStructuralSetup()
• Maybe: Flush only this branch

Diff at https://github.com/franzpoeschel/openPMD-api/compare/defaults-upon-close...fix-parallel-hdf5?expand=1

[github-advanced-security]

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.