E8M0 round_float() returns values the format cannot represent

[sw23]

Reporting this issue in case it's helpful - feel free to close/disregard if the code is working as intended. This description was generated by AI:

TL;DR

round_float() documents "Returns: A float which is one of the values in the
format." For ocp_e8m0 it does not. Any input that rounds below 2^-127 —
E8M0's smallest value, since the format has no subnormals and no zero — is
returned unchanged, in every rounding mode, with sat either way.
encode_float() then maps that unrepresentable value onto code 0xFF, which
is NaN.

>>> from gfloat import round_float, encode_float, RoundMode
>>> from gfloat.formats import format_info_ocp_e8m0 as fi
>>> fi.smallest
5.877471754111438e-39                       # 2^-127
>>> round_float(fi, 2.0**-128, RoundMode.TiesToEven, sat=True)
2.938735877055719e-39                       # 2^-128 - not in the format
>>> encode_float(fi, _)
255                                         # NaN

The overflow direction is handled — round_float clamps to fi.max or returns
NaN as sat dictates. Underflow has no matching branch.

Environment

gfloat	0.5.2 (current PyPI release)
Python	3.12.13
platform	irrelevant; round.py is pure Python

Minimal case

from gfloat import RoundMode, encode_float, round_float
from gfloat.formats import format_info_ocp_e8m0 as fi

for rnd in RoundMode.TiesToEven, RoundMode.TowardPositive, RoundMode.TowardZero:
    for sat in (True, False):
        r = round_float(fi, 2.0**-128, rnd, sat)
        print(f"{rnd.name:15} sat={sat!s:5} -> {r!r}  encode -> 0x{encode_float(fi, r):02X}")

TiesToEven      sat=True  -> 2.938735877055719e-39  encode -> 0xFF
TiesToEven      sat=False -> 2.938735877055719e-39  encode -> 0xFF
TowardPositive  sat=True  -> 2.938735877055719e-39  encode -> 0xFF
TowardPositive  sat=False -> 2.938735877055719e-39  encode -> 0xFF
TowardZero      sat=True  -> 2.938735877055719e-39  encode -> 0xFF
TowardZero      sat=False -> 2.938735877055719e-39  encode -> 0xFF

TowardPositive is the clearest contradiction: rounding 2^-128 toward
+infinity in a format whose smallest value is 2^-127 can only give
2^-127, and instead the input comes back untouched.

A value inside the lowest binade shows the shape of it — rounding up rescues
the result, rounding down does not:

round_float(fi, 0.75 * 2**-127, TiesToEven,     sat=True) -> 2^-127   ok
round_float(fi, 0.75 * 2**-127, TowardZero,     sat=True) -> 2^-128   not in the format

Full reproducer (see below): exits non-zero when the bug is present.

Where it comes from

gfloat/round.py, round_float(). The subnormal clamp is conditional:

        # Effective precision, accounting for right shift for subnormal values
        if fi.has_subnormals:
            expval = max(expval, 1 - bias)

For a format with subnormals, that max is what stops expval from going
below the format's floor, and anything smaller then rounds to zero correctly.
For has_subnormals=False there is no lower clamp at all, so
result = isignificand * 2.0**expval is reconstructed at whatever exponent the
input had.

The function then has an # Overflow block:

    amax = -fi.min if sign else fi.max
    if result > amax:
        ...

and no underflow counterpart. round_ndarray() has the same gap.

Suggested fix

Symmetry with the overflow branch, i.e. after rounding:

    if not fi.has_subnormals and result != 0 and abs(result) < fi.smallest:
        # No subnormals and no zero: there is nothing below `smallest`.
        result = fi.smallest if sat else <NaN / ValueError, per the overflow rule>

What the non-sat answer should be is a genuine design question rather than an
obvious one, and it is the maintainers' call:

• clamping to fi.smallest in both cases is the most useful for MX scale
  selection, and matches what the OCP MX reference implementations do;
• mirroring the overflow branch exactly (NaN when the format has one, else
  ValueError) is the most internally consistent.

Either is better than the present behaviour, where the documented postcondition
is violated silently and the value later becomes NaN inside encode_float().

Scope

• Affected: any FormatInfo with has_subnormals=False. Among the shipped
  formats that is ocp_e8m0 alone, but the code is generic — a user-defined
  format declared without subnormals hits it too (the reproducer includes one).
• Affected functions: round_float() and round_ndarray().
• Not affected: formats with subnormals, and the overflow direction of all
  formats.
• Practical impact: E8M0 is the shared scale of the OCP MX formats. A block
  scale that underflows silently becomes NaN, which then poisons every element
  of the block on dequantisation.

Full Reproducer

#!/usr/bin/env python3
"""Reproducer: gfloat's round_float() returns values the target format cannot
represent when the format has no subnormals.

    pip install gfloat
    python3 gfloat_e8m0_underflow.py     # exit 1 if the bug is present

round_float() documents "Returns: A float which is one of the values in the
format." For ocp_e8m0 - which has no subnormals, no zero and a smallest value
of 2^-127 - any input that rounds below 2^-127 is returned unchanged, in every
rounding mode and with sat either way. encode_float() then maps the
unrepresentable value onto code 0xFF, which is NaN.
"""

import sys

from gfloat import FormatInfo, RoundMode, decode_float, encode_float, round_float
from gfloat.formats import format_info_ocp_e8m0

NAN_CODE = 0xFF


def representable(fi, value):
    return any(decode_float(fi, code).fval == value for code in range(2**fi.k)
               if decode_float(fi, code).fval == decode_float(fi, code).fval)


def check(fi, value, rnd, sat, failures):
    result = round_float(fi, value, rnd, sat)
    code = encode_float(fi, result)
    ok = representable(fi, result)
    print(f"  round_float({value!r:>24}, {rnd.name:14}, sat={sat!s:5}) -> "
          f"{result!r:>24}  encode -> 0x{code:02X}"
          f"{'' if ok else '   <-- NOT IN THE FORMAT' + (' (NaN)' if code == NAN_CODE else '')}")
    return failures + (not ok)


def main():
    fi = format_info_ocp_e8m0
    failures = 0

    print(f"format {fi.name}: k={fi.k} precision={fi.precision} "
          f"has_subnormals={fi.has_subnormals}")
    print(f"  smallest = {fi.smallest!r} (2^-127), max = {fi.max!r} (2^127)")
    print(f"  code 0x00 -> {decode_float(fi, 0).fval!r},  "
          f"code 0xFF -> {decode_float(fi, NAN_CODE).fval!r}\n")

    print("inputs below the smallest representable value:")
    for value in (2.0**-128, 2.0**-130, 5e-40):
        for rnd in (RoundMode.TiesToEven, RoundMode.TowardPositive,
                    RoundMode.TowardZero):
            for sat in (True, False):
                failures = check(fi, value, rnd, sat, failures)
        print()

    print("a value between 2^-128 and 2^-127: rounding up saves it, "
          "rounding down does not")
    for rnd in (RoundMode.TiesToEven, RoundMode.TowardPositive, RoundMode.TowardZero):
        failures = check(fi, 0.75 * 2.0**-127, rnd, True, failures)

    print("\nfor contrast, the overflow direction is handled:")
    for sat in (True, False):
        result = round_float(fi, 2.0**128, RoundMode.TiesToEven, sat)
        note = "clamped to max" if sat else "NaN, the format has no infinity"
        print(f"  round_float(2^128, TiesToEven, sat={sat!s:5}) -> {result!r}  ({note})")

    # The same gap reaches any format declared without subnormals, not just
    # the shipped E8M0.
    print("\na user-defined no-subnormal format hits it too:")
    custom = FormatInfo("demo_e4m0", k=4, precision=1, bias=7, is_signed=False,
                        domain=fi.domain, has_nz=False, num_high_nans=1,
                        has_subnormals=False, is_twos_complement=False)
    failures = check(custom, custom.smallest / 2, RoundMode.TiesToEven, True, failures)

    from importlib.metadata import version
    print(f"\ngfloat {version('gfloat')}")
    if failures:
        print("BUG PRESENT")
        return 1
    print("no failures")
    return 0


if __name__ == "__main__":
    sys.exit(main())