Conversion

Every conversion reduces to one function, Microfloats.cvt. Constructors, convert and broadcasts only resolve defaults and call it, so there is one place to specialize a conversion and one place for a device backend to override it.

DirectionScalarVector
into a Microfloatcvt(T, x, mode, policy)cvt(SVector{N,T}, xs, mode, policy), cvt(NVector{T,N}, xs, mode, policy)
out of a Microfloatcvt(F, x)cvt(SVector{N,F}, xs)

Narrowing takes a rounding mode and an overflow policy as positional arguments, so methods can dispatch on them. Widening into Float16, BFloat16, Float32 or Float64 takes neither: every Microfloat is exactly representable in BFloat16.

julia> using Microfloats: cvt, SAT

julia> cvt(Float8_E4M3FN, 1000f0, RoundNearest, SAT)
Float8_E4M3FN(448.0)

julia> cvt(Float32, Float8_E4M3FN(1.5))
1.5f0
Microfloats.cvt — Function
cvt(::Type{T}, x, mode::RoundingMode, policy::OverflowPolicy) -> T

Central conversion funnel. Every scalar conversion into a Microfloat — constructors, convert, broadcasts, and the packed vector paths — reduces to a call of this function, with the rounding mode and overflow policy as positional, dispatchable arguments.

cvt is the extension surface for optimized conversions. To specialize, add a method on any subset of (T, typeof(x), mode, policy):

  • Bit-twiddling / table specializations add ordinary methods, e.g. Microfloats.cvt(::Type{Float8_E4M3}, x::Float4_E2M1FN, ::RoundingMode, ::OverflowPolicy). See @cvt_table for a generated lookup-table shortcut.
  • Device backends (package extensions) use overlay method tables (e.g. CUDACore.@device_override) on exactly the (T, source, mode, policy) signatures the hardware supports natively; every other combination falls through to the portable methods below.

The always-correct reference path is cvt_generic; specialized methods that need a partial fallback should call it (not cvt, which on overlay method tables would recurse into the override itself).

source
cvt(::Type{F}, x::Microfloat) -> F

Widening half of the conversion funnel: every conversion out of a Microfloat into Float16, BFloat16, Float32 or Float64 reduces to a call of this method, the same way conversions into a Microfloat reduce to the four-argument form.

Every Microfloat is exactly representable in BFloat16 (at most 7 significand bits and 8 exponent bits), and so in Float32 and Float64: widening never rounds, which is why this form takes no rounding mode or overflow policy. Float16 has a narrower exponent range than some formats (Float8_E8M0FNU); it receives the exact Float32 value rounded by Float16(::Float32).

@microfloat registers a method per type that computes it with a branch-free bit-twiddle over the destination's bits where one is cheap (see max_twiddle_cost), and with the generic lookup otherwise; Float64 extends the Float32 result. Like the narrowing form this is the extension surface for device backends, which override it for the destinations their hardware widens to natively.

source
cvt(::Type{SVector{N,T}}, xs::NTuple{N,Any}, mode, policy) -> SVector{N,T}
cvt(::Type{NVector{T,N}}, xs::NTuple{N,Any}, mode, policy) -> NVector{T,N}

Vector forms of the conversion funnel: convert N source lanes in one call, into one byte per lane (SVector) or densely packed (BitPacking.NVector). The SVector default is lanewise (cvt_lanes); the NVector default converts through the SVector form and packs. Packing is layout only and folds away, so specialize the SVector form for multi-lane hardware conversions (e.g. PTX cvt x2 instructions in device overlays): that serves both destinations. Specialize the NVector form only for conversions that work on dense storage directly, such as bit-twiddling over packed sources.

Source containers (SVector, NVector, any StaticArray vector) normalize to NTuple first; packed→packed specializations may intercept the NVector-source signature before it is unpacked.

source
cvt(::Type{SVector{N,F}}, xs::NTuple{N,T}) -> SVector{N,F}

Vector form of the widening funnel: N lanes of microfloat T to N wide lanes (F is Float16, BFloat16, Float32 or Float64). Packed (NVector) and unpacked (SVector) sources normalize to a tuple of lanes; the default is lanewise through the scalar widening funnel. Specialize it for multi-lane hardware conversions, e.g. PTX cvt.rn.bf16x2.e4m3x2 in device overlays. Packing and unpacking a source is layout only and folds away, so a specialization on the tuple form serves every source container.

source
Microfloats.cvt_generic — Function
cvt_generic(::Type{T}, x::Float32, mode::RoundingMode, policy::OverflowPolicy) -> T

The generic reference implementation behind cvt: bit-level rounding from Float32 into any Microfloat layout, for every supported rounding mode and overflow policy. Specialized cvt methods (and device overrides) call this directly when their fast path does not apply.

source
Microfloats.cvt_twiddle — Function
cvt_twiddle(::Type{T}, x::Float32, mode::RoundingMode, policy::OverflowPolicy) -> T

Branch-free narrowing from Float32, the default cvt method for Float32 sources (and so for every source that converts to Float32 first). Results, and errors, are identical to cvt_generic, which it calls for modes it does not cover.

Target normals round the Float32 bits directly: an integer add rounds the significand at the target's last significand bit, carrying into the exponent, and a subtraction rebiases. Target subnormals round |x| scaled to units of the smallest subnormal, which the FPU does exactly. Everything else is a select, so the only branches are the error checks, which the compiler removes for formats that cannot fail (e.g. signed formats with NaN).

source
Microfloats.cvt_lanes — Function
cvt_lanes(::Type{SVector{N,T}}, xs::NTuple{N,Any}, mode, policy) -> SVector{N,T}
cvt_lanes(::Type{NVector{T,N}}, xs::NTuple{N,Any}, mode, policy) -> NVector{T,N}

Reference lanewise implementation of the vector conversion funnel: converts each lane through the scalar cvt funnel (so per-lane specializations and device overrides still apply), then builds the vector. Vectorized cvt specializations call this when their fast path does not cover the requested combination.

source
Microfloats.WideFloat — Type
WideFloat

The floating-point destinations of the widening funnel: Float16, BFloat16, Float32 and Float64.

source

Vectors of lanes

Two static vector types hold N values of one format:

  • SVector{N,T} (StaticArrays) gives each value its own byte.
  • NVector{T,N} (BitPacking) packs the values densely, bitwidth(T) bits each.

For 8-bit formats the two have the same bits. They differ for narrower formats, and both layouts occur in hardware: a 4-bit pair is one byte, while a 6-bit pair is two bytes with each value in the low six bits of its own byte. The aliases name the layout NVIDIA's packed types (__nv_fp8x2_e4m3, __nv_fp6x2_e2m3, __nv_fp4x2_e2m1, …) and the PTX cvt instructions use:

AliasTypeStorage
Float8x2_E4M3FN, Float8x4_E4M3FNNVector{Float8_E4M3FN,N}UInt16, UInt32
Float8x2_E5M2, Float8x4_E5M2NVector{Float8_E5M2,N}UInt16, UInt32
Float8x2_E8M0FNU, Float8x4_E8M0FNUNVector{Float8_E8M0FNU,N}UInt16, UInt32
Float6x2_E2M3FN, Float6x4_E2M3FNSVector{N,Float6_E2M3FN}one byte per value
Float6x2_E3M2FN, Float6x4_E3M2FNSVector{N,Float6_E3M2FN}one byte per value
Float4x2_E2M1FN, Float4x4_E2M1FNNVector{Float4_E2M1FN,N}UInt8, UInt16
Float16x2, BFloat16x2, …SVector{N,Float16}, SVector{N,BFloat16}two bytes per value

Densely packed 6-bit storage, as in memory, is NVector{Float6_E2M3FN,N}.

Both vector types construct from any static vector of lanes, and widen back:

julia> using Microfloats: SVector, NVector, SAT

julia> xs = SVector(0.5f0, 1f0, -6f0, 100f0);

julia> packed = NVector{Float4_E2M1FN,4}(xs; overflow=SAT);

julia> sizeof(packed)
2

julia> Tuple(SVector{4,Float32}(packed))
(0.5f0, 1.0f0, -6.0f0, 6.0f0)

Packing is layout only. The packed narrowing form converts through the SVector form and packs, and packed widening sources unpack to a tuple of lanes, so a specialization of the unpacked form serves both.

Specializing a conversion

Optimized implementations are ordinary methods on the funnel, chosen by dispatch:

  1. Microfloats.cvt_generic is the bit-level reference path. Microfloats.cvt_twiddle computes the same results branch-free and is the default for Float32 sources, and so for every source that converts to Float32 first.
  2. Microfloats.@cvt_table compiles the generic results for one pair of microfloat types into a bit-twiddle or a lookup table. Every built-in pair has one.
  3. Hand-written methods.
  4. Device overrides in package extensions.

@cvt_table runs every source bit pattern through the generic path and fits the results with linear pieces (i << k) + c over the source bits i. Exact widenings, such as Float4_E2M1FN to Float8_E4M3FN, are one piece over the source normals and one per binade of source subnormals; saturated and NaN results are constant pieces. A combination with few pieces becomes a branch-free twiddle, which vectorizes and so also serves packed sources and destinations of any length:

# Float4_E2M1FN => Float8_E4M3FN, any rounding mode and overflow policy
i = reinterpret(UInt8, x) & 0x07
t = ifelse(i >= 0x02, (i << 2) + 0x30, ifelse(i >= 0x01, (i << 3) + 0x28, 0x00))
t |= (reinterpret(UInt8, x) & 0x08) << 4

Everything else is a lookup. Microfloats.max_twiddle_cost sets the cutoff; the CUDA extension lowers it, since on the device a cached table load beats all but the shortest twiddles.

Widening is generated the same way, without registration: @microfloat fits the conversions of each new type into Float16, BFloat16 and Float32 over the destination's bits. Normals are one linear piece. Zero and all subnormals share one piece evaluated on the FPU: the subnormal significand placed under the exponent of $2^{1-\text{bias}}$, minus $2^{1-\text{bias}}$, is the exact value, normalized. Float64 extends the Float32 result.

Microfloats.@cvt_table — Macro
@cvt_table Src => Dst

Register an optimized method on the conversion funnel cvt for converting microfloat Src values to microfloat Dst.

Expands to a @generated method of Microfloats.cvt. Once per (mode, policy) combination actually used, it runs every Src bit pattern through cvt_generic and compiles the resulting table into the cheapest equivalent code, so results are identical to the generic path:

  • a branch-free bit-twiddle when the table is piecewise linear in a few pieces, as for exact widenings like Float4_E2M1FN => Float8_E4M3FN (each piece is a shift and an add, selected by comparing the source bits);
  • otherwise a single 2^bitwidth(Src)-entry table lookup.

Which counts as "a few" is max_twiddle_cost, which device backends lower.

Combinations where the generic path throws (e.g. signed source into unsigned target) keep the runtime path and its errors.

Invoke after both types are defined. Microfloats registers methods for all pairs of built-in types; user-defined @microfloat types can opt in:

@microfloat MyFloat6 exponent=3 significand=2
Microfloats.@cvt_table MyFloat6 => Float8_E4M3
Microfloats.@cvt_table Float8_E4M3 => MyFloat6

To hand-optimize a pair instead, define the Microfloats.cvt method for it directly rather than invoking @cvt_table for that pair.

source
Microfloats.max_twiddle_cost — Function
max_twiddle_cost() -> Int

Largest cost of a bit-twiddle that generated conversion methods (see @cvt_table) prefer over a table lookup: its pieces, plus one if it moves a sign bit. Each piece is a shift, an add, a compare and a select. On CPUs the twiddle vectorizes where a lookup is a gather, and wins by a wide margin up to about twice this many pieces. Device backends override it: on GPUs a load from a small constant table hits the read-only cache and beats all but the shortest twiddles.

source

CUDA

With CUDACore loaded, kernels and broadcasts lower conversions to the native PTX cvt instructions where the compile target has them, and run the generic path otherwise. The choice is made when the kernel is compiled.

ConversionOperandsTarget
to Float8_E4M3FN, Float8_E5M2Float32, Float16sm_89 and newer
to Float8_E4M3FN, Float8_E5M2BFloat16sm_100 and newer, arch or family target
to Float6_E2M3FN, Float6_E3M2FN, Float4_E2M1FNFloat32, Float16, BFloat16sm_100 and newer, arch or family target
to Float8_E8M0FNU (RoundToZero, RoundUp)Float32, BFloat16sm_100 and newer, arch or family target
from Float8_E4M3FN, Float8_E5M2Float16sm_89 and newer
from any of the six formatsFloat16, BFloat16, Float32sm_100 and newer, arch or family target

Float8_E8M0FNU has no native Float16 form. Float32 widens through BFloat16.

Hardware narrowing always saturates, so native narrowing applies to the SAT policy with RoundNearest (or the two listed modes for Float8_E8M0FNU); everything else takes the branch-free cvt_twiddle path. Vector forms use one instruction per two lanes for any even N. Native and generic results agree bit for bit, apart from NaN payloads.