Microscaling

MXFP4

MXFP4 is a microscaling format using FP4 elements (E2M1, no NaN/Inf), with E8M0 scale factors, each scaling contiguous element blocks of 32.

MXFP6_E2M3

MXFP6_E2M3 is a microscaling format using FP6 elements (E2M3, no NaN/Inf), with E8M0 scale factors, each scaling contiguous element blocks of 32.

MXFP6_E3M2

MXFP6_E3M2 is a microscaling format using FP6 elements (E3M2, no NaN/Inf), with E8M0 scale factors, each scaling contiguous element blocks of 32.

MXFP8_E4M3

MXFP8_E4M3 is a microscaling format using FP8 elements (E4M3, including NaN), with E8M0 scale factors, each scaling contiguous element blocks of 32.

MXFP8_E5M2

MXFP8_E5M2 is a microscaling format using FP8 elements (E5M2, including NaN/Inf), with E8M0 scale factors, each scaling contiguous element blocks of 32.

MXINT8

MXINT8 is a microscaling format using INT8 elements (Int8, scaled down by 64), with E8M0 scale factors, each scaling contiguous element blocks of 32.

NVFP4

NVFP4 is a microscaling format using FP4 elements (E2M1, no NaN/Inf), with E4M3 scale factors, each scaling contiguous element blocks of 16.

BlockFormat{E,S,k}

A block format specifies the element type, scale type, and number of elements per block.

quantize(V::AbstractArray, format::BlockFormat{E,S,k}; method=GenericMethod(), axis=:column)

Quantize the input array V to the given block format format.

Arguments

• V::AbstractArray: The input array to quantize.
• format::BlockFormat{E,S,k}: The block format to quantize to.
• method::Method: The method to use for quantization.
• axis::Symbol: The axis to quantize along. Must be :column or :row.

If :row, the first two dimensions are transposed such that the blocks are contiguous along the first dimension.

Returns

• X::AbstractArray: The scale factors.
• P::AbstractArray: The quantized values.