MatrixSign
Documentation for MatrixSign, largely based on the paper The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm
API
MatrixSign.msign — Functionmsign(X::AbstractArray, method=PolarExpress; kws...)Return the sign of X using the PolarExpress method as the default.
MatrixSign.msign! — Functionmsign!(X::AbstractArray, method=PolarExpress; kws...)Return the sign of X using the PolarExpress method as the default, mutating X in-place.
Methods
MatrixSign.PolarExpress — TypePolarExpressUses a variable-step quintic Newton-Schulz iteration to achieve accuracy close to or equal to SVDMethod given enough steps, depending on the numerical precision.
Reference: The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm
MatrixSign.JordanMethod — TypeJordanMethodUses a 5-step quintic Newton-Schulz iteration to compute a coarse approximation of the sign of a matrix, with singular values roughly ending up in the interval $[0.7, 1.2]$.
Reference: Muon: An optimizer for hidden layers in neural networks
MatrixSign.SVDMethod — TypeSVDMethodUses a costly singular value decomposition to compute the sign of a matrix.
Let $\bold{M} = U\Sigma V^T$ be the singular value decomposition of $\bold{M}$. Then the sign, or more generally polar factor, of $\bold{M}$ is given by $UV^T$.