MatrixSign
Documentation for MatrixSign, largely based on the paper The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm
API
MatrixSign.msign — Function
msign(X::AbstractArray, method=PolarExpress; kws...)Return the sign of X using the PolarExpress method as the default.
MatrixSign.msign! — Function
msign!(X::AbstractArray, method=PolarExpress; kws...)Return the sign of X using the PolarExpress method as the default, mutating X in-place.
Methods
MatrixSign.PolarExpress — Type
PolarExpressUses a variable-step quintic Newton-Schulz iteration to achieve accuracy close to or equal to SVDMethod given enough steps, depending on the numerical precision.
Reference: The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm
MatrixSign.JordanMethod — Type
JordanMethodUses a 5-step quintic Newton-Schulz iteration to compute a coarse approximation of the sign of a matrix, with singular values roughly ending up in the interval $[0.7, 1.2]$.
Reference: Muon: An optimizer for hidden layers in neural networks
MatrixSign.SVDMethod — Type
SVDMethodUses a costly singular value decomposition to compute the sign of a matrix.
Let $\bold{M} = U\Sigma V^T$ be the singular value decomposition of $\bold{M}$. Then the sign, or more generally polar factor, of $\bold{M}$ is given by $UV^T$.