diff --git a/docs/quant_4bit_formats.md b/docs/quant_4bit_formats.md index 7b07058..ead5d1b 100644 --- a/docs/quant_4bit_formats.md +++ b/docs/quant_4bit_formats.md @@ -4,9 +4,9 @@ This document specifies the **Q40NL**, **Q41NL**, **Q42NL**, and **Q43NL** forma **Q40NL** is a **4.5 bits/weight** format with **non-linear decode**, designed to improve tail reconstruction while keeping the same storage as classic Q40/Q4_0 and IQ4_NL. **Q40NL** uses **4-bit signed symmetric codes** with a **per-block FP16 scale** and a non-linear decode function that allocates more resolution toward the tails. **Q40NL** is a **drop-in replacement** for Q40/Q4_0, with the same 18-byte block size (32 weights) and no external element LUTs. -**Q41NL** is an **alternative 4.5 bits/weight** format with the same block layout, per-block FP16 scale and kernels as Q40NL (18 bytes per 32 weights; no element LUTs). It differs only in the non-linearity, using \(f_{41}(x)=x\,|x|\) with inverse \(f_{41}^{-1}(y)=\mathrm{sign}(y)\sqrt{|y|}\), which further increases tail emphasis compared to Q40NL’s \(f(x)=\tfrac12(x|x|+x)\). Like Q40NL, it is a drop-in replacement for Q40/Q4_0 and IQ4_NL. +**Q41NL** is an **alternative 4.5 bits/weight** format with the same block layout, per-block FP16 scale and kernels as Q40NL (18 bytes per 32 weights; no element LUTs). It differs only in the non-linearity, using $\(f_{41}(x)=x\,|x|\)$ with inverse $\(f_{41}^{-1}(y)=\mathrm{sign}(y)\sqrt{|y|}\)$, which further increases tail emphasis compared to Q40NL’s $\(f(x)=\tfrac12(x|x|+x)\)$. Like Q40NL, it is a drop-in replacement for Q40/Q4_0 and IQ4_NL. -**Q42NL** extends the Q4*NL family with a **parametric non-linearity**: 18 bytes per 32 weights (16 nibbles + 1 FP8(E5M2) scale + 1 int8 curve parameter **c**). The decode is \(y=(1-c)x+c|x|x\), where **c** is optimized per-block to minimize MSE. This allows each block to adapt its nonlinearity between linear (c=0) and quadratic-tail (c=1), achieving better reconstruction than fixed-curve Q40NL/Q41NL. +**Q42NL** extends the Q4*NL family with a **parametric non-linearity**: 18 bytes per 32 weights (16 nibbles + 1 FP8(E5M2) scale + 1 int8 curve parameter **c**). The decode is $\(y=(1-c)x+c|x|x\)$, where **c** is optimized per-block to minimize MSE. This allows each block to adapt its nonlinearity between linear (c=0) and quadratic-tail (c=1), achieving better reconstruction than fixed-curve Q40NL/Q41NL. **Q43NL** uses the same parametric approach as Q42NL but stores a **per-block FP16 scale** (instead of FP8) for higher precision: 19 bytes per 32 weights (16 nibbles + 2 FP16 scale + 1 int8 curve **c**). Both Q42NL and Q43NL support three optimization methods for finding optimal **c**: gradient-based (6x faster, 99.5-99.7% quality), coarse-to-fine grid (1.5x faster, 99.97% quality), or full grid search (slowest, 100% quality).