NBL

Neural Block Linearization replacing self-attention with LMMSE approximations for faster LLM inference.

LLM inference acceleration via selective replacement of self-attention with linear LMMSE approximations. CCA-based error bounding identifies low-error layers without fine-tuning. Achieves ~32% speed-up on Llama-8B with <1% accuracy loss. Supports quantization, LoRA fine-tuning, and speculative decoding.