LION

Framework converting linear-attention Transformers into bidirectional RNN equivalents.

Framework converting linear-attention Transformers to bidirectional RNN equivalents for fast training and efficient inference. Provides three variants (LION-Lit, LION-D, LION-S) for image classification and masked language modeling, with configurable size, masking, patch order, and format. Achieves competitive accuracy with reduced training time.