Labs21
Stay curious.
Small models. Questions worth following.
A cinematic journey past the solar limb, Earth, and Moon, ending with Earth above Lumen 1.9.
Labs21 model
Lumen 1.9
A small model built to keep going when the question gets difficult.
Start hereStay Curious.
Explore Labs21
04 places to startBuilt here. Still learning.
I'm building and updating these models on an RTX 5080. The aim is simple: keep making them more useful, and make them affordable to run.
Bounded Dynamic Highway
Saga is moving to a looped network that replaces attention, RoPE, and large KV caches with bounded hidden dynamics. Each block keeps four recent hidden states, mixes them across grouped temporal branches, then repeats the same physical depth.
- State
- 4 cached
- Branches
- 12 temporal
- Depth
- 28 × 2 passes
- Target
- RTX 5080