Research

What we're making next, and what we've learned so far training small language models from scratch. Every number here comes from our own lab notebook.

Coming next

Our next release is a whole family at once.

What we've learned

Short notes from our experiments, newest first. Some of these are wins; most are things that surprised us.

How we work

The rules we hold ourselves to, learned the hard way.

From scratch

Every Loom starts from random weights. We never fine-tune someone else's model.

Tests first

We write the tests and the pass marks before training, and keep them out of the training data.

Report the misses

When a model misses its bar, we say so on its card instead of moving the bar.

Write it down

Every run, number and mistake goes into the lab notebook, so nothing is learned twice.