I started this to work out what PyTorch, tinygrad and TVM are actually doing underneath, and it turned into something you can train small models with.
The central idea is that a model is not Python code, it is a graph of tensor operations — the IR. That same graph is what gets evaluated, what gets differentiated, what gets written to disk, and what gets handed to a backend. So you can save a model and load it again with no model code present at all, because the structure was never in the Python in the first place.
Reverse-mode autodiff is one file you can follow from top to bottom. That was the point: most frameworks hide the interesting part behind a wall of C++, and the whole reason to write your own is to be able to read it.
The core is Rust, exposed to Python through maturin, with the API deliberately shaped like PyTorch so the examples are legible to anyone who has used it. It is not trying to beat ONNX Runtime on speed and the README says so.