What Is TiGrIS
TiGrIS, the Tiled Graph Inference Scheduler, is an ahead-of-time compiler for ML inference on embedded devices. You give it a trained model and the memory your device has. It plans the inference so that it fits, or tells you what stops it.
The problem
Section titled “The problem”Weights can stay in flash, but activations need RAM while the model runs. A single intermediate tensor can outgrow the device’s SRAM even when every weight fits, and reusing an arena cannot shrink a tensor below its own size. The usual answer is a smaller model. TiGrIS keeps the model and changes how its computation is laid out in memory.
How it works
Section titled “How it works”The compiler splits the graph into stages and cuts spatial operators into tiles, by rows or as a 2D grid, so only a small working set is live at any moment. Consecutive tiled stages form chains whose intermediates never leave fast memory. Every decision is made on the host, against the budget you set: fast memory, an optional slower pool such as PSRAM, and flash. Nothing is analyzed or replanned on the device, and a model that cannot meet the budget is refused with the reason instead of compiled best-effort.
The result runs one of two ways today:
- On the TiGrIS runtime. The compiler writes a plan, a
.tgrsfile, and the portable C runtime executes it in arenas you own, with no heap allocation during inference. The kernel library is chosen when the C glue is generated: reference C, CMSIS-NN or ESP-NN. - As a step before the runtime you ship. The compiler rewrites the model itself into row bands and writes the same format back: a TFLite file for TFLite Micro, or an ExecuTorch program for Cortex-M. That runtime then runs it in less RAM with the same outputs.
The compiler reads ONNX, TFLite and PyTorch .pt2 files.
When it helps little
Section titled “When it helps little”- The model already fits. The plan then holds about what a good tensor planner holds; the gain is kilobytes, not a deployment.
- Latency comes first. Tiles recompute the rows they share, and a slow pool adds traffic.
- The model is outside what the compiler accepts: dynamic shapes, int16 activations, or operators missing from the operator support table.
TiGrIS does not train or fine-tune, does not quantize a float model for you, and does not replace kernel libraries: it calls them.
Next: Quickstart takes a model from file to C source, and Core Concepts covers plans, memory pools and tiling in more depth.