# Compiling a TFLite Model

TiGrIS compiles a `.tflite` file as it is. `tigris analyze` and `tigris compile`
accept it in place of an ONNX model; there is no separate conversion step.

```bash
tigris analyze model.tflite -m 256K
tigris compile model.tflite -m 256K -o model.tgrs
```

The model is read into the same pipeline an ONNX model goes through, so
tiling, memory budgets and code generation behave the same way. A model that
uses an operator or a form the compiler cannot run is refused before
compiling, with the operator's index, its TFLite name and the reason:

```text
operator 12 STRIDED_SLICE: a zero stride
```

## What the plan keeps from the model

- **Interface dtypes.** An int8 model takes int8 inputs and returns int8
  outputs, as TFLite hands them over; a float32 model stays float32. An int8
  model the converter wrote with uint8 inputs and outputs keeps them: the plan
  holds int8 and converts at the boundary. ARG_MAX
  and ARG_MIN outputs and index inputs are int32, and comparison results are
  bool.
- **Interface shapes.** Inputs and outputs keep TFLite's channels-last axis
  order, for example `1x96x96x3` for an image.
- **Results.** A committed one-operator corpus checks int8 and float32 results
  against TFLite Micro. GELU uses TFLite's reference kernels because TFLite Micro
  does not register it; its int8 results are bit-exact and its float results use
  a float tolerance. Other reference-kernel exceptions are listed with the
  corpus in the compiler repository. The MLPerf Tiny reference
  models (keyword spotting, visual wake words, image classification and
  anomaly detection) produce TFLite Micro's int8 outputs byte for byte, tiled
  and untiled.

## Model requirements

- Custom operators do not convert, except `TFLite_Detection_PostProcess`.
- Activations int8 or float32. An int8 plan can contain float sections after
  `DEQUANTIZE`, ending at `QUANTIZE` or a float model output. Bool and int32
  tensors appear only where the operators below produce or take them, and
  detection results can also be float32 in an int8 model. int16 activations, uint8 activations other
  than at the model boundary, and hybrid models (int8 weights with float32
  activations) do not convert.
- int8 activations quantized per tensor; weights constant, int8 weights
  symmetric, per channel or per tensor.
- Fused activations `NONE`, `RELU` or `RELU6`.
- Shapes fixed in the file. `SHAPE`, `ZEROS_LIKE`, `FILL` and
  `BROADCAST_ARGS` convert when everything they read is fixed in the file:
  they become constants. So does arithmetic on such constants through
  `STRIDED_SLICE`, `SLICE`, `PACK`, `CONCATENATION`, `RESHAPE`, `SQUEEZE`,
  `EXPAND_DIMS` and int32 `ADD`, `SUB` and `MUL`, which is how Keras exports a
  `Reshape` that keeps the batch axis. A `FILL` of a value computed at run time
  does not convert.

## Supported operators

int8 and float32 unless the table says otherwise. Operators that take
indices, axes, padding or bounds need them as constants in the file.

### Convolution, pooling and matrix products

| Operator | Limits |
|---|---|
| `CONV_2D`, `DEPTHWISE_CONV_2D`, `TRANSPOSE_CONV` | |
| `FULLY_CONNECTED` | shuffled weight format refused |
| `BATCH_MATMUL` | operands of equal rank |
| `AVERAGE_POOL_2D`, `MAX_POOL_2D` | |
| `L2_POOL_2D` | float32 only, as in TFLite Micro |

### Elementwise

| Operator | Limits |
|---|---|
| `ADD`, `SUB`, `MUL`, `DIV`, `MAXIMUM`, `MINIMUM`, `SQUARED_DIFFERENCE` | operands broadcast from the right, as in TFLite; `ADD`, `SUB` and `MUL` also on int32, wrapping on overflow, without a fused activation |
| `ABS`, `RSQRT` | |
| `NEG`, `EXP`, `LOG`, `SQRT`, `SQUARE`, `FLOOR`, `CEIL`, `ROUND`, `SIN`, `COS`, `FLOOR_DIV`, `FLOOR_MOD` | float32 only, as in TFLite Micro |
| `ADD_N` | int8 inputs share one quantization; refused where the int8 accumulator could overflow, from 16 inputs up |
| `RELU`, `RELU6`, `LOGISTIC`, `TANH`, `HARD_SWISH`, `LEAKY_RELU`, `ELU`, `PRELU`, `GELU` | `PRELU` slope constant |

### Normalization

| Operator | Limits |
|---|---|
| `SOFTMAX` | over the last axis; beta 1 |
| `LOG_SOFTMAX` | over the last axis |
| `L2_NORMALIZATION` | over the last axis; no fused activation |

### Reductions and scans

| Operator | Limits |
|---|---|
| `MEAN` | constant axes |
| `SUM`, `REDUCE_MAX`, `REDUCE_MIN` | constant axes, adjacent to each other |
| `REDUCE_ALL` | bool input; constant adjacent axes |
| `CUMSUM` | one constant axis |
| `ARG_MAX`, `ARG_MIN` | int32 output, used as a model output or as the indices of the operators below |

### Recurrent

| Operator | Limits |
|---|---|
| `SVDF` | a bias, which TFLite Micro requires; float32 with Relu or Relu6, or int8 with int16 state and time weights |
| `UNIDIRECTIONAL_SEQUENCE_LSTM` | every gate; no peepholes, projection or layer normalization, which TFLite Micro does not run; tanh cell activation; float32, or int8 with a symmetric int16 cell and per-tensor weights |

### Control flow

| Operator | Limits |
|---|---|
| `IF` | a one-element bool condition; float32 or int32 operands; no variables or recurrent operators inside a branch |
| `WHILE` | float32 or int32 loop variables, from inputs or constants; a condition subgraph giving one bool; no variables or recurrent operators inside; a loop starting from a constant does not compress with `-c lz4` |

### Detection

| Operator | Limits |
|---|---|
| `TFLite_Detection_PostProcess` (custom) | float32 anchors in the file; box encodings `[1, boxes, 4]`, float32 or dequantized from int8; fast or regular non-max suppression; one class per detection in the fast form; its four float32 results are model outputs no operator reads |

The detections match TFLite Micro's bit for bit, including its order among
equal scores. In the fast form, rows past the detection count are zero, where
TFLite Micro leaves them unwritten. The operator runs untiled and holds working
memory of about 41 bytes per box during its stage.

### Comparison, logic and selection

| Operator | Limits |
|---|---|
| `EQUAL`, `NOT_EQUAL`, `LESS`, `LESS_EQUAL`, `GREATER`, `GREATER_EQUAL` | bool output; int8 input scales below 1; int32 inputs |
| `LOGICAL_AND`, `LOGICAL_OR`, `LOGICAL_NOT` | bool |
| `SELECT_V2` | int8 values and output share one quantization |
| `CAST` | bool to float32, or bool to int8 with scale 1 and zero point 0; int32 to float32 |

### Shape and data movement

| Operator | Limits |
|---|---|
| `RESHAPE`, `SQUEEZE`, `EXPAND_DIMS`, `TRANSPOSE` | |
| `CONCATENATION`, `SPLIT`, `SPLIT_V`, `PACK`, `UNPACK` | int8 inputs share the output's quantization |
| `PAD`, `PADV2`, `MIRROR_PAD` | |
| `SLICE`, `STRIDED_SLICE` | rank 4 at most; no ellipsis, new axes or offset; no zero stride; a dropped axis needs a positive stride |
| `GATHER`, `GATHER_ND`, `EMBEDDING_LOOKUP` | the table computed or constant; indices constant, or int32 from a model input, `ARG_MAX`/`ARG_MIN` or int32 `ADD`/`SUB`/`MUL`, also through `CAST`, `RESHAPE` and `TRANSPOSE`; an int64 model input that only feeds indices is read as int32 unless the model records a range beyond it; an index outside its axis stops the run with an error |
| `REVERSE_V2` | adjacent axes |
| `BROADCAST_TO`, `DYNAMIC_UPDATE_SLICE` | start indices constant, or int32 from a model input, `ARG_MAX`/`ARG_MIN` or int32 `ADD`/`SUB`/`MUL`; starts are clamped so the update fits, as in TFLite |
| `SPACE_TO_DEPTH`, `DEPTH_TO_SPACE`, `SPACE_TO_BATCH_ND`, `BATCH_TO_SPACE_ND` | rank-4 input |
| `RESIZE_BILINEAR`, `RESIZE_NEAREST_NEIGHBOR` | not `align_corners` and `half_pixel_centers` together |
| `QUANTIZE`, `DEQUANTIZE` | float32 or int8 to int8, int8 to float32; uint8 to int8 behind a model input and int8 to uint8 before a model output |

## Memory

Reductions, scans, `ARG_MAX`/`ARG_MIN` and the index-based data movement
operators (`GATHER`, `GATHER_ND`, `EMBEDDING_LOOKUP`, strided slices,
`MIRROR_PAD`, `REVERSE_V2`, `DYNAMIC_UPDATE_SLICE`) tile in bands along an axis
they do not reduce, index or reverse. A constant table can tile independent
batches of indices computed at run time. Where no valid band exists, the
operator runs on whole tensors, so its stage needs room for its input and output
at once. The same holds for a broadcast that is not a simple
repetition, such as `[1, 6, 6, 4]` against `[6, 1, 4]`, and for `SVDF` and
`UNIDIRECTIONAL_SEQUENCE_LSTM`. A `RESHAPE` tiles when each band of its input
rows maps onto whole rows of its output. Contiguous gathers, stride-1 slices and
the elementwise, convolution and pooling operators tile.
`tigris analyze` reports which stages tile and how much fast memory the plan
needs; see [When a Model Does Not Fit](/0.12.0/guides/model-does-not-fit/).

## State

Resource variables (`VAR_HANDLE`, `READ_VARIABLE`, `ASSIGN_VARIABLE`) in float32
models keep their values from one invocation to the next, as in TFLite Micro.
A `CALL_ONCE` subgraph that assigns constants sets their initial values. The
variable tensors of `SVDF` and `UNIDIRECTIONAL_SEQUENCE_LSTM` are kept the same
way, starting at zero, or at the zero point for an int8 hidden state. The plan
holds all of them in a state buffer the caller supplies next to the arenas:
`tigris_state_required()` gives its size, `tigris_state_init()` writes the
initial values, and `tigris_run_with_state()` runs on it. In Python,
`Session.reset_state()` starts the variables over. A `tigris codegen`
application allocates the buffer itself; the generated core states its size as
`<CORE>_STATE_BYTES` and runs through `<core>_run_with_state()`.

## Control flow

An `IF` runs the subgraph its condition selects, and a `WHILE` runs its body for
as long as its condition subgraph gives true, as in TFLite Micro. Each subgraph is
planned as a model of its own under the same memory budget and runs as its own
stages; the operands are copied in and the results copied out. A `WHILE` has no
iteration limit, as in TFLite Micro. Control flow may nest inside a branch or loop
body, up to the runtime's `TIGRIS_MAX_SUBGRAPH_DEPTH` (4 by default) operators
deep and `TIGRIS_MAX_SUBGRAPHS` (16 by default) graphs in all.

In int8 models the converter keeps the boundary of an `IF` or `WHILE` in
float32, with a `QUANTIZE` and a `DEQUANTIZE` on either side, each subgraph in its
own encodings. The plan keeps those conversions and computes each one as TFLite
Micro does, so the int8 results match bit for bit. The float32 boundary values
take four bytes per element in slow memory, and the conversions can tile with the surrounding operators.

## Not yet supported

int8 resource variables are refused by name.