This content is for v0.15.0. Switch to the latest version for up-to-date documentation.
tigris inspect
Read what a TFLite, ONNX or PyTorch model contains, or what an ExecuTorch program
or a compiled .tgrs plan records, without compiling or executing anything.
tigris inspect MODEL [OPTIONS]The format is detected from the file contents, so the extension does not matter.
Options
Section titled “Options”| Flag | Type | Description |
|---|---|---|
MODEL |
path | TFLite model, ONNX model, PyTorch program (.pt2), ExecuTorch program (.pte) or .tgrs plan |
-v, --verbose |
flag | Adds the per-step or per-stage tables and the full tensor listings |
--json |
flag | Complete, versioned metadata as JSON |
Models
Section titled “Models”For a model, inspect reports the interface, what each operator type costs, the
largest activations and, for TFLite, how the model is quantized and whether it
converts to a plan.
tigris inspect ds_cnn_i8.tfliteds_cnn_i8.tflite TFLite, model main, schema 3, 11 operators, 41.23 KiB input serving_default_input:0 1x49x10x1 int8 output StatefulPartitionedCall_1:0 1x12 int8
operators count weights MACs CONV_2D 5 19.75 KiB 83% 2.37 M 89% DEPTHWISE_CONV_2D 4 3.25 KiB 14% 288.00 K 11% FULLY_CONNECTED 1 768 B 3% 768 <1% MEAN 1 8 B <1% total 11 23.76 KiB 2.66 M
largest activations 1x25x5x64 int8 7.81 KiB step 0, CONV_2D 1x25x5x64 int8 7.81 KiB step 1, DEPTHWISE_CONV_2D 1x25x5x64 int8 7.81 KiB step 2, CONV_2D
quantization int8 activations; int8 weights per channel, symmetricconverts yes-
weightscounts every constant an operator type reads, where it is read. A weight that reaches its operator through other operators on constants, such as an ONNXQuantizeLinearandDequantizeLinearpair, counts at the operator that uses it. -
MACscounts the multiply-accumulates of convolutions, fully connected layers and matrix products from their shapes. It is a measure of size, not of latency. -
largest activationslists the three largest tensors the model computes or takes as input, and the step that writes each. -
convertssays whetheranalyzeandcompileaccept a TFLite model, and if not, every reason. For the TFLite Micro Ethos-U example, whose single operator is a custom one:converts no, 1 reasonoperator 0 CUSTOM:ethos-u: not supportedFor an ONNX model,
analyzenames the operators it cannot compile.
With -v, a steps table lists every operator with its output shape and size, the
weights it reads and its MACs, followed by the constants.
An ONNX model is read as it declares itself: shapes are not inferred and external
tensor data is not loaded. A figure that needs an undeclared shape shows as ?.
A PyTorch program saved with torch.export.save is exported to ONNX with
torch.onnx.export, which needs torch and onnxscript, and reported as that
ONNX model. The ONNX keeps the quantize and dequantize operators of a converted
PT2E program and declares float32 at both ends; analyze and compile fold them
into int8 operators:
mbv2.pt2 PyTorch program, exported to ONNX, model main_graph, IR 10, opsets ai.onnx 20, 340 operators, 3.14 MiB input x 1x3x96x96 float32 output dequantize_per_tensor_78 1x1000 float32
operators count weights MACs Conv 52 456.91 KiB 27% 10.65 M 89% Gemm 1 1.22 MiB 73% 1.28 M 11% QuantizeLinear 67 274 B <1% Reshape 1 16 B <1% Clip 35 8 B <1% DequantizeLinear 173 Add 10 AveragePool 1 total 340 1.67 MiB 11.93 M
largest activations 1x48x48x48 float32 432.00 KiB step 21, Conv 1x48x48x48 float32 432.00 KiB step 22, Clip 1x48x48x48 float32 432.00 KiB step 24, DequantizeLinear
initializers 306 dense, 0 sparse, 0 external
Declared metadata: shapes are not inferred, external data is not loaded; ? marks a figure an undeclared shape hides.ExecuTorch programs
Section titled “ExecuTorch programs”For an ExecuTorch program lowered for Cortex-M, inspect reports its method,
the kernels it was lowered for, its interface and the RAM the program records:
tigris inspect mbv2.ptembv2.pte ExecuTorch program, 64 instructions, CMSIS-NN DSP kernels (Cortex-M4, M7, M33, M35P) input 0 1x96x96x3 int8, dim order 0,2,3,1 output 0 1x1000 int8
ExecuTorch RAM on Cortex-M, from the program model memory 135.00 KiB activations and CMSIS-NN scratch method allocator 23.75 KiB ExecuTorch's records for the method- The first line counts the instructions of the method
forwardand names the kernel class the program was lowered for. - Each input and output shows its shape in memory order, the order in which a
.binfile fortigris runholds the elements.dim orderfollows when PyTorch’s order differs: this input is the 1x3x96x96 tensor PyTorch takes, kept channels last. model memoryis the planned buffer andmethod allocatorwhat ExecuTorch takes from the method allocator to load the method, both for a 32-bit Cortex-M target.analyzeadds the temp allocator and whatexportsaves.
--json reports format pte and a program object with the method, the
instructions, the cmsis_nn_classes, the inputs and outputs with their
shape in PyTorch’s sizes and their dim_order, the memory in bytes and the
operators.
Compiled plans
Section titled “Compiled plans”For a .tgrs plan, inspect reports the plan schema, the interface in stored axis
order and declared dtype, the weights per operator type, the memory the plan needs and
whether a default runtime build can load it. This is the interface
tigris run expects its inputs in.
tigris inspect mobilenet.tgrsmobilenet.tgrs plan, model mobilenet_v1_matched, schema 12, 31 operators, 13 stages, 12 tiled, 3.18 MiB input input 1x128x128x3 float32 stored int8, NHWC output l33_dq 1x10 float32 stored int8, model order
operators count weights Conv 15 3.03 MiB 98% DepthwiseConv 13 62.97 KiB 2% Flatten 1 GlobalAveragePool 1 Softmax 1 total 31 3.09 MiB
memory fast arena 64.00 KiB unscheduled 384.00 KiB weights 3.09 MiB read in place (XIP), uncompressed
runtime build default limits sufficefast arenais what the runtime requires for the plan’s fast arena. For an LZ4 plan it includes the buffer that decompressed weights occupy, sized for the largest tensor alignment the runtime supports, so on a target with smaller alignment it is an upper bound.unscheduledis the activation peak the compiler recorded for the model run as one stage.state, shown for a stateful plan, is the memory kept between runs.runtime buildnames each-Dlimit the plan needs above the runtime’s default, or says the defaults suffice.
With -v, a stages table lists each stage’s operators, its peak before tiling, the
weights it reads and how it tiles, followed by every operator’s parameters and the
stored tensors. A stage tiled on both spatial axes shows as N tiles of H x W, halo K,
one tiled along a single axis as N tiles, axis A, halo K, and the first stage of a
chain as chain of N with its tile height. A schema 10 tile record states neither the
tile count nor the halo, so such a stage shows as tiled with its axis. A plan compiled without
--keep-names names only the model, its inputs and
its outputs: its operators show as (unnamed) and its other tensors by their id,
for example tensor 3.
A plan’s schema number says which plan format it uses; it does not by itself name a runtime version.
JSON output
Section titled “JSON output”--json prints everything inspect reads, with exact byte counts and without weight
values, including the cost, quantization and requirement figures above. The top-level
inspection_version field versions the document, so scripts can check it before reading
further. Version 2 lists each tile record by the fields its schema states: a schema 11
record adds tile_extent, num_tiles, original_extent, halo, band_axis,
band_kind and band_mode to the scheduling choices, and a schema 10 record leaves
those seven null.
tigris inspect model.tgrs --jsonWhat inspect does not do
Section titled “What inspect does not do”- It does not compile, execute or validate kernel numerics.
- It does not infer ONNX shapes or load external ONNX tensor data.
- It needs no native library, so it works on every platform
tigris-mlinstalls on. See Installation.