Integration

Step-by-step guide to integrating the TiGrIS C runtime into your embedded application.

1. Add to your build

CMake (as a subdirectory):

add_subdirectory(tigris-runtime)
target_link_libraries(my_app PRIVATE tigris_runtime)

Manual: Copy tigris-runtime/src/ and tigris-runtime/include/ into your project. Add the source files to your build system and set the include path. With GCC and Clang the runtime’s CMake build compiles the runtime with -ffp-contract=off; a build that compiles src/*.c itself with those compilers needs the same flag.

Link only the kernels a plan uses

By default the runtime links every operator kernel. tigris codegen writes <stem>_kernels.h, which enables only the kernels the plan uses. Set TIGRIS_OP_CONFIG to its absolute path before adding the runtime:

set(TIGRIS_OP_CONFIG ${CMAKE_CURRENT_SOURCE_DIR}/generated/vww_kernels.h)
add_subdirectory(tigris-runtime)

Without CMake, compile the runtime sources with -DTIGRIS_OP_CONFIG='"/path/to/vww_kernels.h"'. On Cortex-M7 at -Os, the VWW app with CMSIS-NN links 68,827 bytes of TiGrIS code instead of 109,660. A plan that needs a kernel the build leaves out fails to load with TIGRIS_ERR_KERNEL_DISABLED.

2. Include headers

#include "tigris.h"
#include "tigris_loader.h"
#include "tigris_mem.h"
#include "tigris_executor.h"
#include "tigris_iface.h"            /* model inputs and outputs */
#include "tigris_kernels_s8.h"       /* int8 reference backend */
#include "tigris_kernels.h"          /* f32 reference backend */

Pick the kernel header matching your model’s dtype. For accelerated int8 backends, use tigris_kernels_esp_nn.h (ESP32 family) or tigris_kernels_cmsis_nn.h (Cortex-M family) instead of tigris_kernels_s8.h.

3. Load the plan

tigris_plan_t plan;
tigris_error_t err = tigris_plan_load(plan_buf, plan_buf_len, &plan);
if (err != TIGRIS_OK) {
    /* report tigris_error_str(err) and stop */
}

plan_buf is the .tgrs file content, either memory-mapped from flash or loaded into a buffer. The loader is zero-copy and zero-alloc: all pointers in plan refer directly into plan_buf. Keep plan_buf alive for the lifetime of plan.

Align the plan base to 16 bytes when an optimized backend reads uncompressed XIP weights directly from it: CMSIS-NN on DSP-enabled Cortex-M (__ARM_FEATURE_DSP), and ESP-NN on ESP32-S3, where tigris_esp_nn_prepare() refuses a plan with misaligned weights. Give embedded arrays and linker sections an explicit alignment, and preserve it for custom flash mappings or loaded buffers.

4. Initialize memory

Allocate a fast buffer (SRAM), a slow buffer (PSRAM), and a tensor pointer array. Then initialize the memory manager:

uint32_t required_fast = tigris_fast_arena_required(&plan);
#if defined(TIGRIS_HAS_CMSIS_NN)
required_fast = tigris_cmsis_nn_fast_arena_required(&plan);
#endif
if (required_fast == UINT32_MAX) {
    /* invalid or unrepresentable fast-buffer requirement; stop */
}
if (fast_capacity < required_fast) {
    /* caller-provided fast buffer cannot cover the requirement; stop */
}

tigris_mem_t mem;
static void *tensor_ptrs[TIGRIS_MAX_TENSORS];
tigris_mem_error_t merr = tigris_mem_init(
    &mem, tensor_ptrs, plan.header->num_tensors,
    fast_buf, fast_capacity, slow_buf, slow_size);
if (merr != TIGRIS_MEM_OK) {
    /* report tigris_mem_error_str(merr) and stop */
}

tigris_fast_arena_required() returns the activation figure plus compressed-weight storage. For a plan of schema 11 or later the activation figure is the plan’s scheduled peak, the fast bytes its largest stage or chain holds at once, and a run refuses a smaller fast arena with TIGRIS_EXEC_ERR_MEM before any stage runs. For an earlier plan it is the compiled budget, and the run is not checked up front. Both helpers assume that the fast-buffer base satisfies TIGRIS_TENSOR_ALIGN; align the allocation rather than relying on extra capacity to absorb leading padding. For CMSIS-NN, tigris_cmsis_nn_fast_arena_required() also adds the exact adapter workspace. Use the helper that matches the selected backend rather than reproducing its arithmetic in application code.

When tigris compile and analyze print no slow memory row for a plan, it needs no slow buffer: pass NULL with a slow_size of 0. A generated core sets <NAME>_CORE_SLOW_ARENA_BYTES to 0 for such a plan. Give any other plan a slow buffer of at least the slow memory figure. The figure assumes the inputs are written as in step 6.

5. Prepare an accelerated backend

#if defined(TIGRIS_HAS_ESP_NN)
if (tigris_esp_nn_prepare(&plan, &mem) != 0) {
    /* backend workspace allocation failed; stop */
}
#elif defined(TIGRIS_HAS_CMSIS_NN)
if (tigris_cmsis_nn_prepare(&plan, &mem) != 0) {
    /* arena too small for CMSIS-NN scratch; stop */
}
#endif

Call the preparation function for the selected accelerated backend exactly once after tigris_mem_init() and before inference, and always check its result. The reference int8 and float32 dispatch functions need no preparation.

CMSIS-NN preparation carves a 16-byte-aligned region from the top of the fast buffer for vendor scratch and scalar-to-per-channel quantization expansion. Query that reservation with tigris_cmsis_nn_scratch_required(), or query the complete buffer with tigris_cmsis_nn_fast_arena_required(). ESP-NN obtains aligned workspace through the platform allocator during initialization. Neither backend allocates model-dependent workspace during inference after successful preparation. Call tigris_cmsis_nn_deinit() or tigris_esp_nn_deinit() after the last inference when the workspace is no longer needed.

6. Set model inputs

Hand each input to tigris_input_write() in the dtype the model declares. It converts the data and places the input with tigris_input_alloc(): in the fast arena when the plan reads its inputs in place (the condition is under tigris_input_alloc), above room for the stage’s decompressed weights, and in the slow buffer otherwise. Code that fills inputs itself, as generated code does, calls tigris_input_alloc() and writes the stored dtype into the buffer it places.

for (uint16_t i = 0; i < plan.header->num_model_inputs; i++) {
    uint16_t tidx = plan.model_inputs[i];
    /* inputs[i]: your preprocessed data, tigris_iface_bytes(&plan, tidx) bytes */
    tigris_error_t ierr = tigris_input_write(
        &plan, &mem, tidx, inputs[i], tigris_iface_bytes(&plan, tidx));
    if (ierr != TIGRIS_OK) {
        /* TIGRIS_ERR_MEM: the arena cannot hold the input;
           report tigris_error_str(ierr) and stop */
    }
}

Write the inputs one after another, so they sit together. A run consumes the inputs it reads in place: write every input again before the next run. An input the caller allocated beforehand with tigris_mem_alloc_slow() is filled where it is. A plan that reads its inputs in place then copies it instead, and the slow buffer must hold the inputs and the outputs, more than the slow memory figure. Inputs placed in the fast arena another way are read in place only when the room for the decompressed weights is free directly below them.

7. Run inference

static tigris_executor_workspace_t executor_workspace;

tigris_exec_stats_t stats;
tigris_exec_error_t eerr = tigris_run_with_workspace(
    &plan, &mem, tigris_dispatch_kernel_s8, NULL, &stats,
    &executor_workspace);
if (eerr != TIGRIS_EXEC_OK) {
    /* report tigris_exec_error_str(eerr) and stop */
}

Pass the dispatch function for your chosen backend. The int8 and accelerated dispatchers route float section operators to the portable float kernels. The user_ctx parameter (NULL above) is forwarded to every kernel call.

Provide one workspace for each inference that may run concurrently. Static storage is appropriate for a single inference task; separate tasks or model instances need separate workspaces. The compatibility entry point tigris_run() owns one process-global workspace and is therefore not re-entrant or safe for concurrent inference.

After the call, mem.fast_peak and mem.slow_peak are the measured high-water marks of the core fast and slow arenas during that run. Backend workspace reserved outside the fast arena must be counted separately.

Plans that keep state

A model with resource variables, SVDF or LSTM keeps values from one inference to the next. Its plan states how many bytes of state it needs; the application supplies that buffer next to the arenas, initializes it once, and runs through tigris_run_with_state():

size_t state_size = tigris_state_required(&plan);   /* 0 for a stateless plan */
static uint8_t state[STATE_BYTES];                  /* at least state_size */
if (tigris_state_init(&plan, state, sizeof(state)) != TIGRIS_EXEC_OK) {
    /* report and stop */
}

eerr = tigris_run_with_state(&plan, &mem, tigris_dispatch_kernel_s8, NULL, &stats,
                             workspace, workspace_size, state, sizeof(state));

Each run reads the values the previous run left and writes back the ones it assigns. Call tigris_state_init() again to start over. The other entry points refuse a stateful plan with TIGRIS_EXEC_ERR_STATE. A tigris codegen core states the size as <NAME>_STATE_BYTES and runs through <name>_run_with_state().

Executor limits and task-stack sizing

The executor workspace is bounded by compile-time plan limits:

Definition	Default	Accepted values
TIGRIS_MAX_TENSORS	512	1-65,535
TIGRIS_MAX_STAGE_INPUTS	16	1-65,535
TIGRIS_MAX_STAGE_OUTPUTS	16	1-65,535
TIGRIS_MAX_CHAIN_STAGES	16	2-65,535
TIGRIS_MAX_SPATIAL_OPS_PER_STAGE	8	1 or greater
TIGRIS_MAX_SUBGRAPHS	16	1-65,535
TIGRIS_MAX_SUBGRAPH_DEPTH	4	1-65,535

Override these definitions consistently for every runtime translation unit. Lower limits reduce the workspace; higher limits accept larger plans and increase it. The loader returns TIGRIS_ERR_PLAN_LIMITS before execution when a plan exceeds the configured limits; tigris analyze, compile and inspect name the -D options a plan needs above the defaults. sizeof(tigris_executor_workspace_t) and tigris_executor_workspace_size() are the authoritative workspace size for a particular build.

Keep the workspace in static, global, or heap-backed storage unless its full size is deliberately included in the task-stack budget. Runtime sources reject variable-length arrays. Build with -DTIGRIS_STACK_USAGE=ON to generate and check GCC stack-usage reports; CI rejects an unbounded dynamic frame or any individual runtime function frame above 1,024 bytes. Those reports exclude caller and vendor-library frames, C-library internals, RTOS context, interrupt nesting, and instrumentation. Size the final task stack from the complete call chain and confirm it with the target’s stack high-water measurement under the largest supported model and worst interrupt load.

8. Read outputs

for (uint16_t i = 0; i < plan.header->num_model_outputs; i++) {
    uint16_t tidx = plan.model_outputs[i];
    int8_t *output = (int8_t *)mem.tensor_ptrs[tidx];
    uint32_t size = plan.tensors[tidx].size_bytes;
    if (output == NULL) { /* inference did not produce this output; stop */ }
    /* process output */
}

Model outputs are in the slow buffer after inference completes. A run that read its inputs in place leaves them in the fast arena, where they stay valid until the next allocation there: read them before writing the next inputs. tigris_output_read() converts an output to the dtype the model declares.

Complete example

Minimal POSIX integration that loads a plan from file, runs inference, and prints first output values. This example fills each input with bytes of 1 in the dtype the model declares, expects stored int8 outputs and uses the int8 reference backend. A float section output is stored as float32; check each tensor’s dtype before casting it, or use tigris_output_read(). For a float32 plan, replace tigris_dispatch_kernel_s8 with tigris_dispatch_kernel:

#define _POSIX_C_SOURCE 200112L

#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include "tigris.h"
#include "tigris_loader.h"
#include "tigris_mem.h"
#include "tigris_executor.h"
#include "tigris_iface.h"
#include "tigris_kernels_s8.h"

static tigris_executor_workspace_t executor_workspace;

int main(int argc, char **argv) {
    FILE *f = NULL;
    void *plan_storage = NULL;
    void *fast_buf = NULL;
    void *slow_buf = NULL;
    void **tensor_ptrs = NULL;
    int result = 1;

    if (argc != 2) {
        fprintf(stderr, "usage: %s model.tgrs\n", argv[0]);
        return 2;
    }

    f = fopen(argv[1], "rb");
    if (f == NULL || fseek(f, 0, SEEK_END) != 0) {
        fprintf(stderr, "could not open or seek plan\n");
        goto cleanup;
    }
    long end = ftell(f);
    if (end <= 0 || (unsigned long)end > UINT32_MAX ||
        fseek(f, 0, SEEK_SET) != 0) {
        fprintf(stderr, "invalid plan size\n");
        goto cleanup;
    }
    uint32_t file_size = (uint32_t)end;

    size_t alignment = TIGRIS_TENSOR_ALIGN;
    if (alignment < sizeof(void *)) alignment = sizeof(void *);
    if (posix_memalign(&plan_storage, alignment, file_size) != 0) {
        fprintf(stderr, "could not allocate aligned plan buffer\n");
        goto cleanup;
    }
    if (fread(plan_storage, 1, file_size, f) != file_size) {
        fprintf(stderr, "could not read complete plan\n");
        goto cleanup;
    }
    if (fclose(f) != 0) {
        f = NULL;
        fprintf(stderr, "could not close plan file\n");
        goto cleanup;
    }
    f = NULL;

    /* Parse plan (zero-copy into the aligned storage) */
    tigris_plan_t plan;
    tigris_error_t err = tigris_plan_load(
        (const uint8_t *)plan_storage, file_size, &plan);
    if (err != TIGRIS_OK) {
        fprintf(stderr, "load failed: %s\n", tigris_error_str(err));
        goto cleanup;
    }
    if (plan.header->num_model_inputs == 0 ||
        plan.header->num_model_outputs == 0) {
        fprintf(stderr, "plan has no model input or output\n");
        goto cleanup;
    }

    uint32_t fast_size = tigris_fast_arena_required(&plan);
    if (fast_size == UINT32_MAX) {
        fprintf(stderr, "invalid fast-buffer requirement\n");
        goto cleanup;
    }
    uint32_t slow_size = 512u * 1024u;
    if (fast_size == 0 ||
        posix_memalign(&fast_buf, alignment, fast_size) != 0 ||
        posix_memalign(&slow_buf, alignment, slow_size) != 0) {
        fprintf(stderr, "could not allocate aligned arenas\n");
        goto cleanup;
    }
    tensor_ptrs = calloc(plan.header->num_tensors, sizeof(void *));
    if (tensor_ptrs == NULL) {
        fprintf(stderr, "could not allocate tensor pointer table\n");
        goto cleanup;
    }

    tigris_mem_t mem;
    tigris_mem_error_t merr = tigris_mem_init(
        &mem, tensor_ptrs, plan.header->num_tensors,
        fast_buf, fast_size, slow_buf, slow_size);
    if (merr != TIGRIS_MEM_OK) {
        fprintf(stderr, "memory init failed: %s\n",
                tigris_mem_error_str(merr));
        goto cleanup;
    }

    /* Write every model input; the runtime places it. */
    for (uint16_t i = 0; i < plan.header->num_model_inputs; i++) {
        uint16_t in_idx = plan.model_inputs[i];
        uint32_t in_bytes = tigris_iface_bytes(&plan, in_idx);
        void *input = in_bytes != 0 ? malloc(in_bytes) : NULL;
        if (input == NULL) {
            fprintf(stderr, "could not allocate input staging\n");
            goto cleanup;
        }
        memset(input, 1, in_bytes);
        err = tigris_input_write(&plan, &mem, in_idx, input, in_bytes);
        free(input);
        if (err != TIGRIS_OK) {
            fprintf(stderr, "input write failed: %s\n", tigris_error_str(err));
            goto cleanup;
        }
    }

    tigris_exec_stats_t stats;
    tigris_exec_error_t eerr = tigris_run_with_workspace(
        &plan, &mem, tigris_dispatch_kernel_s8, NULL, &stats,
        &executor_workspace);
    if (eerr != TIGRIS_EXEC_OK) {
        fprintf(stderr, "inference failed: %s\n", tigris_exec_error_str(eerr));
        goto cleanup;
    }

    uint16_t out_idx = plan.model_outputs[0];
    int8_t *output = (int8_t *)mem.tensor_ptrs[out_idx];
    if (output == NULL || plan.tensors[out_idx].size_bytes < 5) {
        fprintf(stderr, "output is missing or shorter than five bytes\n");
        goto cleanup;
    }
    if (printf("Output[0..4]: %d %d %d %d %d\n"
               "Fast arena peak: %lu bytes\n",
               output[0], output[1], output[2], output[3], output[4],
               (unsigned long)mem.fast_peak) < 0) {
        fprintf(stderr, "could not write output\n");
        goto cleanup;
    }

    result = 0;

cleanup:
    if (f != NULL && fclose(f) != 0) result = 1;
    free(tensor_ptrs);
    free(slow_buf);
    free(fast_buf);
    free(plan_storage);
    return result;
}

ESP-IDF deployment

On ESP32 targets, embed the .tgrs plan in the app with a small assembly file that places it on a 16-byte boundary (.balign 16 followed by .incbin), and pass its start symbol to tigris_plan_load(). EMBED_FILES does not align, and ESP-NN reads filters in place. The plan lands in .rodata, which is memory-mapped flash, so the loader references it with no RAM copy and no partition table entry. The ESP-IDF tutorial walks a working project.

Store the plan on a dedicated flash partition and reach it with esp_partition_mmap() instead when you want to swap models without rebuilding the firmware, or when the plan makes the app image too large for the app partition.

Allocate the fast arena with heap_caps_malloc(MALLOC_CAP_INTERNAL) and, when available, the slow buffer with heap_caps_malloc(MALLOC_CAP_SPIRAM). Keep the executor workspace outside the task stack unless it is explicitly budgeted there. Do not use a fixed stack recommendation across models, backends, and ESP-IDF releases: start from compiler stack reports for the complete firmware, include RTOS and interrupt margin, and validate the chosen task size with the ESP-IDF high-water-mark APIs under worst-case inference load.

Error handling

Check every return code. All API functions return typed error enums:

Loader errors (tigris_error_t): See API Reference for the full error enum.

Memory errors (tigris_mem_error_t): See API Reference for the full error enum.

Executor errors (tigris_exec_error_t): See API Reference for the full error enum.

Use tigris_error_str(), tigris_mem_error_str(), and tigris_exec_error_str() to convert error codes to human-readable strings.