User-Defined Functions

NSPL user-defined functions (UDFs) are domain-owned functions written in Roto and invoked anywhere an ordinary expression function can be used. They execute in process over Arrow columns: one UDF invocation receives every message of a batch that reaches it, or, inside a conditional arm, only the messages that select that arm, and column methods perform vectorized work without serializing rows.

UDF creation is trusted-code administration, like uploading a WASM resource. Roto code is native JIT-compiled code, not a sandbox for third-party programs.

See Choosing An Extension Tier before choosing between builtins, Roto UDFs, and WASM processors.

Roto execution is synchronous. Nervix therefore runs every UDF-bearing expression on the node’s extension workers, including small Arrow batches and UDFs used while constructing branch keys, so native UDF work never occupies an asynchronous runtime worker or the data workers that relay bodies are encoded and decoded on.

The Roto language itself — syntax, types, functions, and test blocks — is covered by the generated Roto Language Reference, downloaded from upstream at the exact Roto release Nervix embeds for ROTO_0_13; the language is developed at NLnetLabs/roto. The column types and operations available inside a UDF body are the Nervix-provided catalog described on this page, not part of the Roto standard library.

Declaration

CREATE [IF NOT EXISTS] UDF <name>
  WITH ROTO_0_13
  ARGS (<argument> <type> [OPTIONAL], ...)
  RETURNS <type> [OPTIONAL]
  [VOLATILE]
  CODE $roto$
    fn <name>(...) -> ... {
        ...
    }
  $roto$;

A declaration has between one and eight arguments. Argument and return types use the schema type grammar: all scalar types, VEC<T>, and ARRAY<T, n> are accepted. Code is limited to 64 KiB. The declared signature must exactly match the named Roto entry function; implicit casts are never inserted.

Dollar-quoted strings ($$...$$ and $tag$...$tag$) are verbatim and may span lines. They are valid wherever NSPL accepts a string literal.

For example:

CREATE UDF risk_band
  WITH ROTO_0_13
  ARGS (score F64)
  RETURNS STRING
  CODE $roto$
    fn risk_band(score: F64Column) -> StringColumn {
        when_s(score.gt_s(0.9), "critical")
            .when_s(score.gt_s(0.7), "high")
            .otherwise_s("normal")
    }
  $roto$;

The Roto column names for scalar declarations are U8Column, I8Column, U16Column, I16Column, U32Column, I32Column, U64Column, I64Column, F32Column, F64Column, BoolColumn, StringColumn, BytesColumn, and DatetimeColumn. A one-level sequence uses VecU8Column through VecDatetimeColumn, including VecBytesColumn; deeper nesting uses AnyColumn. A bytes column carries Arrow Binary values directly.

Calling UDFs

Calls use the dedicated, case-insensitive udf:: qualifier and compose with fields, literals, builtins, and other call sites:

CREATE JUNCTION scoring FROM raw_events
  UNBRANCHED
  TO scored_events
    SET band = udf::risk_band(message.score)
    WHERE udf::risk_band(message.score) != 'normal'
    FLUSH IMMEDIATE
    ON MESSAGE ERROR LOG;

The NSPL compiler validates exact arity and types when the consuming model is created. Use an explicit AS or TRY_CAST conversion when conversion is intended. An unqualified call never resolves to a UDF. The separate namespace allows a UDF name to equal a builtin name without changing either call when a later Nervix version adds builtins. Results retain the maximum sensitivity of their arguments.

Roto test blocks execute during CREATE UDF. If any test rejects, creation fails with Roto test block failed and the UDF is not persisted.

The web console’s Create → Roto UDF form edits ordered arguments and the result with the exact schema types, optional flags, volatility, and a source editor that keeps the function and its test blocks verbatim. It renders the completed Model as canonical NSPL and shows Completed only after compilation and all Roto tests succeed. A rejected test leaves the entered declaration and source in the form for correction.

Nulls, errors, and volatility

OPTIONAL controls boundary nullability; it does not change the Roto column type.

A null row in a non-optional argument strictly produces a null result row without exposing that row to the UDF.
An optional argument exposes its validity bitmap so the body can use null-aware operations.
A non-optional result may contain nulls only for strict-propagation rows or rows carrying a side error. Any other null is a batch error.
Checked column operations and reject_where_s add per-row errors to the same channel used by builtins, so the owning route’s ON MESSAGE ERROR policy applies.
A Roto trap, invalid result type or row count, or watchdog expiry is a whole-batch error.

The watchdog detects an overrun after native code returns; Roto provides no in-process preemption. A non-terminating UDF occupies one extension worker permanently. The watchdog converts a late completion into a whole-batch error, but it cannot reclaim a worker from native code that never returns. A node whose extension workers are all held, and whose wait queue for them is full, refuses further UDF-bearing executions, which the owning processor’s general error policy handles; the node’s relay, control and bulk work keeps its own workers throughout.

Treat trusted authorship, code review, and passing Roto test blocks as admission policy. The extension class runs one job per available CPU but one, shared with JAQ transformations, and is not an NSPL setting. Limit concurrent UDF-bearing paths or isolate them onto separate Nervix processes when one workload could hold every extension worker. See Capacity Planning For Branched Graphs.

Non-terminating or third-party code must stay on the isolated WASM processor path.

UDFs are deterministic unless declared VOLATILE. Only a volatile UDF can call now(), rand_f64(), or uuid_v4(). now() uses the execution-local domain clock. Deterministic calls may be reused within one compiled expression; volatile calls are evaluated at every occurrence. User code is never executed by constant folding.

Roto column operations

Column methods operate elementwise and preserve the input row count. A base method takes another column of the identical type; an _s method broadcasts a Roto scalar.

The ROTO_0_13 catalog currently includes:

integer and floating columns: add, sub, mul, div, their _s forms, eq, lt, gt, their _s forms, and batch min/max;
boolean columns: and, or, not, plus string-column select;
string columns: trim, contains_s, regexp_replace, and the slow-path get;
I64Column.cast_f64() and DatetimeColumn.lt_s(Timestamp);
VecStringColumn.contains_s;
string coalesce, when_s(...).when_s(...).otherwise_s(...), and reject_where_s;
ColumnBuilder.bool, push, push_null, and finish for explicit per-row fallback code;
volatile now, rand_f64, and uuid_v4.

Arithmetic overflow, division by zero, and explicit rejection produce per-row errors. Invalid scalar configuration such as a malformed regular expression fails the batch.

Use column operations whenever possible. get and a builder are the explicit slow path for logic that has no vectorized kernel.

Introspection and lifecycle

SHOW UDFS;
DESCRIBE UDF risk_band;
SHOW CREATE UDF risk_band;
DROP UDF risk_band;

DESCRIBE UDF reports the language tag, signature, volatility, content hash, and referencing nodes. SHOW CREATE preserves the Roto source bytes and chooses a safe dollar-quote delimiter. Dropping a UDF is rejected while a stored model references it.

Changing a UDF body means dropping the old UDF and recreating it, and the drop is rejected until every referencing model has been removed. While a node references a UDF, that node’s UDF semantics are fixed by the UDF content hash reported by DESCRIBE UDF. A UDF body change quiesces the domain because its callers are spread across the graph.

Roto source is persisted; native JIT output is rebuilt in memory during domain activation. A server that does not support a stored language tag rejects activation rather than reinterpreting the source.

Nervix does not currently expose per-UDF invocation, latency, or row-error metrics.