Neural Network Primitives (BNNS)

BNNSArray(A::AbstractArray)

A GC-safe BNNSNDArrayDescriptor view of a Julia array A, suitable for passing to BNNS routines via Ref. The wrapper keeps a reference to the backing array so it is not collected while the descriptor is alive; pass the underlying descriptor with Base.cconvert/Ref only inside a GC.@preserve block guarding A.

BNNS descriptors are layout aware. Julia stores arrays in column-major order, so this constructor reports the array using a column-major-friendly BNNS layout:

• 1D Vector -> BNNSDataLayoutVector
• 2D Matrix -> BNNSDataLayoutColumnMajorMatrix (BNNS size = (rows, cols)).

Only Float16, Float32 and Int32 dense, contiguous arrays are supported here; other element types or strided/transposed arrays should use the raw LibAccelerate layer directly.

bnns_transpose(A::Array, dim0, dim1) -> Array

Swap Julia dimensions dim0 and dim1 of A (1-based) via BNNSTranspose, equivalent to a permutedims that exchanges those two axes. Works for every element type BNNS can describe: Float16, Float32, Int8–Int64, UInt8–UInt64 and Bool.

bnns_copy!(dest::Array, src::Array) -> dest

Copy src into the equally-sized dest via BNNSCopy. For equal element types this is a plain element copy.

dest and src may have different element types, in which case BNNS converts: Float16 ↔ Float32 (the usual way to move data in and out of a half-precision graph), Int8/Int16/Int32/UInt8 → Float32, and Float32 → Int32. Any other pair throws an ArgumentError without calling BNNS.

bnns_reduce(func::Symbol, input::Array; dim=1) -> Array

Reduce input along Julia dimension dim with func (:sum, :mean, :max, :min, :sumsquare, :l1, :l2, :product, :logsumexp) via BNNSDirectApplyReduction. The reduced axis collapses to length 1 and the result has the element type of input.

Supported element types: Float32, Float16 (computed in half precision, so sums saturate at floatmax(Float16) = 65504) and Int32. For Int32 only the reductions that are exact in integers are offered (:sum, :max, :min, :sumsquare, :l1, :product); :mean, :l2 and :logsumexp throw.

bnns_topk(input::Array, K; dim=1) -> (values, indices)

Top-K values and their 0-based indices along Julia dimension dim via BNNSDirectApplyTopK. values has the element type of input, indices is Int32. Comparable to sort-based partialsortperm per slice.

Supported element types: Float32, Float16, Int8, Int16, Int32, UInt8, UInt16 (BNNS rejects the wider integer types).

bnns_in_topk(input::Array, targets::Array{Int32}, K; dim=1) -> Array{Bool}

For each batch column, test whether the targets class index is among the top-K scores of input along Julia dimension dim (BNNSDirectApplyInTopK). input may be Float32 or Float16.

bnns_layout_rank(layout::BNNSDataLayout) -> Int

Rank (number of dimensions) encoded by a BNNSDataLayout constant, via BNNSDataLayoutGetRank.

bnns_data_size(A::Array) -> Int

Number of bytes of tensor data described by A (BNNSNDArrayGetDataSize).

bnns_tensor_allocation_size(A::Array) -> Int

Bytes required to allocate a BNNSTensor describing A (BNNSTensorGetAllocationSize). Uses the modern BNNSTensor struct (rank + shape/stride), distinct from the legacy BNNSNDArrayDescriptor.

BNNSRandomGenerator([seed]) -> BNNSRandomGenerator

A BNNS random number generator handle (AES-CTR method). Construct with an optional 64-bit seed for reproducibility (BNNSCreateRandomGeneratorWithSeed, or BNNSCreateRandomGenerator when omitted). The handle is destroyed automatically by a finalizer (BNNSDestroyRandomGenerator).

Use with bnns_random_fill_uniform!, bnns_random_fill_normal!, bnns_random_fill_uniform_int!, bnns_random_fill_categorical! and the bnns_random_state/bnns_random_state! round-trip.

bnns_random_fill_uniform!(g::BNNSRandomGenerator, A::Array, lo=0f0, hi=1f0) -> A

Fill A (Float32 or Float16) with i.i.d. uniform samples on [lo, hi) (BNNSRandomFillUniformFloat). For Float16 the samples are rounded to half precision, so a value can round up to exactly hi.

bnns_random_fill_uniform_int!(g::BNNSRandomGenerator, A::Array{<:Integer}, lo, hi) -> A

Fill integer array A with i.i.d. uniform samples on the half-open range [lo, hi) (BNNSRandomFillUniformInt). A may be Int8, Int16, Int32, Int64, UInt8, UInt16, UInt32 or UInt64; the range must fit the element type.

bnns_random_fill_normal!(g::BNNSRandomGenerator, A::Array, mean=0f0, stddev=1f0) -> A

Fill A (Float32 or Float16) with i.i.d. Gaussian samples (BNNSRandomFillNormalFloat).

bnns_random_fill_categorical!(g::BNNSRandomGenerator, out::Array{T}, probs::Array{T}; log_probs=false) -> out

Draw categorical samples (0-based category indices, stored as floating point) into out using per-category weights probs (BNNSRandomFillCategoricalFloat). Pass log_probs=true if probs holds log probabilities. T is Float32 or Float16; out and probs must share it (BNNS silently mis-samples mixed precisions).

bnns_random_state(g::BNNSRandomGenerator) -> Vector{UInt8}

Snapshot the generator's internal state (BNNSRandomGeneratorStateSize + BNNSRandomGeneratorGetState). Restore it with bnns_random_state!.

bnns_random_state!(g::BNNSRandomGenerator, state::Vector{UInt8}) -> g

Restore a generator state captured by bnns_random_state (BNNSRandomGeneratorSetState).

BNNSNearestNeighbors(max_samples, n_features, n_neighbors; T=Float32) -> BNNSNearestNeighbors

A brute-force k-nearest-neighbours index (BNNSCreateNearestNeighbors) holding up to max_samples reference points of dimension n_features, answering n_neighbors-NN queries. Destroyed automatically (BNNSDestroyNearestNeighbors).

Add reference points with bnns_knn_load! and query with bnns_knn_query.

bnns_knn_load!(knn::BNNSNearestNeighbors, data::Matrix{Float32}) -> Int

Append reference samples to the index (BNNSNearestNeighborsLoad). data is n_features × n_new_samples (each column is one sample, matching BNNS's feature-major layout). Returns the number of samples loaded.

bnns_knn_query(knn::BNNSNearestNeighbors, sample_number) -> (indices, distances)

Return the n_neighbors nearest reference points to the (0-based) loaded sample sample_number (BNNSNearestNeighborsGetInfo): their 0-based indices (Vector{Int32}) and Float32 distances.

BNNSGraphCompileOptions(; single_thread=nothing, generate_debug_info=nothing,
                          optimization=nothing, log_mask=nothing,
                          output_path=nothing, output_fd=nothing) -> BNNSGraphCompileOptions

Options controlling BNNSGraphCompileFromFile, backed by BNNSGraphCompileOptionsMakeDefault and destroyed by a finalizer (BNNSGraphCompileOptionsDestroy). Any keyword left nothing keeps the BNNS default. optimization is :performance or :ir_size. Individual fields can also be read/written with the accessor functions below.

BNNSGraph(filename; func=nothing, options=BNNSGraphCompileOptions()) -> BNNSGraph

Compile the compiled Core ML model (.mlmodelc directory, or the model.mil inside it) at filename — optionally only the named func inside it — into an executable graph via BNNSGraphCompileFromFile. Throws if BNNS cannot compile the model (unsupported op, malformed program, missing file). The returned handle feeds BNNSGraphContext and the graph-introspection helpers, and the compiled graph's memory is released by a finalizer once the graph and every context made from it are unreachable.

.mlmodelc is what Xcode / xcrun coremlcompiler compile produce from an .mlpackage; only ML Program models (MIL), not the older NeuralNetwork format, are accepted by BNNS.

BNNSGraphContext(g::BNNSGraph) -> BNNSGraphContext

An executable context for a compiled BNNSGraph (BNNSGraphContextMake), destroyed by a finalizer (BNNSGraphContextDestroy). It holds the mutable execution state (dynamic shapes, streaming state), keeps its graph alive, and must be used by one thread at a time; make one context per task for concurrent inference. Feed it to bnns_graph_run / bnns_graph_run!, or to the low-level bnns_graph_execute!.

BNNSGraphArgument

Description of one argument of a graph function, as returned by bnns_graph_arguments:

• name::String
• intent::Symbol — :in, :out or :inout
• eltype::DataType — Julia element type (Float16, Float32, Int32, Bool, …)
• shape::Dims — the shape as written in the model (MIL / row-major order)
• size::Dims — size of the Julia Array to pass for it: reverse(shape)

A dimension of 0 (in either tuple) is dynamic and not bound yet.

bnns_graph_arguments(c::BNNSGraphContext; func=nothing) -> Vector{BNNSGraphArgument}
bnns_graph_arguments(g::BNNSGraph; func=nothing)

Names, intents, element types and shapes of every argument of func, in execute order (outputs first). Given a context, shapes reflect any batch size / dynamic shapes already set on it.

Memory layout: reverse the dimensions

Core ML / MIL tensors are row-major; Julia arrays are column-major. A MIL tensor of shape [N, C, H, W] therefore has exactly the memory layout of a Julia Array of size (W, H, C, N). The graph functions here use that correspondence — they pass Julia's memory to BNNS as is, with no copy — so every argument is a Julia array whose size is the reverse of the model's shape (BNNSGraphArgument.size). In particular a MIL matrix [rows, cols] is a Julia (cols, rows) matrix, i.e. the transpose; use permutedims (or pass mil_order=true to bnns_graph_run) when you want model index order. BNNS graphs assume contiguous storage and ignore custom strides, so this is the only zero-copy mapping.

bnns_graph_run(c, inputs...; func=nothing, mil_order=false, workspace=UInt8[]) -> Dict{String,Array}

Run func on inputs (name => Array pairs, or a single Dict/NamedTuple), allocating the outputs, and return them keyed by output name.

g = AppleAccelerate.BNNSGraph("classifier.mlmodelc")
c = AppleAccelerate.BNNSGraphContext(g)
AppleAccelerate.bnns_graph_arguments(c)          # names, eltypes, sizes
out = AppleAccelerate.bnns_graph_run(c, "image" => img)
probs = out["probabilities"]

By default arrays use the zero-copy reversed-dimension layout (bnns_graph_arguments): pass size == BNNSGraphArgument.size. With mil_order=true inputs and outputs instead have the model's own shape (BNNSGraphArgument.shape) and index order — out[n, c, h, w] means what it means in the model — at the cost of one permutedims copy per array.

If the graph has dynamic dimensions, bind them first with bnns_graph_context_set_batch_size! or bnns_graph_context_set_dynamic_shapes!; an output whose shape is still unknown throws.

bnns_graph_run!(c, outputs, inputs; func=nothing, workspace=UInt8[]) -> outputs

Run func of the graph behind context c, reading inputs and writing into the preallocated outputs. Both are collections of name => Array (a pair, a vector or tuple of pairs, a Dict, or a NamedTuple); together they must supply every argument of the function exactly once. Each array must be a dense Array whose element type and size match bnns_graph_arguments — note the reversed-dimension layout described there. Nothing is copied: BNNS reads and writes the arrays' memory directly.

Pass a reusable page-aligned workspace from bnns_graph_workspace to keep repeated calls allocation-free. A context must not be run from two threads at once.

bnns_graph_workspace(c; func=nothing) -> Vector{UInt8}

A page-aligned scratch buffer of the size func currently needs (bnns_graph_context_workspace_size), for the workspace keyword of bnns_graph_run! / bnns_graph_execute!. BNNSGraphContextExecute requires page alignment, which an ordinary Vector{UInt8} does not guarantee. Reusing one buffer across calls makes execution allocation-free on the BNNS side; without one BNNS allocates its own scratch on every call. The memory is released when the vector is garbage collected.

bnns_graph_context_workspace_size(c, func=nothing) -> Int

Workspace size (bytes) required to execute func (BNNSGraphContextGetWorkspaceSize). Query it again after changing the batch size or dynamic shapes. Allocate a suitable buffer with bnns_graph_workspace.

bnns_graph_context_set_batch_size!(c, n; func=nothing) -> c

Set the batch size of a graph whose only dynamic dimension is a shared leading (MIL-order) batch dimension (BNNSGraphContextSetBatchSize). For anything more general use bnns_graph_context_set_dynamic_shapes!.

bnns_graph_context_set_dynamic_shapes!(c, shapes; func=nothing) -> Vector{Pair{String,Dims}}

Bind concrete input shapes to a graph compiled with dynamic dimensions (BNNSGraphContextSetDynamicShapes). shapes is a collection of name => dims pairs for (some of) the graph's inputs; dims is given the way Julia sees the array, i.e. size(A) of the array you will pass to bnns_graph_run (the reverse of the MIL shape — see bnns_graph_arguments). Inputs that are not mentioned keep the model's default shape.

Returns the resulting shape of every argument, in execute order, again in Julia order. A 0 in an output shape means BNNS cannot bound that dimension from the input shapes alone (it depends on input values).

Argument names of func (BNNSGraphGetArgumentNames).

bnns_graph_execute!(c, arguments::Vector{bnns_graph_argument_t}; func=nothing, workspace=UInt8[]) -> c

Low-level execute (BNNSGraphContextExecute): run func with raw argument buffers, ordered as bnns_graph_argument_names reports them (outputs first). The caller must keep the memory behind every argument alive for the call. Prefer bnns_graph_run / bnns_graph_run!, which build and validate the arguments from Julia arrays.

workspace must be page-aligned — get one from bnns_graph_workspace; leave it empty to let BNNS allocate its own scratch.