This is not a list of things we have read about. It is the set of instruction sets, toolchains and boards we have shipped against — including the ones whose documentation is wrong, whose compilers refuse to vectorise the obvious loop, and whose delegates silently fall back to the CPU while reporting success.
If your platform is not listed, ask anyway. The techniques transfer; only the vocabulary changes.
Stack
CPU & SIMD
Vector ISAs we write by handDSP & NPU
Accelerators we offload ontoGPU & parallel compute
Where the wide work goesAI runtimes & toolchains
How the model reaches the metalVision & signal
The pipelines around the modelPlatforms & silicon
Boards we work on dailyHow we choose where the work goes
The accelerator that wins is rarely the one on the marketing slide. The decision comes out of the roofline, not the datasheet:
- Compute-bound, regular, small kernel → the vector unit on the CPU. Lowest latency, no transfer cost, no driver in the path.
- Compute-bound, large batch, high arithmetic intensity → the GPU or NPU, once the transfer cost is amortised across enough work.
- Streaming, fixed-point, always on → the DSP, where the power per operation is an order of magnitude better than the application processor.
- Bandwidth-bound → nothing accelerates it until the data layout changes. This is the most common finding, and the cheapest to fix.
- Latency-bound by synchronisation → the fix is in the pipeline structure, not in any kernel. Also common, also cheap.
Being told your problem is a memcpy rather than a missing NPU is a good outcome. It is a smaller invoice and a faster fix.
Not sure which accelerator is right?
That is exactly what the performance audit answers, on your hardware, in a couple of weeks.