aitl.ai NEON · SVE2 · DSP · GPU · NPU
AI ThinkLab builds edge-AI systems that read the real world — number plates, container codes, whatever the camera is pointed at — and makes them run on the device instead of in a data centre. The same craft is available as engineering: profiling, hand-tuned NEON, DSP, GPU and NPU kernels, and quantised networks, wherever the compute budget is already fixed.
Shipping products
Recognition problems that sound solved until you try to do them outdoors, at speed, in the rain, on hardware that costs less than a laptop. Licensed per device, deployed by you or your integrator.
Plate detection and reading from live camera feeds under conditions that break most systems — motion blur, headlight glare, rain, darkness, oblique angles and dirty plates. The whole pipeline runs on the edge device, so there is no server room, no bandwidth bill and no dependency on a connection that will eventually drop.
Reading container and ISO type codes off moving boxes at gates, cranes and yard portals. Every read is validated against the ISO 6346 check digit before it leaves the device, so a misread is caught at the camera instead of propagating into the terminal system and costing somebody an afternoon.
Reading wagon identifiers at rail portals and yard entries, where the train does not stop, the number is painted on a curved and frequently filthy surface, and the answer the operator needs is a rake in running order rather than a bag of unrelated numbers.
Inspection on a moving line, where the line rate is a commercial commitment rather than an aspiration. Surface defects, presence and absence, orientation, count and dimensional checks — with a deterministic per-part latency instead of a best effort.
Face and person recognition for consented access control, attendance and occupancy — running entirely on the camera or a local box. Because nothing is sent anywhere, the privacy question has a simple answer instead of a long one.
What we do
We are not a general software house. Everything below is the same craft applied at a different level of the stack — understanding exactly what the hardware can do, and then making the software ask for it.
Hand-tuned Arm NEON and SVE2 intrinsics for the kernels that dominate your profile — convolution, filtering, colour conversion, matrix and FFT work. We fix data layout first, then vectorise, then unroll and software-pipeline against the real issue width of your core.
Moving the heavy loops off the application processor and onto the DSP that is already sitting idle on your SoC. Kernel porting, fixed-point conversion, VLIW scheduling, DMA-driven tiling and the RPC plumbing that keeps the two sides in step.
OpenCL, Vulkan compute and CUDA kernels written for the memory system you actually have. Workgroup sizing, local-memory tiling, texture paths, zero-copy buffer sharing with the camera and display pipelines, and removal of the hidden synchronisation that stalls mobile GPUs.
Vision pipelines rebuilt end to end: ISP output straight into an accelerated operator graph with no redundant copies, no format churn and a deterministic frame budget. Detection, tracking, stereo depth, OCR, stitching, registration and classical feature work.
Making a model fit the device instead of asking for a bigger device. Post-training and quantisation-aware INT8/INT4 conversion, structured pruning, operator fusion, graph surgery, knowledge distillation and layer substitution — each step measured against your accuracy floor, not a benchmark set.
Getting the optimised model onto the board and keeping it there: runtime selection, delegate and backend configuration, memory arena planning, multi-model scheduling, OTA-safe model packaging and the regression harness that proves the next model release did not silently regress.
The premise
Almost every embedded SoC ships with far more parallel compute than the software running on it ever touches. The vector unit idles while a scalar loop grinds. The DSP sleeps through the entire product life. The GPU renders a UI and nothing else.
That gap is where we work. Before anyone talks about a faster processor, a bigger battery or a smaller model, there is usually a factor of several sitting unclaimed inside the design that has already been through EMC, thermal and cost review.
How we work
Optimisation without measurement is guesswork with extra steps. Every engagement follows the same five stages, and you can stop after any of them.
We instrument the real workload on the real board. Cycle counts, cache misses, memory bandwidth, thermal headroom and per-stage latency — never a synthetic benchmark. You get the profile even if you stop here.
A roofline for each hot stage tells us what is compute-bound, what is bandwidth-bound and what is simply badly scheduled. That is what decides whether the win is in SIMD, in the DSP, on the GPU, or in the data layout.
Layout, tiling and buffer lifetime come before intrinsics. Most of the first big speed-up comes from stopping the copies and giving the vector unit contiguous data to chew on.
Then the hand-written work: NEON/SVE2 kernels, HVX offload, OpenCL or Vulkan compute, quantised graphs and fused operators — each one landed behind a switch so it can be compared against the reference path.
Bit-exactness or a stated accuracy delta, a soak test under thermal load, a regression harness wired into your CI, and documentation an engineer on your team can maintain after we leave.
Technology
Vendor toolchains disagree with each other, documentation lags the hardware, and the fastest path is rarely the one in the getting-started guide. Knowing where each toolchain lies to you is most of the job.
Where it ships
Gate and portal automation where a box does not stop and a misread costs a person an afternoon. Container and wagon identity, damage and seal evidence, hazmat placards, and truck-to-box correlation — feeding the terminal system as structured events rather than photographs somebody has to look at.
Plate recognition for open-road tolling, car parks, gated sites and enforcement, running on the pole rather than in a control room. Designed for the lane where the light is bad, the plate is dirty and the connection is not guaranteed.
Inspection and metrology pipelines where a line rate is a commercial commitment. Deterministic latency, repeatable results and graceful behaviour when the part on the belt is not the part in the training set.
Always-on detection in a milliwatt budget: cascaded models, wake-on-motion, on-sensor pre-processing and models small enough to live in SRAM.
Perception stacks inside a fixed frame budget — lane and object detection, driver monitoring, surround view and sensor fusion on automotive SoCs, with determinism and safety-process evidence in mind.
Signal and image processing for constrained, ruggedised and often air-gapped platforms. On-premises engagement, contracting entity and jurisdiction to suit your procurement, and code handed over in full.
Working together
You should not have to commit to a programme to find out whether the performance is there. Most relationships begin with a fixed-fee audit and grow from what it finds.
The starting point for almost everyone. We profile your workload on your own hardware and hand back a written roofline analysis, a ranked list of opportunities with estimated gains, and the effort each one costs. You keep the analysis whether or not you continue.
A model, a board and a number written into the statement of work before we start: this latency, this power envelope, this accuracy floor. Quantisation, delegate and runtime selection, memory planning, kernel work — whatever it takes to land the target, with the acceptance test agreed up front.
Language and vision-language models running locally on embedded silicon — no cloud, no per-token bill, no data leaving the device. Low-bit weight quantisation, KV-cache and context strategy, prompt and decode scheduling, and an honest measurement of what the board can and cannot sustain.
Before the BOM is frozen: can the cheaper part carry this workload, does the NPU actually support these operators, and what will the thermal behaviour be at the third minute rather than the third second? A short engagement that regularly saves a hardware respin.
Modern AI running inside your own perimeter — on-premises, air-gapped, or in a jurisdiction you choose. Open-weight language and vision models sized to hardware you already own or can buy outright, with no inference leaving the building, no per-token bill, and no vendor able to deprecate the model underneath you.
The documentation trail regulated products need for an AI component — dataset and model provenance, verification and validation evidence, accuracy and robustness argumentation, and the traceability that safety and quality auditors ask for. Prepared alongside the engineering, not retrofitted after it.
The lab
Every embedded team quantises the same models for the same boards, privately, and throws the numbers away. We run a standing research programme and publish what comes out of it — latency, energy per inference and accuracy delta, measured after the board is hot, with the method written down so you can disagree using evidence.
How far INT4 and mixed-precision schemes can be pushed on hardware whose compiler was designed for INT8 — where the accuracy actually goes, which layers refuse to cooperate, and when the dequantisation overhead quietly eats the gain.
Small language and vision-language models on embedded SoCs: memory-bandwidth limits on decode, KV-cache strategies inside a fixed arena, speculative decoding when there is no second accelerator, and what a usable tokens-per-second actually feels like in a product.
Common questions
It depends entirely on where you are starting. Scalar C compiled with default flags against a vector unit that is sitting idle usually has several times headroom; code that a competent team has already tuned may have twenty per cent left. We will not quote you a number before we have profiled the workload — the audit exists precisely so that both sides are arguing about a measured roofline instead of a hope.
Almost never, and that is the point. Most embedded boards ship with a DSP, a GPU and a vector unit that the software touches lightly or not at all. The first place we look is the silicon you have already paid for and already qualified.
Kernel-level and pipeline-level work is bit-exact or provably equivalent, so no. Quantisation and pruning do move accuracy, which is why we ask for your accuracy floor and your validation set at the start and report the delta at every step. If a technique cannot stay inside your floor, we say so rather than shipping it.
You do. Deliverables are work-for-hire under the engagement agreement, handed over as source with build instructions and documentation. Anything we bring in from our own pre-existing libraries is identified up front and licensed to you perpetually.
Most of our work is international. Development happens remotely against hardware we hold or hardware you ship us, our working day overlaps Europe, Asia-Pacific and the US morning, and we travel for commissioning, bring-up and acceptance testing. Where data cannot leave a country or a building, we work on your infrastructure instead of ours.
Yes to both. We routinely work under mutual NDA before any technical detail is exchanged, and for defence and regulated work we can operate on your premises or on isolated infrastructure you control.
A 45-minute technical call, no charge — you describe the workload, the target hardware and the number you need to hit. If it looks tractable we scope a performance audit; if it does not, we will tell you that on the call.
Send us the workload, the target board and the number you have to hit. If it is tractable we will say how; if it is not, we will say that too.