Lab 8.5 — STM32 vs Pi 5 vs Jetson Benchmark

← Course 2 syllabus · Bonus Module 8 · Prev: « Lab 8.4 · Next: Lab 9.1 »

Goal

Run the same workload on all three platforms — the STM32 (CMSIS-DSP), the Raspberry Pi 5 (NumPy / onnxruntime), and the Jetson Orin Nano (TensorRT) — and measure latency, throughput, and power, then build a latency-vs-power Pareto view and reason about when each platform is the right choice for embedded DSP. This is the capstone of the bonus module and the payoff of the whole course: you’ll have real numbers, taken on your own bench, for the classical-microcontroller pipeline versus the two edge accelerators, and a defensible answer to “what should I run this on?” — the exact judgment a DSP/firmware engineer is paid for. It closes the loop back to the classical pipeline of Modules 5–7.

Equipment & parts

  • STM32 Nucleo-L476RG with a CMSIS-DSP build of the workload (FFT and/or the classifier).
  • Raspberry Pi 5 and Jetson Orin Nano with the Module 8 pipelines already deployed.
  • WANPTEK PSU as the measured supply for the boards you can power from it, so you can read supply current directly; and/or the Fluke 117 in series for a clean current reading.
  • A stopwatch source that is on the device: the STM32’s DWT cycle counter (CYCCNT), and time.perf_counter_ns() / CUDA events on the Pi and Jetson. Do not time over a serial print — you’ll measure the UART, not the DSP.

Safety & don’t-break-it

  • Measuring current means breaking the supply loop. To read board current with the Fluke, put it in series on the DC input (red lead in the A jack), exactly as in Lab 0.1 — never across the rail. The Pi 5 and Jetson draw amps; make sure it’s within the Fluke’s fused A range (the 10 A input), and move the lead back to VΩ when done.
  • Inrush and brown-out. The Pi 5 and Jetson have large input capacitance and real inrush; a current-limited WANPTEK set too low will brown them out and they’ll boot-loop. Set the current limit high enough for the platform’s peak (check the board’s PSU spec) but still bounded, and expect a current spike at boot.
  • Don’t power the Jetson/Pi from a sagging rail. Undervoltage on these boards causes silent throttling and corrupt filesystems. If the WANPTEK can’t hold the voltage under load, power the board from its proper supply and measure current with a clamp or an inline USB-C power meter instead.
  • Fix the clocks and thermals on the Pi and Jetson (jetson_clocks, a fan, a cool room) before timing — a thermal-throttled run makes the accelerator look worse than it is and ruins the comparison.
  • STM32 side: standard 3.3 V discipline; the Nucleo is USB-powered and low-risk.

Background

Pick one workload, run it everywhere. Two good choices:

  1. A 1024-point real FFT — the pure-DSP kernel, directly comparable to Lab 6.3. CMSIS-DSP arm_rfft_fast_f32 on the STM32, np.fft.rfft on the Pi, cupyx/torch on the Jetson.
  2. The keyword classifier from Lab 8.2 — a realistic learned-DSP workload (the STM32 runs only the feature stage or a tiny quantized net; the accelerators run the full model).

Latency. The per-operation wall time, measured on-device after warm-up. Report the distribution, not just the mean:

\[ \text{latency: } \ \bar{t}, \ p_{95}, \ p_{99} \ \text{ over } N \ge 1000 \text{ runs}. \]

For real time, the relevant number is the tail (\(p_{99}\)), because one late frame is a dropped frame.

Throughput. Operations completed per second under sustained load,

\[ \text{throughput} \;=\; \frac{N_{\text{ops}}}{T_{\text{total}}} \quad [\text{inferences/s or FFTs/s}]. \]

Throughput ≠ \(1/\text{latency}\) once the platform pipelines or batches (the Jetson’s batched path from Lab 8.1 has high throughput but higher single-item latency).

Power and energy per inference. Estimate board power from the DC input:

\[ P \;=\; V_{\text{in}} \cdot I_{\text{in}}, \]

with \(V_{\text{in}}\) the supply voltage and \(I_{\text{in}}\) the current read on the WANPTEK display or the Fluke in series. Measure idle power and loaded power (running the workload in a tight loop); the workload’s marginal power is \(P_{\text{load}} - P_{\text{idle}}\). The figure of merit that fairly compares a 0.5 W microcontroller to a 15 W GPU is energy per inference:

\[ E_{\text{inf}} \;=\; \frac{P_{\text{load}}}{\text{throughput}} \;=\; P_{\text{load}} \cdot \bar{t}_{\text{eff}} \quad [\text{J/inference}]. \]

A platform can be slower yet more energy-efficient, or faster but far hungrier — the Pareto point depends on whether your product is latency-bound or battery-bound.

The Pareto view. Plot each platform (and each precision variant) as a point in (power, latency). The Pareto frontier is the set of points not dominated on both axes — those are the only rational choices; everything behind the frontier is strictly worse. The STM32, Pi 5, and Jetson typically land at three different corners: low power/high latency, balanced, and low latency/high power.

Procedure

Part A — Build the identical workload on all three.

  1. Fix the exact workload (say, 1024-pt f32 rFFT) and the exact input. STM32: CMSIS-DSP with the DWT cycle counter around the transform, converted to time via the core clock (80 MHz). Pi 5: perf_counter_ns around np.fft.rfft. Jetson: CUDA events (or perf_counter_ns with torch.cuda.synchronize()), timing both single-shot and batched.
  2. Warm up each (discard the first runs — cold caches, cuFFT plan, JIT), then time ≥1000 runs and record \(\bar t\), \(p_{95}\), \(p_{99}\).

Part B — Throughput.

  1. Run each platform flat-out for a fixed wall-clock window (e.g. 10 s) and count completed operations → throughput. On the Jetson, sweep batch size and record the throughput/latency curve.

Part C — Power.

  1. For each board: at the bench, set the supply and read idle current (booted, doing nothing), then loaded current (tight workload loop). Compute \(P_{\text{idle}}\), \(P_{\text{load}}\). Use the WANPTEK’s current display for a quick read and the Fluke in series for a trustworthy one; reconcile the two.
// STM32 timing — DWT cycle counter (illustrative)
CoreDebug->DEMCR |= CoreDebug_DEMCR_TRCENA_Msk;
DWT->CYCCNT = 0; DWT->CTRL |= DWT_CTRL_CYCCNTENA_Msk;
uint32_t t0 = DWT->CYCCNT;
arm_rfft_fast_f32(&fft, in, out, 0);
uint32_t cyc = DWT->CYCCNT - t0;      // /80e6 -> seconds

Part D — Energy per inference and the Pareto plot.

  1. Combine: \(E_{\text{inf}} = P_{\text{load}} / \text{throughput}\) for each platform/precision. Tabulate everything.
  2. Plot (power, latency) for every platform and precision variant; mark the Pareto frontier. Annotate each point with its energy/inference.

Deliverable & expected results

  • A full benchmark table (below) and a latency-vs-power Pareto scatter with the frontier drawn.
  • A one-page recommendation: which platform for a battery sensor node, which for a low-latency perception loop, which for a fixed-function tone detector — with the numbers backing each call.
Platform / precision Latency \(\bar t\) (p99) Throughput Load power Energy/inf
STM32 (CMSIS-DSP, f32) … (…) … … …
Pi 5 (NumPy / onnx int8) … (…) … … …
Jetson (TensorRT fp16, single) … (…) … … …
Jetson (TensorRT int8, batched) … (…) … … …

Order-of-magnitude expectations to reconcile against: the STM32 has the lowest power (sub-watt) and, for a small FFT, a competitive latency because there’s no OS jitter and no host↔︎device copy — but its throughput ceiling is low. The Pi 5 has no copy overhead and a fast CPU, so at tiny model sizes it is often within striking distance of the Jetson on latency while drawing a few watts. The Jetson wins throughput and large-model latency decisively but pays the highest idle and load power and a real copy/launch overhead that makes it a poor choice for a single tiny FFT.

Analysis & reconciliation

Explain each platform’s position on the Pareto plot from first principles: the STM32 is compute-bound on a slow-but-deterministic core with no memory-hierarchy surprises (predict its FFT cycles from the Lyons \(O(N\log N)\) operation count and the M4F’s cycles-per-FLOP); the Pi 5 is a fast OoO CPU with cache effects and OS scheduling jitter (hence a wider \(p_{99}\) tail); the Jetson only shines when the workload is big enough or batched enough to hide the fixed host↔︎device and launch cost — exactly the single-frame-loses / batched-wins result you already saw in Lab 8.1. Reconcile the two power readings (WANPTEK display vs Fluke in series) and attribute any gap to the display’s coarser resolution and the meter’s burden voltage. Then state the engineering conclusion plainly: for a fixed-frequency tone detector the STM32 + Goertzel (Lab 6.5) is unbeatable on energy and determinism; for a latency-critical, large learned model the Jetson is the only option; the Pi 5 is the pragmatic middle when you want learned DSP without a GPU’s power budget. This is the whole course’s thesis in one plot — the classical embedded pipeline of Modules 5–7 is not obsolete, it’s a point on the Pareto frontier, and picking the right point for the problem is the real skill.

Going further

  • Add fixed-point vs floating-point STM32 variants (CMSIS-DSP q15/q31) and show the latency/accuracy trade Kuo describes.
  • Add a duty-cycled measurement: real sensor nodes sleep between inferences, so integrate energy over a realistic duty cycle — the STM32’s deep-sleep current can flip the energy verdict entirely.
  • Put a proper inline USB-C power meter or a shunt+scope on the Pi/Jetson for a time-resolved power trace (see the boot inrush and the per-inference power spikes).
  • Extend the Pareto plot to a third axis (accuracy, from Lab 8.2) — the real deployment decision is a 3-way latency/power/accuracy trade.