End-of-line test system: when automating functional testing pays for itself

Since 2024, a CAN sensor node we design has left our lab only after an automated end-of-line test. In one sequence the bench loads the bootloader and firmware, checks current draw, LEDs and CAN communication, calibrates the sensor against a reference, writes the calibration and the serial number to flash and stores one report per unit. That adds up to almost 600 reports in two years, at a few hundred units a year – a volume the usual payback rules of thumb would not justify. The bench has been there since the first batch, because cycle time was never the reason.

For many OEMs the test bench sits at a contract manufacturer, and what comes back with the boards is a pass/fail label and, at best, a spreadsheet. Using the numbers from two benches we built for CAN sensor nodes, this guide covers when an automated end-of-line test system pays off, what changes when you introduce one, what the first batches teach and what to ask of the supplier who designs and builds it.

Cycle time matters less than it looks

On the first bench the automated part of the sequence takes about 20 seconds, yet one to two minutes pass between units: most of the cycle is loading, unloading and connecting. The second bench tests a CANopen sensor node side by side with a reference unit on the same bus; there a sequence of about 150 steps runs in a minute, and a batch of 12 units is done in under half an hour, handling included. Once measurement is automated, the fixture becomes the bottleneck, and that is where the next investment should go.

The costs that weigh more are not about time:

• Repeatability. Two operators read the same instrument at different points of a transient, interpret a limit differently, skip a step when the line is busy. The outcome starts to depend on who runs the test, not only on the board.
• Traceability. A spreadsheet with a PASSED row for every check says the board passed. It does not say what the reading was or which limit it was judged against. That was the first thing we changed when the second bench moved from spreadsheet reports to test software: every step now records the measured value, its unit, and the lower and upper limits.
• Escapes. Every defective board that reaches a customer costs more than stopping it at the bench would have: shipping, replacement, failure analysis and trust.

Four signs your manual test has hit its limit

1. Every unit must be programmed or calibrated. If end-of-line means loading firmware, writing a serial number or storing a calibration in flash, the test is already a sequence that must run identically on every unit. Automating it pays at a few hundred units a year: we did it from the first batch on both benches.
2. Different operators, different results. If the reject rate changes with the shift or the person, the procedure is no longer the reference: the operator is.
3. Customers or auditors ask for evidence. A shipment that travels with a certificate of conformance, an ISO 9001 quality system or a tender asks for measurement history by serial number, and for the limits each board was released against. Paper answers slowly; a database answers in seconds.
4. A field failure cannot be traced. It happened to us with a firmware update applied in the field, which erased the calibration and the node address on some sensors running an earlier version. The firmware was fixed, and the end-of-line plan now includes a step that updates the firmware and checks that the calibration survives. A test that does not record what it wrote to the unit cannot tell you what was lost.

What an automated end-of-line test system changes

An automated bench is not a faster manual test. It is a test described once and then run the same way every time. The sequence fixes the steps – set the supply, wait, read the current, send a CAN frame, check the answer – and the limit of every measurement. The operator no longer decides when to read or whether a value is acceptable: they load the board, start the cycle and read the verdict.

This is the model behind TestHive, our production and end-of-line test software, presented on the Nexilica systems page and on its own site, testhive.nexilica.com. In TestHive a test is a graph of steps authored in a visual builder and run against the instruments on the bench or against a simulator:

• Bench instruments driven over SCPI – for example the Keithley 2110 multimeter and the Keithley 2280S and TTi CPX400DP power supplies – alongside CAN and CANopen, serial ports and fixture I/O.
• Per-unit traceability: every run is recorded against the unit under test, organized by product, batch and unit, and keyed to the serial number. The report identifier is derived from the product, the serial number and the start time, so two benches never issue the same number.
• Reports on demand in HTML, CSV, XLSX and PDF, with the measured value, unit and limits of every step and the identifiers an ISO 9001 quality system requires. Reports are rendered from the stored results, so a template fix applies to every past run as well.
• A built-in simulator: the same sequence runs with no instrument attached, so it can be written and verified before the bench exists.

What the first batches taught us

• Serial numbers must be unique across variants. On the first bench the same number existed on two variants of the same product; serial numbers now come from a central register.
• A PASSED does not say how many attempts it took. A later review of almost 500 reports found about 6% of units programmed more than once: the reports said PASSED, and only a counter in the firmware told how many times the unit had been on the bench. Recording failed and repeated runs as well is what makes first-pass yield measurable.
• The bench must recognize a unit that is already programmed. One board reached the test with a serial number already written to flash: the blank-device check flagged it instead of overwriting it.
• The measurement ground needs its own wire. About 100 mΩ of needle and cable is enough to shift the converter ground by 22 mV: 0.7% on a 3.3 V rail and several percent on a low reference. A dedicated Kelvin connection removes the error at the source.
• Console messages are not a verdict. A healthy board can print ERROR and FAIL on its console while it boots: the decision has to rest on a precise signal, such as the login prompt, not on searching for a word.
• Limits are approved by whoever owns the product. Failing a board on a criterion the customer has not approved is worse than not measuring it: until a limit is confirmed, the value is recorded without being judged.
• The first real batch tests the operator too. The first production batch on the new software turned up six usability problems, from saving reports to a checkbox placed too close to the Delete button, all fixed the next day.

What to ask of the supplier who builds it

An end-of-line test system is a project, not a purchase, and the deliverables decide how useful it stays after commissioning:

• A test specification you own – which parameters are measured, with which limits and why – written before the fixture is drawn.
• A fixture designed with the board. Accessible test points, reachable connectors and an exposed programming interface cost little at the electronic design stage and a lot afterwards.
• A sequence your team can maintain. In TestHive the person who knows the unit builds the sequence without writing code, and it saves as a JSON file: a new limit or a new variant does not wait for a programmer.
• Data you can export by serial number, in formats your quality system and your contract manufacturer can read.

Lead times depend on what already exists. A new bench, with a bed-of-nails fixture and dedicated instruments, reached its first bring-up about 8 weeks after the order, part of it spent waiting for sample boards and product documentation. When libraries, instruments and fixtures already existed, the first version of our second bench was ready in under two weeks.