The last gate of an OEM or ODM smart-glasses development programme is not the moment the factory starts producing. It is the moment a controlled production run has proved that the unit you approved is the unit that will ship, at the volume you ordered, on the tooling that has already been paid for. Everything before this stage is designed to make a good unit possible. This stage is designed to find out whether good units survive repetition.
Most programmes do not fail at the engineering bench, where a single unit is carefully assembled and tested by people who know exactly what to look for. They fail in the gap between the last engineering sample and the first shipping carton, because that gap contains the transition from one-off build to repeatable build. The purpose of a pilot run is to make that transition deliberately, under supervision, with the ability to stop and correct — rather than to discover it in your first customer shipment.
This article walks through what a pilot run is for, what it catches that an engineering sample run cannot, how a pilot is inspected and signed off, and what you should settle with your supplier before the transition to volume production happens.
Development process and mass production: the pilot run
A pilot run is a production run performed on production tooling, at production volume, using production materials and production line labour, before the programme is released to unrestricted mass production. The three qualifiers are the whole definition. Remove any one of them and you have something else.
If the tooling used is not the production tooling, you have not tested the tooling. If the quantity is a handful of units rather than a production quantity, you have not tested the line. If the parts come from a pilot allocation or a sample stock, you have not tested the supply chain at all. The value of the stage lies in removing exactly these qualifications.
In practice the pilot answers four questions that no earlier stage can answer. First, can the line physically produce this unit repeatedly, at the intended rate, without the kind of intervention that only a skilled hand can hide? Second, do the components that arrive from the supply chain perform like the components that were approved during engineering validation, or have they drifted? Third, does the finished unit still match the golden sample once it has come off a production line rather than off an engineering bench? Fourth, does the packaging and documentation chain behave the same way at volume, in the sequence a warehouse will actually follow?
For the buyer, this stage is the first point at which the risk profile changes character. Up to this point you have been buying engineering judgement. At this point you are buying process discipline. The unit may be fine and the process may still be wrong, and the difference between those two situations is exactly what the pilot is built to reveal.
A pilot run also produces the first defensible answer to the question every channel procurement manager eventually asks: what does a unit from this programme actually cost to land in our warehouse, including the packaging and documentation steps we specified? That number is only meaningful once the process is stable, which is why asking for it earlier tends to produce a number that changes.
What "production volume" means in practice
Production volume is a line concept, not a marketing one. It means the quantity at which the line runs in its normal configuration, with the normal operators, in the normal sequence, at the normal cadence, without the engineering team standing at the station watching every unit. If the pilot is run at a quantity so small that the line is idle for most of the run, or so large that additional shifts and temporary labour are involved, the pilot is not representative of the steady state in either direction.
The reason this matters is that many of the failure modes discussed below are frequency-dependent. A tool that produces a marginally out-of-tolerance part once in every few hundred shots will not appear in a small run. A mis-stated carton quantity that occurs when a packer changes shift will not appear when the same person assembles the whole run. A tolerance that holds at one ambient condition and drifts at another will not appear if the pilot runs in a single comfortable session. The pilot is designed to be boring, and boring is what repeatable actually looks like.
The standing of the tooling at this point
By the time a pilot run is scheduled, the tooling has been completed and verified, and its cost has been committed. This makes the pilot the point of maximum irreversibility in the whole development process. Earlier stages still allow a buyer to change direction cheaply. At the pilot, a change of specification means re-tooling, and a re-tool is a new commercial event that has to be re-agreed.
This is why the pilot stage exists as a separate gate rather than being folded into the first production order. Once the tooling is paid for and the parts are allocated, the buyer's realistic options reduce to accept, screen, or re-run. A gate that exists while the decision is still cheap is worth more than any assurance offered afterwards.
What the pilot produces for the buyer
A well-run pilot leaves behind a documentary package, and that package is the actual deliverable of the stage. It should contain the pilot production record, the inspection record with results against the written criteria, the disposition of every finding, the retained physical samples, the signed transition approval, and the updated production documentation package. Together these form the evidence base we will rely on for the first purchase order for volume and for any later conversation about consistency between shipments.
We will be direct about the boundary of this stage. A pilot run is a verification activity, not a commercial one. The pilot order is a real order with real terms, and it should be handled as one, but its function is to close the final gate of the development process. A programme that treats the pilot as a discount opportunity to press for a better unit price has changed the meaning of the stage, and the change shows up later as a rushed transition rather than a clean one.
The pilot run is where the development process stops being a document set and becomes a repeatable operation. Everything a brand-side buyer is really buying at this point is not the headset, and not the firmware, but the certainty that the same thing will arrive in the warehouse a second time, a third time, and then several thousand times.

What a pilot run catches that a sample run cannot
An engineering sample run is performed under ideal conditions. Parts are selected, fitted, and checked by the specific engineers who built the programme. Each unit that passes is, by construction, a unit that a person made work. This is exactly why it cannot tell you what happens in the volume case.
The following failures are structural to a one-off build and essentially invisible to it. Each one is a substitution or repetition effect, and each one is routinely found in a pilot.
Component substitution after allocation. During engineering, the exact parts specified for the programme are available because someone reserved them. In the volume case, parts are allocated by the supplier against forecast, against other programmes, and against price and supply position. A part can be replaced with a functionally similar one that has a different acoustic characteristic, a different radio module, a different charge controller tolerance, or a different display behaviour under low light. Nothing in the engineering record changes, because nothing in the part number changed. The pilot run is the first point at which the actual production allocation is in the building.
The tell for this class of problem is a unit that passes every functional test but behaves slightly differently from the golden sample in ways that are not captured by a pass or fail check. This is why the golden sample comparison in the pilot is done by a person who has been given permission to say that something is different, rather than by a checklist that only asks whether something is within tolerance. A substituted component that still passes tolerance but sounds different will be accepted by a checklist and rejected by a golden sample.
Tooling wear and first-run drift. Injection moulding dies, metal stamping tools, and soft tooling all behave differently on the first shots and differently again after a few hundred cycles. Parts produced from a tool on its first run can pass every incoming inspection and still assemble with a slightly different fit, which changes the acoustic chamber, the antenna clearance, or the pressure of a button. An engineering sample built on a freshly commissioned tool is the worst possible environment for detecting this, because the tool has not yet entered its working range.
A further wrinkle here is that a tool can look correct and still be wrong. Dimensions, weight, and appearance can all sit within the agreed range while the surface finish, the draft, or the moulding pressure behaves differently under repeated cycles. The only reliable test is the repetition itself, which is why the pilot volume is not a decorative number.
Assembly drift under line conditions. On an engineering bench, one assembler performs one operation with unlimited time and full attention, and can compensate for a marginal part by hand. On a line, the operation is divided, timed, and repeated by several people with different habits, and the compensation that used to happen silently in engineering simply does not happen. Fit issues, adhesive application variance, screw torque drift, and cable routing differences all appear here and nowhere earlier.
The practical consequence for a buyer is that a defect with a wide tolerance window will spread across the run rather than clustering, while a defect caused by a specific part or a specific station will cluster. The pilot inspection should record the distribution, not just the count, because the distribution tells you whether the run needs re-running, screening, or nothing at all.
Packaging and fulfilment behaviour. Packaging is designed and reviewed as a design, but it is exercised at volume, in warehouse sequence, under the pressure of a shipping deadline. Insert errors, wrong accessories in the carton, missing or duplicated documents, language or label mix-ups, and carton counts that do not match order quantities all show up at this stage. For a brand, these are the failures that reach the end customer with your name on the box.
This category also covers the documentation layer that sits alongside the physical carton. A production document pack that was correct at engineering stage can contain a superseded revision by the time the line runs at volume, because revisions travel faster than people. Asking for the document revision that was actually used in the pilot run, rather than the latest revision on file, is a small question that prevents an expensive misunderstanding.
Yield distribution. An engineering run tells you that a good unit is achievable. A pilot run tells you what proportion of units are good without intervention. The distribution of failures across a line tells you something that no single hand-built unit can: whether a problem is systematic, which affects the whole run, or isolated, which affects a small number of units and can be screened.
A useful way to think about this is that an engineering sample answers "can this design be built at all", while a pilot run answers "can this design be built by this line, this way, repeatedly, at this volume". The second question is the one that determines whether your first shipment is a product or an apology, and it is the question we will insist on seeing answered before any volume order is released.
We will say plainly that a supplier who cannot explain why a pilot failed, in terms of a specific process step and a specific root cause, has not gained much from the stage. The value of a pilot is not that it passed; it is that it produced a defensible answer to a question nobody had asked before the run started.

How a pilot is inspected and signed off
A pilot run is judged against two references at the same time: the golden sample, and the written inspection criteria. Neither is optional, and neither does the other's job.
The golden sample is the signed reference unit that defines the boundary between acceptable and not acceptable. It was established earlier in the programme, and it is what a production unit is physically compared against when a question arises about whether a difference is a defect or a variation. The golden sample defines what normal looks like. The written inspection criteria define what is being measured, how it is measured, and what the pass condition is. A golden sample alone leaves the supplier to decide, per unit, what counts as a difference, which is exactly where a production line drifts over time.
Inspection during a pilot is normally broader than the routine final inspection used on a running production order, because the point of the pilot is to find out what the line produces, not merely to reject what is obviously wrong. Pairing behaviour, audio output, charging behaviour, any camera function and its control behaviour, and the physical fit of the product are all examined. Physical samples are retained from the run so that findings can be reproduced rather than argued about.
The reason the pilot inspection is deliberately wider than the production inspection is a structural one. The production inspection is designed to protect a stable process, and it can therefore be narrow and fast. The pilot inspection is the mechanism that establishes the process is stable, and it has to be broad enough to find the things nobody thought to check. A pilot inspection that simply repeats the routine final check has duplicated a routine and missed the point.
The disposition decision
When the pilot finds a failure, the response is to fix it and re-run, not to ship anyway. This is the single most important behaviour to look for in a supplier, and it is worth writing into the development agreement explicitly rather than assuming it. A pilot that fails and is re-run costs time. A pilot that fails and is ignored costs the same money and none of the knowledge, and it moves the discovery into the customer channel where the correction is worth many times more.
The disposition needs to be one of a small number of explicit choices, and the choice should be recorded with a reason attached:
- Rework. The units can be brought into conformance without a new build. Appropriate where the defect is a correctable assembly or packaging operation.
- Screen and accept. The defective units can be identified and removed, and the remaining population is acceptable. Appropriate only for isolated, detectable, clearly-bounded failures.
- Scrap and re-run. The run is not usable and a new pilot run is required. Appropriate for systematic failures and for anything affecting the tooling or the firmware build.
- Re-run the whole pilot. The process has been corrected and the pilot must be repeated in full to demonstrate the correction works.
The option that does not exist is shipping anyway with a note. That option converts a controlled failure into an uncontrolled one and permanently removes the evidence needed to find the root cause. It is also the option most likely to be proposed informally, which is exactly why the alternatives should be written down before the run rather than improvised during it.
What the sign-off document contains
The transition from pilot to volume production is a signed event, and the document that carries it should be readable by someone outside the programme. It should confirm that the pilot ran on production tooling at production volume, that the inspection was performed against the agreed criteria and the golden sample, that the findings were closed with a recorded disposition, and that the production run is authorised to proceed.
It should also name the person signing, state what they are authorised to commit to, and reference the revisions of the specification, the inspection criteria, and the production documentation that were used during the run. A signature by a production manager who cannot authorise a specification change or a tooling commitment is a signature without a decision behind it, and it is worth asking about before it is requested rather than after.
The transition approval is also the point at which responsibility changes shape. Before it, the programme is a joint piece of work with shared unknowns. After it, there is a product with a defined configuration, a documented inspection method, and a supply relationship. The evidence we will rely on later, when the first shipment is questioned, is exactly this document set, and it is worth checking at the time that it can be read by someone who was not in the programme.
The documents that fall out of this stage — pilot inspection record, disposition of findings, retained samples, signed transition approval, and the updated production documentation package — are what the buyer receives, and they are the evidence base for the first production purchase order and for any later discussion about consistency.
What the buyer should end up holding is a file that lets a third party understand what was produced and how it was judged, without needing a phone call to the factory to interpret it. If that file is thin, the transition to volume is an assumption rather than a decision.

Questions to ask about moving into production
The most useful thing a buyer can do before a pilot run is to settle the commercial and procedural questions in advance, because several of them only become visible once a failure has already happened.
-
Who absorbs the cost of a re-run? If the pilot fails for a reason attributable to the supplier's engineering, line setup, or tooling, the re-run and the resulting scrap should sit with the supplier. If it fails because a requirement changed after the freeze, or because a buyer-side decision arrived late, the cost treatment needs to be decided in advance rather than at the point of the argument. Ask for this in writing, tied to the re-run trigger, before the pilot is scheduled.
-
What exactly triggers a re-run, as opposed to rework or screening? Get the written list. A trigger that says "significant deviation" is not a trigger; a trigger that names the inspection criteria and the defect classes that cannot be screened out is.
-
What is the pilot order, and how does it sit against the first purchase order? Ask whether the pilot units are separate from the first production order or are counted against it, what the incoterms and packaging specification are, and how spares are handled for the transition period. A pilot that is quietly treated as a small first order changes the meaning of the stage.
-
What documentation comes out of the pilot? Ask for the inspection record, the finding dispositions, the retained-sample arrangement, the signed transition approval, and the production documentation package, and ask to receive them as part of the pilot deliverable rather than as a later favour.
-
What is checked on every unit after the transition, and what is checked on a sample? Ask for the final inspection criteria in writing and confirm that they match the pilot criteria, so that the pilot is not a one-off standard that the running line quietly does not apply.
-
Who signs the transition, and what are they authorised to commit to? A production manager signing a transition they cannot authorise on tooling spend or specification change is a signature without a decision behind it.
-
What happens to the tooling if the programme stops after the pilot? This is the mirror image of the tooling question at the engineering stage and it deserves the same written answer, because the tooling is by then fully paid for and the sunk cost is at its largest.
-
What claim is the product allowed to make, and what evidence supports it? Ask what the product documentation and packaging are permitted to state, and what certification evidence sits behind each claim. If the answer is that a claim is stated in the marketing material only, that is a live risk for a channel partner and should be resolved before the volume order, not after a complaint.
-
What spares and rework stock are held, and under what arrangement? Ask how many spare units are held against the first production run, where they are stored, and what the arrangement is for warranty replacement of the first shipments. This is where the MOQ conversation and the warranty conversation actually meet, and it is better to have it before the first carton ships than during a claim.
-
How will you tell me that something has changed after the transition? Ask what notice the supplier will give if a component is substituted, a firmware build changes, or a documentation revision is issued. A supplier with no change-notification process will change something, and you will find out from a customer rather than from a note.
-
What is the trade-off you would make if the pilot and the schedule conflicted? Ask the question in a way that produces an answer rather than a principle. Every supplier will say the pilot takes priority. The informative version is what they would sacrifice if it did not, and whether that sacrifice would be tooling, schedule, quantity, or documentation.
The commercial vocabulary is straightforward once the process is understood, and the sequence we will ask for is the one that puts the risk decisions before the volume order rather than after it. A pilot order carries MOQ, incoterms, lead time, and negotiation on tooling cost and spares exactly like a production order does. The difference is the purpose and the paper trail. Once the transition is signed, the programme becomes an ordinary supply relationship with an ordinary purchase order behind it, and the value of the pilot stage is that the ordinary part arrived without surprises.
We will also note what this stage is not. It is not a place to negotiate a lower unit price, because the quantity involved does not carry the economics of a production run and a discount negotiated there either means little or signals that the pilot will be treated as a small order. It is not a place to introduce new requirements, because a requirement introduced after the freeze belongs to a new version of the product and carries new tooling and new firmware work. And it is not a place to skip the inspection because the run looks clean, because looking clean is the outcome the inspection exists to produce.
Further reading on the wider development method sits in the pillar overview at /oem-odm-process, and the document side of the process is covered in /blog/development-process-documentation. The end-to-end sequence is described in /blog/development-process-how-it-works, and the planning question of what happens to capacity once volume starts is treated separately at /procurement/production-capacity.



