Smart eyewear OEM/ODM manufacturer · Shenzhen, China +86 19835872450 sales@cynrin.com
Smart Glasses Technology

Smart Glasses AI Features: Quality Control In Practice

AI feature quality control is unusual in a way that is worth naming before the specifics, because it explains why the discipline in this category is…

By CYNRIN Engineering & Sourcing Team/ Mar 28, 2023/ 14 min read
Smart Glasses AI Features: Quality Control In Practice

AI feature quality control is unusual in a way that is worth naming before the specifics, because it explains why the discipline in this category is different from every other quality control discipline in the range.

Every other quality control discipline in a hardware product checks something that is physically present. The cell either holds charge or it does not. The camera either focuses or it does not. An AI feature is different: it is a behaviour that depends on a model, a network, a service, a language pair, an input and a moment. The unit can be assembled correctly and the feature can still be broken, and the failure is frequently invisible in the factory because the factory's network, the factory's account and the factory's language configuration are not the customer's. So smart glasses ai features smart glasses quality control is a question about testing the behaviour at the point where it is used, rather than at the point where it is built.

Smart glasses ai features: four test layers and the acceptance script

Smart glasses ai features: why bench tests fail

The buyer who has to design this regime

The reader is a brand quality engineer or a sourcing manager who has to specify what the supplier will be held to, and who has discovered that the usual incoming inspection regime does not apply. It applies to hardware. It cannot apply to a behaviour that depends on a service.

That is the practical problem this article solves. The buyer has three things they can ask for: a factory test regime that the factory runs, a buyer acceptance regime that the buyer runs, and a shared regime that both run. Most of the confusion in this category comes from not deciding which of the three a given check belongs to, and then discovering halfway through a production run that a check nobody owned was being performed by somebody who had no access to the account or the network state it needed.

Three reasons the factory test is not enough, and what each one produces

1. The account and the subscription. A feature that requires an account, a login or a subscription can be perfectly functional in a factory and completely non-functional for a customer who has not signed in, or whose trial has expired, or who is on the wrong plan. The symptom is a customer who says the feature does not work, and the factory's evidence that it does is worthless. The check that catches it is a fresh-account test and an expired-plan test, and both need credentials the factory does not hold.

2. The network and the region. A feature that runs in a cloud service is subject to the service's availability, the customer's region, and the local network. A factory test on one network in one country proves nothing about a customer in another. The symptom is a cluster of returns concentrated in one market, which reads like a market problem and is usually a network or a region configuration problem.

3. The language and the input. A translation or voice feature that works in the factory's configuration may not handle the customer's language, the customer's accent, the customer's noise environment, or a phrase outside the test set. This is not a defect; it is a scope boundary, and it has to be written down. The check that catches it is the buyer's own phrase set, run in the buyer's own environment.

Each of the three produces a distinct failure, and none of them is visible to a factory test. That is the whole argument for the four-layer structure below.

AI-generated photograph illustrating smart glasses ai features: quality control in practice — image 1 of 3

The four test layers and what each one can prove

Layer one: the hardware

Does the microphone capture, does the processor run, is the network radio functional, does the power management behave. This is a factory test and it is the layer most quality control systems already cover. It is worth doing well, because a failure here is the only failure in the whole stack that a replacement unit fixes.

Layer two: the service

Does the endpoint respond, does the account authorise, does the subscription state permit the call, is the region supported. This layer requires live credentials in a configuration resembling the customer's. It cannot be run as a factory test with the factory's own account, because the factory's account is entitled and the customer's is not — and the difference between an entitled account and an unentitled one is exactly what the customer experiences.

Layer three: the behaviour

The command set, the latency, the accuracy, the handling of out-of-scope requests, the degradation under noise. This layer has to be run on a script rather than demonstrated, because a demonstration is a single favourable instance and a behaviour is a distribution. The script is the buyer's, for the reason given later in this article.

Layer four: the experience

Pairing, first-run setup, the application's handling of a failed request, what the customer sees when the network drops. This is the layer that produces returns, and it is the layer nobody tests, because it requires a person who has never seen the product.

The acceptance script and who owns each line

What each layer can and cannot prove

The value of separating the layers is that each one produces a different kind of evidence, and each one has a different owner.

Layer one produces a test record against a specification, and the factory can produce it. Layer two produces an endpoint and account log, and it needs a buyer account. Layer three produces a result sheet per unit, and the script has to be the buyer's. Layer four produces session observation notes, and the user has to be the buyer's.

This gives a clean answer to the question buyers ask most often about this category: how much of the regime can be delegated to the factory. The honest answer is the bottom layer, plus the parts of the service layer that can be exercised with a test account the brand provisions and the factory is authorised to use.

The acceptance script, and who has to own it

The eight items

  1. A command set of twenty, written by the buyer, not supplied by the supplier. Each is a thing the feature must do in your use case, and the script is run on every sample, not once.
  2. Latency, measured. Time from the end of speech to the start of the response, in the environment it will be used. A feature that is instant in a quiet office is not the same in a street.
  3. The network-dependency result. With connectivity off, what still works. This one test reveals the architecture better than any specification.
  4. Failure behaviour, in four cases. Unclear input, out-of-scope request, a request in a second language, and a request made in a loud environment. What the user sees in each case is the difference between a feature and a defect generator.
  5. Account and subscription state. Tested at least twice: once in a fresh account, once in an account with the subscription lapsed.
  6. Region and network variation. At least one test from a different network and, if the product is sold in multiple markets, a test on a network in each of them.
  7. The upgrade path. From the shipped firmware version to the current one, and what a customer on the older version experiences. This is the test that is skipped and then regretted.
  8. The data position. Where the audio and the resulting data are processed and stored, and what a customer sees in the application. This is a legal question that quality control cannot answer, and it has to be answered before the test is meaningful.

The quality control gate sequence

Why the buyer has to write the script

This is the single most contested point in the whole regime, and it is worth arguing properly rather than asserting.

The supplier knows which paths work. That is genuine expertise and it is why the supplier should be asked to advise on the script. But the buyer is the only party that knows what the customer will actually ask for, and the twenty commands are supposed to be the twenty things the customer will actually do. A script the supplier writes measures the supplier's product against the supplier's assumptions, and it will pass every time.

The way to get both is to have the buyer write the twenty commands first and the supplier comment on them before the test runs. The buyer keeps ownership of the list; the supplier gets the right to say that three of them are not achievable in this configuration, which is itself valuable information because it is an early warning about what the listing can say.

Making the script an acceptance standard

A script that lives in a spreadsheet and not in the purchase order is a test rather than a standard. The threshold has to be written down in advance, and it has to be a number the buyer is willing to fail a lot over. The realistic thresholds are simple: a number of the twenty commands succeeding in the specified environment, the enumerated offline functions that survive, the latency ceiling, and the four failure cases producing the agreed behaviour.

The consequence of failing has to be agreed as well. In this category the usual arrangement is that a failed pilot lot is not shipped, and that a second evaluation round is funded by the supplier. That second clause is the one worth negotiating, because an AI feature is the part of the product most likely to fail a first acceptance run, and a supplier who will not fund the second round is telling the buyer how much confidence they place in the first.

AI-generated photograph illustrating smart glasses ai features: quality control in practice — image 2 of 3

Where incoming inspection ends and behavioural testing begins

The dividing line, and why it matters commercially

Incoming inspection is a per-lot gate against a specification. It works when the specification describes something physically present, and it does not work for behaviour. The dividing line is therefore not a stage in the process. It is whether the check can be run by a person holding a unit against a written specification, with no network, no account and no service involved.

Everything to the left of that line is incoming inspection and belongs in the supply agreement as a release criterion. Everything to the right of it is acceptance testing and belongs in the programme plan, with the buyer's resources committed and the buyer's script attached.

Test layer, environment required and the artefact it produces

What a lot release actually requires

A lot release for a camera model in our range is straightforward: the unit matches the published specification for the model — resolution tier, cell capacity, charge behaviour, runtime band, charging method — and the per-model certificate register at certification checklist and the battery transport documentation at battery transport are on file for that configuration.

For a lot that carries an AI feature, the same release has to add an artefact that has no precedent in hardware: the result sheet from the acceptance script, tied to a specific firmware build and a specific account configuration. Without that artefact, the lot is released on its hardware merits and the feature position is unknown, which is the situation this whole article is written to prevent.

There is a second practical point about the line. The scripts are cheap in units and expensive in coordination — twenty commands across two units of the shipped build, with the network-off run, in the buyer's environment. The coordination cost is the reason to schedule the acceptance testing early, on the pilot tier rather than after the run tier, and it is the reason a buyer should reserve the test units rather than receive whatever units are left.

Who pays for the acceptance testing, and what has to be negotiated

The cost split in this regime is unusual, and buyers who have run hardware programmes tend to mis-price it. The hardware layer is already inside the unit price. The service layer costs the brand a test account, a small amount of traffic and the time of whoever maintains the account. The behaviour layer costs the brand the twenty commands, the environment, and the people who write the observation sheet. The experience layer costs the brand a stranger's afternoon.

In other words, the largest part of the acceptance cost falls on the buyer, not on the factory, and it falls on the buyer's people rather than on its budget. That is why three commercial points are worth settling before the order rather than during the first campaign.

The first is whether the supplier funds a second evaluation round if the first one fails. An AI feature is the part of the product most likely to fail a first acceptance run, and a supplier who will not fund the second round is telling the buyer how much confidence the supplier places in the first.

The second is whether the supplier can run the script at all, which depends on which layers it is authorised to execute. A supplier with the account configuration can run the behaviour layer; a supplier without it cannot, and the whole regime then depends on the buyer travelling to the factory or the factory travelling to the buyer's environment.

The third is the artefact format. If the result sheet has to be retyped into a buyer's format by the factory's staff after each run, the record stops being trustworthy within a quarter. A result sheet in a defined layout, with the build identifier on it, is what makes the evidence usable across a production run rather than across a single campaign.

AI-generated photograph illustrating smart glasses ai features: quality control in practice — image 3 of 3

Four ways an AI quality control regime produces false confidence

Testing the demo path. A scripted demo runs the happy path, in the factory's configuration, with the factory's account. Every one of those is a controlled variable, and the interesting failures are all in the uncontrolled ones. This is the most common error and the most benign, because it usually shows up in the field rather than in the audit.

Treating a cloud-dependent feature as a hardware function. The moment a feature depends on a service, it has a dependency that the factory does not control, and the test regime has to reflect that. This is the single most common structural error, and it is the one that turns a service outage into a warranty claim.

Not testing the shipped firmware version. The version on the sample and the version on the production unit can differ, and the difference is where the surprise lives. The check is cheap — record the build identifier on every result sheet — and the omission is expensive, because a behaviour change between builds is indistinguishable from a defect when it reaches the customer.

Letting the supplier own the acceptance criteria. This is the one that produces a lot release with no behavioural evidence behind it, because the criteria were written by the party being measured.

Who owns a defect once it leaves the factory

The escalation problem behind them

All four of these errors share a consequence, and it is the one that matters commercially. A defect whose owner is unclear cannot be routed. A service outage, an expired plan and a language misconfiguration all present to the customer as the same thing — the feature does not work — and only two of them are fixed by replacing a unit. The ownership matrix is therefore not a technical diagram. It is the rule that decides whether the warranty budget gets spent on units that were never faulty, and it is why the triage has to happen before the replacement rather than after it.

The production position we can hold, and the one we cannot

What our published position covers

The boundary we hold: we publish the AI features as the catalogue states them, and we will not attach a model name, a language count, a latency, an offline claim or a subscription position to them, because none of those is published to us for the models in this range. We will not repeat a marketing phrase as though it were a specification, and our catalogue's export certification is a company-level capability rather than a per-model certification row, so it is not presented as one.

What we will do is tell you the published configuration accurately, and tell you which of the eight acceptance items require a live service and a customer-like account rather than a bench test — because those are the ones a factory quality control regime will not catch.

We will also be direct about the commercial questions that sit behind the feature: whether it runs on the device or in a service, whether a subscription is required, which language pairs in which direction, and where the data goes. Those four answers determine your retail proposition, and they are not engineering questions.

What we cannot do, stated plainly

We cannot publish an accuracy figure, a latency, a language count, an offline behaviour, a processing location or a data path for any model, because we have not measured them against a stated condition. We also cannot supply a pass rate for the behaviour layer of the script, because the script is the buyer's and the result depends on the buyer's phrases, the buyer's environment and the buyer's account configuration. A supplier who publishes a generic pass rate for this layer is publishing a number produced by a script nobody can see.

What we can do is run the test with you. Send us the twenty commands, the environments and the account configuration, and we will run them on the units you nominate, on the build you specify, and record the results as they come out. The measurement sheet that settles this kind of question is at measuring AI features, the test protocol for a sample is at testing AI features before ordering, and the upgrade considerations that the seventh acceptance item depends on are in AI features upgrade considerations. The specification reading method that produces the rows in the first place is at AI features datasheet reading method.

The job the feature has to do

Tell us what the feature is meant to do and we will tell you what to include in the acceptance script, which of the eight items we can run with you, and which of them need your account and your environment to be closed. The platform background is under AI features technology, and the related application area is at AI content creation solutions.


Related reading

Tell us what the feature is meant to do and we will tell you what to include in the acceptance script.

Chat on WhatsApp
Your experience on this site will be improved by allowing cookies.