Ask a modern anomaly detector to find a scratch on a metal surface, a crack in a casting, or a stain on fabric, and it will do an excellent job. Ask that same class of model to notice that a microplate is sitting in the wrong nest, rotated ninety degrees, or still wearing its lid, and performance falls off a cliff. This is not a tuning problem. It is a structural gap in how visual anomaly detection is built and benchmarked, and it happens to sit exactly where lab automation faults live.

Two kinds of "wrong"

It helps to separate anomalies into two families. Structural (or textural) anomalies are local: a defect appears in a small patch of pixels that looks different from the material around it. A scratch, a dent, a discoloration, a missing thread. You can detect these by learning what "normal texture" looks like and flagging regions that deviate. The rest of the image can be ignored.

Logical (or arrangement) anomalies are global. Every individual object in the scene can look perfectly normal, yet the configuration is wrong: the right parts in the wrong places, the wrong count of an otherwise-fine object, a valid item in an invalid position, or two things that should never co-occur. Nothing is locally defective. The relationship is the defect.

That distinction matters because most of the anomaly-detection literature — and most of the models shipped from it — was optimized for the structural case.

The published gap

The clearest public evidence comes from MVTec LOCO, a benchmark released specifically to separate these two failure modes. It reports scores for structural defects and logical defects side by side, on the same scenes. The pattern in the literature is consistent: methods that score in the mid-80s AUROC on structural/textural defects drop toward the low-60s on the logical-anomaly split. Same detectors, same images, roughly twenty points of AUROC lost the moment the fault becomes about arrangement rather than texture.

The uncomfortable summary: the state of the art can reliably tell you a part is scratched, and struggles to tell you the parts are in the wrong order. For a lab deck, the wrong order is the whole problem.

We cite these figures as public research, not as AssayEye results. The point is not our number; it is that the field's best general-purpose detectors are weakest precisely on the fault class that dominates a workcell deck.

Lab placement faults are logical anomalies

Walk through the faults that actually stop an automated run and notice how few of them are textural:

• Rotated plate — a perfectly good plate, oriented A1-to-the-wrong-corner. Every pixel is "normal."
• Wrong nest — the right labware, seated in the position meant for something else.
• Lid on — a valid lid, present when the next step needs it absent.
• Wrong count — nine tip boxes staged where the method expects ten; each box is fine.
• Hotel/stacker occupancy — a slot that should be empty is full, or vice versa.

None of these is a scratch. They are relationships between known objects and known positions. A texture-first detector, dropped onto this deck, is being asked the one question it was never trained to answer well.

Why factory-floor benchmarks measure the wrong thing

Most off-the-shelf industrial-vision tooling is validated against the structural case, because that is where the historical demand lived: surface inspection, defect grading, cosmetic QC. Those benchmarks reward pixel-level sensitivity to local deviation. Score well there and you have proven you can find a scratch — you have proven almost nothing about whether you can catch a lid-on plate in the wrong nest. Picking a vision stack on factory-floor defect metrics, and then pointing it at a liquid-handler deck, is optimizing for the wrong test.

How we approach it

Our engineering response is to stop treating deck verification as an open-ended anomaly-detection problem, and instead collapse most of it into questions a deterministic system can answer with high confidence. Two ideas do the work.

Geometry first, learning second

The deck is not an unconstrained natural scene. It is a known layout: fixed nest positions, known labware footprints, a defined method that says what belongs where at each step. That structure lets us pose most checks as geometry rather than as learned appearance — is an object present at this coordinate, does its footprint and orientation match the expected part, is the count right, is the lid plane present. These are deterministic checks with clear pass/fail logic, not probabilistic guesses, which is what you want on the safety-relevant path that decides whether to release or hold a run.

A constrained scene that collapses logical faults into position checks

Because the scene is constrained and the expected configuration is known ahead of time, a "logical" fault — wrong arrangement — reduces to a "positional" question we can actually verify: is this object where the method says it should be, in the state the method says it should be in. The hard, global, relationship-reasoning problem becomes a set of local, checkable assertions against a known plan. A small learned layer is then reserved for the genuinely subtle cases where deterministic geometry is not enough — a not-fully-dropped tip, a questionable seal, a mis-seated plate that is only slightly proud of its nest — rather than being asked to carry the whole load.

This is a design philosophy, not a benchmark claim. We are pre-launch and forming a design-partner cohort, and we will publish honest numbers on real decks as we validate them. What we are confident stating today is the shape of the problem: the faults that matter most on a workcell are the faults generic detectors handle worst, and building for that reality means putting deterministic geometry first and using learning where it genuinely earns its place.

See how the smart visual sensors fit together on the platform overview, or book a demo and bring a deck we can reason about.

A modern automated lab is a choreography problem. A scheduler decides what runs when, and a workcell of instruments — liquid handlers, readers, sealers, thermocyclers, hotels — executes the plan while a robotic arm shuttles labware between them. Biosero Green Button Go Orchestrator, driving devices defined in Workflow Designer, is a common conductor for that choreography: it sequences steps, resolves resource conflicts, and keeps a labware map of what should be where at every moment.

That map is the quiet assumption underneath everything. The orchestrator knows a plate should be seated flat in nest B3 with its lid removed, because the previous step told it so. What it does not have is a way to confirm that the plate is actually there, actually flat, and actually unlidded before the next command fires.

The gap: the scheduler moves labware but cannot see it

Most faults in an unattended run are not logic errors — they are physical ones. A plate lands a few millimeters proud of its nest. A gripper rotates labware 180 degrees. A lid that should have been removed is still on. A tip box reads as full but has three empty columns. The scheduler's internal state still says "pass," so it proceeds, and a small placement error becomes a crashed arm, a ruined plate, or — worse — a run that completes and reports clean numbers built on a mistake nobody caught.

The orchestrator knows what it intended to do. It has no independent way to check what actually happened on the deck. That blind spot is exactly where a run goes quietly wrong.

Sensors on the instruments help, but each one sees only its own interior. What is missing is an outside observer of the shared deck — an eye that looks at the physical scene the way a scientist leaning over the bench would, and answers a plain question: does reality match the plan?

How AssayEye slots in

AssayEye is designed to register as a Device Driver in Workflow Designer, so a verification step drops into a workflow like any other instrument action. You place it wherever a physical assumption matters — after a transfer, before a seal, ahead of an arm move — and it runs a small, purpose-built visual check against the region of the deck you care about. Inference runs on-prem, on your hardware, so images never leave the lab.

The check returns a structured result: a pass/fault status plus the specific faults it found and where. From there the workflow branches like any conditional step. This is the integration model we are building toward, and it is deliberately boring by design — no new control plane, just a step the orchestrator already knows how to schedule.

The pass / fault branch

The pattern is a fork. A pass releases the run to continue. A fault sends it down a hold-and-notify branch: pause the arm before its next move, alert the scientist with the annotated reason, and record the event to Data Services / your LIMS so the fault is captured alongside the run record instead of lost.

# Verification step, wired as a Device Driver in Workflow Designer
result = assayeye.check(
    station="thermocycler-1",
    checks=["plate_present", "lid_off", "orientation"],
)

if result.status == "pass":
    orchestrator.continue_run()          # release the next move
else:
    orchestrator.hold(arm)               # stop before the fault propagates
    orchestrator.notify(result.faults)    # alert the scientist, with reasons
    data_services.record(result)        # write the event to LIMS / Data Services

The pseudocode is illustrative — real names follow the Green Button Go step and driver conventions — but the shape is the point: one call, one branch, no bespoke integration code.

Practical tips for a reliable check

Three things separate a check you trust from one you learn to ignore:

• Fix the camera. Mount it rigidly with a stable view of the deck and consistent lighting. A camera that drifts between runs turns every comparison into guesswork.
• Define ROIs, not "the whole scene." Scope each visual sensor to the exact region it judges — this nest, that tip position, this seal edge — so a check answers one question well instead of many questions vaguely.
• Set tolerance to the failure that hurts. Tune how tight "seated correctly" has to be around the physical error you are actually trying to prevent, then hold that line. Too loose misses faults; too tight cries wolf and gets muted.

Start with the one step where a physical mistake costs the most, wire a single verification driver into that branch, and expand from there. If you want the full driver-level view of how AssayEye registers inside Green Button Go, see Integrations — or book a demo and we will map it onto one of your own workflows.

It is the first question every automation lead asks us, and the honest answer is unsatisfying: it depends on the check and the scene. "How many images" is really two questions in a trench coat — what are you asking the eye to decide, and how tightly is the scene controlled when it decides it. Get specific about both and the number falls out. Below is how we think about it while we stand up a smart visual sensor on a new deck.

Some checks need zero training images

A large share of deck verification is geometry and rules, not learned appearance. Is a plate present in nest B3? Is it square to the nest, or rotated past a few degrees? Is a lid sitting on a plate that should be open? Is a tip column full or empty? These reduce to fiducials, edges, occupancy in a known region of interest, and simple color or contrast thresholds against a calibrated reference.

For that class of check, the "training set" is one careful setup pass: you define the regions, you register the camera to deck coordinates, and you write the rule. There is no model to fit, so there is nothing to overfit. This is where we start on almost every integration, because it is the most inspectable and the easiest to reason about when it fires a hold.

Presence and placement: a small reference set, if the scene is fixed

Once you want robustness to normal variation — a slightly different plate vendor, a shadow that moves with the day — a handful of reference images of a known-good deck starts to earn its keep. Our working design target, which we are still validating with design partners, is on the order of ~20 reference images of a good deck under a fixed camera and controlled lighting to anchor a presence or placement check.

Treat that ~20 as a target we are measuring against, not a promise. It holds only because the scene is pinned: same mount, same focus, same illumination, same layout. Under those conditions a small set captures "what right looks like" well enough to flag the obvious wrongs — missing labware, wrong nest, lid-on, mis-seated tips.

The count climbs when the scene moves

Every degree of freedom you leave open multiplies the images you need. Ambient light that changes through the day, a camera on a moving arm instead of a fixed post, mixed labware colors, reflective seals, decks that get reconfigured between protocols — each one widens the distribution the check has to cover, and the reference set has to widen with it. This is the single biggest driver of "why does it need so many images," and it is almost always a scene problem, not a model problem.

Subtle faults need a learned layer on your own runs

A few faults resist rules entirely. Bubbles and foam, a meniscus at a target level, a partial seal, faint well-imaging defects — these are appearance judgments where the boundary between pass and fail is genuinely learned. For those we fit a small model on your good runs, on-prem, so it learns your labware, your liquids, and your lighting rather than a stock dataset. The image count here is honestly higher and case-specific; we scope it per check during a pilot rather than quote a universal figure.

Constrain the scene and the number shrinks

The most useful lever is not more data — it is a tighter scene. A short checklist we walk with every design partner:

• Mount the camera fixed, with locked focus and a known view of the deck.
• Control the lighting: consistent, diffuse, and shielded from room and daylight swings.
• Pin the layout so regions of interest map to stable deck coordinates.
• Standardize labware where you can; note the variants you genuinely need to support.
• Capture references and edge cases during normal operation, not in a staged one-off.
• Start with geometry and rule checks; add a learned layer only where a fault is truly subtle.

So the real answer is a range: zero for geometry, roughly a couple dozen references for fixed-scene presence and placement as a target we are validating, and a scoped learned layer for the subtle stuff. If you tell us the checks you care about and how controlled your workcell is, we can give you a grounded estimate rather than a number pulled from the air.

The fastest way to need fewer images is to give the eye less to be surprised by.