The uncomfortable open secret of the manipulation data market is the discard rate. Datasets are marketed by episode count, but what a lab can actually train on is episode count multiplied by the fraction that survives their own quality screening, and that fraction varies enormously between suppliers. The industry's scaling results, including the log-linear data-to-performance relationships NVIDIA researchers have reported, are drawn through usable episodes. Noise does not sit on the curve; it just sits on your storage bill.
Quality in this trade is not a vague virtue. It decomposes into a short list of concrete, checkable properties, and almost all of them are decided at capture time. Here is our list, in rough order of how expensive each one is to get wrong.
Synchronisation accuracy
Everything in a multi-stream episode assumes the streams agree about time. At 30 frames per second, a single frame of misalignment is 33 milliseconds, and a hand in normal motion travels a centimetre or more in that window. Around the moment of contact, which is precisely where manipulation learning lives, a one-frame error means the wrist view and the overhead view describe two different events. Worse, sync errors are usually systematic: one drifting camera taints every episode in a session. We hold sub-frame tolerance, verify it against shared events at both ends of every session, and record the measured drift in the session metadata rather than asserting perfection.
Calibration you can verify, not trust
Camera extrinsics fail silently. A bumped tripod or a re-rigged arm changes the geometry of the whole station while the footage keeps looking fine, and every 3D quantity derived downstream inherits the error. The quality marker to look for is not "we calibrate", which everyone says, but whether calibration artefacts ship with the data, per session, so your team can check reprojection error yourselves. Ours do. A supplier reluctant to include them is telling you something.
Episode boundaries and segmentation discipline
Training pipelines treat episode boundaries as truth. If episodes start wherever the operator happened to press record, the model learns that tasks begin in arbitrary mid-states. Boundary discipline is enforced physically on the floor, neutral start and end positions bracketing every attempt, and it is one of the easiest things to check in a sample batch: scrub to the first and last seconds of twenty episodes and see whether they look like conventions or accidents.
Label taxonomy, agreed before capture
Labels are only as useful as their consistency. A disciplined corpus uses a controlled vocabulary fixed before capture: the same phase names meaning the same observable events in every episode, boundaries anchored to things visible in frame, and inter-annotator agreement measured and reported rather than presumed. The failure mode we see in loosely run corpora is semantic drift, where "grasp" quietly means three different things across three annotation shifts. Once that drift is baked in, no amount of downstream cleverness recovers it.
Success, failure, and the space between
Binary success flags throw away signal in both directions. Real outcomes are graded: fully successful, acceptable with defects, recovered after an error, failed with a cause. Failure episodes are not waste, they are some of the most informative data in a corpus, but only when labelled with a failure class; unlabelled failures mixed into nominal successes are contamination. We also deliberately retain recovery behaviour, the fumble that gets corrected, because deployed robots spend a meaningful share of their time in exactly that regime. This grading matters most on the hardest material; perceptual outcomes on cloth, which we cover in the deformables note, cannot be scored any other way.
Coverage beats raw count
Five hundred near-identical episodes of the same object on the same table compress, in information terms, to a handful. What scales performance is structured variation: object instances, layouts, lighting states, start conditions, swept deliberately against a schedule. This is why serious capture is planned as a coverage matrix rather than a stopwatch target, and why two corpora with identical episode counts can differ several-fold in what a policy extracts from them. When you evaluate a dataset, ask to see the variation schedule, not just the total.
Capture hygiene
The mundane failures are the common ones. Crew reflections in a stainless surface. A demonstrator's logo t-shirt appearing in four thousand episodes, ready to become a spurious feature. Auto-exposure pumping during the reach. Object instances swapped mid-session without a metadata note. Hygiene is unglamorous by definition: controlled wardrobe, closed sets, exposure locked, resets done to protocol, and metadata complete enough that any episode can be traced to its session, station, rig configuration, and demonstrator. Episodes that violate protocol get cut, and the rejection rate is reported to the client, not hidden.
The acceptance test we recommend
Whatever supplier you are evaluating, us included, run the same test. Take a sample batch before committing. Check sync against contact events by eye and by tooling. Compute reprojection errors from the shipped calibration. Scrub episode boundaries. Re-annotate a random slice and measure agreement against the shipped labels. Then train a small probe policy, or at minimum fit a validation metric, and compare against your baseline. A day of screening on a sample batch has saved more programme budgets than any contract clause we have seen. If the batch is egocentric, our note on what egocentric manipulation data should contain doubles as a checklist; if the tasks are dexterous, the failure patterns in fine motor task families are where screening pays off fastest.