Architectural limits we know about.

Nothing in the pipeline checks “did the LLM extract every step the user asked for, in the right action types?” If the LLM silently drops a step or misclassifies one, the orchestrator and constraint checker see a smaller, well-formed spec and won’t object. The visual surface (column 2, extracted spec) exists specifically so you can catch this before the pipeline runs further. The user’s eye is the only check.

What fixing this looks like: an automated retry-with-feedback loop at extraction time would cost real tokens and may not improve precision enough to justify until we have an eval set to measure against.

If your instruction contains the substring “100uL” multiple times in different roles (e.g. “Add 100uL of sample, then mix at 100uL volume”), the LLM picks one occurrence as the cite and the verifier trusts that pick. There’s no formal check that the cited instance is the right instance.

What fixing this looks like: a positional cite format (cite by character offset instead of substring) would eliminate the class entirely. Real schema change.

A populated value carries a provenance object. A null value carries nothing. So step.source = None could mean “the LLM correctly inferred there was no source mentioned” or “the LLM forgot to extract a source that was actually there.” Both look identical to downstream stages. Mitigated by the orchestrator’s gap detectors flagging missing required fields, so protocols-with-required-source aren’t silently broken — but truly-optional null fields are unauditable.

What fixing this looks like: a separate OmittedField Provenance subtype that carries the reasoning for the null. Modest schema addition.

The Opentrons simulator catches code-execution problems (loading failures, well-out-of-range, pipette mismatch). It does not validate that the script does what the user asked for. The user’s eye is the final arbiter on intent fidelity.

LocationRef.wells is a list, but LocationRef.wells_provenance is ONE Provenance for the whole list. The verifier checks each well against any cite entry via substring match. When wells extracted from multi-bullet instructions have cites that don’t literally name each well, false-positive fabrication fires per missing well. The detector deduplicates these into one Gap, but the gap modal’s “Current:” label shows the last offending element instead of the full field state.

What fixing this looks like: migrate wells_provenance from one Provenance to List[Provenance] aligned with wells. Per-well grounding, per-well verification. Cascading schema change — touches the extractor prompt, the verifier, the apply layer, and the visual surface. Highest-impact fix on the roadmap.

Substring matching, no semantic understanding. Cite says “top row”, value is A1 → false positive. Synonyms, paraphrases, abbreviations all hit this. Fix requires either per-element cite alignment (see the wells fix above) or a semantic check; both non-trivial.

When you click Accept on a fabrication gap, the system restates the provenance as source=“inferred” with the suggester’s reasoning. The architecture doc says “value untouched.” But the code also overwrites confidence with the suggester’s confidence — the user endorsed the reasoning, not necessarily the suggester’s confidence calibration. Unauthorized rewrite.

What fixing this looks like: one-line code change — preserve the existing confidence and only update source/reasoning/review_status. Trivial.

The live demo runs on a single small Fly machine; pipeline state is per-process. If someone’s mid-run when you hit Start, you get a “demo busy, try in a few minutes” page. Lifting this requires session-keyed thread bridges and a worker queue.

Per-IP rate limit on POST /start. Defense in depth on top of bring-your-own-key — even though visitors pay their own Anthropic costs, we don’t want a bot to exhaust the demo machine’s CPU. Plenty of headroom for a real human clicking around.

The orchestrator’s detect → suggest → review → apply loop runs at most 3 passes. If gaps still remain, the pipeline halts with an error. In practice convergence happens in 1 pass for most protocols; the cap is a safety net against pathological loops.

The thread-bridge between the orchestrator and the browser confirmation handler assumes one pipeline per process. Two concurrent users would race for the bridge. The single-pipeline lock prevents this from breaking, but a multi-user shape requires session-keyed bridges — a real refactor.

Pipeline state lives in memory during a run; static HTML reports are written to the local disk on the Fly machine. Cloud-scale deployment needs a database for run history and object storage for artifacts.

Bring-your-own-key is the only access-control mechanism today. Anyone with an Anthropic key can run the demo. Acceptable at portfolio scale; not at paid-tier scale.

The three pre-orchestrator confirmation modals (initial contents, source containers, labware assignments) write a state_log entry only when the user aborts. On success, the downstream spec reflects their choices but no audit-trail entry records what they kept versus edited. Audit gap.

What fixing this looks like: three lines per modal — after each successful confirmation, write state_log[“stage_2_5_<modal>_confirmed”] = <choices>. Trivial.